A performance Brokerage for Heterogeneous Clouds

6.10 Is the Broker Profitable?

In this section we investigate conditions under which the broker is profitable. We recall that the model has 6 parameters: NUM_CPUS, order_rate, min_charge, spread, peak and tail. Initially we are concerned with identifying combinations of parameter values for which the broker is profitable - if any exist. And to limit our search we begin with values of parameters we may plausibly expect to see based on observations of current providers.

We begin by considering values of NUM_CPUS of 1, 2 and 4, matching, for example, the degree of heterogeneity found in the instance types C4, M2 and C1 respectively. We set order_rate = 1 as this is the smallest possible integer value we can use. We consider 2 values for min_charge: 10 and 60, which we refer to as GCE and EC2 as it corresponds to the minimum charge on those Clouds respectively. Next, we define a narrow workload as one with a peak of up to a 1.03, and tail of up to 1.1, whilst a wide workload is defined as up to 1.1 for the peak and 1.25 for the tail. Finally, for NUM_CPUS = 2 or 4 we define a spread of 1.1 and 1.37 – as seen on the M2 and C1 family respectively running bzip2³⁹.
For each combination of parameters we run 30 simulations and we report a 95% CI for E[PROFIT], in the table below. We also calculate and report the markup, or profit margin, which is the percentage increase/decrease of revenues over costs.
Table : Markup for each combination of parameters simulated. We highlight when the broker is profitable.

NUM_CPUS	min_charge	Wide/narrow	PROFIT/LOSS	Markup
1	GCE	Narrow	LOSS	-13%
		Wide	LOSS	-11%
	EC2	Narrow	LOSS	-3%
		Wide	LOSS	-0.5%
2	GCE	Narrow	LOSS	-12%
		Wide	LOSS	-11%
	EC2	Narrow	LOSS	-1.2%
		Wide	PROFIT	+0.5%
4	GCE	Narrow	LOSS	-10%
		Wide	LOSS	-8%
	EC2	Narrow	PROFIT	+2.5%
		Wide	PROFIT	+4%

From our initial results we observe that a loss is made whenever we have a minimum charge of only 10 minutes, as found on GCE. We suggest that this is because a low minimum charge produces low effective costs for clients, which shrinks the margin for profit as well as reducing demand. Essentially, it is simply cheaper for the clients to find performance for themselves. Further, when the simulations were re-run but with per minute pricing, an even larger loss of over -30% is made.

The smallest loss when using a GCE minimum charge occurs when we have NUM_CPUS = 4 and workloads with a wide profile. Is it possible for GCE to become profitable in this case by increasing the order_rate? We recall the order_rate is the average number of orders arriving at the broker per minute; however an order will only complete successfully if the broker price is less than the effective price for the client, or if the client is risk-averse. We run 30 simulations setting order_rate to 2, then with order_rate = 3 and a final 30 with the order_rate = 4. The broker makes a loss in each case. We do not try any higher values as we note that the average job submission rate in the Google trace data for jobs greater than 10 minutes is 2.97. As such, in order to be profitable the broker requires an order_rate that is at least 25% above the Google submission for jobs of the same durations.
Further, the broker also makes a loss whenever NUM_CPUs at the CeX is 1 i.e. all instances are homogeneous. In this case there is insufficient variation for the broker to profit from at the given level of demand. However, a small loss is made when we have EC2 minimum charges and workloads with a wide profile, and indeed the broker becomes profitable when we increase the order_rate to 2.
The broker can, however, make a profit, and does so when we have NUM_CPUS = 2, with EC2 minimum charges and workloads with a wide profile. Arguably, maintaining homogeneity on the CeX will be difficult, indeed, it will almost certainly only come about if mandated as different sellers are highly likely to have different hardware. However, mandating homogeneous hardware will likely reduce liquidity on the CeX. As such, future CeX will likely offer opportunities for the broker as soon as they become minimally heterogeneous, assuming an EC2 minimum charge and useful benchmark workload with a wide profile.
In the case NUM_CPUS = 4 the broker is profitable using EC2 minimum charges for both narrow and wide workload profiles. We have observed heterogeneity to an extent of NUM_CPUS = 4 on EC2, in both the M1 and C1 class, and indeed we also find it on GCE on both the general purpose and high-cpu classes. We may well expect to find instance types on the CeX to be heterogeneous to this degree. In this case, variation and hence profitability for the broker is largely determined by the performance spread i.e. differences between CPU models, although wide workloads are more profitable than narrow. This is useful for the broker, as they do not have to limit the benchmarks offered to those with a wide profile, as in the case NUM_CPUS = 2. As a consequence the broker can concentrate on offering useful benchmarks.
Of the parameter values considered the broker is most profitable in the case of NUM_CPUS = 4, EC2 minimum charges and workloads with a wide profile. Over a 48 hour period, on average, the broker receives 2989 orders requesting a total of 3447 instances, of which 84% are satisfied. Note that satisfaction requires the broker to not only have the instance available but also that the price offered is acceptable to the client. The returns however are relatively small. On average, every $1 of costs generates $1.04 in revenue. Clearly, this would be a low margin high volume business.
The broker is, however, exposed to gaming. As part of our performance model we randomly generate the CPU rankings, in terms of best to worst, for each workload in order to simulate different rankings found empirically. Suppose a client has determined a negative correlation between two workloads, say w1 and w2. They can request a w1 at tranche C, whilst using it for w2 at a higher level. What would the effect on profitability be if all clients only requested tranche C? We run 30 simulations with NUM_CPUS =4, EC2 minimum charges and workloads with a wide profile. The broker still profits, and we have a 95% CI for E[PROFIT] of ($320, $390), but with a smaller expected profit than without gaming, where we have a 95% CI of ($540, $590). Finally, is it more profitable for the broker to offer a ‘high-value’ service only, by, for example, only selling instances in tranche A? We run 30 simulations using theses parameters; however, every request from a client is for tranche A only. In this case we obtain a 95% CI for E[PROFIT] of ($670, $790). As such, selling tranche A only produces the highest profit.
In our model we explicitly assume that NUM_CPUS = 4 will produce more variation than NUM_CPUS = 2. It is possible that this is not always true. On a CeX where NUM_CPUS > 4 we would expect there to be both more variation than for NUM_CPUS = 4, and also less gaps with the range. The latter makes is easier for the broker to offer multiple tranches. As such, increased heterogeneity will typically lead to more profit.