A Workload-Specific Performance Brokerage for Infrastructure Clouds John O’Loughlin Submitted in fulfillment of the requirements for the award of the degree Doctorate of Philosophy within the University of Surrey.
Department of Computer Science, University of Surrey
I confirm that the submitted work is my own work and that I have clearly identified and fully acknowledged all material that is entitled to be attributed to others (whether published or unpublished) using the referencing system set out in the programme handbook. I agree that the University may submit my work to means of checking this, such as the plagiarism detection service Turnitin® UK. I confirm that I understand that assessed work that has been shown to have been plagiarised will be penalised.
5.2 Performance Variation on EC2, Rackspace and GoGrid 136
5.3 Performance Consistency 145
5.4 Workload Specific Variation 150
5.5 Location Dependence 155
5.6 Performance Refresh: The C4 family 159
5.7 Longitudinal Variation 163
5.8 Performance Correlations 169
5.9 Performance Interference from co-location 173
5.10 Summary and Discussion 176
21.6 An On-demand Performance Brokerage 181
6.1 High Level Overview of Broker Operation 183
6.2 The Cloud Exchange (CeX) 188
6.3 Sellers on the CeX 190
6.4 Instance Performance 192
6.5 Performance Tranches and Pricing 198
6.6 The Broker’s Clients 200
6.7 Demand and Pool Management 207
6.8 Model Parameters 209
6.9 Simulation Events and Collected Statistics 209
6.10 Is the Broker Profitable? 213
6.11 Summary 216
22.7 Critique 219
7.1 Critique of Problem Choice 220
7.2 Critique of Chapter 5 224
7.3 Critique of Chapter 6 227
7.4 Chapter Summary 242
24.8 Conclusions and Future Work 245
8.1 Thesis Structure and Summary 246
8.2 Research Problem and Questions 248
8.3 Conclusions 256
8.4 Future Work 258
A1: CPU Architecture and Performance 280
A2: Additional Performance Results 291
A3: SLA Pricing and Correlated Risks 293
A4: Risk Diversification and Instance Seeking 300
This thesis is dedicated to Natasha, without whose support, commitment and love it would not have been possible. And to Dylan, a constant source of love, joy and inspiration, especially as he claims to have completed two PhDs before the age of four.
This is also dedicated with love to Mum, Dad and to Nana. Also to Siobhan, Caroline, Harry and Felisa.
Finally, love to Daniel, Jordon, Finley, Chloe, Alice, Amaia, Saira and Naila, and in the hope that this will prove an inspiration to you all, showing what hard work can bring.
I would like to thank Dr Lee Gillam for giving me the opportunity to work with him on this PhD, in doing so he has helped me to fulfill a long held ambition. In particular, I thank him for his support and encouragement in matters both academic and beyond, enabling me to achieve more than I thought possible at the start. He promised me fame, fortune and hard work, and without doubt has delivered on the latter. I’m sure the former will be forthcoming in time. Finally, I fully absolve him should a spherical cow be found in this document.
I would like to acknowledge and thank the University of Surrey for payment of fees.
Finally, I would like to thank a number of colleagues past and present. In particular, Gary Dear for being a constant source of encouragement and support, even if it did prove somewhat expensive in coffee, and Bin Li for many Cloud related discussions.
Major Cloud providers offer a seemingly unlimited supply of compute resources for rent on-demand, with instances (virtual machines) being particularly popular amongst a range of service offerings. However, variation in performance across supposedly identical instances, supported by heterogeneous hardware, sold at the same price leads to variation in workload execution costs. Indeed, users pay more to have work delivered slower whilst missing out on the concomitant benefits of completing work faster and receive no performance assurances.
To address price, performance, and assurance, we propose and evaluate a broker that re-prices cloud provider instances according to measured performance levels to offer performance-assured instances. Such a broker does not impose changes on Cloud provider business models, but to be viable the broker must be profitable and yet profitable Cloud brokers seem not to exist in the literature.
We investigate broker profitability through simulations that model a commodity exchange analogously to extant financial exchanges, with sellers characterised in accordance with extant major Cloud providers and the broker modelled as a so-called Zero Intelligence (ZI) trading agent. Instance performance data are drawn from extensive benchmarking of Amazon’s Elastic Compute Cloud (EC2), and a Google workload trace comprising some 650,000 jobs provides for buyer willingness.
The maximum profit margin that could be achieved by the proposed broker across multiple demand profiles is 4%, and achieving this requires 4 different types of hardware that exhibit an average performance degrade of 52% from best to worst. A loss is made under a variety of other conditions. At best, such a broker would support a low margin high volume business, leaving it sensitive to market competition, vagaries in demand, exchange transaction fees and gaming strategies that clients may be able to employ. As such, we question the viability of brokers proposed elsewhere which claim to offer performance services of various kinds, despite profitability not having been evaluated and, worse, without operational costs addressed.
Original contributions from this research include: (1) quantification and characterisation of performance variation amongst Cloud instances; (2) a model of instance performance that is qualitatively similar to results found empirically in both cross-sectional and longitudinal studies; (3) an exposition of the underestimation of risk in extant performance improvement strategies; (4) a Cloud broker offering performance-assured instances and the set of assumptions to be met for profitability; and (5) a strategy for minimising risks to performance due to correlated instance usage.