Experiments, Lessons Learned, Best Practices & Tutorials
At Flux7 Labs, as part of our consulting, we found ourselves solving the same problems in cost and performance optimization for our clients. We have productized some of these learnings into our product VyScale. A major feature of VyScale is its management of spot instances. We have been working closely with users of spot instances and Amazon to come up with strategies to use spot instances. As part of that, today, we hosted a Hangout on spot instances. The idea was to get people out from working in silos and offering a platform to share knowledge and ideas.
Compute As A Utility
Our CEO, Aater Suleman, guided the discussion. He referenced the following Youtube video by Stephen Fry in which Stephen compares the cloud to power plants replacing individual generators at factories and spurring the industrial revolution. Aater envisioned that spot instances are the natural next step to the process. The need for spot instances is the same as the need for variable pricing in the energy market. Both these markets have a variable demand while having the expenses fixed. Therefore spot instances or a variable pricing scheme is something that all providers will need to eventually add as it solves a real problem.
The CEO of Teevity, Nicholas Fonrose, had some interesting points to add to that question. Right now all other providers are struggling to catch up to Amazon while differentiating from Amazon in some other way. For example they have the minute pricing strategies, Google is easing the process of migrating across regions and availability zones. It doesn’t look like any other cloud provider have spot instances on their horizon any time soon.
Our CTO, Ali Hussain, pointed out that a better cross region support puts a provider in a place for better spot instances support. One of the biggest problems with the current spot instances is that the market place for spot instances is not mature.
The conversation led to the idea of compute brokers of multi-cloud setups that will find the cheapest spot instances in all regions. Nicholas pointed out some major limitations on compute brokers including how it differs from a general utility like electricity. The first difference is the “Data Gravity”. There is a cost of both resources and time to get the data in place for starting a computation on a different compute farm. Other issues includes that cloud computing is not a simple commodity with a variety of different services and that while the demand is elastic, the supply for computing is not nearly as variable as supply for electricity.
The conversation led into some of the rules of spot instances charges and the future of Amazon’s strategies. The following slide shows the facts discussed:
Preventing Loss Of Work
We discussed detecting an early termination and what that enables us to do? Amazon already has early termination notification as a beta feature, but what this feature was capable of was a food for thought. Similarly there was an interest in Amazon allowing rebids. If you want to run a batch job over a long period then as you get further and further into the job you are willing to pay more to prevent losing work due to termination. While this creates a valid customer need for increasing your bid for spot instances, Sanket Dangi, a consultant at 8K miles who has blogged about spot strategies, pointed out that it can lead to fairness issues with some customers hogging the spot instances and other customers not getting the opportunity to participate in the spot marketplace. This is similar to how in the interest of fairness downtown parking meters have maximum time limits on them. Another feature Amazon could provide for solving this problem would be to allow suspending a spot instance instead of terminating on losing the bid price.
Vishnu Sreekumar, who handles operations at Yahoo, pointed out the instance sizing issues and the corresponding price discrepancy in the Spot market. The discussion moved into the future features that Amazon could provide to improve better instance sizing.
Until Next Time!
Stay tuned for a video of the Hangout. It was overall a very useful and thought provoking conversation. The learnings and knowledge sharing came out to be extremely useful and we at Flux7 Labs would encourage more of that. Although we hit several technical difficulties which hindered the presence of an open live audience and the team leads on spot instances at Amazon at the Hangout, we do believe this experiment was a success and plan on expanding our community engagement through similar panel discussions. Please keep your eyes peeled for announcements of similar events.