Preview of Docker Benchmarking at Flux7
At Flux7, we have been working closely with the Docker team to do performance benchmarking of Docker. Right now, we’ve been trying experiments with native, Docker and KVM with virtualized hardware. We’ve tried a mixture of both micro and macro benchmarks to get a general feel for Docker.
I feel we are already sitting on some very good data. Rather than let perfect be the enemy of good, I’ll get the ball rolling on the conversation about the performance of Docker with a preview of some of the results. Keep in mind, though, the expectation is that this is very much a work in progress.
So before we begin looking at data, I’ll give away the punchline: Docker performed in line with our expectation as an alternative to VMs with very little overhead. We are very excited to see and partake in the changes Docker will bring along with it to the DevOps landscape.
At the same time, we have found ourselves tweaking many knobs in all three environments: Docker, Native and VM to have our results make sense. As a vested interest, it pointed to a lot of needed consultants to help you get started, especially in the beginning stages of Docker as research is carried out and best practices are developed. The way I see it, Docker is in a position similar to AWS: revolutionary; easy to get started on; but to enjoy many of the benefits, you need to have someone guide you and ramp you up to really unlock the benefits or use the technology.
So, now to start sharing some results. One of the first macro benchmarks we looked at was Sysbench configured with default database size. While in our runs, Docker and KVM completed the benchmark in the same duration. There was a 2x difference in the CPU usage when using KVM vs Docker.
[Tweet “Docker performed in line with our expectation as an alternative to VMs with very little overhead”]
The figure below shows a screenshot of CPU usage with Docker at nearly 1.5%. In the case of Docker, since under the hood the processes are running directly on the host, we can see the CPU usage of the Sysbench and the time for requests mysqld from mysqld separately from the host:
The next figure shows the CPU usage when running the same benchmark from within KVM. The CPU usage for the various processes running under KVM is combined under the single process for qemu. We can see the additional overhead in managing a guest kernel in the increased CPU.
While this increase in CPU does not seem significant, these differences start to add up. It points to why at Flux7 we have used Docker to create highly productive development environments that mimic the distributed nature of the services in production. Not only does the extra CPU overhead from each instance add up, but every time you launch a VM, you have to reserve the entire memory space for the running kernel. On the flipside, Docker does run the existing kernel you are using in the host OS, so there is practically zero additional memory overhead. This allows a regular laptop to easily run five or six services for testing.
As mentioned earlier, we are flushing out our microbenchmark results because they create a basis for a higher-level understanding. But, because of the intricacies of the setup, we want to be sure we present an apples to apples comparison to the readers. As a result, we are unable to publish the results right now. We are appreciative of the Docker team in helping us with our research. And the preliminary results are promising for Docker. As we flesh them out, we will continue to share them much like we have shared our benchmarks on AWS.
In the meantime, we hope you’ll find our research useful. We’d love to hear any question or comments you have. If you are currently investigating how you can make your dev team more productive, contact us now to find out more about the high productivity developer workflows we have setup for clients using Docker, Jenkins, Vagrant and Chef.
Update: Performance of Docker vs VMs
(Talk given by Ali Hussain, CTO and Co-Founder at Cloud Open)