Pat Gelsinger, VMware CEO, said that VM managed Containers could be 8% faster than bare metal during the VMworld keynote (@kitcolbert). On the surface, this comment defies logic: bare metal should be the theoretical limit for performance like the speed of light. While I don’t know the specifics of his test case, the claim of improved performance is credible. Let’s explore how.
RackN specializes in bare metal workloads so let me explain how it’s possible in the right cases that containers in VMs benchmark faster than containers alone.
The crux of the argument comes down to two factors:
- Operating systems degrade when key resources are depleted
- CPUs are optimized for virtualization (see NUMA architecture)
Together, these factors conspire to make VMs a design necessity on large bare metal systems.
A large RAM and CPU core system can become saturated with container workloads even in the 10s of containers. In these cases, the performance cost for operating system to manage resources starts to take away from the performance. Since typical hypervisor hosts have a lot of resources, the risk of over saturation is very high.
The solution on high resource hosts is to leverage a hypervisor to partition the resources into multiple operating system instances. That eliminates over saturation and improves throughput for the host. We’re talking about 10 vms with 10 containers instead of 1 host with 100 containers.
In addition to simple partitioning, most CPUs are optimized for virtualization. That means that they can run multiple virtualization operating systems on the same host with minimal overhead. The non-virtualized host does not get to leverage this optimization.
Due to these factors AND with the right tuning, it would be possible to demonstrate improved container performance for hosts that were optimized for running a hypervisor. The same would not hold true for systems that are size optimized for only container workloads. Since the container optimized machines are also much cheaper, the potential performance gain is likely an not a good ROI.
While bare metal will eventually come; this strange optimization reinforces why we expect to see hypervisors continue to be desired in container deployments for a long time.
Three weeks ago, Kubernetes leaders met for a very busy day to reflect and plan how the community was being growing. I was humbled to be part of the Kubernetes Leadership Summit due to my work as the 
That made sense for a multi-vendor project that was deeply integrated with the physical infrastructure and virtualization technologies. The cost of that decision has been high for everyone because we did not converge to shared practices that would drive ease of operations, upgrade or tuning. We ended up with waves of vendors vying to have the the fastest, simplest and openest version.