stoney cloud: Nested virtualization

Abstract

This document describes how to set up nested virtualization on a default installation (until it gets enabled by default).

Nested virtualization permits to start another hypervisor (in our case KVM) inside a virtual machine.

With the current version of Qemu/KVM you can expect a performance of about 25% of the host performance (concerning CPU and Memory access) in a guest within a guest (L2). In the future Qemu/KVM should support Intels Virtual EPT which should give 80% of the host performance for L2.

See:

• http://events.linuxfoundation.org/sites/events/files/cojp13_nakajima.pdf
• https://docs.google.com/file/d/0BzyAwvVlQckedmpobUY1Sm0zNWc/edit

For Intel CPUs we are going to enable Shadow VMCS which requires a sufficiently new CPU, otherwise it gets ignored. This should give some performance increase, disable it if it causes problems.

Setup

Login as root, then execute the following:

cat > /etc/modprobe.d/nested-virtualization.conf << EOF
options kvm-intel nested=Y enable_shadow_vmcs=Y
options kvm-amd nested=Y
EOF

You either have to reboot or rmmod/modprobe the corresponding module if no VM is running

VM configuration

The following is largely untested, therefore this is more an instruction on how to figure out what the correct setting is.

First try to start any VM and execute the following to see whether virtualization is available in the guest:

grep vmx /proc/cpuinfo

If there is support, the output should be something like this:

flags		: [...] vmx [...]

If it does not get enabled automatically by libvirt as soon as the hypervisor supports nested virtualization (it should, though), you have to change to XML description of the VM. To that end, use virsh edit on the node after having it defined it and replace the lines:

  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>

with

  <cpu mode='host-model'>
    <model fallback='allow'/>
    <feature policy='require' name='vmx'/>
  </cpu>