Libvirt external snapshot with GlusterFS

With recent versions of Libvirt and Qemu it is possible to create external disk snapshots while the current and future volumes are connected natively to the GlusterFS cluster via gfapi (not via FUSE).

For this test, the following versions have been used:

• GlusterFS: 3.4.2
• Qemu: 2.0.0
• Libvirt: 1.2.3

Part 1: Preparation

VM-Name

d4572522-eae4-4e3e-ab36-618ae4a91fb4

Original GlusterFS Image Path

virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2

Prepare an XML, here named

snap.xml

:

<domainsnapshot>
  <name>snap</name>
  <disks>
    <disk name='vda' type='network'>
      <driver type='qcow2'/>
      <source protocol='gluster' name='virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2'>
        <host name='10.1.120.11'/>
      </source>
    </disk>
  </disks>
</domainsnapshot>

Part 2: Create the snapshot using virsh

Note: passing --quiesce in addition to the options already specified makes libvirt contact the qemu-guest-agent it expects to run within the VM and calling freeze/thaw as appropriate to get a more-than-crash-consistent image (on Windows using VSS).

virsh snapshot-create d4572522-eae4-4e3e-ab36-618ae4a91fb4 snap.xml --disk-only --atomic

Ex.

vm-test-02 ~ # virsh snapshot-create d4572522-eae4-4e3e-ab36-618ae4a91fb4 snap.xml --disk-only --atomic
Domain snapshot snap created from 'snap.xml'

Part 3: Check

Libvirt

vm-test-02 ~ # virsh snapshot-list d4572522-eae4-4e3e-ab36-618ae4a91fb4
 Name                 Creation Time             State
------------------------------------------------------------
 snap                 2014-05-02 15:16:09 +0200 disk-snapshot

vm-test-02 ~ # virsh snapshot-info d4572522-eae4-4e3e-ab36-618ae4a91fb4 --current
Name:           snap
Domain:         d4572522-eae4-4e3e-ab36-618ae4a91fb4
Current:        yes
State:          disk-snapshot
Location:       external
Parent:         -
Children:       0
Descendants:    0
Metadata:       yes

VM Image Path

vm-test-02 ~ # virsh dumpxml d4572522-eae4-4e3e-ab36-618ae4a91fb4 | grep "\.qcow2"
      <source protocol='gluster' name='virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2'>

Backing File Path

Unfortunately, the domblkinfo command still does not understand natively attached glusterfs volumes, but qemu-img does:

vm-test-02 ~ # qemu-img info gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
image: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 2.6M
cluster_size: 65536
backing file: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false

Cleanup

Unfortunately libvirt does not know how to remove an external snapshot and still has some issues when trying to blockcommit on a file with native-gluster backing file:

vm-test-02 ~ # virsh blockcommit d4572522-eae4-4e3e-ab36-618ae4a91fb4 vda
error: invalid argument: top 'virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2' in chain for 'vda' has no backing file

vm-test-02 ~ # qemu-img info /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2 
image: /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 45G
cluster_size: 65536
backing file: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2
backing file format: qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false

We therefore have to do it via Qemu, below is the example for doing it offline, but the same should be possible via qemu-monitor commands (since qemu-2.0 can block-commit from the top-level image):

virsh shutdown d4572522-eae4-4e3e-ab36-618ae4a91fb4
qemu-img commit gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
virsh edit d4572522-eae4-4e3e-ab36-618ae4a91fb4