Difference between revisions of "Libvirt external snapshot with GlusterFS"
From stoney cloud
| [unchecked revision] | [unchecked revision] |
(→Cleanup) |
(→Cleanup) |
||
| Line 89: | Line 89: | ||
</pre> | </pre> | ||
| − | === Cleanup === | + | === Cleanup/Commit (Offline) === |
Unfortunately libvirt does not know how to remove an external snapshot and still has some issues when trying to <code>blockcommit</code> on a file with native-gluster backing file: | Unfortunately libvirt does not know how to remove an external snapshot and still has some issues when trying to <code>blockcommit</code> on a file with native-gluster backing file: | ||
| Line 120: | Line 120: | ||
rm /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2 | rm /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2 | ||
| − | # | + | # unregister the snapshot in libvirt |
| + | virsh snapshot-delete d4572522-eae4-4e3e-ab36-618ae4a91fb4 --current --metadata | ||
</source> | </source> | ||
| + | === Cleanup/Commit (Online) === | ||
| + | |||
| + | Patches for traversing the block chain with direct gluster as backing file are already on the libvirt mailinglist, the same goes for patches fixing the glusterfs storage pool. They will probably become part of libvirt-1.2.5. | ||
| + | Until then, one has to bypass libvirt to execute a block commit. | ||
| + | |||
| + | <pre> | ||
| + | ~ # virsh qemu-monitor-command e43a2954-3914-465f-9391-9e63b52ec2f5 --pretty '{"execute":"block-commit", "arguments": { "device":"drive-virtio-disk0", "base": "gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.qcow2", "top": "gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.snap01.qcow2" } }' | ||
| + | { | ||
| + | "return": { | ||
| + | |||
| + | }, | ||
| + | "id": "libvirt-28" | ||
| + | } | ||
| + | |||
| + | |||
| + | vm-test-02 ~ # virsh blockjob e43a2954-3914-465f-9391-9e63b52ec2f5 vda | ||
| + | Block Commit: [100 %] | ||
| + | ~ # | ||
| + | </pre> | ||
| + | |||
| + | The block job is running on 100% for a long time until finished. | ||
| + | |||
| + | Check the image again: | ||
| + | <pre> | ||
| + | ~ # virsh qemu-monitor-command --hmp e43a2954-3914-465f-9391-9e63b52ec2f5 "info block" | ||
| + | drive-ide0-0-1: /var/virtualization/iso/a473e8ec-3327-4e24-a19b-e1e7a5a791e0.iso (raw, read-only) | ||
| + | Removable device: locked, tray closed | ||
| + | |||
| + | drive-virtio-disk0: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.qcow2 (qcow2) | ||
| + | </pre> | ||
| + | |||
| + | Qemu has now only the original image open again. | ||
| + | |||
| + | Have libvirt forget about the snapshot: | ||
| + | <source lang='bash'> | ||
| + | virsh snapshot-delete e43a2954-3914-465f-9391-9e63b52ec2f5 --current --metadata | ||
| + | </source> | ||
| + | |||
| + | '''TODO''': update the XML | ||
[[Category:Snippets]] | [[Category:Snippets]] | ||
Revision as of 18:12, 8 May 2014
With recent versions of Libvirt and Qemu it is possible to create external disk snapshots while the current and future volumes are connected natively to the GlusterFS cluster via gfapi (not via FUSE).
For this test, the following versions have been used:
- GlusterFS: 3.4.2
- Qemu: 2.0.0
- Libvirt: 1.2.3
Contents
Part 1: Preparation
- VM-Name
- d4572522-eae4-4e3e-ab36-618ae4a91fb4
- Original GlusterFS Image Path
- virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2
snap.xml:
<domainsnapshot> <name>snap</name> <disks> <disk name='vda' type='network'> <driver type='qcow2'/> <source protocol='gluster' name='virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2'> <host name='10.1.120.11'/> </source> </disk> </disks> </domainsnapshot>
Part 2: Create the snapshot using virsh
Note: passing --quiesce in addition to the options already specified makes libvirt contact the qemu-guest-agent it expects to run within the VM and calling freeze/thaw as appropriate to get a more-than-crash-consistent image (on Windows using VSS).
virsh snapshot-create d4572522-eae4-4e3e-ab36-618ae4a91fb4 snap.xml --disk-only --atomic
Ex.
vm-test-02 ~ # virsh snapshot-create d4572522-eae4-4e3e-ab36-618ae4a91fb4 snap.xml --disk-only --atomic Domain snapshot snap created from 'snap.xml'
Part 3: Check
Libvirt
vm-test-02 ~ # virsh snapshot-list d4572522-eae4-4e3e-ab36-618ae4a91fb4 Name Creation Time State ------------------------------------------------------------ snap 2014-05-02 15:16:09 +0200 disk-snapshot vm-test-02 ~ # virsh snapshot-info d4572522-eae4-4e3e-ab36-618ae4a91fb4 --current Name: snap Domain: d4572522-eae4-4e3e-ab36-618ae4a91fb4 Current: yes State: disk-snapshot Location: external Parent: - Children: 0 Descendants: 0 Metadata: yes
VM Image Path
vm-test-02 ~ # virsh dumpxml d4572522-eae4-4e3e-ab36-618ae4a91fb4 | grep "\.qcow2"
<source protocol='gluster' name='virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2'>
Backing File Path
Unfortunately, the domblkinfo command still does not understand natively attached glusterfs volumes, but qemu-img does:
vm-test-02 ~ # qemu-img info gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
image: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 2.6M
cluster_size: 65536
backing file: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2
backing file format: qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
Cleanup/Commit (Offline)
Unfortunately libvirt does not know how to remove an external snapshot and still has some issues when trying to blockcommit on a file with native-gluster backing file:
vm-test-02 ~ # virsh blockcommit d4572522-eae4-4e3e-ab36-618ae4a91fb4 vda
error: invalid argument: top 'virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2' in chain for 'vda' has no backing file
vm-test-02 ~ # qemu-img info /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
image: /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2
file format: qcow2
virtual size: 200G (214748364800 bytes)
disk size: 45G
cluster_size: 65536
backing file: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.qcow2
backing file format: qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
We therefore have to do it via Qemu, below is the example for doing it offline, but the same should be possible via qemu-monitor commands (since qemu-2.0 can block-commit from the top-level image):
virsh shutdown d4572522-eae4-4e3e-ab36-618ae4a91fb4 qemu-img commit gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2 # edit the XML to point to the original image again virsh edit d4572522-eae4-4e3e-ab36-618ae4a91fb4 # remove the snapshot rm /var/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/de7c7c4e-8664-4ac7-b559-53fd52ec461c.snap01.qcow2 # unregister the snapshot in libvirt virsh snapshot-delete d4572522-eae4-4e3e-ab36-618ae4a91fb4 --current --metadata
Cleanup/Commit (Online)
Patches for traversing the block chain with direct gluster as backing file are already on the libvirt mailinglist, the same goes for patches fixing the glusterfs storage pool. They will probably become part of libvirt-1.2.5. Until then, one has to bypass libvirt to execute a block commit.
~ # virsh qemu-monitor-command e43a2954-3914-465f-9391-9e63b52ec2f5 --pretty '{"execute":"block-commit", "arguments": { "device":"drive-virtio-disk0", "base": "gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.qcow2", "top": "gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.snap01.qcow2" } }'
{
"return": {
},
"id": "libvirt-28"
}
vm-test-02 ~ # virsh blockjob e43a2954-3914-465f-9391-9e63b52ec2f5 vda
Block Commit: [100 %]
~ #
The block job is running on 100% for a long time until finished.
Check the image again:
~ # virsh qemu-monitor-command --hmp e43a2954-3914-465f-9391-9e63b52ec2f5 "info block"
drive-ide0-0-1: /var/virtualization/iso/a473e8ec-3327-4e24-a19b-e1e7a5a791e0.iso (raw, read-only)
Removable device: locked, tray closed
drive-virtio-disk0: gluster://10.1.120.11/virtualization/vm-templates/5b77d2f6-061f-410c-8ee7-9e61da6f1927/bbf7796f-90bf-45ab-8645-2894f5dae727.qcow2 (qcow2)
Qemu has now only the original image open again.
Have libvirt forget about the snapshot:
virsh snapshot-delete e43a2954-3914-465f-9391-9e63b52ec2f5 --current --metadata
TODO: update the XML
