Changes

Jump to: navigation, search

stoney conductor: VM Backup

1,834 bytes removed, 14:43, 27 June 2014
/* Communication through backend */
Furthermore there is an control instance, which can independently call these three sub-processes for a given machine. Like that, the stoney cloud is able to handle different cases:
=== Backup a single machine ===
The procedure for backing up a single machine is very simple. Just call the three sub-processes (snapshotcreate-, merge export- and retaincommitSnapshot) one after the other. So the control instance would do some very basic stuff:
<source lang="c">
object machine = args[0];
=== Sub-Processes ===
See also [[Libvirt_external_snapshot_with_GlusterFS]]
==== Snapshot createSnapshot ====
For the commands see [[Libvirt_external_snapshot_with_GlusterFS#Part_2:_Create_the_snapshot_using_virsh]]
For the workflow see [[stoney_conductor:_prov-backup-kvm#createSnapshot]]
==== Merge exportSnapshot ====# Check if Simply copy the VM <code>vm-001</code> is runningunderlying image to the backup location#* If not, start the VM in paused state: <syntaxhighlight source lang="bash">virsh start cp --paused vm-001<p /syntaxhighlight># Merge the live-disk-image (<codepath>/path/<to>/images/vm-001<image>.qcow2</code>) with its backing store file (<codepath>/path/<to>/retain/vm-001.qcow2</codebackup>): <syntaxhighlight lang="bash">virsh qemu-monitor-command vm-001 --hmp "block_stream drive-virtio-disk0"</syntaxhighlight>#* '''Please note:''' If a VM has more than just one disk-image, repeat this step for every image. Just increase the number at the end of the command. So command to merge the second disk image would be: <syntaxhighlight lang="bash"location>virsh qemu-monitor-command vm-001 --hmp "block_stream drive-virtio-disk1"</syntaxhighlight># If the machine is running in paused state (means we started it in 1. because it was not running), stop it again: #* <syntaxhighlight lang="bash">virsh shutdown vm-001</syntaxhighlightsource>
'''Please noteFor the workflow see [[stoney_conductor:''' After these steps, the live_prov-diskbackup-image <code>/path/to/image/vm-001.qcow2</code> no longer contains a reference to the image at the retain location (<code>/path/to/retain/vm-001.qcow2</code>). This is important for the [[kvm#Retain | retain processexportSnapshot]].
==== commitSnapshot ====
For the commands see [[Libvirt_external_snapshot_with_GlusterFS#Cleanup.2FCommit_.28Online.29]]
 See also: For the workflow see [[stoney_conductor:_prov_backup_kvm#Merge | Merge workflow]] ==== Retain ====# Move the all the files in from the retain directory (<code>/path/to/retain/</code>) to the backup directory (<code>/path/to/backup/</code>)## Move the VMs state file to the backup directory##* <source lang="bash">mv /path/to/retain/vm_prov-001.state /path/to/backup/vm-001.state</source>kvm## Move the VMs disk image to the backup directory##* <source lang="bash">mv /path/to/retain/vm-001.qcow2 /path/to/backup/vm-001.qcow2</source>##** '''Please note:''' If the VM <code>vm-001</code> has more than just one disk image, repeat this step for each disk image## Move the VMs XML description file to the backup directory##* <source lang="bash">mv /path/to/retain/vm-001.xml /path/to/backup/vm-001.xml</source>## Move the VMs backend entry file to the backup directory##* <source lang="bash">mv /path/to/retain/vm-001.backend /path/to/backup/vm-001.backend</source>  See also [[stoney_conductor:_prov_backup_kvm#Retain | Retain workflowcommitSnapshot]]
== Communication through backend ==
[[File:Daemon-communication.png|800px|thumbnail|none|Figure 1: Communication between the control instance and the prov-backup-kvm daemon through the LDAP backend]]
 
You can modify/update this workflow by editing [[File:Daemon-communication.xmi]] (you may need [http://uml.sourceforge.net/ Umbrello UML Modeller] diagram programme for KDE to display the content properly).
=== Control-Instance Daemon Interaction for creating a Backup with LDIF Examples ===
</pre>
==== Step 06: Start the Merge export Process (Control instance daemon) ====With the setting of the '''sstProvisioningMode''' to '''mergeexport''', the Control instance daemon tells the Provisioning-Backup-KVM daemon to merge export the backing file disk image back into to the current disk imagebackup location.
<pre>
# The attribute sstProvisioningState is set to zero by the fc-brokerd, when sstProvisioningMode is modified to
# merge export (this way the Provisioning-Backup-VKM daemon knows, that it must start the merging export process).
dn: ou=20121002T010000Z,ou=backup,sstVirtualMachine=kvm-005,ou=virtual machines,ou=virtualization,ou=services,o=stepping-stone,c=ch
changetype: modify
-
replace: sstProvisioningMode
sstProvisioningMode: mergeexport
</pre>
==== Step 07: Starting the Merge export Process (Provisioning-Backup-KVM daemon) ====As soon as the Provisioning-Backup-KVM daemon receives the merge export command, it sets the '''sstProvisioningMode''' to '''mergingexporting''' to tell the Control instance daemon and other interested parties, that it is merging exporting the virtual machine or virtual machine templatedisk images.
<pre>
# The attribute sstProvisioningMode is set to merging exporting by the Provisioning-Backup-VKM daemon.
dn: ou=20121002T010000Z,ou=backup,sstVirtualMachine=kvm-005,ou=virtual machines,ou=virtualization,ou=services,o=stepping-stone,c=ch
changetype: modify
replace: sstProvisioningMode
sstProvisioningMode: mergingexporting
</pre>
==== Step 08: Finalizing the Merging export Process (Provisioning-Backup-KVM daemon) ====As soon as the Provisioning-Backup-KVM daemon has executed the merge export command, it sets the '''sstProvisioningMode''' to '''mergedexported''', the '''sstProvisioningState''' to the current timestamp (UTC) and '''sstProvisioningReturnValue''' to zero to tell the Control instance daemon and other interested parties, that the merging export of the virtual machine or virtual machine template disk-images is finished.
<pre>
# The attribute sstProvisioningState is set with the current timestamp by the Provisioning-Backup-VKM daemon, when
-
replace: sstProvisioningMode
sstProvisioningMode: mergedexported
</pre>
==== Step 09: Start the Retain commit Process (Control instance daemon) ====With the setting of the '''sstProvisioningMode''' to '''retaincommit''', the Control instance daemon tells the Provisioning-Backup-KVM daemon to retain (copy and then delete) all commit the necessary files changes from the overlay file to the configured backup location.underlying disk-image
<pre>
# The attribute sstProvisioningState is set to zero by the fc-brokerd, when sstProvisioningMode is modified to
# retain commit (this way the Provisioning-Backup-VKM daemon knows, that it must start the retaining commit process).
dn: ou=20121002T010000Z,ou=backup,sstVirtualMachine=kvm-005,ou=virtual machines,ou=virtualization,ou=services,o=stepping-stone,c=ch
changetype: modify
-
replace: sstProvisioningMode
sstProvisioningMode: retaincommit
</pre>
==== Step 10: Starting the Retain commit Process (Provisioning-Backup-KVM daemon) ====As soon as the Provisioning-Backup-KVM daemon receives the retain commit command, it sets the '''sstProvisioningMode''' to '''retainingcomitting''' to tell the Control instance daemon and other interested parties, that it is retaining committing changes from the necessary files overlay disk-images back to the configured backup locationunderlying ones.
<pre>
# The attribute sstProvisioningMode is set to retaining comitting by the Provisioning-Backup-VKM daemon.
dn: ou=20121002T010000Z,ou=backup,sstVirtualMachine=kvm-005,ou=virtual machines,ou=virtualization,ou=services,o=stepping-stone,c=ch
changetype: modify
replace: sstProvisioningMode
sstProvisioningMode: retainingcommitting
</pre>
==== Step 11: Finalizing the Retaing commit Process (Provisioning-Backup-KVM daemon) ====As soon as the Provisioning-Backup-KVM daemon has executed the retain commit command, it sets the '''sstProvisioningMode''' to '''retainedcomitted''', the '''sstProvisioningState''' to the current timestamp (UTC) and '''sstProvisioningReturnValue''' to zero to tell the Control instance daemon and other interested parties, that the retaining comitting of all the necessary files changes from the overlay disk-images back to the configured backup location underlying ones is finisheddone.
<pre>
# The attribute sstProvisioningState is set with the current timestamp by the Provisioning-Backup-VKM daemon, when
-
replace: sstProvisioningMode
sstProvisioningMode: retainedcomitted
</pre>
==== Step 12: Finalizing the Backup Process (Control instance daemon) ====
As soon as the Control instance daemon notices, that the attribute '''sstProvisioningMode''' ist set to '''retainedcommitted''', it sets the '''sstProvisioningMode''' to '''finished''' and the '''sstProvisioningState''' to the current timestamp (UTC). All interested parties now know, that the backup process is finished, there for a new backup process could be started.
<pre>
# The attribute sstProvisioningState is updated with current time by the fc-brokerd, when sstProvisioningMode is
#* Remove the old backup leaf (the "yesterday-leaf"), and add a new one (the "today-leaf")
#* After this step, the machines are ready to be backed up
# Call the BackupKVMWrapperKVMBackupWrapper.pl script with the machines list as a parameter# Wait for the BackupKVMWrapperKVMBackupWrapper.pl script to finish
# Go again through all machines and update the backup subtree a last time
#* Check if the backup was successful, if yes, set sstProvisioningMode = finished (see also TBD)
[[File:wrapper-interaction.png|500px650px|thumbnail|none|Figure 2: How the two wrapper interact with the LDAP backend]] You can modify/update this workflow by editing [[File:wrapper-interaction.xmi]] (you may need [http://uml.sourceforge.net/ Umbrello UML Modeller] diagram programme for KDE to display the content properly).
* If for some reason something does not work at all, the whole backup process can be deactivated by simply disabling the LDAPKVMWrapper cronjob
= Restore =
'''Attention:''' The restore process is not yet defined / nor implemented. The following documentation is about the old restore process.
== Basic idea ==
The restore process, similar to the backup process, can be divided into three sub-processes:
The actual KVM-Restore process is controlled completely by the Control instance daemon via the OpenLDAP directory. See [[#OpenLDAP Directory Integration|OpenLDAP Directory Integration]] the involved attributes and possible values.
[[File:Daemon-interaction-restore.png|thumb|500px650px|none|Figure 3: Communication between all involved parties during the restore process]]
You can modify/update these interactions by editing [[File:Restore-Interaction.xmi]] (you may need [http://uml.sourceforge.net/ Umbrello UML Modeller] diagram programme for KDE to display the content properly).
== Current Implementation (Restore) ==
'''Attention''': The restore process is not yet defined / nor implemented. The following documentation is about the old restore process.   * Since the prov-backup-kvm daemon is not running on the vm-nodes (c.f. [[stoney_conductor:_Backup#State_of_the_artCurrent_Implementation_.28Backup.29]]), the restore process does not work when clicking the icon in the webinterface. * Resolving the conflicts in the backend and XML description file is not yet done** Actually all steps not executed by prov-backup-kvm are not yet properly implemented (c.f. [[stoney_conductor:_prov_backup_kvm#Restore]])* The implementation is done, but the last step from the [[#Restore_2 | restore process ]] is different:** The <code>virsh restore</code> command is not executed with the <code>--xml</code> option, the XML from the state file is taken when restoring the machine. Therefore the conflicts are not properly resolved. *** --[[User:Pat|Pat]] ([[User talk:Pat|talk]]) 09:41, 29 October 2013 (CET): Currently the [http://search.cpan.org/~danberr/Sys-Virt-1.1.3/lib/Sys/Virt.pm Sys::Virt] library does not support the --xml parameter when restoring a domain
=== How to manually restore a machine from backup ===
3,368
edits