Difference between revisions of "stoney conductor: prov-backup-kvm"
(→Overview) |
(→createSnapshot) |
||
(50 intermediate revisions by 2 users not shown) | |||
Line 2: | Line 2: | ||
The '''Provisioning-Backup-KVM Daemon''' is written in Perl and uses the mechanisms described under [[stoney core: OpenLDAP directory data organisation]]. | The '''Provisioning-Backup-KVM Daemon''' is written in Perl and uses the mechanisms described under [[stoney core: OpenLDAP directory data organisation]]. | ||
− | = State of the | + | = Workflow = |
+ | == Backup == | ||
+ | This is the simplified workflow for the Provisioning-Backup-KVM Daemon. The Subroutines (create-, export- and commitSnapshot) are shown later. | ||
+ | |||
+ | [[File:KVM-Backup-Workflow.png|thumb|none|650px|Figure 1: Simplified prov-backup-kvm workflow]] | ||
+ | |||
+ | You can modify/update this workflow by editing [[File:KVM-Backup-simple.xmi]] (you may need [http://uml.sourceforge.net/ Umbrello UML Modeller] diagram programme for KDE to display the content properly). | ||
+ | |||
+ | === createSnapshot === | ||
+ | [[File:KVM-Backup-Workflow-Snapshot.png|thumb|none|650px|Figure 2: Detailed workflow for the createsSnaphshot process]] | ||
+ | |||
+ | You can edit this workflow with the following file (you may need umbrello to modify it): [[File:KVM-Backup-Workflow-detailed.xmi]] | ||
+ | |||
+ | See also: [[stoney_conductor:_Backup#createSnapshot | createSnapshot: Basic idea]] | ||
+ | |||
+ | === exportSnapshot === | ||
+ | [[File:KVM-Backup-Workflow-Merge.png|thumb|none|500px|Figure 2: Detailed workflow for the exportSnapshot process]] | ||
+ | |||
+ | You can edit this workflow with the following file (you may need umbrello to modify it): [[File:KVM-Backup-Workflow-detailed.xmi]] | ||
+ | |||
+ | See also: [[ stoney_conductor:_Backup#exportSnapshot | exportSnapshot: Basic idea]] | ||
+ | |||
+ | === commitSnapshot === | ||
+ | [[File:KVM-Backup-Workflow-Retain.png|thumb|none|500px|Figure 3: Detailed workflow for the commitSnapshot process]] | ||
+ | |||
+ | You can edit this workflow with the following file (you may need umbrello to modify it): [[File:KVM-Backup-Workflow-detailed.xmi]] | ||
+ | |||
+ | See also [[stoney_conductor:_Backup#commitSnaphsot | commitSnapshot: Basic idea]] | ||
+ | |||
+ | == Restore == | ||
+ | ''' Restore is currently not implemented ''' | ||
+ | |||
+ | = Configuration = | ||
+ | == Global == | ||
+ | <pre> | ||
+ | # Copyright (C) 2013 stepping stone GmbH | ||
+ | # Switzerland | ||
+ | # http://www.stepping-stone.ch | ||
+ | # support@stepping-stone.ch | ||
+ | # | ||
+ | # Authors: | ||
+ | # Pat Kläy <pat.klaey@stepping-stone.ch> | ||
+ | # | ||
+ | # Licensed under the EUPL, Version 1.1. | ||
+ | # | ||
+ | # You may not use this work except in compliance with the | ||
+ | # Licence. | ||
+ | # You may obtain a copy of the Licence at: | ||
+ | # | ||
+ | # http://www.osor.eu/eupl | ||
+ | # | ||
+ | # Unless required by applicable law or agreed to in | ||
+ | # writing, software distributed under the Licence is | ||
+ | # distributed on an "AS IS" basis, | ||
+ | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either | ||
+ | # express or implied. | ||
+ | # See the Licence for the specific language governing | ||
+ | # permissions and limitations under the Licence. | ||
+ | # | ||
+ | |||
+ | |||
+ | |||
+ | [Global] | ||
+ | # If true the script logs every information to the log-file. | ||
+ | LOG_DEBUG = 1 | ||
+ | |||
+ | # If true the script logs additional information to the log-file. | ||
+ | LOG_INFO = 1 | ||
+ | |||
+ | #If true the script logs warnings to the log-file. | ||
+ | LOG_WARNING = 1 | ||
+ | |||
+ | #If true the script logs errors to the log-file. | ||
+ | LOG_ERR = 1 | ||
+ | |||
+ | # The environment indicates the hostname (fqdn) on which the prov-backup-kvm | ||
+ | # daemon is running | ||
+ | ENVIRONMENT = <STONEY-CLOUD-NODE-NAME> | ||
+ | |||
+ | # All information related to the database (backend) the daemon connects to | ||
+ | [Database] | ||
+ | BACKEND = LDAP | ||
+ | SERVER = <STONEY-CLOUD-LDAP-SERVER> | ||
+ | PORT = <STONEY-CLOUD-LDAP-PORT> | ||
+ | ADMIN_USER = <STONEY-CLOUD-LDAP-BINDDN> | ||
+ | ADMIN_PASSWORD = <STONEY-CLOUD-LDAP-BIND-PASSWORD> | ||
+ | SERVICE_SUBTREE = <STONEY-CLOUD-LDAP-SERVICE-SUBTREE> | ||
+ | |||
+ | # A cookie file will be used to be able to restart the daemon without | ||
+ | # processing every entry again (they appear as new if the daemon is started) | ||
+ | COOKIE_FILE = <STONEY-CLOUD-LDAP-COOKIE-FILE> | ||
+ | |||
+ | # The default cookie just contains an empty CSN, in that way, all entries | ||
+ | # are processed | ||
+ | DEFAULT_COOKIE = rid=001,csn= | ||
+ | |||
+ | # The search filter for the database. Only process entries found with this | ||
+ | # filter | ||
+ | SEARCH_FILTER = (&(entryCSN>=%entryCSN%)(objectClass=*)) | ||
+ | |||
+ | # Indicates the prov-backup-kvm configuration which applies for every | ||
+ | # VM-Pool and every VM if not overwritten by a VM-Pool- or VM-specific | ||
+ | # configuration | ||
+ | STONEY_CLOUD_WIDE_CONFIGURATION = <STONEY-CLOUD-LDAP-PROV-BACKUP-KVM-DEFAULT-CONFIGURATION> | ||
+ | |||
+ | # Configuration concerining the provisioning module | ||
+ | [Service] | ||
+ | |||
+ | # The modus should always be selfcare | ||
+ | MODUS = selfcare | ||
+ | |||
+ | # Which TransportApi is used to execute the commands on the destination system | ||
+ | # TransportApi can be "LocalCLI" or "CLISSH" | ||
+ | TRANSPORTAPI = LocalCLI | ||
+ | |||
+ | # The name of the provisioning service | ||
+ | SERVICE = Backup | ||
+ | |||
+ | # The name of the provisioning type | ||
+ | TYPE = KVM | ||
+ | |||
+ | # The syslog tag (normally service-type) | ||
+ | SYSLOG = Backup-KVM | ||
+ | |||
+ | # All information concerning the gateway (TransportApi) | ||
+ | [Gateway] | ||
+ | HOST = localhost | ||
+ | USER = provisioning | ||
+ | DSA_FILE = none | ||
+ | |||
+ | # Service specific configuration which is not present in the backend | ||
+ | [Backup] | ||
+ | |||
+ | # Which command is used to export files | ||
+ | EXPORT_COMMAND = cp -p | ||
+ | </pre> | ||
+ | |||
+ | == Backend == | ||
+ | ''' Currently the backend configuration is not active. If it is active it will be (maybe some minor modifications) as the following: ''' | ||
+ | |||
+ | In the backend, you need to have at least one configuration which applies for the whole stoney cloud. This configuration is referenced in the [[#Global|global configuration]]. You are able to overwrite the stoney-cloud-wide configuration for | ||
+ | * A VM-Pool | ||
+ | * A single VM | ||
+ | The configuration which applies for the VM is evaluated in the following way: | ||
+ | # Check if the VM has a VM-specific configuration | ||
+ | #* If yes, this one applies | ||
+ | #* If not, continue | ||
+ | # Check if the VM-Pool has a specific configuration | ||
+ | #* If yes, this one applies | ||
+ | #* If not, continue | ||
+ | # The stoney-cloud-wide configuration applies | ||
+ | |||
+ | === Mandatory Configuration-Parameters === | ||
+ | * '''sstBackupNumberOfIterations''': An integer value how many backup iterations should be kept. Default is 1 (for disaster recovery). | ||
+ | * '''sstBackupRootDirectory''': The path to the backup root directory where all iterations of disk-images and state files are stored. Default is file:///var/backup/virtualization. | ||
+ | * '''sstBackupRetainDirectory''': The path to the local retain directory where the temporary snapshots (disk-image and state file) are stored. Default is file:///var/virtualization/retain. | ||
+ | * '''sstRestoreVMWithoutState''': Boolean value which indicates whether or not to restore a virtual machine without the state. Default is FALSE (most often we want to restore the state together with the virtual machine). | ||
+ | * '''sstBackupRamDiskLocation''': Path to the RAM-Disk. Default is /mnt/ramdisk. Because this attribute can be set for the whole FOSS-Cloud, for a specific VM-Pool, for a specific virtual machine or a specific virtual machine template, this attribute is independent from the VM-Nodes. There for no guarantee can be given, that this RAM-Disk exists on all the VM-Nodes. A check for its existence is mandatory! | ||
+ | * '''sstVirtualizationVirtualMachineForceStart''': Force start VM in the case of not being able to restore the VM State during the backup process. TRUE or FALSE, default is FALSE. Attention: If set to TRUE, this could lead to file system inconsistencies in the virtual machine. | ||
+ | * '''sstVirtualizationBandwidthMerge''': Bandwidth of the disk merging process (specifies the maximum I/O rate to allow in Megabyte/s). Default is 0 (unlimited). Integer Attribute, single value. | ||
+ | * '''sstVirtualizationDiskImageFormat''': The format for the new disk image that is created during the backup process. Default is qcow2. | ||
+ | * '''sstVirtualizationDiskImageOwner''': The owner for the new disk image that is created during the backup process. Default is root. | ||
+ | * '''sstVirtualizationDiskImageGroup''' : The group for the new disk image that is created during the backup process. Default is vm-storage. | ||
+ | * '''sstVirtualizationDiskImagePermission''': The permission (in octal representation) for the new disk image that is created during the backup process. Default is 660 (equivalent to 0660). | ||
+ | * '''sstVirtualizationDiskImageDirectoryOwner''': The owner for the new directory where the disk image is located. Default is root. | ||
+ | * '''sstVirtualizationDiskImageDirectoryGroup''': The group for the new directory where the disk image is located. Default is vm-storage. | ||
+ | * '''sstVirtualizationDiskImageDirectoryPermission''': The permission (in octal representation) for the new directory where the disk image is located. Default is 770 (equivalent to 0770). | ||
+ | |||
+ | === Optional Configuration-Parameters === | ||
+ | * '''sstBackupExcludeFromBackup''': Do we want to exclude a virtual machine from the default backup plan? Default is FALSE. | ||
+ | * '''sstVirtualizationVirtualMachineSequenceStop''': Multiple dependencies for the stopping order can be defined. Example: a web VM depends on the corresponding database VM. IA5String, multi valued. This attribute must exist in all of the virtual machine entries, that are to be stopped in a certain order. Example (0,1,2, ... is the order, UUID1, UUID2, ... is the uuid of a virtual machine): | ||
+ | ** 0: UUID1 | ||
+ | ** 1: UUID2 | ||
+ | ** 2: UUID3 | ||
+ | * '''sstVirtualizationVirtualMachineSequenceStart''': Multiple dependencies for the starting order can be defined. Example: a database VM must be started before the corresponding web VM. IA5String, multi valued. This attribute must exist in all of the virtual machine entries, that are to be started in a certain order. Example (0,1,2, ... is the order, UUID1, UUID2, ... is the uuid of a virtual machine): | ||
+ | ** 0: UUID3 | ||
+ | ** 1: UUID2 | ||
+ | ** 2: UUID1 | ||
+ | |||
+ | = Exit codes = | ||
+ | The following list defines the return codes and their meaning for the KVM-Backup script: | ||
+ | <pre> | ||
+ | use constant | ||
+ | { | ||
+ | SUCCESS_CODE => 0, | ||
+ | ERROR_CODE => 1, | ||
+ | |||
+ | TRUE => 1, | ||
+ | FALSE => 0, | ||
+ | |||
+ | # Specific error codes | ||
+ | TEMPLATE_NOT_READABLE => 101, | ||
+ | NO_MACHINE => 102, | ||
+ | NO_XML_DESCRIPTION => 103, | ||
+ | CANNOT_CREATE_SNAPSHOT => 104, | ||
+ | NO_STATE_INFORMATION => 105, | ||
+ | CANNOT_START_MACHINE => 106, | ||
+ | NO_BACKUP_LOCATION => 107, | ||
+ | UNCONSISTENT_BACKUP => 108, | ||
+ | CANNOT_GET_SNAPSHOT => 109, | ||
+ | CANNOT_DELETE_SNAPSHOT => 110, | ||
+ | CANNOT_UPDATE_XML => 111, | ||
+ | NO_SPACE_LEFT => 112, | ||
+ | }; | ||
+ | </pre> | ||
= Next steps = | = Next steps = | ||
+ | * Implement restore | ||
+ | |||
+ | = Source Code = | ||
+ | The source code is located in our GitHub Repository: | ||
+ | |||
+ | https://github.com/stoney-cloud/stoney-conductor/tree/master/prov-backup-kvm | ||
+ | |||
+ | = Links = | ||
− | [[Category:stoney conductor]] | + | [[Category:stoney conductor]][[Category:Provisioning Modules]] |
Latest revision as of 15:39, 27 June 2014
Contents
Overview
The Provisioning-Backup-KVM Daemon is written in Perl and uses the mechanisms described under stoney core: OpenLDAP directory data organisation.
Workflow
Backup
This is the simplified workflow for the Provisioning-Backup-KVM Daemon. The Subroutines (create-, export- and commitSnapshot) are shown later.
You can modify/update this workflow by editing File:KVM-Backup-simple.xmi (you may need Umbrello UML Modeller diagram programme for KDE to display the content properly).
createSnapshot
You can edit this workflow with the following file (you may need umbrello to modify it): File:KVM-Backup-Workflow-detailed.xmi
See also: createSnapshot: Basic idea
exportSnapshot
You can edit this workflow with the following file (you may need umbrello to modify it): File:KVM-Backup-Workflow-detailed.xmi
See also: exportSnapshot: Basic idea
commitSnapshot
You can edit this workflow with the following file (you may need umbrello to modify it): File:KVM-Backup-Workflow-detailed.xmi
See also commitSnapshot: Basic idea
Restore
Restore is currently not implemented
Configuration
Global
# Copyright (C) 2013 stepping stone GmbH # Switzerland # http://www.stepping-stone.ch # support@stepping-stone.ch # # Authors: # Pat Kläy <pat.klaey@stepping-stone.ch> # # Licensed under the EUPL, Version 1.1. # # You may not use this work except in compliance with the # Licence. # You may obtain a copy of the Licence at: # # http://www.osor.eu/eupl # # Unless required by applicable law or agreed to in # writing, software distributed under the Licence is # distributed on an "AS IS" basis, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either # express or implied. # See the Licence for the specific language governing # permissions and limitations under the Licence. # [Global] # If true the script logs every information to the log-file. LOG_DEBUG = 1 # If true the script logs additional information to the log-file. LOG_INFO = 1 #If true the script logs warnings to the log-file. LOG_WARNING = 1 #If true the script logs errors to the log-file. LOG_ERR = 1 # The environment indicates the hostname (fqdn) on which the prov-backup-kvm # daemon is running ENVIRONMENT = <STONEY-CLOUD-NODE-NAME> # All information related to the database (backend) the daemon connects to [Database] BACKEND = LDAP SERVER = <STONEY-CLOUD-LDAP-SERVER> PORT = <STONEY-CLOUD-LDAP-PORT> ADMIN_USER = <STONEY-CLOUD-LDAP-BINDDN> ADMIN_PASSWORD = <STONEY-CLOUD-LDAP-BIND-PASSWORD> SERVICE_SUBTREE = <STONEY-CLOUD-LDAP-SERVICE-SUBTREE> # A cookie file will be used to be able to restart the daemon without # processing every entry again (they appear as new if the daemon is started) COOKIE_FILE = <STONEY-CLOUD-LDAP-COOKIE-FILE> # The default cookie just contains an empty CSN, in that way, all entries # are processed DEFAULT_COOKIE = rid=001,csn= # The search filter for the database. Only process entries found with this # filter SEARCH_FILTER = (&(entryCSN>=%entryCSN%)(objectClass=*)) # Indicates the prov-backup-kvm configuration which applies for every # VM-Pool and every VM if not overwritten by a VM-Pool- or VM-specific # configuration STONEY_CLOUD_WIDE_CONFIGURATION = <STONEY-CLOUD-LDAP-PROV-BACKUP-KVM-DEFAULT-CONFIGURATION> # Configuration concerining the provisioning module [Service] # The modus should always be selfcare MODUS = selfcare # Which TransportApi is used to execute the commands on the destination system # TransportApi can be "LocalCLI" or "CLISSH" TRANSPORTAPI = LocalCLI # The name of the provisioning service SERVICE = Backup # The name of the provisioning type TYPE = KVM # The syslog tag (normally service-type) SYSLOG = Backup-KVM # All information concerning the gateway (TransportApi) [Gateway] HOST = localhost USER = provisioning DSA_FILE = none # Service specific configuration which is not present in the backend [Backup] # Which command is used to export files EXPORT_COMMAND = cp -p
Backend
Currently the backend configuration is not active. If it is active it will be (maybe some minor modifications) as the following:
In the backend, you need to have at least one configuration which applies for the whole stoney cloud. This configuration is referenced in the global configuration. You are able to overwrite the stoney-cloud-wide configuration for
- A VM-Pool
- A single VM
The configuration which applies for the VM is evaluated in the following way:
- Check if the VM has a VM-specific configuration
- If yes, this one applies
- If not, continue
- Check if the VM-Pool has a specific configuration
- If yes, this one applies
- If not, continue
- The stoney-cloud-wide configuration applies
Mandatory Configuration-Parameters
- sstBackupNumberOfIterations: An integer value how many backup iterations should be kept. Default is 1 (for disaster recovery).
- sstBackupRootDirectory: The path to the backup root directory where all iterations of disk-images and state files are stored. Default is file:///var/backup/virtualization.
- sstBackupRetainDirectory: The path to the local retain directory where the temporary snapshots (disk-image and state file) are stored. Default is file:///var/virtualization/retain.
- sstRestoreVMWithoutState: Boolean value which indicates whether or not to restore a virtual machine without the state. Default is FALSE (most often we want to restore the state together with the virtual machine).
- sstBackupRamDiskLocation: Path to the RAM-Disk. Default is /mnt/ramdisk. Because this attribute can be set for the whole FOSS-Cloud, for a specific VM-Pool, for a specific virtual machine or a specific virtual machine template, this attribute is independent from the VM-Nodes. There for no guarantee can be given, that this RAM-Disk exists on all the VM-Nodes. A check for its existence is mandatory!
- sstVirtualizationVirtualMachineForceStart: Force start VM in the case of not being able to restore the VM State during the backup process. TRUE or FALSE, default is FALSE. Attention: If set to TRUE, this could lead to file system inconsistencies in the virtual machine.
- sstVirtualizationBandwidthMerge: Bandwidth of the disk merging process (specifies the maximum I/O rate to allow in Megabyte/s). Default is 0 (unlimited). Integer Attribute, single value.
- sstVirtualizationDiskImageFormat: The format for the new disk image that is created during the backup process. Default is qcow2.
- sstVirtualizationDiskImageOwner: The owner for the new disk image that is created during the backup process. Default is root.
- sstVirtualizationDiskImageGroup : The group for the new disk image that is created during the backup process. Default is vm-storage.
- sstVirtualizationDiskImagePermission: The permission (in octal representation) for the new disk image that is created during the backup process. Default is 660 (equivalent to 0660).
- sstVirtualizationDiskImageDirectoryOwner: The owner for the new directory where the disk image is located. Default is root.
- sstVirtualizationDiskImageDirectoryGroup: The group for the new directory where the disk image is located. Default is vm-storage.
- sstVirtualizationDiskImageDirectoryPermission: The permission (in octal representation) for the new directory where the disk image is located. Default is 770 (equivalent to 0770).
Optional Configuration-Parameters
- sstBackupExcludeFromBackup: Do we want to exclude a virtual machine from the default backup plan? Default is FALSE.
- sstVirtualizationVirtualMachineSequenceStop: Multiple dependencies for the stopping order can be defined. Example: a web VM depends on the corresponding database VM. IA5String, multi valued. This attribute must exist in all of the virtual machine entries, that are to be stopped in a certain order. Example (0,1,2, ... is the order, UUID1, UUID2, ... is the uuid of a virtual machine):
- 0: UUID1
- 1: UUID2
- 2: UUID3
- sstVirtualizationVirtualMachineSequenceStart: Multiple dependencies for the starting order can be defined. Example: a database VM must be started before the corresponding web VM. IA5String, multi valued. This attribute must exist in all of the virtual machine entries, that are to be started in a certain order. Example (0,1,2, ... is the order, UUID1, UUID2, ... is the uuid of a virtual machine):
- 0: UUID3
- 1: UUID2
- 2: UUID1
Exit codes
The following list defines the return codes and their meaning for the KVM-Backup script:
use constant { SUCCESS_CODE => 0, ERROR_CODE => 1, TRUE => 1, FALSE => 0, # Specific error codes TEMPLATE_NOT_READABLE => 101, NO_MACHINE => 102, NO_XML_DESCRIPTION => 103, CANNOT_CREATE_SNAPSHOT => 104, NO_STATE_INFORMATION => 105, CANNOT_START_MACHINE => 106, NO_BACKUP_LOCATION => 107, UNCONSISTENT_BACKUP => 108, CANNOT_GET_SNAPSHOT => 109, CANNOT_DELETE_SNAPSHOT => 110, CANNOT_UPDATE_XML => 111, NO_SPACE_LEFT => 112, };
Next steps
- Implement restore
Source Code
The source code is located in our GitHub Repository:
https://github.com/stoney-cloud/stoney-conductor/tree/master/prov-backup-kvm