== prov-backup-rsnapshot ==
Install the [[stoney_backup:_prov-backup-rsnapshot | prov-backup-rsnasphot ]] daemon script using the package manager:
<pre>
emerge -va sys-apps/sst-prov-backup-rsnapshot
==== Provisioning global configuration ====
The global configuration for the provisioning daemon (which was installed with the first provisioning module and the <code>sys-apps/sst-provisioning</code> package) applies to all provisioning modules running on the server. This configuration therefore contains information about the provisioning daemon itself and no information at all about the specific modules. /etc/Provisioning/Global.conf
<pre>
# Copyright (C) 2012 stepping stone GmbH
#
/* If you want, you can override the log information from the global configuration file this might be useful for debugging */
[Global]
# If true the script logs every information to the log-file.
[Database]
BACKEND = LDAP
SERVER = ldaps://ldapm.stoney-cloudtombstone.org
PORT = 636
ADMIN_USER = cn=Manager,dc=stoney-cloud,dc=org
/* Specifies the service itself. As it is the prov-backup-rsnapshot module, the SERVICE is "Backup" and the TYPE is "Rsnapshot".
* The MODUS is as usual selfcare and the TRANSPORTAPI is LocalCLI. This is because the daemon is running on the same host as the
* backup accounts are provisioned and the commands can be executed on this host using the cli. * For more information about MODUS and TRANSPORTAPI see https://int.stepping-stone.ch/wiki/provisioning.pl#Service_Konfiguration
*/
[Service]
</pre>
= stoney = backup Service Software Configuration =TBD: PKL Bitte kurz eine übersicht als Liste geben, welche Scripts für was sind:* '''writeAccountSize.pl''': Is used for ...* '''snapshot.pl''': Is used for ... == Write Backup Account Size utils ==TBD: PKL Install the backup utils (bitte bei allen Scripts angeben, aus welchem Päckli/ebuild es stammtmultiple scripts which help you to manage and monitor your backup server and backup accounts)using the package manager. If you have already installed For more information about the [[Backup_(Server_Setup)#rsnapshot | rsnapshot]] script, you also have scripts please see the writeAccountSize script. Otherwise follow [[Backup_(Server_Setup)#rsnapshot stoney_backup:_Service_Software | these instructions (installation only)stoney backup Service Software]] === Write Backup Account Size Configuration === /etc/backup-utils/writeAccountSizepage.conf
<pre>
[Global]INCOMING_DIRECTORY = /incomingACCOUNT_SIZE_FILE = /etc/backupSizeSNAPSHOTS = 1 [Syslog]SYSLOG = rsnapshot [Directory]LDAP_SERVER = ldaps://ldapm.tombstone.chLDAP_PORT = 636LDAP_BIND_DN = cn=Manager,dc=stoneyemerge -cloud,dc=orgLDAP_BIND_PW = <password>LDAP_BASE_DN = ou=accounts,ou=backup,ou=services,dc=stoneyva sys-cloud,dc=orgLDAP_PERSON_BASE = ou=people,dc=stoneyapps/sst-cloud,dc=orgLDAP_RESELLER_BASE = ou=reseller,ou=configuration,ou=backup,ou=services,dc=stoney-cloud,dc=orgLDAP_EMAIL_ATTRIBUTE = mail [Notification]EMAIL_SENDER = stepping stone GmbH Supprt <support@stepping-stone.ch>EMAIL_ALERT_THRESHOLD = 85 Salutation_Default_de-CH = Liebe Kundin / Lieber KundeSalutation_m_de-CH = Sehr geehrter HerrSalutation_f_de-CH = Sehr geehrte FrauSalutation_Default_en-GB = Dear customerSalutation_m_en-GB = Dear Mr.Salutation_f_en-GB = Dear Mrs. [MAIL]host = mail.stepping-stone.chport = 587username = support@stepping-stone.chpassword = <password>utils
</pre>
=== Write Backup Account Size Test ===
/usr/libexec/backup-utils/writeAccountSize.pl -U 4000080 -d
<pre>
Debug modus was turned on
Debug sub checkUsersHomeDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersHomeDirectory: The $localUsersHomeDirectory /var/backup/080/4000080/home/4000080 exists
Debug sub checkUsersIncomingDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersIncomingDirectory: $localUsersIncomingDirectory: /incoming
Debug sub checkUsersIncomingDirectory: $localIncomingPath: /var/backup/080/4000080/home/4000080/incoming
Debug sub checkUsersIncomingDirectory: The $localIncomingPath /var/backup/080/4000080/home/4000080/incoming exists
Total Quota: 1048576 kilobytes
Total used Space: 0 kilobytes
Incoming Size: 0 kilobytes
Debug sub getSnapshotsSize: $localUsedQuota: 0
Debug sub getSnapshotsSize: $localSnapshotsSize: 0
Debug writeAccountSize: Working on /var/backup/080/4000080/etc/backupSize
Debug: wrote 1024 0 0 to /var/backup/080/4000080/etc/backupSize
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstBackupSize => 0
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstIncrementSize => 0
Alert Threshold: 85 %
Calculated value: 0
</pre>
Now write some data (200 megaytes in this example) into the users incoming directory and then execute the script again:
dd if=/dev/zero of=/var/backup/080/4000080/home/4000080/incoming/test.zeros bs=1024k count=200
chown 4000080:4000080 /var/backup/080/4000080/home/4000080/incoming/test.zeros
/usr/libexec/backup-utils/writeAccountSize.pl -U 4000080 -d
<pre>
Debug modus was turned on
Debug sub checkUsersHomeDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersHomeDirectory: The $localUsersHomeDirectory /var/backup/080/4000080/home/4000080 exists
Debug sub checkUsersIncomingDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersIncomingDirectory: $localUsersIncomingDirectory: /incoming
Debug sub checkUsersIncomingDirectory: $localIncomingPath: /var/backup/080/4000080/home/4000080/incoming
Debug sub checkUsersIncomingDirectory: The $localIncomingPath /var/backup/080/4000080/home/4000080/incoming exists
Total Quota: 1048576 kilobytes
Total used Space: 204800 kilobytes
Incoming Size: 204800 kilobytes
Debug sub getSnapshotsSize: $localUsedQuota: 204800
Debug sub getSnapshotsSize: $localSnapshotsSize: 0
Debug writeAccountSize: Working on /var/backup/080/4000080/etc/backupSize
Debug: wrote 1024 200 0 to /var/backup/080/4000080/etc/backupSize
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstBackupSize => 209715200
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstIncrementSize => 0
Alert Threshold: 85 %
Calculated value: 19.53125
</pre>
Everything seems to be working fine!
== Snapshots ==
We use rsnapshot - remote filesystem snapshot utility for the actual snapshots and a handful of wrapper scripts, that do things like:
* Read the users and their settings from the LDAP directory.
* Execute rsnapshot according to the users settings.
* Write the backup quotas backup (incoming), iterations (.snapshots) and free space to the users local backupSize file and update the LDAP directory.
* Inform the reseller, customer or user (depending on the settings in the LDAP directory) via mail, if the quota limit has been reached.
* Depending on the users settings in the LDAP directory, warning mail will be sent to the reseller, customer or user, if a backup was not executed on time.
=== rsnaphot configuration directory ===
The users individual rsnapshot configurations are stored under <code>/etc/rsnapshot</code>. Please make sure, that the directory exists:
ls -al /etc | grep rsnapshot
drwx------ 2 root root 64 30. Aug 20:20 rsnapshot
If not, create it:
mkdir /etc/rsnapshot
chmod 700 /etc/rsnapshot
=== snapshot.pl Configuration ===
The snapshot.pl script is responsible for the execution of rsnapshot according to the users settings.
/etc/backup-utils/snapshot.conf
<pre>
[General]
MaxParallelProcesses = 5
Rsnapshot_command = /usr/bin/nice -n 19 /usr/bin/rsnapshot -c /etc/rsnapshot/rsnapshot.conf.%uid% %interval%
[LDAP]
Host = ldaps://ldapm.tombstone.ch
Port = 636
User = cn=Manager,dc=stoney-cloud,dc=org
Password = <Password>
CA_Path = /etc/ssl/certs
Accounts_Base = ou=accounts,ou=backup,ou=services,dc=stoney-cloud,dc=org
</pre>
Legend:
* '''%uid%''': The backup account and login uid as a numeric number. For example: 4000205.
* '''%interval%''': The backup level to be executed. Possible values are hourly, daily, weekly, monthly and yearly.
=== snapshot.pl Test ===
Before adding the necessary cronjob entries, we need to make sure, that we've configured the snapshot.pl script correctly:
/usr/libexec/backup-utils/snapshot.pl --interval daily -d
If everything worked as planned, you should receive feedback looking roughly like:
<pre>
INFO: Starting rsnapshot for interval daily with maximum 5 parallel processes
INFO: Executing snapshot for 4000080
INFO: Executing snapshot for 4000079
INFO: Snapshot process for 4000079 finished in 0.18 seconds with status 0
INFO: Snapshot process for 4000080 finished in 0.19 seconds with status 0
INFO: rsnapshot for all backups done. Took 0.24 seconds
</pre>
Just to make sure, that everything did work out fine, execute <code>writeAccountSize.pl</code> againg:
/usr/libexec/backup-utils/writeAccountSize.pl -U 4000080 -d
<pre>
Debug modus was turned on
Debug sub checkUsersHomeDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersHomeDirectory: The $localUsersHomeDirectory /var/backup/080/4000080/home/4000080 exists
Debug sub checkUsersIncomingDirectory: $localUsersHomeDirectory: /var/backup/080/4000080/home/4000080
Debug sub checkUsersIncomingDirectory: $localUsersIncomingDirectory: /incoming
Debug sub checkUsersIncomingDirectory: $localIncomingPath: /var/backup/080/4000080/home/4000080/incoming
Debug sub checkUsersIncomingDirectory: The $localIncomingPath /var/backup/080/4000080/home/4000080/incoming exists
Total Quota: 1048576 kilobytes
Total used Space: 409600 kilobytes
Incoming Size: 204800 kilobytes
Debug sub getSnapshotsSize: $localUsedQuota: 409600
Debug sub getSnapshotsSize: $localSnapshotsSize: 204800
Debug writeAccountSize: Working on /var/backup/080/4000080/etc/backupSize
Debug: wrote 1024 200 200 to /var/backup/080/4000080/etc/backupSize
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstBackupSize => 209715200
DEBUG: Successfully executed the following modifications for entry uid=4000080,ou=accounts,ou=backup,ou=services,o=stepping-stone,c=ch: sstIncrementSize => 209715200
Alert Threshold: 85 %
Calculated value: 39.0625
</pre>
As you can see, the total used space has risen to 39.0625.
=== Cronjobs ===
After making sure, that everything worked as planned, you can update your crontab entry:
crontab -e
<pre>
...
# Rsnapshot for all users
30 22 * * * /usr/libexec/backup-utils/snapshot.pl --interval daily
15 22 * * sun /usr/libexec/backup-utils/snapshot.pl --interval weekly
00 22 1 * * /usr/libexec/backup-utils/snapshot.pl --interval monthly
...
</pre>
== schedule warning ==
To install the new schedule warning script you have to execute the following commands:
<pre>
cd /var/work/
git clone --recursive https://github.com/stepping-stone/backup-surveillance.git
cd backup-surveillance/bin/
ln -s ../perl-utils/lib/PerlUtil/ PerlUtil
</pre>
=== Configuration ===
vi /var/work/backup-surveillance/etc/config.conf<pre>Please refer to the configuration sections for the different scripts in [XML]SCHEDULE_FILE = %homeDirectory%/incoming/%computerName%/.sepiola_backup/scheduler.xmlSCHEDULE_XSD = %configpath%/../etc/schema/scheduler_schema.xsdBACKUP_ENDED_FILE = %homeDirectory%/incoming/%computerName%/.sepiola_backup/backupEnded.xmlBACKUP_ENDED_XSD = %configpath%/../etc/schema/backupended_schema.xsdBACKUP_STARTED_FILE = %homeDirectory%/incoming/%computerName%/.sepiola_backup/backupStarted.xmlBACKUP_STARTED_XSD = %configpath%/../etc/schema/backupstarted_schema.xsd [TEMPLATE]Salutation_Default_de-CH = Liebe Kundin / Lieber KundeSalutation_m_de-CH = Sehr geehrter HerrSalutation_f_de-CH = Sehr geehrte FrauSalutation_Default_en-GB = Dear customerSalutation_m_en-GB = Dear Mr.Salutation_f_en-GB = Dear Mrs. [LDAP] SERVER = ldapsstoney_backup://ldapm.tombstone.chPORT = 636DEBUG = 1 ADMIN_DN = cn=Manager,dc=foss-cloud,dc=org ADMIN_PASSWORD = <Password> BACKUP_BASE = ou=accounts,ou=_Service_Software | stoney backup,ou=services,dc=foss-cloud,dc=orgPEOPLE_BASE = ou=people,dc=foss-cloud,dc=orgRESELLER_BASE = ou=reseller,ou=configuration,ou=backup,ou=services,dc=foss-cloud,dc=orgSCOPE = sub [MAILService Software]]mailTo = host = mail.stepping-stone.chport = 587username = password = from = </pre>
= Links =