====== Learnexa ====== ===== Backup Solution on Backblaze ===== {{:learnexa_backup.jpg|}} ==== Why BackBlaze ? ==== * No Storage administration and Maintenance * Unlimited storage scalability * Lower prices * Excellent for decentralized cloud servers backup * Highly reliable and Highly available * Can create n number of buckets (Each server has its own bucket - to easily differentiate backups) * Encryption both in-transit (HTTPS-API) and at-rest (via duplicity) ==== Pricing information of BackBlaze ==== {{:learnexa_backup_pricing.jpg|}} {{:learnexa_backup_pricing2.jpg|}} Sample Calculations ------------------- Monthly Fixed Charge - 500GB * 0.005 = $2.5 (No upload charges) Recovery Charges - 500GB * 0.05 = $25 (Download charges - $0.05 per GB) ===== Backup Tool ===== We are considering duplicity (http://duplicity.nongnu.org/) for our backup solution. The tool is command line based solution and is really easy to manage (backup's, restore's and storage life-cycle management) via wrapper called duply (http://duply.net). ==== Key Features ==== - Easy to use - Encrypted and signed archives - Bandwidth and space efficient - Standard file format - Choice of remote protocol (AWS S3, Google Storage, BackBlaze) For more detailed information on its features, please visit; http://duplicity.nongnu.org/features.html ==== Backup Statergy ==== * Full Backup on Sunday's and incremental backups on Week days including Saturday (via Cron and Duply) ==== Installation Instruction ==== Created Installation Script for the CentOS 6.x (only works with python2.7) Its available at : ftp.expertus.com/duplicity_install ==== Configuration Steps ==== The Configuration of Duplicity is really easy, the steps are mentioned in the following links; * http://duplicity.nongnu.org/docs.html * http://duply.net/wiki/index.php/Duply-documentation 1. Create a profile; duply /path/to/ create 2. Edit configuration file; vi /path/to//conf 3. Make the following changes (comment GPG_KEY) Change the parameter accordingly; GPG_PW='_GPG_PASSWORD_' # Remember the password - This is crucial for recovery - Back it up in password manager GPG_OPTS='--compress-algo=bzip2 --personal-cipher-preferences AES256,AES192' SOURCE='/' TARGET='b2://account_id[:application_key]@bucket_name/[some_dir/]' # Make sure to keep a backup Application Key on password manager MAX_AGE=1Y MAX_FULL_BACKUPS=5 MAX_FULLBKP_AGE=2W #how old a full backup must be before a new full backup will be created DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE " VOLSIZE=10 # Default 25MB DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE" Pre Backup File (script) /usr/bin/mysqldump --all-databases -u root -ppw> /tmp/sqldump-$(date '+%F') Post Backup File (script) /bin/rm /tmp/sqldump-$(date '+%F') 4. Make the backup of only specific folders vi /path/to//Exclude - + 5. Setup a cronjob Cronjob # Sundays at 0:00 and old backups are deleted. Incremental backups are performed every day Monday - Saturday. 0 0 * * 7 /usr/bin/duply /path/to/ full_verify_purge --force 0 0 * * 1-6 /usr/bin/duply /path/to/ incr 6. To restore Latest file duply /path/to/ restore /mnt/restore Specific file for with specific version/revision by date duply /path/to/ fetch etc/passwd /root/pw 4D 7. To check status and to send email based on below command or monitor via Zabbix. duply /path/to/ status ==== List of Servers which are currently being managed by the backup solution ==== Production site is already being backed up via Opsource as mentioned below in this wiki. repos.exphosted.com (184.106.207.71) performexa.com (52.74.47.244) Asterisk (production) (50.57.232.54) meeting.expertus.com (107.170.80.241) lrssandbox01.expertusone.com (192.241.162.211) ===== Opsource system state backup ===== Plan: Rec. Enterprise. \\ Benefits: - Located on a different storage system but same location - Choice of location for secondary data backup (locations limited) - Storage calculated after compression and dedup. - No offsite data delivery(disk mail etc) - Restore manually - Backup schedules - NetApp @ RAID-5. - Minor IO performance hit. === Disk Usage as of May 14,2014 === ~93.6G (without dedup and compression) Estimated cost: - **Enterprise: $0.0164/GB/day = 0.0164* 95 * 30 = $46.74 for 30 days** === Retention Period=== day 1-30, daily backup, 12:00-6:00AM \\ === Steps to enable backup === Scroll to the bottom of this page. === Introduction plan *COMPLETE* === |Server|Date|Type| RP| |prodapp01|05/17/2014|System|30 d| |prodapp02|05/17/2014|System|60 d | |prodcollab01|05/17/2014|System| 30 d| |prddb-mast01|05/17/2014|System| 30 d| |prddb-slv01|05/17/2014|System| 30 d| |loadbalancer|05/17/2014|System| 30 d| * Verified servers being backup as scheduled - 19th May 14* === Future plan === |prddb-mast01||MySQL - more frequently, have to evaluate| |prddb-slv01||MySQL - more frequently, have to evaluate| ===== Manual Scripted Backup ===== === Server details === Server IP: 192.168.225.134 \\ Credentials: KeePass \\ Network: Destination NAT via port 8889. \\ Shares: /srv/qb (For QB backups) \\ /media/learnexannkp (for Learnexa backups) \\ ===DB (we will have to increase this once we are out) === -> everyday on site /data00/backups \\ -> everyday off site (SC office) 1 4 * * * rsync -avz --rsh='ssh -p8889' /data00/backups/*.sql.* root@scvpn.exphosted.com:/media/learnexabkp/db/ ==App=== -> every 15 days offsite. (SC office) 1 1 */15 * * tar zcf - deploy --listed-incremental=/app1backup.snar | ssh root@scvpn.exphosted.com -p 8889 'cd /media/learnexabkp/app; cat - > `date '+%Y%m%d%H%M'`.tar.gz' === Future === Discuss backup for svn repo, tracker db, meeting db etc. ==== Instructions for OpSource BaaS as of May 16th 2014==== ** Only the Primary Administrator OR a User with the Server and Backup role assigned by the Primary Administrator can enable Cloud Backups ** Process at high level: \\ - Enable Backup via UI - Request Agent, a unique agent for the system is created. This cannot be shared across systems. - Install Agent. - Configure Agent and Schedule. === Enable Backup === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7df0&fromProcRevs=adb4dff1011878273f6150eafe64ef2f \\ Snapshot: {{:article:how_to_enable_a_server_for_cloud_backup.pdf|Click here}} === Add Backup Client === https://community.opsourcecloud.net/View.jsp?procId=54253c46401a7f1a980184bd227c60bd&from=Browse_fccddea63a95186586515c6ed34efa01 \\ Snapshot: {{:article:how_to_add_a_backup_client_to_a_cloud_server.pdf|Click here}} === Install Agent=== https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7c1d&fromProcRevs=adb4dff1011878273f6150eafe64ef2f \\ Snapshot: {{:article:how_to_install_the_linux_file_backup_client_on_your_cloud_server.pdf|Click here}} === Configure Agent === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7a7b&fromProcRevs=d06838faa8a99da58d62f6db377baa71%7C12ac42a6b460c91f2beaa552191976c4 \\ Snapshot: {{:article:how_do_i_do_a_backup_now_for_my_server_linux_.pdf|Click here}} === Recovery via Agent === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7b5a&fromProcRevs=d06838faa8a99da58d62f6db377baa71 \\ Snapshot: {{:article:how_to_run_a_self-service_file_or_database_config_restore_linux_.pdf|Click here}} === Recovery via UI === == Create a new server with backup - RECOMMENDED== https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7b40&fromProcRevs=6b790184a2899e2ef5108cc91bb37ed5 \\ Snapshot: {{:article:how_to_initiate_a_full_system_out_of_place_server_restore_using_the_administrative_ui.pdf|Click here}} == In place == Note: This is a destructive process, existing files will be overwritten. This should be a last resort. \\ https://community.opsourcecloud.net/View.jsp?procId=82cb50c37384727c4d0187bdd09bff52&from=Browse_fccddea63a95186586515c6ed34efa01 \\ Snapshot: {{:article:how_to_initiate_a_full_system_in_place_server_restore_using_the_administrative_ui.pdf|Click here}} === If all fails, Recovery via Support === Create a support ticket with SEV1 priority. \\ Turn around time depends on amount of storage, typically between 1-4 hours range. === Manage Backup === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7bea&fromProcRevs=d06838faa8a99da58d62f6db377baa71%7Cd06838faa8a99da58d62f6db377baa71 \\ Snapshot: {{:article:how_to_view_and_manage_the_backup_service_for_a_cloud_server.pdf|Click here}} === Disable Backup === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7c2c&fromProcRevs=d06838faa8a99da58d62f6db377baa71 \\ Snapshot: {{:article:how_to_disable_cloud_backup_for_a_server.pdf|Click here}} === Disable/Uninstall Client === https://community.opsourcecloud.net/View.jsp?id=79f956c70c57bca5--76da58ae-13851990be4--7ba2&fromProcRevs=d06838faa8a99da58d62f6db377baa71%7C12a8d7017201dc74acc47e545401babb \\ Snapshot: {{:article:how_to_uninstall_a_backup_client_on_a_linux_cloud_server.pdf|Click here}}