Site Tools


Writing /var/www/dokuwiki/data/cache/c/cbf34937568e728f9c2057416066841a.metadata failed
Hotfix release available: 2025-05-14b "Librarian". upgrade now! [56.2] (what's this?)
Hotfix release available: 2025-05-14a "Librarian". upgrade now! [56.1] (what's this?)
New release available: 2025-05-14 "Librarian". upgrade now! [56] (what's this?)
Hotfix release available: 2024-02-06b "Kaos". upgrade now! [55.2] (what's this?)
Hotfix release available: 2024-02-06a "Kaos". upgrade now! [55.1] (what's this?)
New release available: 2024-02-06 "Kaos". upgrade now! [55] (what's this?)
Hotfix release available: 2023-04-04b "Jack Jackrum". upgrade now! [54.2] (what's this?)
Hotfix release available: 2023-04-04a "Jack Jackrum". upgrade now! [54.1] (what's this?)
New release available: 2023-04-04 "Jack Jackrum". upgrade now! [54] (what's this?)
Hotfix release available: 2022-07-31b "Igor". upgrade now! [53.1] (what's this?)
Hotfix release available: 2022-07-31a "Igor". upgrade now! [53] (what's this?)
New release available: 2022-07-31 "Igor". upgrade now! [52.2] (what's this?)
New release candidate 2 available: rc2022-06-26 "Igor". upgrade now! [52.1] (what's this?)
New release candidate available: 2022-06-26 "Igor". upgrade now! [52] (what's this?)
Hotfix release available: 2020-07-29a "Hogfather". upgrade now! [51.4] (what's this?)
New release available: 2020-07-29 "Hogfather". upgrade now! [51.3] (what's this?)
New release candidate 3 available: 2020-06-09 "Hogfather". upgrade now! [51.2] (what's this?)
New release candidate 2 available: 2020-06-01 "Hogfather". upgrade now! [51.1] (what's this?)
New release candidate available: 2020-06-01 "Hogfather". upgrade now! [51] (what's this?)
Hotfix release available: 2018-04-22c "Greebo". upgrade now! [50.3] (what's this?)
Hotfix release available: 2018-04-22b "Greebo". upgrade now! [50.2] (what's this?)
backup

Learnexa

Backup Solution on Backblaze

Why BackBlaze ?

  • No Storage administration and Maintenance
  • Unlimited storage scalability
  • Lower prices
  • Excellent for decentralized cloud servers backup
  • Highly reliable and Highly available
  • Can create n number of buckets (Each server has its own bucket - to easily differentiate backups)
  • Encryption both in-transit (HTTPS-API) and at-rest (via duplicity)

Pricing information of BackBlaze

Sample Calculations
-------------------

Monthly Fixed Charge - 500GB * 0.005 = $2.5 (No upload charges)

Recovery Charges     - 500GB * 0.05  = $25  (Download charges - $0.05 per GB)

Backup Tool

We are considering duplicity (http://duplicity.nongnu.org/) for our backup solution. The tool is command line based solution and is really easy to manage (backup's, restore's and storage life-cycle management) via wrapper called duply (http://duply.net).

Key Features

  1. Easy to use
  2. Encrypted and signed archives
  3. Bandwidth and space efficient
  4. Standard file format
  5. Choice of remote protocol (AWS S3, Google Storage, BackBlaze)

For more detailed information on its features, please visit; http://duplicity.nongnu.org/features.html

Backup Statergy

  • Full Backup on Sunday's and incremental backups on Week days including Saturday (via Cron and Duply)

Installation Instruction

Created Installation Script for the CentOS 6.x (only works with python2.7)

Its available at : ftp.expertus.com/duplicity_install

Configuration Steps

The Configuration of Duplicity is really easy, the steps are mentioned in the following links;

1. Create a profile;
           duply /path/to/<profile> create
2. Edit configuration file;
           vi /path/to/<profile>/conf
3. Make the following changes (comment GPG_KEY)
            
            Change the parameter accordingly;
            
            GPG_PW='_GPG_PASSWORD_' # Remember the password - This is crucial for recovery - Back it up in password manager
            GPG_OPTS='--compress-algo=bzip2 --personal-cipher-preferences AES256,AES192'
            SOURCE='/'
            TARGET='b2://account_id[:application_key]@bucket_name/[some_dir/]' # Make sure to keep a backup Application Key on password manager
            MAX_AGE=1Y
            MAX_FULL_BACKUPS=5
            MAX_FULLBKP_AGE=2W #how old a full backup must be before a new full backup will be created
            DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
            VOLSIZE=10 # Default 25MB
            DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE"
            
            Pre Backup File (script)
            /usr/bin/mysqldump --all-databases -u root -ppw> /tmp/sqldump-$(date '+%F')
            Post Backup File (script)
            /bin/rm /tmp/sqldump-$(date '+%F')
            
4. Make the backup of only specific folders

            vi /path/to/<profile>/Exclude
 
            -<folder_to_ignore> 
            +<folder_to_backup>
5. Setup a cronjob            
             
Cronjob     
       
# Sundays at 0:00 and old backups are deleted. Incremental backups are performed every day Monday - Saturday. 

0 0 * * 7 /usr/bin/duply /path/to/<profile> full_verify_purge --force
0 0 * * 1-6 /usr/bin/duply /path/to/<profile> incr

6. To restore

Latest file

duply /path/to/<profile> restore /mnt/restore

Specific file for with specific version/revision by date

duply /path/to/<profile> fetch etc/passwd /root/pw 4D

7. To check status and to send email based on below command or monitor via Zabbix.

duply /path/to/<profile> status 

List of Servers which are currently being managed by the backup solution

Production site is already being backed up via Opsource as mentioned below in this wiki.

repos.exphosted.com            (184.106.207.71)
performexa.com                 (52.74.47.244)
Asterisk (production)          (50.57.232.54)
meeting.expertus.com           (107.170.80.241)
lrssandbox01.expertusone.com   (192.241.162.211)

Opsource system state backup

Plan: Rec. Enterprise.
Benefits:

  1. Located on a different storage system but same location
  2. Choice of location for secondary data backup (locations limited)
  3. Storage calculated after compression and dedup.
  4. No offsite data delivery(disk mail etc)
  5. Restore manually
  6. Backup schedules
  7. NetApp @ RAID-5.
  8. Minor IO performance hit.

Disk Usage as of May 14,2014

~93.6G (without dedup and compression) Estimated cost:

  1. Enterprise: $0.0164/GB/day = 0.0164* 95 * 30 = $46.74 for 30 days

Retention Period

day 1-30, daily backup, 12:00-6:00AM

Steps to enable backup

Scroll to the bottom of this page.

Introduction plan *COMPLETE*

ServerDateType RP
prodapp0105/17/2014System30 d
prodapp0205/17/2014System60 d
prodcollab0105/17/2014System 30 d
prddb-mast0105/17/2014System 30 d
prddb-slv0105/17/2014System 30 d
loadbalancer05/17/2014System 30 d

* Verified servers being backup as scheduled - 19th May 14*

Future plan

prddb-mast01MySQL - more frequently, have to evaluate
prddb-slv01MySQL - more frequently, have to evaluate

Manual Scripted Backup

Server details

Server IP: 192.168.225.134
Credentials: KeePass
Network: Destination NAT via port 8889.
Shares: /srv/qb (For QB backups)

      /media/learnexannkp (for Learnexa backups) \\
      

DB (we will have to increase this once we are out)

→ everyday on site /data00/backups
→ everyday off site (SC office)

1 4 * * * rsync -avz --rsh='ssh -p8889' /data00/backups/*.sql.* root@scvpn.exphosted.com:/media/learnexabkp/db/
App

→ every 15 days offsite. (SC office)

1 1 */15 * * tar zcf - deploy --listed-incremental=/app1backup.snar | ssh root@scvpn.exphosted.com -p 8889 'cd /media/learnexabkp/app; cat - > `date '+%Y%m%d%H%M'`.tar.gz'

Future

Discuss backup for svn repo, tracker db, meeting db etc.

Instructions for OpSource BaaS as of May 16th 2014

Only the Primary Administrator OR a User with the Server and Backup role assigned by the Primary Administrator can enable Cloud Backups Process at high level:

  1. Enable Backup via UI
  2. Request Agent, a unique agent for the system is created. This cannot be shared across systems.
  3. Install Agent.
  4. Configure Agent and Schedule.

Enable Backup

Add Backup Client

Install Agent

Configure Agent

Recovery via Agent

Recovery via UI

In place

Note: This is a destructive process, existing files will be overwritten. This should be a last resort.
https://community.opsourcecloud.net/View.jsp?procId=82cb50c37384727c4d0187bdd09bff52&from=Browse_fccddea63a95186586515c6ed34efa01
Snapshot: Click here

If all fails, Recovery via Support

Create a support ticket with SEV1 priority.
Turn around time depends on amount of storage, typically between 1-4 hours range.

Manage Backup

Disable Backup

Disable/Uninstall Client

backup.txt · Last modified: 2018/08/31 16:16 (external edit)