Page 2 of 7 FirstFirst 1234 ... LastLast
Results 11 to 20 of 61

Thread: My Notes for Installing Zimbra OSE on Ubuntu Server 10.04 LTS

  1. #11
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Backup

    Zimbra Backup

    There are various methods of backup that can be utilized but these are the ones I will be using:


    1. Backup to local folder using RSync
    2. Backup individual mailboxes
    3. Backup using LVM snapshots


    If I have time, I will also explore the setup of an additional server to serve as a hot-spare so-to-speak. I'd like to see a backup server configured to receive backups from the production server and in the event the production server goes dark, spin up the backup server to be its replacement.

    For the 1st method of using RSync, there is one major catch using RSync alone in that you cannot backup files currently in use. That means you have to shutdown the services in order to get a complete backup. With this in mind, my solution is to utiliize two scripts. The 1st script will backup what it can while the server is "online" and the 2nd script will run during off-hours to shutdown the services (make it offline) and perform a complete backup.

    The online script will be scheduled to run multiple times throughout the day and will not cause any downtime.

    The offline script will be scheduled once per day at a time that will cause the least amount of impact because the server will go offline for about 3 to 5 minutes. It will then compress the backup into a single archive and move it to an offsite server.

    My offsite storage will be on a Windows 2008 server. The 1st thing to do will be the configuration of the Windows server to allow the Ubuntu server to connect and manage files.




    Backup - RSync while online

    rsync-online.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : rsync-online.sh
    ## Version       : 1.1
    ## Date          : 2011-11-04
    ## Author        : LHammonds
    ## Compatibility : Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE
    ## Purpose       : Partial backup while Zimbra is online.
    ## Run Frequency : Several times throughout the day.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##   0 = success
    ##   1 = rsync failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-21 LTH Created script.
    ## 2011-11-04 LTH Moved common variables and functions to external file.
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/rsync.log"
    TARGET="${BACKUPDIR}/zimbra"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    ERRORFLAG=0
    RETURNVALUE=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_cleanup()
    {
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
      echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup exit code: ${ERRORFLAG}" >> ${LOGFILE}
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    if [ -f ${LOCKFILE} ]; then
      ## rsync lock file detected.  Abort script.
      f_sendmail "Online RSync Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup mode started." >> ${LOGFILE}
    
    ## Output the current version to a text file which will be included in the backup.
    ## NOTE: You can also view /opt/zimbra/.install_history
    su - zimbra -c "zmcontrol -v > ${ZIMBRADIR}/conf/zimbra_version.txt"
    
    ## rsync will copy what it can to the target location.
    rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGET} 1>/dev/null 2>&1
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## ERROR: Send email notification.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Online RSync failed. ${ZIMBRADIR} -> ${TARGET}" >> ${LOGFILE}
      f_sendmail "Online RSync Backup Failure - rsync" "ERROR: Online rsync failed. ${ZIMBRADIR} -> ${TARGET}, RETURN VALUE = ${RETURNVALUE}"
      ERRORFLAG=${ERRORFLAG} + 1
    fi
    ## Document statistics.
    echo "`date +%Y-%m-%d_%H:%M:%S` - Disk usage for both source and target:" >> ${LOGFILE}
    echo "`du -sh ${ZIMBRADIR}`" >> ${LOGFILE}
    echo "`du -sh ${TARGET}`" >> ${LOGFILE}
    echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup mode completed." >> ${LOGFILE}
    
    ## Perform cleanup routine.
    f_cleanup
    ## Exit with the combined return code value.
    exit ${ERRORFLAG}
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  2. #12
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Full Offline Backup

    Backup - RSync while offline

    Although this backup takes the services down, the RSync job just prior to taking the services down ensures the vast majority of the data is already backed up. The only thing that has to be done after the services are down is to copy the changes in the last minute and the files that could not be copied while the services were running.

    The actual downtime is typically between 3 to 5 minutes of my server and I doubt it will require longer downtime time even as the data store increases.

    rsync-offline.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : rsync-offline.sh
    ## Version       : 1.4
    ## Date          : 2012-01-09
    ## Author        : LHammonds
    ## Purpose       : Complete backup while Zimbra is offline.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE
    ## Requirements  : p7zip-full (if ARCHIVEMETHOD=tar.7z)
    ## Run Frequency : Once per day after hours.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##    0 = success
    ##    1 = rsync failure
    ##    2 = archive failure
    ##    4 = archive purge failure
    ##    8 = configuration error
    ##   16 = crontab backup error
    ##   32 = crontab restore warning
    ##   64 = mount warning
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-21 LTH Created script.
    ## 2011-10-28 LTH Added 7-Zip compression (gunzip optional)
    ## 2011-11-04 LTH Move common variables and funtions to external file.
    ## 2011-11-08 LTH Added zimbra subfolder to remote site for organization.
    ## 2012-01-09 LTH Bugfix for f_PurgeOldestArchive
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/rsync.log"
    TEMPLOG="${TEMPDIR}/rsync-offline-temp.$$"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    TARGETDIR="${BACKUPDIR}/zimbra"
    OFFSITEBACKDIR="${OFFSITEDIR}/zimbra"
    ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_zimbra-backup.${ARCHIVEMETHOD}"
    ZMCRONFILE="${TEMPDIR}/zimbra-crontab.$$"
    ERRORFLAG=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    function f_PurgeOldestArchive()
    {
      ## Purpose: Delete the oldest archive on the remote site.
      ## Return values:
      ##    0 = Success
      ##    1 = Cannot delete file
      ##    9 = Configuration error, path empty
    
      ## Variable Error Check. *
      if [ ${OFFSITEBACKDIR} = "" ]; then
        ## Make darn sure the path is not empty since we do NOT
        ## want to start purging files from a random location.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${TEMPLOG}
        return 9
      fi
      ## Get the name of the oldest file.
      OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1`
      FILESIZE=`ls -la ${OFFSITEDIR}/${OLDFILE} | awk '{print $5}' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
      if [ "${OLDESTFILE}" = "" ]; then
        ## Error. Filename variable empty.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${TEMPLOG}
        return 9
      else   
        FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${TEMPLOG}
        rm "${OFFSITEBACKDIR}/${OLDESTFILE}"
        if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then
          ## File still exists.  Return error.
          return 1
        else
          return 0
        fi
      fi
    }
    
    function f_cleanup()
    {
      echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup exit code: ${ERRORFLAG}" >> ${TEMPLOG}
      ## Copy temporary log file into main log file.
      cat ${TEMPLOG} >> ${LOGFILE}
    
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
      ## Email the log to the administrator.
      EMAILBODY=""
      while read FILELINE; do
        EMAILBODY=${EMAILBODY}${FILELINE}"\n"
      done < ${TEMPLOG}
      f_sendmail "Offline RSync Backup Log" "${EMAILBODY}"
      if [ -f ${TEMPLOG} ]; then
        rm ${TEMPLOG}
      fi
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code
      f_cleanup
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    if [ -f ${LOCKFILE} ]; then
      # rsync lock file detected.  Abort script.
      f_sendmail "Offline RSync Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup started." >> ${TEMPLOG}
    
    ## If the 7-Zip archive method is specified, make sure the package is installed.
    if [ "${ARCHIVEMETHOD}" = "tar.7z" ]; then
      if [ ! -f "/usr/bin/7za" ]; then
        ## Required package (7-Zip) not installed.
        echo "`date +%Y-%m-%d_%H:%M:%S` - CRITICAL ERROR: 7-Zip package not installed.  Please install by typing 'aptitude -y install p7zip-full'" >> ${TEMPLOG}
        ERRORFLAG=2
        f_emergencyexit ${ERRORFLAG}
      fi
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Partition status:" >> ${TEMPLOG}
    df -h >> ${TEMPLOG}
    
    ## Output the current version to a text file to be included in the backup.
    ## NOTE: You can also view /opt/zimbra/.install_history
    su - zimbra -c "zmcontrol -v > ${ZIMBRADIR}/conf/zimbra_version.txt"
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Begin pre-shutdown rsync command." >> ${TEMPLOG}
    rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGETDIR} 1>/dev/null 2>&1
    
    ## Backup the zimbra crontab schedule.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Backup and disable Zimbra crontab schedule." >> ${TEMPLOG}
    crontab -u zimbra -l > ${ZMCRONFILE}
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong with the backup of the crontab file.
      ## This error will be considered fatal and terminate the script.
      f_sendmail "RSync Offline Backup Error - Crontab" "Could not backup crontab for the zimbra user account.  RETURN VALUE = ${RETURNVALUE}"
      ERRORFLAG=$((${ERRORFLAG} + 16))
      f_emergencyexit ${ERRORFLAG}
    else
      ## Backup successful, now disable the zimbra crontab schedule.
      echo "" > ${TEMPDIR}\emptyfile.$$
      crontab -u zimbra ${TEMPDIR}\emptyfile.$$
      rm ${TEMPDIR}\emptyfile.$$ 1>/dev/null 2>&1
      ## Add a copy of the crontab file to the Zimbra folder to be backed up.
      cp ${ZMCRONFILE} ${ZIMBRADIR} 1>/dev/null 2>&1
    fi
    
    StartTime="$(date +%s)"
    
    ## Stop Zimbra Services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." >> ${TEMPLOG}
    /etc/init.d/zimbra stop 1>/dev/null 2>&1
    
    ## Kill any orphaned Zimbra processes.
    pkill -9 -u zimbra 1>/dev/null 2>&1
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Space consumed in ${ZIMBRADIR} = `du -sh ${ZIMBRADIR} | awk '{ print $1 }'`" >> ${TEMPLOG}
    
    ## Now that Zimbra is offline, rsync can quickly copy all remaining files to the backup target.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Begin post-shutdown rsync command." >> ${TEMPLOG}
    rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGETDIR} 1>/dev/null 2>&1
    RETURNVALUE=$?
    
    ## Quickly start the Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." >> ${TEMPLOG}
    /etc/init.d/zimbra start 1>/dev/null 2>&1
    
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## rsync command failed.  Send warning email.
      f_sendmail "RSync Offline Backup Failure - rsync" "rsync failed with return value of ${RETURNVALUE}"
      ERRORFLAG=$((${ERRORFLAG} + 1))
    fi
    
    ## Calculates downtime of Zimbra services.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra downtime: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${TEMPLOG}
    
    ## Restore the zimbra crontab schedule.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Restore Zimbra crontab schedule." >> ${TEMPLOG}
    crontab -u zimbra ${ZMCRONFILE}
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Could not restore the zimbra crontab schedule.
      ## This is considered a non-fatal error and the script will continue.
      f_sendmail "Offline RSync Backup Warning - Crontab" "Could not restore the zimbra crontab schedule using ${ZMCRONFILE}  RETURN VALUE = ${RETURNVALUE}"
      ERRORFLAG=$((${ERRORFLAG} + 32))
    else
      ## Zimbra crontab schedule restored.  Remove temp file.
      rm ${ZMCRONFILE}
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Space consumed in ${TARGETDIR} = `du -sh ${TARGETDIR} | awk '{ print $1 }'`" >> ${TEMPLOG}
    
    ## Show status of Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" >> ${TEMPLOG}
    su - zimbra -c "zmcontrol status" >> ${TEMPLOG}
    
    ## Compress the backup into a single file based on archive method specified.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TEMPDIR}/${ARCHIVEFILE}" >> ${TEMPLOG}
    case "${ARCHIVEMETHOD}" in
    tar.7z)
      tar -cpf - ${TARGETDIR} | 7za a -si -mx=9 -w${TEMPDIR} ${TEMPDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
      ## Restore using one of the following commands (do not uncomment, only for notation):
      ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C / -xf -
      ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C ${TEMPDIR}/restore --strip-components=1 -xf -
      ;;
    tgz)
      tar -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
      ## Restore using one of the following commands (do not uncomment, only for notation):
      ## tar -C / -xzf ${TEMPDIR}/${ARCHIVEFILE}
      ## tar -C ${TEMPDIR}/restore --strip-components=1 -xzf ${TEMPDIR}/${ARCHIVEFILE}
      ;;
    *)
      tar -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
      ;;
    esac
    
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## tar command failed.  Send warning email.
      f_sendmail "Offline RSync Backup Failure - tar" "tar failed with return value of ${RETURNVALUE}"
      ERRORFLAG=$((${ERRORFLAG} + 2))
    fi
    
    ## Mount the Windows share folder. ##
    f_mount
    sleep 2
    if [ ! -f ${OFFSITETESTFILE} ]; then
      ## Could not find expected file on remote site.  Assuming failed mount.
      ERRORFLAG=$((${ERRORFLAG} + 64))
      f_sendmail "Offline RSync Backup - Win Mount Error" "${HOSTNAME} could not detect ${OFFSITETESTFILE}"
    else
      ## Remote site is available.
      FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
      BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'`
    
      ## Make sure space is available on the remote server to copy the file.
      if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then
        ## Not enough free space available.  Purge existing backups until there is room.
        ENOUGHSPACE=0
        while [ ${ENOUGHSPACE} -eq 0 ]
        do
          f_PurgeOldestArchive
          RETURNVALUE=$?
          case ${RETURNVALUE} in
          1)
            ## Cannot purge archives to free up space.  End program gracefully.
            echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEDIR} and cannot purge old archives.  Script aborted." >> ${TEMPLOG}
            ## Stop and exit the script with an error code.
            ERRORFLAG=$((${ERRORFLAG} + 4))
            f_emergencyexit ${ERRORFLAG}
            ;;
          9)
            ## Configuration error, end program gracefully.
            echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${TEMPLOG}
            ## Stop and exit the script with an error code.
            ERRORFLAG=$((${ERRORFLAG} + 8))
            f_emergencyexit ${ERRORFLAG}
            ;;
          esac
          FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
          if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then
            ## Enough space is now available.
            ENOUGHSPACE=1
          else
            ## Not enough space is available yet.
            ENOUGHSPACE=0
          fi
        done  
      fi
    
      ## Copy the backup to an offsite storage location.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${TEMPLOG}
      cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
      if [ ! -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then
      ## NON-FATAL ERROR: Copy command did not work.  Send email notification.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed. ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not exist!" >> ${TEMPLOG}
        f_sendmail "Offline RSync Backup Failure - Remote Copy" "Remote copy failed. ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not exist\n\nBackup file still remains in this location: ${HOSTNAME}:${TEMPDIR}/${ARCHIVEFILE}"
      else
        ## Remove local copy of the compressed backup file
        rm ${TEMPDIR}/${ARCHIVEFILE}
      fi
    fi
    ## Unmount the Windows shared folder.
    f_umount
    
    ## Calculate total time for backup.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${TEMPLOG}
    echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup completed." >> ${TEMPLOG}
    
    ## Perform cleanup routine.
    f_cleanup
    ## Exit with the combined return code value.
    exit ${ERRORFLAG}
    Example output in the log file:
    Code:
    2011-10-28_17:43:21 - Offline RSync backup started.
    2011-10-28_17:43:21 --- Partition status:
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/mapper/LVG-root   3842104   1467924   2179008  41% /
    none                   2025164       188   2024976   1% /dev
    none                   2030020         0   2030020   0% /dev/shm
    none                   2030020       296   2029724   1% /var/run
    none                   2030020         0   2030020   0% /var/lock
    none                   2030020         0   2030020   0% /lib/init/rw
    none                   3842104   1467924   2179008  41% /var/lib/ureadahead/debugfs
    /dev/sda1               182331     33053    139550  20% /boot
    /dev/mapper/LVG-bak  240306696   3333320 224766480   2% /var/backup
    /dev/mapper/LVG-opt  120152260   3287176 110761636   3% /opt
    /dev/mapper/LVG-temp 240306696    799184 227300616   1% /var/temp
    2011-10-28_17:43:22 --- Begin pre-shutdown rsync command.
    2011-10-28_17:45:10 --- Backup and disable Zimbra crontab schedule.
    2011-10-28_17:45:10 --- Stopping Zimbra services.
    2011-10-28_17:45:54 --- Space consumed in /opt/zimbra = 3.3G
    2011-10-28_17:45:58 --- Begin post-shutdown rsync command.
    2011-10-28_17:47:05 --- Starting Zimbra services.
    2011-10-28_17:48:32 --- Zimbra downtime: 0 hour(s) 3 minute(s) 22 second(s)
    2011-10-28_17:48:32 --- Restore Zimbra crontab schedule.
    2011-10-28_17:48:33 --- Space consumed in /temp/zimbra = 3.4G
    2011-10-28_17:48:35 --- Status of Zimbra services:
    Host mail.mydomain.com
    	antispam                Running
    	antivirus               Running
    	ldap                    Running
    	logger                  Running
    	mailbox                 Running
    	mta                     Running
    	snmp                    Running
    	spell                   Running
    	stats                   Running
    	zmconfigd               Running
    2011-10-28_17:48:50 --- Compressing archive: /var/temp/2011-10-28_zmbackup.tar.7z
    2011-10-28_18:10:09 --- Copying archive file to offsite location.
    2011-10-28_18:23:38 --- Total backup time: 0 hour(s) 38 minute(s) 28 second(s)
    2011-10-28_18:23:39 - Offline RSync backup completed.
    2011-10-28_18:23:39 - Offline RSync backup exit code: 0

    To test the online script, make sure you are root and type the following commands:
    Code:
    nohup ./rsync-online.sh &
    tail -f /var/temp/rsync-online*.log
    The command above will start the script in the background and will allow it to run even if you log off the system.

    The tail command will display the last 10 lines of the log. The -f option will keep the command active and continuously show the end of the log...thus you can sit there and just watch the entries show up in the log as they are written (works with any log file). Also note that this is the temporary log file which everything is placed until the very end...which is then copied to the main log file shared between both online and offline scripts.

    Press CTRL+C to stop watching the log file.

    Reference: OSE Backup Procedures (multiple)

    NOTE: Synchronize files with rsync
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  3. #13
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Individual Mailbox Backup

    Backup - Individual Mailbox

    The 2nd method involves backing up individual mailboxes. This process can occur while the server is online and thus will not cause any downtime. It can be run as often as you want but you should consider the performance impact on the server as well as how long each run will take. I plan to schedule it once per day during non-peak hours.

    mailbox-backup.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-backup.sh
    ## Version       : 1.1
    ## Date          : 2012-01-09
    ## Author        : LHammonds
    ## Purpose       : Backup individual mailbox accounts.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE
    ## Requirements  : Zimbra must be online, must be run as root user.
    ## Run Frequency : Once or more per day.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##    0 = Success (or non-critical failure)
    ##    1 = Root access failure
    ##    2 = Archive creation failure
    ##    4 = Archive purge error
    ##    8 = configuration error
    ##   16 = Offsite mount failure
    ##   32 = Offsite copy failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-23 LTH Created script.
    ## 2011-11-05 LTH Move common variables and functions to external file.
    ## 2011-11-06 LTH Additional error checks and better log info.
    ## 2012-01-09 LTH Bugfix f_PurgeOldestArchive
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-backup.log"
    HISTORYFILE="${TEMPDIR}/mailbox-backup-size-history.log"
    TARGETDIR="${BACKUPDIR}/mailbox"
    OFFSITEBACKDIR="${OFFSITEDIR}/mailbox"
    ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_mailbox-all.tar"
    LOCKFILE="${TEMPDIR}/mailbox-backup.lock"
    EXCEPTIONS="spam.ppnvqogp0@${MYDOMAIN};ham.iki6sotcy@${MYDOMAIN};virus-quarantine.qvj6nc_jl@${MYDOMAIN}"
    RETURNVALUE=0
    UCOUNT=0
    ERRORFLAG=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_PurgeOldestArchive()
    {
      ## Purpose: Delete the oldest archive on the remote site.
      ## Return values:
      ##    0 = Success
      ##    1 = Cannot delete file
      ##    9 = Configuration error, path empty
    
      ## Variable Error Check. *
      if [ ${OFFSITEBACKDIR} = "" ]; then
        ## Make darn sure the path is not empty since we do NOT
        ## want to start purging files from a random location.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${LOGFILE}
        return 9
      fi
      ## Get the name of the oldest file.
      OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1`
      FILESIZE=`ls -la ${OFFSITEDIR}/${OLDFILE} | awk '{print $5}' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
      if [ "${OLDESTFILE}" = "" ]; then
        ## Error. Filename variable empty.
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${LOGFILE}
        return 9
      else   
        FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'`
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${LOGFILE}
        rm "${OFFSITEBACKDIR}/${OLDESTFILE}"
        if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then
          ## File still exists.  Return error.
          return 1
        else
          return 0
        fi
      fi
    }
    
    function f_cleanup()
    {
      if [ -f "${LOCKFILE}" ]; then
        ## Remove run file since this script is complete.
        rm ${LOCKFILE}
      fi
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
      f_cleanup
      f_sendmail "Zimbra Mailbox Backup Error" "EXIT CODE = ${1}"
      echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted.  EXIT CODE: ${1}" >> ${LOGFILE}
      ## Write script name and error code to the system log.
      logger "${SCRIPTNAME}: ERROR CODE = ${1}"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      ERRORFLAG=$((${ERRORFLAG} + 1))
      echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted.  ERROR CODE ${ERRORFLAG}: Script must be run as root user instead of $(whoami)." | tee -a ${LOGFILE}
      f_emergencyexit ${ERRORFLAG}
    fi
    
    if [ -f "${LOCKFILE}" ]; then
      ## Last call to this script is still running or failed to remove its lock file.
      ## As an additional check, see if the file is older than today...if so, we should
      ## probably send an email notification of a problem that may need manual interention.
      FILEDATE=$(stat -c %y ${LOCKFILE})
      FILEDATE=${FILEDATE%% *}
      if [ "${FILEDATE}" != "$(date +%Y-%m-%d)" ]; then
        ## Lock file not created today, might need to be manually deleted.  Send email notification.
        f_sendmail "Zimbra Mailbox Backup Warning" "Warning: This script cannot run if it detects this lock file: ${LOCKFILE}\n\nThis file should only exist while this script is running which should not take more than a day.\n\nSystem Date: $(date +%Y-%m-%d)\nLock File Date: ${FILEDATE}\n\nIf you determine that the file should be removed, do so by typing this command on the server's console:  rm ${LOCKFILE}"
      fi
      exit 0
    else
      ## Create the "script is running" lock file and process the script.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ${HOSTNAME}:${SCRIPTNAME} is currently running." > ${LOCKFILE}
    fi
    
    ## Record the start time of the backup process.
    STARTTIME="$(date +%s)"
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup started." >> ${LOGFILE}
    if [ -d "${TARGETDIR}" ]; then
      ## Purge existing archives.
      rm ${TARGETDIR}/*.tgz 1>/dev/null 2>&1
    else
      ## Make the folder since it does not exist.
      mkdir -p ${TARGETDIR} 1>/dev/null 2>&1
    fi
    for ACCT in `su - zimbra -c "zmprov -l gaa"`
    do
      ## Check to see if current account should be skipped.
      if echo "${EXCEPTIONS}" | grep -q ${ACCT}
      then
        ## Exception found, skip this account.
        echo "" > /dev/null
      else
        ## Backup user account.
        UCOUNT=$((UCOUNT+1))
        ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} getRestURL "//?fmt=tgz" > ${TARGETDIR}/${ACCT}.tgz
        RETURNVALUE=$?
        if [ ${RETURNVALUE} -ne 0 ]; then
          ## Something went wrong.
          echo "`date +%Y-%m-%d_%H:%M:%S` --- Error on ${ACCT}, RETURN VALUE = ${RETURNVALUE}" >> ${LOGFILE}
          ERRORFLAG=$((ERRORFLAG+1))
        else
          ## Calculate archive size.
          MAILBOXSIZE=$(stat -c %s ${TARGETDIR}/${ACCT}.tgz)
          ## Comment out the below line if you do not want details in the log file.
          echo "`date +%Y-%m-%d_%H:%M:%S` --- ${ACCT}, ${MAILBOXSIZE} bytes" >> ${LOGFILE}
        fi
      fi
    done
    echo "`date +%Y-%m-%d_%H:%M:%S` --- ${UCOUNT} accounts processed." >> ${LOGFILE}
    
    ## Calculate mailbox backup time.
    FINISHTIME="$(date +%s)"
    ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})"
    HOURS=$((${ELAPSEDTIME} / 3600))
    ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600))
    MINUTES=$((${ELAPSEDTIME} / 60))
    SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Backup time for ${UCOUNT} mailboxes: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE}
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Setting file permissions on ${TARGETDIR}/*.tgz" >> ${LOGFILE}
    chmod 0600 ${TARGETDIR}/*.tgz
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Creating a single file for archiving: ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
    tar -cf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error creating ${TEMPDIR}/${ARCHIVEFILE}, Return Value: ${RETURNVALUE}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+2))
    fi
    ARCHIVESIZE=$(stat -c %s ${TEMPDIR}/${ARCHIVEFILE})
    if [ ! -f "${HISTORYFILE}" ]; then
      ## Create history log and include header description as 1st line, field separator = space character.
      echo "Date Archive SizeInBytes" > ${HISTORYFILE}
    fi
    ## Append archive size to the history log.
    echo "`date +%Y-%m-%d` ${ARCHIVEFILE} ${ARCHIVESIZE}" >> ${HISTORYFILE}
    
    ## Mount the offsite Windows share folder.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Mounting ${OFFSITEDIR}" >> ${LOGFILE}
    f_mount
    if [ -f ${OFFSITETESTFILE} ]; then
      ## Remote site is online / available.
      if [ ! -d ${OFFSITEBACKDIR} ]; then
        ## Make the backup folder since it does not exist yet.
        mkdir -p ${OFFSITEBACKDIR}
      fi
      if [ -f ${TEMPDIR}/${ARCHIVEFILE} ]; then
        ## Make sure space is available on the remote server to copy the file.
        FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
        BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'`
        if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then
          ## Not enough free space available.  Purge existing backups until there is room.
          ENOUGHSPACE=0
          while [ ${ENOUGHSPACE} -eq 0 ]
          do
            f_PurgeOldestArchive
            RETURNVALUE=$?
            case ${RETURNVALUE} in
            1)
              ## Cannot purge archives to free up space.  End program gracefully.
              echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEBACKDIR} and cannot purge old archives.  Script aborted." >> ${TEMPLOG}
              ## Stop and exit the script with an error code.
              ERRORFLAG=$((${ERRORFLAG} + 4))
              f_emergencyexit ${ERRORFLAG}
              ;;
            9)
              ## Configuration error, end program gracefully.
              echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${TEMPLOG}
              ## Stop and exit the script with an error code.
              ERRORFLAG=$((${ERRORFLAG} + 8))
              f_emergencyexit ${ERRORFLAG}
              ;;
            esac
            FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
            if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then
              ## Enough space is now available.
              ENOUGHSPACE=1
            else
              ## Not enough space is available yet.
              ENOUGHSPACE=0
            fi
          done
        fi
    
        echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying ${TEMPDIR}/${ARCHIVEFILE} to ${OFFSITEBACKDIR}/" >> ${LOGFILE}
        cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1
      fi
    else
      ## Remote site is offline / unavailable.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error: Remote site is unavailable: ${OFFSITEBACKDIR}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+16))
    fi
    
    if [ -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then
      ## Remote copy worked.  Remove local archive.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Copied ${ARCHIVESIZE} bytes." >> ${LOGFILE}
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Removing ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
      rm ${TEMPDIR}/${ARCHIVEFILE}
    
      FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'`
      ## Comment out the below line if you do not want to receive statistic emails.
      f_sendmail "Zimbra Individual Mailbox Backup" "${UCOUNT} accounts backed up.\n\nTotal archive size: $((${ARCHIVESIZE}/1024)) kb\n\nAvailable Backup Space: ${FREESPACE} kb"
    
      ## Uncomment the following 2 lines if you do not wish to have a local copy of individual mailboxes.
    #  rm ${TARGETDIR}/*.tgz
    #  rmdir ${TARGETDIR}
    else
      ## Remote copy failed.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Error copying to ${OFFSITEBACKDIR}/${ARCHIVEFILE}" >> ${LOGFILE}
      ERRORFLAG=$((ERRORFLAG+32))
    fi
    
    ## Calculate total time for backup.
    FINISHTIME="$(date +%s)"
    ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})"
    HOURS=$((${ELAPSEDTIME} / 3600))
    ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600))
    MINUTES=$((${ELAPSEDTIME} / 60))
    SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE}
    
    if [ ${ERRORFLAG} -ne 0 ]; then
      f_sendmail "Zimbra Individual Mailbox Backup Error" "${ERRORFLAG} errors detected while trying to backup ${UCOUNT} individual mailboxes."
    fi
    ## Unmount the offsite backup location.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Dismounting ${OFFSITEDIR}" >> ${LOGFILE}
    f_umount
    ## Perform cleanup routine.
    f_cleanup
    ## Exit with the combined return code value.
    echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup complete. Exit code: ${ERRORFLAG}" >> ${LOGFILE}
    exit ${ERRORFLAG}

    Sample output of mailbox-backup.log
    Code:
    2011-11-06_15:48:02 - Individual mailbox backup started.
    2011-11-06_15:48:11 --- admin@mydomain.com, 6781 bytes
    2011-11-06_15:48:13 --- lhammonds@mydomain.com, 1066 bytes
    2011-11-06_15:48:16 --- jsmith@mydomain.com, 1067 bytes
    2011-11-06_15:48:18 --- jdoe@mydomain.com, 1066 bytes
    2011-11-06_15:48:21 --- administrator@mydomain.com, 1069 bytes
    2011-11-06_15:48:23 --- foobar@mydomain.com, 1067 bytes
    2011-11-06_15:48:25 --- jvorhees@mydomain.com, 1068 bytes
    2011-11-06_15:48:28 --- conferencerooms@mydomain.com, 1067 bytes
    2011-11-06_15:48:30 --- test@mydomain.com, 1068 bytes
    2011-11-06_15:48:48 --- 9 accounts processed.
    2011-11-06_15:48:48 --- Backup time for 9 mailboxes: 0 hour(s) 0 minute(s) 46 second(s)
    2011-11-06_15:48:48 --- Setting file permissions on /var/backup/mailbox/*.tgz
    2011-11-06_15:48:48 --- Creating a single file for archiving: /var/temp/2011-11-06-15-48_mailbox-all.tar
    2011-11-06_15:48:48 --- Mounting /mnt/backup
    2011-11-06_15:48:49 --- Copying /var/temp/2011-11-06-15-48_mailbox-all.tar to /mnt/backup/mailbox/
    2011-11-06_15:48:49 --- Copied 61440 bytes.
    2011-11-06_15:48:49 --- Removing /var/temp/2011-11-06-15-48_mailbox-all.tar
    2011-11-06_15:48:49 --- Total backup time: 0 hour(s) 0 minute(s) 47 second(s)
    2011-11-06_15:48:49 --- Dismounting /mnt/backup
    2011-11-06_15:48:49 - Individual mailbox backup complete. Exit code: 0
    Sample output of mailbox-backup-size-history.log
    Code:
    Date Archive SizeInBytes
    2011-11-06 2011-11-06-11-21_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-17_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-21_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-22_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-32_mailbox-all.tar 61440
    2011-11-06 2011-11-06-15-48_mailbox-all.tar 61440
    Sample email notification
    Code:
    Subject: Zimbra Individual Mailbox Backup
    
    9 accounts backed up.
    
    Total archive size: 60 kb
    
    Available Backup Space: 28440616 kb
    
    Server: mail
    Program: /var/scripts/prod/mailbox-backup.sh
    Log: /var/temp/mailbox-backup.log
    Sample error email notification
    Code:
    Subject: Zimbra Individual Mailbox Backup Error
    
    1 errors detected while trying to backup 9 individual mailboxes.
    
    Server: mail
    Program: /var/scripts/prod/mailbox-backup.sh
    Log: /var/temp/mailbox-backup.log
    Sample /var/log/messages
    Code:
    Nov  6 11:13:01 mail administrator: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 1
    Nov  6 11:15:02 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 2
    Nov  6 11:18:03 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 4
    Nov  6 11:23:04 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 8
    Nov  6 11:33:05 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 16
    Nov  6 11:43:06 mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE = 32
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  4. #14
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Restore

    Zimbra Restore

    I see the need to restore from at least several different scenarios. That means coming up with a restore plan and scripts to help speed up restore time and reduce errors during emergency scenarios. This also means testing each of these restore plans; otherwise you are playing Russian roulette with more than one bullet in the revolver! Always remember that the vast majority of administrators involuntarily removed from their job are typically related to data loss that cannot be recovered.

    Here are the scenarios I am going to cover and test:


    1. Restore individual mailbox from a local copy - we will restore a mailbox account using the latest individual mailbox backup which is stored on the local server. This can be done at any time and will not impact server uptime.
    2. Restore individual mailbox from an offsite location - we will restore a mailbox account using one of the archives located on the offsite storage. This can be done at any time and will not impact server uptime.
    3. Restore existing server from a local copy (fastest, but potentially problematic) - we will use the local folder used for doing daily rsync backups. The caveat here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open and in-use files. This will cause the server to be offline, so be sure to coordinate and plan accordingly.
    4. Restore existing server from an offsite location (slower, but more reliable) - we will restore the server using one of the archives located on the offsite storage. This will cause the server to be offline, so be sure to coordinate and plan accordingly.
    5. Restore to new server (production server unavailable) - we will assume the production server is offline and inaccessible. We will configure a new server and restore using one of the archives located on the offsite storage.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  5. #15
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Disaster Scenario #1

    Scenario #1 Restore individual mailbox from a local copy

    This process will assume you are wanting to restore a mailbox back to the way it was when last backed up. This means anything new since the backup will be wiped out. If this is not desired, a simple solution (which will be my standard protocol) will be to create a new mailbox and restore to it.

    If creating a new mailbox, be sure to set a password on it and test it by logging into the account...this also creates the mailbox store when accessed the 1st time. Then copy the user's mailbox archive to look like the archive for the restore account.

    Example #1 - Restore the admin mailbox.

    Code:
    /var/scripts/prod/mailbox-restore-local.sh admin@mydomain.com
    Onscreen result:
    Code:
    2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com
    Recorded event in the log: /var/temp/mailbox-restore.log
    Code:
    2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com
    Example #2 - Restore the admin mailbox to a temporary account called "restore".

    1. Login to the admin console: h t t p s ://mail.mydomain.com:7071
    2. Create a new account called restore and set the password
    3. Login to the webmail using the restore account: h t t p s ://mail.mydomain.com
    4. Once logged in to verify the empty account works, logout
    5. Access the Ubuntu server. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass)
    6. Type the following:
      Code:
      
      cp /var/backup/mailbox/admin@mydomain.com.tgz /var/backup/mailbox/restore@mydomain.com.tgz
      /var/scripts/prod/mailbox-restore-local.sh restore@mydomain.com
      


    mailbox-restore-local.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-restore-local.sh
    ## Version       : 1.0
    ## Date          : 2011-11-06
    ## Author        : LHammonds
    ## Compatibility : Verified on Ubuntu 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0
    ## Purpose       : Restore individual mailbox accounts.
    ## Run Frequency : Manual as needed.
    ## Exit Codes    : (if multiple errors, value is the addition of codes)
    ##   0 = success
    ##   1 = parameter failure
    ##   2 = unknown mailbox account
    ##   4 = missing archive folder
    ##   8 = missing archive file
    ##  16 = mailbox restore failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-24 LTH Created script.
    ## 2011-11-06 LTH Move common variables and functions to external file.
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-restore.log"
    ARCHIVEDIR="${BACKUPDIR}/mailbox"
    ACCT=$1
    RETURNVALUE=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_showhelp()
    {
      echo ""
      echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN}"
      echo ""
      echo "NOTE: You could create a temporary account and copy the desired archive"
      echo "      to be named the same as the temporary account in order to restore"
      echo "      to a new mailbox and preserve the existing mailbox."
      echo "      Example:"
      echo "        cd ${ARCHIVEDIR}"
      echo "        cp admin@${MYDOMAIN}.tgz restore@${MYDOMAIN}.tgz"
      echo "        ${SCRIPTNAME} restore@${MYDOMAIN}"
      echo ""
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
      echo -e "Individual mailbox restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Check command-line parameters.
    case "$1" in
      "")
        f_showhelp
        f_emergencyexit 1
        ;;
      --help|-h|-?)
        f_showhelp
        f_emergencyexit 1
        ;;
      *)
        ACCT=$1
        ;;
    esac
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "ERROR: The archive folder does not exist!  ${ARCHIVEDIR}"
      f_emergencyexit 4
    fi
    
    ## Check to make sure the archive exists.
    if [ ! -f "${ARCHIVEDIR}/${ACCT}.tgz" ]; then
      ## ERROR: Archive does not exist.
      echo "ERROR: The archive does not exist!   ${ARCHIVEDIR}/${ACCT}.tgz"
      f_emergencyexit 8
    fi
    
    ## Check to make sure parameter is a valid mailbox account name.
    if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT}`
    then
      ## The parameter contains a valid mailbox account.
      echo "" > /dev/null
    else
      ## Invalid mailbox account.
      echo "ERROR: Invalid mailbox: ${ACCT}"
      f_showhelp
      f_emergencyexit 2
    fi
    
    ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} postRestURL "//?fmt=tgz&resolve=reset" ${ARCHIVEDIR}/${ACCT}.tgz
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` - Error restoring ${ACCT}, RETURN VALUE = ${RETURNVALUE}" | tee -a ${LOGFILE}
      f_emergencyexit 16
    fi
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT}" | tee -a ${LOGFILE}
    f_sendmail "Zimbra User Mailbox Restore" "${ACCT} was manually restored."
    
    exit 0
    If you do not supply an account to the script as a parameter, it will display the following help text:
    Code:
    /var/scripts/prod/mailbox-restore.sh
    
    Usage : ./mailbox-restore-local.sh user@mydomain.com
    
    NOTE: You could create a temporary account and copy the desired archive
          to be named the same as the temporary account in order to restore
          to a new mailbox and preserve the existing mailbox.
          Example:
            cd /var/backup/mailbox
            cp admin@mydomain.com.tgz restore@mydomain.com.tgz
            /var/scripts/prod/mailbox-restore.sh restore@mydomain.com
    
    Individual mailbox restore aborted.  EXIT CODE: 1
    NOTE: The above will actually use your domain in all examples because it pulls the domain from the standard.conf file.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  6. #16
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Disaster Scenario #2

    Scenario #2 Restore individual mailbox from an offsite location

    Here is the general outline of the script:

    Parameter #1 (required) = Account in which we will find the archive for.
    Parameter #2 (optional) = Account in which we will restore the selected archive into (default = parameter 1)

    Step 1 - Display all offsite archives (order oldest to newest) and present list to user for selection
    Step 2 - Extract only the account's file from the archive to the temp folder
    Step 3 - Restore archive to desired account

    ----------------------

    /var/scripts/prod/mailbox-restore.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : mailbox-restore.sh
    ## Version       : 1.0
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Compatibility : Verified on Ubuntu 10.04.3 - 10.04.4 LTS, Zimbra 7.1.3 - 7.2.0 OSE
    ## Purpose       : Restore individual mailbox accounts.
    ## Run Frequency : Manual as needed.
    ## Parameters    :
    ##   1 = (Required) Account to search for archive (e.g. jdoe@mydomain.com)
    ##   2 = (Optional) Account to restore archive to (e.g. restore@mydomain.com)
    ## Exit Codes    :
    ##   0 = Success or non-critical problem
    ##   1 = Archive folder does not exist
    ##   2 = Invalid mailbox account
    ##   3 = Mailbox restore failure
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-11-08 LTH Created script.
    #############################################
    
    ## Import common variables and functions.
    source /var/scripts/common/standard.conf
    
    ## Define local variables.
    LOGFILE="${TEMPDIR}/mailbox-restore.log"
    ARCHIVEDIR="${OFFSITEDIR}/mailbox"
    ACCT1=$1
    ACCT2=$2
    RETURNVALUE=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_showhelp()
    {
      echo ""
      echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN} restore@${MYDOMAIN} (optional)"
      echo ""
      echo "NOTE: If you specify a 2nd account, the 1st account data will be"
      echo "      restored to the 2nd account."
      echo ""
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
    
      if [ -f ${REMOTEDIR}/online.txt ]; then
        ## Dismount the remote site.
        f_umount
      fi
      echo -e "Individual mailbox restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    ## Check command-line parameters.
    case "$1" in
      "")
        f_showhelp
        exit 0
        ;;
      --help|-h|-?)
        f_showhelp
        exit 0
        ;;
      *)
        ACCT1=$1
        ACCT2=$2
        ;;
    esac
    
    if [ "${ACCT1}" = "" ]; then
      ## Required parameter not specified.
      f_showhelp
      exit 0
    fi
    
    if [ "${ACCT2}" = "" ]; then
      ## Restore to the same account.
      ACCT2=${ACCT1}
    fi
    
    ## Mount the remote site.
    f_mount
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist!  ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE}
      f_emergencyexit 1
    fi
    
    ## Define user prompt using the special PS3 variable.
    PS3="Type number for the desired archive or 'q' to quit: "
    
    ## Get sorted list of all archives (newest at the bottom).
    FILELIST=$(find ${ARCHIVEDIR}/*.tar -maxdepth 1 -type f | sort -f)
    
    ## Prompt user to select a file to use.
    ## NOTE: If it is a long list, user can scroll up if
    ##       using PuTTY to see older files.
    select GETFILE in ${FILELIST}; do
      if [ "${GETFILE}" != "" ]; then
        FILENAME=${GETFILE}
      fi
      break
    done
    
    if [ "${FILENAME}" = "" ]; then
      ## User opted to quit.
      echo -e "Exiting restore program.\n"
      exit 0
    fi
    
    ## Extract just the mailbox file from the archive.
    tar --extract -C ${TEMPDIR} --file=${FILENAME} --wildcards "*${ACCT1}*" --no-anchored --strip-components=3
    
    if [ ! -f ${TEMPDIR}/${ACCT1}.tgz ]; then
      ## File did not exist in the archive.
      echo -e "No file found matching ${ACCT1}\n"
      exit 0
    fi
    
    ## Check to make sure parameter is a valid mailbox account name.
    if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT2}`
    then
      ## The parameter contains a valid mailbox account.
      echo "" > /dev/null
    else
      ## Invalid mailbox account.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Invalid mailbox: ${ACCT2}, EC=2" | tee -a ${LOGFILE}
      f_showhelp
      f_emergencyexit 2
    fi
    
    ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT2} postRestURL "//?fmt=tgz&resolve=reset" ${TEMPDIR}/${ACCT1}.tgz
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## Something went wrong.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Restore failure on ${ACCT2}, RETURN VALUE=${RETURNVALUE}, EC=3" | tee -a ${LOGFILE}
      f_emergencyexit 3
    fi
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT2}" | tee -a ${LOGFILE}
    f_sendmail "Zimbra User Mailbox Restore" "${ACCT2} was manually restored from ${ACCT1}"
    
    ## Clean up temporary files.
    if [ -f ${TEMPDIR}/${ACCT1}.tgz ]; then
      rm ${TEMPDIR}/${ACCT1}.tgz
    fi
    
    ## Dismount the remote site.
    f_umount
    exit 0
    /var/temp/mailbox-restore.log (sample)
    Code:
    2011-11-08_13:01:39 - ERROR: The archive folder does not exist!  /mnt/backup/configtypo, EC=1
    2011-11-08_15:54:34 - ERROR: Restore failure for admin@mydomain.com, RETURN VALUE=2, EC=3
    2011-11-08_15:55:47 - Restored mailbox --> admin@mydomain.com
    2011-11-08_15:57:50 - Restored mailbox --> lhammonds@mydomain.com
    2011-11-08_16:00:14 - Restored mailbox --> ddiggler@mydomain.com
    2011-11-08_16:01:16 - Restored mailbox --> restore@mydomain.com
    2011-11-08_16:28:38 - ERROR: Invalid mailbox: bogus@mydomain.com, EC=2
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  7. #17
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Disaster Scenario #3

    Scenario #3 Restore existing server from a local copy

    We will use the local folder used for doing daily rsync backups.

    The caveot here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open and in-use files. If the last rsync was the online version, we will need to use a different script to access remote storage.

    This script is intended to be run manually as the root user.

    Here are the instructions to restore the currently running server using the script below:


    1. Before running this, make sure everyone is aware that Zimbra will be going down to be restored.
    2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass)
    3. Run the script by typing /var/scripts/prod/restore-rsync-local.sh
    4. If you missed any of the output on the screen, you can view the log by typing vi /var/temp/restore-rsync.log


    restore-rsync-local.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : restore-rsync-local.sh
    ## Version       : 1.1
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Purpose       : Restore Zimbra using rsync and the local backup.
    ## Compatibility : Verifed on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.3 - 7.2.0 OSE
    ## Requirements  : Must be run as root user, last rsync should be the offline version.
    ## Run Frequency : Manual as needed.
    ## Exit Codes    : Errors displayed to screen.
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-10-18 LTH Created script.
    ## 2011-10-31 LTH Added better logging, error checks, user input.
    ## 2011-11-08 LTH Move common variables and funtions to external file.
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/restore-rsync.log"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    RSYNCLOGFILE="${TEMPDIR}/rsync.log"
    PRODROOT="/opt"
    OLDDIR="${BACKUPDIR}/zimbra.old"
    LOCALBACKUP="${BACKUPDIR}/zimbra/zimbra"
    LOCALBACKUPDEV="/dev/mapper/LVG-bak"
    NOBACKUP=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_cleanup()
    {
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
    }
    
    function f_checkspace()
    {
      ## Need to check and see if there is enough space available to move Zimbra to a backup location.
      FREESPACE=`df -k ${LOCALBACKUPDEV} | grep ${LOCALBACKUPDEV} | awk '{ print $4 }'`
      BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'`
    
      if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then
        echo "There is not enough freespace available to archive production Zimbra."
        echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb"
        echo ""
        echo "To override and not make a backup 1st, use the --nobackup option."
        echo ""
        echo "Example:  ${SCRIPTNAME} --nobackup"
        echo ""
        f_cleanup
        exit 1
      fi
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    if [ -f ${LOCKFILE} ]; then
      # rsync lock file detected.  Abort script.
      echo "Restore rsync aborted"
      echo "This script tried to run but detected the lock file: ${LOCKFILE}"
      echo "Please check to make sure the file does not remain when rsync is not actually running."
      f_sendmail "restore rsync aborted" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    clear
    echo ""
    
    # Determine if we are going to backup existing folder or not.
    if [ "$1" = "--nobackup" ]; then
      NOBACKUP=1
    else
      NOBACKUP=0
      f_checkspace
    fi
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Root user required for restore."
      echo ""
      f_cleanup
      exit 1
    fi
    
    if [ ! -f ${RSYNCLOGFILE} ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Cannot find log: ${RSYNCLOGFILE}"
      echo "This log file is required to determine status of last rsync job."
      echo ""
      f_cleanup
      exit 1
    fi
    
    
    echo "This restore method can only work if the last rsync backup was"
    echo "the 'Offline' version which allows for a complete backup."
    echo ""
    echo "The 'Online' version is only a partial backup and will not work."
    echo ""
    read -p "The next step will try to auto-detect the last backup. Continue (y/n)? "
    if [ "${REPLY}" != "y" ]; then
      echo "Script cancelled."
      echo ""
      f_cleanup
      exit 1
    fi
    echo ""
    
    tail -n1 ${RSYNCLOGFILE} | grep -q "Offline RSync backup exit code"
    if [ $? -eq 0 ]; then
      echo "Last rsync seems to have been a good offline backup"
    else
      echo "WARNING: This script did not see the offline entry at the end of the log file."
    fi
    echo ""
    
    echo "--------------------------------------------------------"
    tail -n5 ${RSYNCLOGFILE}
    echo "--------------------------------------------------------"
    echo ""
    
    read -p "Do you see 'Offline RSync backup exit code: 0' in the log above (y/n)? "
    if [ "${REPLY}" != "y" ]; then
      echo "Script cancelled."
      echo ""
      f_cleanup
      exit 1
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE}
    
    StartTime="$(date +%s)"
    
    ## Stop Zimbra Services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra stop
    
    ## Kill any orphaned Zimbra processes.
    pkill -9 -u zimbra 1>/dev/null 2>&1
    
    if [ ${NOBACKUP} -eq 1 ]; then
      ## Leave current production folder alone.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE}
    else
      ## Move production folder to backup location.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE}
      mv ${ZIMBRADIR} ${OLDDIR}
      RETURNVALUE=$?
      if [ ${RETURNVALUE} -ne 0 ]; then
        ## Could not move Zimbra to backup location.
        ## Document error and terminate script.
        echo ""
        echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE}
        echo ""
        echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE}
        echo ""
        echo "The Zimbra services are currently stopped and an unknown amount of files were moved"
        echo "from the source.  That means there some files are still in ${ZIMBRADIR}"
        echo "and some files were moved to ${OLDDIR}"
        echo "This needs to be resolved before you can start the services back up."
        echo "Either move the files back or delete all files and run the restore."
        f_cleanup
        exit 99
      fi
    fi
    
    ## Now that Zimbra is offline, rsync can restore all files.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files with rsync." | tee -a ${LOGFILE}
    rsync -apogHK ${LOCALBACKUP} ${PRODROOT}
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## rsync command failed.  Display warning message.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: rsync return value = {$RETURNVALUE}" | tee -a ${LOGFILE}
    fi
    
    if [ ! -f ${ZIMBRADIR}/.install_history ]; then
      ## Something is not right...at least this file should exist here.
      ## NOTE: This is not a guarentee all files were restored, it is just a quick check.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s).  Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE}
    fi
    
    ## Fix permissions.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE}
    chown -R zimbra:zimbra ${ZIMBRADIR}
    ${ZIMBRADIR}/libexec/zmfixperms
    
    ## Start Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra start
    
    ## Calculates downtime of Zimbra services.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE}
    
    ## Show status of Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE}
    su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE}
    
    if [ -d ${OLDDIR} ]; then
      echo -e "\n\nDon't forget to purge ${OLDDIR} once everything is running smoothly again.\n\n"
    fi
    
    f_cleanup
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE}
    Sample output from script:
    Code:
    2011-10-18_16:26:40 - Restore started.
    2011-10-18_16:26:40 --- Stopping Zimbra services.
    Host mail.mydomain.com
            Stopping stats...Done.
            Stopping mta...Done.
            Stopping spell...Done.
            Stopping snmp...Done.
            Stopping cbpolicyd...Done.
            Stopping archiving...Done.
            Stopping antivirus...Done.
            Stopping antispam...Done.
            Stopping imapproxy...Done.
            Stopping memcached...Done.
            Stopping mailbox...Done.
            Stopping logger...Done.
            Stopping zmconfigd...Done.
            Stopping ldap...Done.
    2011-10-18_16:27:22 --- Moving Zimbra from /opt/zimbra to /temp/zimbra.old.
    2011-10-18_16:27:22 --- Restoring files with rsync.
    2011-10-18_16:31:53 --- Fixing file ownership and permissions.
    2011-10-18_16:31:55 --- Starting Zimbra services.
    Host mail.mydomain.com
            Starting ldap...Done.
            Starting zmconfigd...Done.
            Starting logger...Done.
            Starting mailbox...Done.
            Starting antispam...Done.
            Starting antivirus...Done.
            Starting snmp...Done.
            Starting spell...Done.
            Starting mta...Done.
            Starting stats...Done.
    2011-10-18_16:33:25 --- Zimbra restore time: 0 hour(s) 6 minute(s) 45 second(s)
    2011-10-18_16:33:25 --- Status of Zimbra services:
    Host mail.mydomain.com
            antispam                Running
            antivirus               Running
            ldap                    Running
            logger                  Running
            mailbox                 Running
            mta                     Running
            snmp                    Running
            spell                   Running
            stats                   Running
            zmconfigd               Running
    2011-10-18_16:33:55 - Restore completed.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  8. #18
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Disaster Scenario #4

    Scenario #4 Restore existing server from an offsite location

    /var/scripts/prod/rsync-restore.sh
    Code:
    #!/bin/bash
    #############################################
    ## Name          : rsync-restore.sh
    ## Version       : 1.0
    ## Date          : 2011-11-08
    ## Author        : LHammonds
    ## Purpose       : Restore Zimbra using rsync from a remote archive.
    ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE
    ## Requirements  : Must be root user, p7zip-full (if ARCHIVEMETHOD=tar.7z)
    ## Run Frequency : Manual as needed.
    ## Exit Codes    :
    ##    0 = Success or non-critical problem
    ##    1 = Archive folder does not exist
    ##    2 = Archive extration failed
    ##    3 = Zimbra restore failure
    ##   99 = Move production to temp directory failure (bad)
    ################ CHANGE LOG #################
    ## DATE       WHO WHAT WAS CHANGED
    ## ---------- --- ----------------------------
    ## 2011-11-08 LTH Created script.
    #############################################
    
    ## Import common variables and functions. ##
    source /var/scripts/common/standard.conf
    
    LOGFILE="${TEMPDIR}/rsync-restore.log"
    LOCKFILE="${TEMPDIR}/rsync.lock"
    ARCHIVEDIR="${OFFSITEDIR}/zimbra"
    OLDDIR="${BACKUPDIR}/zimbra.old"
    NOBACKUP=0
    
    #######################################
    ##            FUNCTIONS              ##
    #######################################
    
    function f_cleanup()
    {
      if [ -f ${LOCKFILE} ];then
        ## Remove lock file so other rsync jobs can run.
        rm ${LOCKFILE} 1>/dev/null 2>&1
      fi
    }
    
    function f_checkspace()
    {
      ## Need to check and see if there is enough space available to move Zimbra to a backup location.
      FREESPACE=`df -k ${BACKUPDIR} | grep ${BACKUPDIR} | awk '{ print $4 }'`
      BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'`
    
      if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then
        echo "There is not enough freespace available to archive production Zimbra."
        echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb"
        echo ""
        echo "To override and not make a backup 1st, use the --nobackup option."
        echo ""
        echo "Example:  ${SCRIPTNAME} --nobackup"
        echo ""
        f_cleanup
        exit 1
      fi
    }
    
    function f_emergencyexit()
    {
      ## Purpose: Exit script as cleanly as possible.
      ## Parameter #1 = Error Code.
    
      if [ -f ${REMOTEDIR}/online.txt ]; then
        ## Dismount the remote site.
        f_umount
      fi
      echo -e "Zimbra restore aborted.  EXIT CODE: ${1}\n"
      exit $1
    }
    
    #######################################
    ##           MAIN PROGRAM            ##
    #######################################
    
    if [ -f ${LOCKFILE} ]; then
      ## Restore lock file detected.  Abort script.
      echo "Restore aborted"
      echo "This script tried to run but detected the lock file: ${LOCKFILE}"
      echo "Please check to make sure the file does not remain when rsync or restore is not actually running."
      f_sendmail "Zimbra Restore aborted" "This script tried to run but detected the lock file: ${LOCKFILE}\n\nPlease check to make sure the file does not remain when rsync/restore is not actually running."
      exit 1
    else
      echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE}
    fi
    
    clear
    echo ""
    
    ## Determine if we are going to backup existing folder or not.
    if [ "$1" = "--nobackup" ]; then
      NOBACKUP=1
    else
      NOBACKUP=0
      ## 
      f_checkspace
    fi
    
    ## Requirement Check: Script must run as root user.
    if [ "$(id -u)" != "0" ]; then
      ## FATAL ERROR DETECTED: Document problem and terminate script.
      echo "ERROR: Root user required for restore."
      echo ""
      f_cleanup
      exit 1
    fi
    
    ## Mount the remote site.
    f_mount
    
    ## Check to make sure the archive folder exists.
    if [ ! -d "${ARCHIVEDIR}" ]; then
      ## ERROR: Archive folder does not exist.
      echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist!  ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE}
      f_cleanup
      f_emergencyexit 1
    fi
    
    ## Define user prompt using the special PS3 variable.
    PS3="Type number for the desired archive or 'q' to quit: "
    
    echo "                Z I M B R A   R E S T O R E"
    echo "                ---------------------------"
    ## Get sorted list of all archives (newest at the bottom).
    FILELIST=$(find ${ARCHIVEDIR}/*.${ARCHIVEMETHOD} -maxdepth 1 -type f | sort -f)
    
    ## Prompt user to select a file to use.
    ## NOTE: If it is a long list, user can scroll up if
    ##       using PuTTY to see older files.
    select GETFILE in ${FILELIST}; do
      if [ "${GETFILE}" != "" ]; then
        FILENAME=${GETFILE}
      fi
      break
    done
    
    if [ "${FILENAME}" = "" ]; then
      ## User opted to quit.
      echo -e "Exiting restore program.\n"
      f_cleanup
      f_umount
      exit 0
    fi
    
    echo -e "\nSelected file: ${FILENAME}\n"
    echo -e "   This restore will shutdown Zimbra and make it"
    echo -e "   unavailable while it is being restored.\n"
    read -p "Are you absolutely sure you wish to restore (y/n)? "
    
    if [ "${REPLY}" != "y" ]; then
      echo -e "\n\nRestore aborted.\n\n"
      f_cleanup
      exit 0
    fi
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE}
    
    StartTime="$(date +%s)"
    
    ## Stop Zimbra Services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra stop
    
    ## Kill any orphaned Zimbra processes.
    pkill -9 -u zimbra 1>/dev/null 2>&1
    
    if [ ${NOBACKUP} -eq 1 ]; then
      ## Leave current production folder alone.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE}
    else
      ## Move production folder to backup location.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE}
      mv ${ZIMBRADIR} ${OLDDIR}
      RETURNVALUE=$?
      if [ ${RETURNVALUE} -ne 0 ]; then
        ## Could not move Zimbra to backup location.
        ## Document error and terminate script.
        echo ""
        echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE}
        echo ""
        echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE}
        echo ""
        echo "The Zimbra services are currently stopped and an unknown amount of files were moved"
        echo "from the source.  That means there some files are still in ${ZIMBRADIR}"
        echo "and some files were moved to ${OLDDIR}"
        echo "This needs to be resolved before you can start the services back up."
        echo "Either move the files back or delete all files and run the restore."
        f_cleanup
        exit 99
      fi
    fi
    
    ## Now that Zimbra is offline, we can restore all files.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files from ${FILENAME}" | tee -a ${LOGFILE}
    
    if [ ! -d ${ZIMBRADIR} ]; then
      mkdir -p ${ZIMBRADIR}
    fi
    
    ## Extract the entire contents of the archive back to its original location.
    case "${ARCHIVEMETHOD}" in
    tar.7z)
      7za x -so -w/${TEMPDIR} ${FILENAME} | tar -C ${ZIMBRADIR} --extract --strip-components=4 --file -
      ;;
    tgz)
      tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME}
      ;;
    *)
      tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME}
      ;;
    esac
    RETURNVALUE=$?
    if [ ${RETURNVALUE} -ne 0 ]; then
      ## tar extract command failed.  Display warning message.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: tar extract return value = {$RETURNVALUE}" | tee -a ${LOGFILE}
      f_cleanup
      f_emergencyexit 2
    fi
    
    if [ ! -f ${ZIMBRADIR}/.install_history ]; then
      ## Something is not right...at least this file should exist here.
      ## NOTE: This is not a guarentee all files were restored, it is just a quick check.
      echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s).  Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE}
    fi
    
    ## Fix permissions.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE}
    chown -R zimbra:zimbra ${ZIMBRADIR}
    ${ZIMBRADIR}/libexec/zmfixperms
    
    ## Start Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE}
    /etc/init.d/zimbra start
    
    ## Calculates downtime of Zimbra services.
    FinishTime="$(date +%s)"
    ElapsedTime="$(expr ${FinishTime} - ${StartTime})"
    Hours=$((${ElapsedTime} / 3600))
    ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600))
    Minutes=$((${ElapsedTime} / 60))
    Seconds=$((${ElapsedTime} - ${Minutes} * 60))
    
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE}
    
    ## Show status of Zimbra services.
    echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE}
    su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE}
    
    if [ -d ${OLDDIR} ]; then
      echo -e "\n\nDon't forget to purge ${OLDDIR} once everything is running smoothly again.\n\n"
    fi
    
    f_cleanup
    ## Dismount the remote site.
    f_umount
    
    echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE}
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  9. #19
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Disaster Scenario #5

    Scenario #5 Restore to new server
    ** UNFINISHED **
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

  10. #20
    Join Date
    Sep 2011
    Location
    Texas
    Posts
    150
    Rep Power
    4

    Default Crontab Schedule

    Crontab

    I would not advise anyone to ever "edit" a live crontab schedule by typing "crontab -e" but rather edit a saved schedule file and then load the schedule file. This will allow you to make backups of the schedule so you can always go back to a known-good schedule or at least back to the way it was before you made a change...assuming you always work with a copy of the schedule 1st.

    Here is my root crontab scheduling file:

    /var/scripts/data/crontab.root

    Code:
    ########################################
    # Name: Crontab Schedule for root user
    # Author: LHammonds
    ############# Update Log ###############
    # 2011-10-29 - LTH - Created schedule
    # 2011-11-22 - LTH - Added shell and path
    # 2012-01-30 - LTH - Added time adjustment
    # 2012-05-02 - LTH - Added check-space
    # 2012-05-05 - LTH - Added check mailbox size
    ########################################
    #
    # Import Active Directory users (run every minute)
    0-59 * * * * /var/scripts/prod/import-ad.sh > /dev/null 2>&1
    #
    # Update the Global Address List for iPhone usage.
    #
    0 23 * * * /var/scripts/prod/gal-sync.sh > /dev/null 2>&1
    #
    # Partial backup of Zimbra to a local folder without taking it offline.
    #
    0 8-18 * * * /var/scripts/prod/rsync-online.sh > /dev/null 2>&1
    #
    # Backup Zimbra to a local folder, archive and store offsite.
    #
    0 23 * * * /var/scripts/prod/rsync-offline.sh > /dev/null 2>&1
    #
    # Backup individual mailboxes to a local folder, archive and store offsite.
    #
    0 6-18 * * * /var/scripts/prod/zm-user-backup.sh > /dev/null 2>&1
    #
    # Daily check for available space on /opt
    #
    0 1 * * * /var/scripts/prod/check-storage.sh opt 5 1 > /dev/null 2>&1
    #
    # Daily check for available space on /var/backup
    #
    0 2 * * * /var/scripts/prod/check-storage.sh bak 5 1 > /dev/null 2>&1
    #
    # Daily check for available space on /var/temp
    #
    0 3 * * * /var/scripts/prod/check-storage.sh temp 5 1 > /dev/null 2>&1
    #
    # Daily check for mailbox size notifications
    #
    0 6 * * * /var/scripts/prod/mailbox-size-notification.sh > /dev/null 2>&1
    Once you have created the file, make sure appropriate permissions are set by typing the following:
    Code:
    chown root:root /var/scripts/data/crontab.root
    chmod 0600 /var/scripts/data/crontab.root
    To enable the root schedule using this file, type the following:

    Code:
    crontab -u root /var/scripts/data/crontab.root
    To disable the root schedule, type the following:
    Code:
    touch /tmp/deleteme
    crontab -u root /tmp/deleteme
    rm /tmp/deleteme
    If you need to modify the schedule, make a backup copy 1st. For example:

    Code:
    cp /var/scripts/data/crontab.root /var/scripts/prod/2011-11-28-crontab.root
    vi /var/scripts/data/crontab.root (make your changes)
    crontab -u root /var/scripts/data/crontab.root
    Last edited by LHammonds; 05-05-2012 at 12:20 PM.
    Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here)

Similar Threads

  1. /tmp filling
    By Nutz in forum Administrators
    Replies: 8
    Last Post: 02-22-2008, 01:00 AM
  2. Replies: 31
    Last Post: 12-15-2007, 08:05 PM
  3. Post instsallation problems
    By Assaf in forum Installation
    Replies: 14
    Last Post: 01-29-2007, 10:38 AM
  4. 3.1 on FC4 problems
    By cohnhead in forum Installation
    Replies: 8
    Last Post: 05-26-2006, 11:16 AM
  5. Zimbra server crashed
    By goetzi in forum Administrators
    Replies: 6
    Last Post: 03-25-2006, 12:00 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •