[SOLVED] Yet Another Backup Script for Community Version

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
LaFong
Outstanding Member
Outstanding Member
Posts: 221
Joined: Fri Sep 12, 2014 11:54 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby LaFong » Mon May 24, 2010 2:15 pm

[quote]NEVER HAD TO DO IT YET EITHER (TOUCH WOOD) - VERY KEEN TO HEAR HOW IT GOES. PLEASE REPLY AND LET US KNOW.
GOOD LUCK :-)[/QUOTE]
IN THE END, AFTER SOME FALSE STARTS, WE WERE ABLE TO GET THE USER'S INBOX RESTORED. I MADE CHANGES TO THE EARLIER POST TO REFLECT OUR EXPERIENCE. (REFERENCE "HTTP://WWW.ZIMBRA.COM/FORUMS/ADMINISTRATORS/15275-SOLVED-YET-ANOTHER-BACKUP-SCRIPT-COMMUNITY-VERSION-30.HTML#POST179918 )
ALSO, I HAVE CREATED A PER-USER BACKUP SCRIPT WHICH DOES THESE THINGS:



  • CREATES A PROCESSED SQL DUMP OF EACH USER, CONSISTING OF THEIR PERSONAL DATABASE, THEIR RECORDS IN THE ZIMBRA MAILBOX AND MAILBOX_METADATA TABLES, WITH A FEW ADDITIONAL SQL COMMANDS ADDED TO THE .SQL FILE. RESTORING THE USER'S DATABASE IS THEN A SIMPLE ONE-LINE COMMAND.

  • CREATES AN LDAP LDIF BACKUP OF EACH USER. RESTORING THEIR LDAP ENTRY IS A ONE-LINE COMMAND.

  • OPTIONALLY, CREATES A SIMPLE RAW, UNPROCESSED, BACKUP OF JUST THE USER'S DATABASE. ALTERNATELY, A 'MAKERAW' OPTION CAN MODIFY A USER'S PROCESSED DB BACK TO A RAW DB. THIS CAN BE USEFUL IN RESTORING INDIVIDUAL MAIL FOLDERS.


THE COMBINATION OF RESTORING THE 3 BACKUPS: USER STORE, DATABASE, AND LDAP LDIF, RESTORES THE USER. I HAVE DONE SEVERAL TEST RESTORES, AND IT APPEARS TO WORK FINE, BUT NO GUARANTEES.

THIS SCRIPT IS MEANT TO BE CALLED BY THE FILESYSTEM BACKUP SCRIPT IN THIS THREAD, THOUGH IT CAN BE RUN ON ITS OWN TOO, OR BE USED WITH A DIFFERENT FILESYSTEM BACKUP METHOD. IT IS CALLED RIGHT AFTER THE COLD RSYNC, AND SO MEANS ADDITIONAL DOWNTIME WHILE THE DB'S AND LDIFS ARE BEING CREATED. FOR ME, ON A SMALL SERVER, IT'S JUST A COUPLE OF MINUTES. THE LDAP BACKUP COULD BE DONE HOT, TO SHAVE SECONDS OR MINUTES OFF THE DOWNTIME FOR LARGER SERVERS.
IF ANYONE IS INTERESTED IN TAKING A LOOK, PM ME. I WANT TO WAIT TO PUT IT IN A POST UNTIL OTHERS HAVE TRIED IT OUT.


myahi
Posts: 1
Joined: Sat Sep 13, 2014 1:13 am

[SOLVED] Yet Another Backup Script for Community Version

Postby myahi » Tue May 25, 2010 9:11 am

i want to make backup just for my zimbra server configuration, it is possible to do it ?
phoenix
Ambassador
Ambassador
Posts: 26779
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

[SOLVED] Yet Another Backup Script for Community Version

Postby phoenix » Tue May 25, 2010 9:19 am

[quote user="myahi"]i want to make backup just for my zimbra server configuration, it is possible to do it ?[/QUOTE]Not just for configuration, use the script in this thread or one from the wiki on this page (you'll have to backup the whole server): Zimbra :: Wiki
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
jetorres
Posts: 22
Joined: Sat Sep 13, 2014 2:52 am

[SOLVED] Yet Another Backup Script for Community Version

Postby jetorres » Wed May 26, 2010 9:23 am

Since I change to the v0.8 of the script the backup is giving me an error during the scp copy section.
I scp to the same server the backup is running.
I added to the script, just before the scp command, and ps -ef command to output all the process running at same time the scp is to be run.
The log I received is:



============================

Sat May 22 19:00:02 AST 2010

Performing FULL backup

============================

full Backup started at: Sat May 22 19:00:02 AST 2010 Old backups found...old week= 19 current week= 20
/media/zimbrabackup/dars/19_Zimbra_Backup_20100515_FULL.1.dar moved to /media/zimbrabackup/burn/

/media/zimbrabackup/dars/19_Zimbra_Backup_20100515_FULL.1.dar.md5 moved to /media/zimbrabackup/burn/ /media/zimbrabackup/dars/19_Zimbra_Backup_20100515_FULL.2.dar moved to /media/zimbrabackup/burn/

/media/zimbrabackup/dars/19_Zimbra_Backup_20100515_FULL.2.dar.md5

...

moved to /media/zimbrabackup/burn/ /media/zimbrabackup/dars/19_Zimbra_Backup_20100521_DIFF.txt.gz moved to /media/zimbrabackup/burn/ Setting date & version Marker into /opt/zimbra/DATE_VERSION.txt Doing a hotsync of /opt/zimbra/ to /media/zimbrabackup/sync/ Disabling the Zimbra crontab...

Stopping the Zimbra server...
Host conecosrv1.conecodepr.com

Stopping stats...Done

Stopping mta...Done

Stopping spell...Done

Stopping snmp...Done

Stopping archiving...Done

Stopping antivirus...Done

Stopping antispam...Done

Stopping imapproxy...Done

Stopping mailbox...Done

Stopping logger...Done

Stopping ldap...Done

Doing a fast cold sync...

Reinstating Zimbra's crontab...

Starting Zimbra...

Host conecosrv1.conecodepr.com

Starting ldap...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting imapproxy...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting snmp...Done.

Starting spell...Done.

Starting mta...Done.

Starting stats...Done.

Service down time was - Hr:0 Min:3 Sec:39 Writing a full backup: 20_Zimbra_Backup_20100522_FULL

into: /media/zimbrabackup/dars/ with file sizes of max: 4096M Saving Unencrtyped Archive...


--------------------------------------------

134789 inode(s) saved

with 12735 hard link(s) recorded

0 inode(s) changed at the moment of the backup 0 inode(s) not saved (no inode/file change) 0 inode(s) failed to save (filesystem error) 0 inode(s) ignored (excluded by filters) 0 inode(s) recorded as deleted from reference backup

--------------------------------------------

Total number of inode considered: 134789

--------------------------------------------

EA saved for 0 inode(s)

--------------------------------------------

Creating MD5 Checksum for 20_Zimbra_Backup_20100522_FULL.1.dar...

/usr/bin/md5sum.textutils -b 20_Zimbra_Backup_20100522_FULL.1.dar > 20_Zimbra_Backup_20100522_FULL.1.dar.md5

Creating MD5 Checksum for 20_Zimbra_Backup_20100522_FULL.2.dar...

/usr/bin/md5sum.textutils -b 20_Zimbra_Backup_20100522_FULL.2.dar > 20_Zimbra_Backup_20100522_FULL.2.dar.md5

Creating MD5 Checksum for 20_Zimbra_Backup_20100522_FULL.3.dar...

/usr/bin/md5sum.textutils -b 20_Zimbra_Backup_20100522_FULL.3.dar > 20_Zimbra_Backup_20100522_FULL.3.dar.md5
UID PID PPID C STIME TTY TIME CMD

root 1 0 0 Apr19 ? 00:00:03 /sbin/init

...

root 3116 1 0 Apr19 ? 00:00:00 /sbin/udevd --daemon

www-data 3345 6348 0 May16 ? 00:00:00 /usr/sbin/apache2 -k start

www-data 3348 6348 0 May16 ? 00:00:00 /usr/sbin/apache2 -k start

www-data 3350 6348 0 May16 ? 00:00:00 /usr/sbin/apache2 -k start

www-data 3351 6348 0 May16 ? 00:00:00 /usr/sbin/apache2 -k start

www-data 3352 6348 0 May16 ? 00:00:00 /usr/sbin/apache2 -k start

root 3431 2 0 Apr19 ? 00:00:00 [kpsmoused]

root 3442 2 0 Apr19 ? 00:00:00 [edac-poller]

postfix 4428 17248 0 20:55 ? 00:00:00 smtpd -n smtp -t inet -u -o stress

root 4670 2 0 Apr19 ? 00:00:00 [kjournald]

root 4671 2 0 Apr19 ? 00:03:28 [kjournald]

root 4672 2 0 Apr19 ? 00:02:46 [kjournald]

root 4673 2 0 Apr19 ? 00:00:00 [kjournald]

root 4674 2 0 Apr19 ? 00:01:48 [kjournald]

root 4675 2 0 Apr19 ? 00:00:00 [kjournald]

root 4973 1 0 Apr19 tty4 00:00:00 /sbin/getty 38400 tty4

root 4974 1 0 Apr19 tty5 00:00:00 /sbin/getty 38400 tty5

root 4976 1 0 Apr19 tty2 00:00:00 /sbin/getty 38400 tty2

root 4978 1 0 Apr19 tty3 00:00:00 /sbin/getty 38400 tty3

root 4980 1 0 Apr19 tty6 00:00:00 /sbin/getty 38400 tty6

root 5182 1 0 Apr19 ? 00:00:00 /usr/sbin/acpid -c /etc/acpi/events -s /var/run/acpid.socket

root 5242 2 0 Apr19 ? 00:00:00 [kondemand/0]

root 5243 2 0 Apr19 ? 00:00:00 [kondemand/1]

root 5244 2 0 Apr19 ? 00:00:00 [kondemand/2]

root 5245 2 0 Apr19 ? 00:00:00 [kondemand/3]

syslog 5333 1 0 Apr19 ? 00:05:44 /sbin/syslogd -u syslog

root 5388 1 0 Apr19 ? 00:00:00 /bin/dd bs 1 if /proc/kmsg of /var/run/klogd/kmsg

klog 5390 1 0 Apr19 ? 00:00:00 /sbin/klogd -P /var/run/klogd/kmsg

113 5412 1 0 Apr19 ? 00:00:00 /usr/bin/dbus-daemon --system

root 5428 1 0 Apr19 ? 00:00:00 /usr/sbin/NetworkManager --pid-file /var/run/NetworkManager/NetworkManager.pid

root 5443 1 0 Apr19 ? 00:00:00 /usr/sbin/NetworkManagerDispatcher --pid-file /var/run/NetworkManager/NetworkManagerDispatcher.pid

root 5456 1 0 Apr19 ? 00:00:00 /usr/bin/system-tools-backends

bind 5483 1 0 Apr19 ? 00:00:06 /usr/sbin/named -u bind

avahi 5550 1 0 Apr19 ? 00:00:46 avahi-daemon: running [conecosrv1.local]

avahi 5551 5550 0 Apr19 ? 00:00:00 avahi-daemon: chroot helper

root 5602 1 0 Apr19 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe

mysql 5645 5602 0 Apr19 ? 00:15:47 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock

root 5646 5602 0 Apr19 ? 00:00:00 logger -p daemon.err -t mysqld_safe -i -t mysqld

backuppc 5736 1 0 Apr19 ? 00:00:54 /usr/bin/perl /usr/share/backuppc/bin/BackupPC -d

backuppc 5751 5736 0 Apr19 ? 00:04:41 /usr/bin/perl /usr/share/backuppc/bin/BackupPC_trashClean

root 5763 1 0 Apr19 ? 00:00:00 /usr/sbin/cupsd

root 5810 1 0 Apr19 ? 00:00:00 /usr/sbin/rinetd

root 5826 1 0 Apr19 ? 00:00:44 /usr/sbin/nmbd -D

root 5829 1 0 Apr19 ? 00:00:00 /usr/sbin/smbd -D

root 5846 5829 0 Apr19 ? 00:00:00 /usr/sbin/smbd -D

root 5847 1 0 Apr19 ? 00:00:04 /usr/sbin/winbindd

root 5866 5847 0 Apr19 ? 00:00:00 /usr/sbin/winbindd

root 5874 1 0 Apr19 ? 00:00:00 /usr/sbin/xinetd -pidfile /var/run/xinetd.pid -stayalive -inetd_compat

root 5942 1 0 Apr19 ? 00:01:42 /usr/sbin/dhcdbd --system

116 5961 1 0 Apr19 ? 00:00:26 /usr/sbin/hald

root 5964 1 0 Apr19 ? 00:00:00 /usr/sbin/console-kit-daemon

root 5965 5961 0 Apr19 ? 00:00:00 hald-runner

root 6041 5965 0 Apr19 ? 00:00:00 hald-addon-input: Listening on /dev/input/event1 /dev/input/event4

116 6048 5965 0 Apr19 ? 00:00:00 hald-addon-acpi: listening on acpid socket /var/run/acpid.socket

root 6050 5965 0 Apr19 ? 00:00:07 hald-addon-storage: no polling on /dev/fd0 because it is explicitly disabled

root 6069 5965 0 Apr19 ? 00:01:14 hald-addon-storage: polling /dev/scd0 (every 16 sec)

root 6095 1 0 Apr19 ? 00:00:00 /usr/sbin/hcid -x -s

root 6112 2 0 Apr19 ? 00:00:00 [btaddconn]

root 6113 2 0 Apr19 ? 00:00:00 [btdelconn]

root 6135 6095 0 Apr19 ? 00:00:00 /usr/lib/bluetooth/bluetoothd-service-audio

root 6142 2 0 Apr19 ? 00:00:00 [krfcommd]

root 6156 6095 0 Apr19 ? 00:00:00 /usr/lib/bluetooth/bluetoothd-service-input

root 6215 1 0 Apr19 ? 00:00:00 /usr/sbin/gdm

root 6218 6215 0 Apr19 ? 00:00:00 /usr/sbin/gdm

root 6222 6218 0 Apr19 tty7 00:12:07 /usr/bin/X :0 -br -audit 0 -auth /var/lib/gdm/:0.Xauth vt7

daemon 6281 1 0 Apr19 ? 00:00:00 /usr/sbin/atd

root 6295 1 0 Apr19 ? 00:00:12 /usr/sbin/cron

root 6348 1 0 Apr19 ? 00:01:08 /usr/sbin/apache2 -k start

root 6371 1 0 Apr19 ? 00:00:01 /opt/likewise/sbin/dcerpcd

root 6435 1 0 Apr19 ? 00:00:03 /usr/sbin/sshd

gdm 6448 6218 0 Apr19 ? 00:45:04 /usr/lib/gdm/gdmgreeter

root 6469 1 0 Apr19 ? 00:00:01 /opt/likewise/sbin/eventlogd --start-as-daemon

root 6502 1 0 Apr19 ? 00:00:00 /opt/likewise/sbin/lwiod --start-as-daemon

root 6537 1 0 Apr19 ? 00:00:10 /opt/likewise/sbin/netlogond --start-as-daemon

root 6575 1 0 Apr19 ? 00:22:54 /opt/likewise/sbin/lsassd --start-as-daemon

root 6687 2 0 Apr19 ? 00:00:01 [cifsoplockd]

root 6688 2 0 Apr19 ? 00:00:00 [cifsdnotifyd]

root 6698 2 0 Apr19 ? 00:10:39 [cifsd]

root 6710 1 0 Apr19 ? 00:00:38 /usr/bin/perl /usr/share/webmin/miniserv.pl /etc/webmin/miniserv.conf

ntp 6757 1 0 Apr19 ? 00:00:48 /usr/sbin/ntpd -p /var/run/ntpd.pid -u 117:130 -g

postfix 7233 17248 0 21:00 ? 00:00:00 cleanup -z -t unix -u

postfix 7234 17248 0 21:00 ? 00:00:00 smtp -n smtp-amavis -t unix -u -o smtp_data_done_timeout 1200 -o smtp_send_xforward_command yes -o disable_dns_lookups yes -o max_use 20

postfix 7237 17248 0 21:00 ? 00:00:00 smtpd -n 127.0.0.1:10025 -t inet -u -o content_filter -o local_recipient_maps -o virtual_mailbox_maps -o virtual_alias_maps -o relay_recipient_maps -o smtpd_restriction_classes -o smtpd_delay_reject no -o smtpd_client_restrictions permit_mynetworks,reject -o smtpd_helo_restrictions -o smtpd_milters -o smtpd_sender_restrictions -o smtpd_recipient_restrictions permit_mynetworks,reject -o mynetworks_style host -o mynetworks 127.0.0.0/8 -o strict_rfc821_envelopes yes -o smtpd_error_sleep_time 0 -o smtpd_soft_error_limit 1001 -o smtpd_hard_error_limit 1000 -o smtpd_client_connection_count_limit 0 -o smtpd_client_connection_rate_limit 0 -o receive_override_options no_header_body_checks,no_unknown_recipient_checks,no_address_mappings

postfix 7238 17248 0 21:00 ? 00:00:00 lmtp -t unix -u

root 7248 11737 0 21:00 ? 00:00:00 ps -ef

root 10510 1 0 Apr19 tty1 00:00:00 /sbin/getty 38400 tty1

root 11730 6295 0 19:00 ? 00:00:00 /USR/SBIN/CRON

root 11732 11730 0 19:00 ? 00:00:00 /bin/sh -c /bin/bash /opt/conecobin/zmbac.sh -f > /var/log/zim_backup.log 2>&1

root 11737 11732 0 19:00 ? 00:00:00 /bin/bash /opt/conecobin/zmbac.sh -f

root 13928 2 0 19:02 ? 00:00:07 [pdflush]

zimbra 14009 1 0 19:03 ? 00:00:12 /opt/zimbra/openldap/libexec/slapd -l LOCAL0 -4 -u zimbra -h ldap://conecosrv1.conecodepr.com:389 -f /opt/zimbra/conf/slapd.conf

zimbra 14475 1 0 19:03 ? 00:00:00 /bin/sh /opt/zimbra/logger/mysql/bin/mysqld_safe --defaults-file=/opt/zimbra/conf/my.logger.cnf --external-locking --log-error=/opt/zimbra/log/logger_mysql_error.log --ledir=/opt/zimbra/logger/mysql/libexec

zimbra 14482 1 0 19:03 ? 00:00:00 /usr/bin/perl /opt/zimbra/libexec/logswatch --config-file=/opt/zimbra/conf/logswatchrc --use-cpan-file-tail --pid-file=/opt/zimbra/log/logswatch.pid --script-dir=/tmp -t /var/log/zimbra.log

zimbra 14521 14475 0 19:03 ? 00:00:54 /opt/zimbra/logger/mysql/libexec/mysqld --defaults-file=/opt/zimbra/conf/my.logger.cnf --basedir=/opt/zimbra/logger/mysql --datadir=/opt/zimbra/logger/db/data --pid-file=/opt/zimbra/logger/db/mysql.pid --skip-external-locking --port=7307 --socket=/opt/zimbra/logger/db/mysql.sock --external-locking

zimbra 14523 14482 0 19:03 ? 00:00:04 /usr/bin/perl /tmp/.swatch_script.14482

zimbra 14534 14523 0 19:03 ? 00:00:01 /usr/bin/perl /opt/zimbra/libexec/zmlogger

zimbra 14654 1 0 19:03 ? 00:00:13 /usr/bin/perl /opt/zimbra/libexec/zmmtaconfig

zimbra 14660 1 0 19:03 ? 00:00:00 /bin/sh /opt/zimbra/mysql/bin/mysqld_safe --defaults-file=/opt/zimbra/conf/my.cnf --external-locking --log-error=/opt/zimbra/log/mysql_error.log --ledir=/opt/zimbra/mysql/libexec

zimbra 14729 14660 0 19:03 ? 00:00:24 /opt/zimbra/mysql/libexec/mysqld --defaults-file=/opt/zimbra/conf/my.cnf --basedir=/opt/zimbra/mysql --datadir=/opt/zimbra/db/data --pid-file=/opt/zimbra/db/mysql.pid --skip-external-locking --port=7306 --socket=/opt/zimbra/db/mysql.sock --external-locking

root 15883 1 0 19:04 ? 00:00:00 /opt/zimbra/libexec/zmmailboxdmgr start -Xms1582m -Xmx1582m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k

zimbra 15884 15883 0 19:04 ? 00:00:40 /opt/zimbra/java/bin/java -Xms1582m -Xmx1582m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml

zimbra 15898 1 0 19:04 ? 00:00:00 /opt/zimbra/memcached/bin/memcached -d -P /opt/zimbra/log/memcached.pid

zimbra 16254 1 0 19:04 ? 00:00:00 nginx: master process /opt/zimbra/nginx/sbin/nginx -c /opt/zimbra/conf/nginx.conf

zimbra 16255 16254 0 19:04 ? 00:00:00 nginx: worker process

zimbra 16256 16254 0 19:04 ? 00:00:00 nginx: worker process

zimbra 16258 16254 0 19:04 ? 00:00:00 nginx: worker process

zimbra 16259 16254 0 19:04 ? 00:00:00 nginx: worker process

zimbra 16282 1 0 19:05 ? 00:00:01 amavisd (master)

zimbra 16320 1 0 19:05 ? 00:00:00 /opt/zimbra/clamav/bin/freshclam --config-file=/opt/zimbra/conf/freshclam.conf -d --checks=12

zimbra 16321 16282 0 19:05 ? 00:00:01 amavisd (ch4-avail)

zimbra 16322 16282 0 19:05 ? 00:00:02 amavisd (ch3-avail)

zimbra 16323 16282 0 19:05 ? 00:00:02 amavisd (ch6-avail)

zimbra 16324 16282 0 19:05 ? 00:00:04 amavisd (ch6-avail)

zimbra 16325 16282 0 19:05 ? 00:00:02 amavisd (ch5-avail)

zimbra 16326 16282 0 19:05 ? 00:00:02 amavisd (ch3-avail)

zimbra 16327 16282 0 19:05 ? 00:00:02 amavisd (ch5-avail)

zimbra 16328 16282 0 19:05 ? 00:00:02 amavisd (ch5-avail)

zimbra 16329 16282 0 19:05 ? 00:00:01 amavisd (ch2-avail)

zimbra 16330 16282 0 19:05 ? 00:00:03 amavisd (ch4-avail)

zimbra 16428 1 0 19:05 ? 00:00:08 /opt/zimbra/clamav/sbin/clamd --config-file=/opt/zimbra/conf/clamd.conf

zimbra 16468 1 0 19:05 ? 00:00:00 /usr/bin/perl /opt/zimbra/libexec/swatch --config-file=/opt/zimbra/conf/swatchrc --use-cpan-file-tail --script-dir=/tmp -t /var/log/zimbra.log

zimbra 16472 16468 0 19:05 ? 00:00:03 /usr/bin/perl /tmp/.swatch_script.16468

zimbra 16480 1 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

zimbra 16482 16480 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

zimbra 16484 16480 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

zimbra 16485 16480 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

zimbra 16487 16480 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

zimbra 16488 16480 0 19:05 ? 00:00:00 /opt/zimbra/httpd-2.2.8/bin/httpd -k start -f /opt/zimbra/conf/httpd.conf

root 17248 1 0 19:05 ? 00:00:00 /opt/zimbra/postfix/libexec/master

postfix 17255 17248 0 19:05 ? 00:00:00 qmgr -l -t fifo -u

zimbra 17256 1 0 19:05 ? 00:00:00 /opt/zimbra/cyrus-sasl/sbin/saslauthd -r -a zimbra

zimbra 17257 17256 0 19:05 ? 00:00:00 /opt/zimbra/cyrus-sasl/sbin/saslauthd -r -a zimbra

zimbra 17258 17256 0 19:05 ? 00:00:00 /opt/zimbra/cyrus-sasl/sbin/saslauthd -r -a zimbra

zimbra 17260 17256 0 19:05 ? 00:00:00 /opt/zimbra/cyrus-sasl/sbin/saslauthd -r -a zimbra

zimbra 17261 17256 0 19:05 ? 00:00:00 /opt/zimbra/cyrus-sasl/sbin/saslauthd -r -a zimbra

zimbra 17333 1 0 19:05 ? 00:00:05 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-proc

zimbra 17335 1 0 19:05 ? 00:00:00 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-cpu

zimbra 17337 1 0 19:05 ? 00:00:00 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-vm

zimbra 17343 1 0 19:05 ? 00:00:00 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-fd

zimbra 17345 1 0 19:05 ? 00:00:01 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-mysql

zimbra 17350 1 0 19:05 ? 00:00:00 /usr/bin/perl -w /opt/zimbra/libexec/zmstat-mtaqueue

root 17679 2 0 May21 ? 00:00:07 [pdflush]

zimbra 17937 17337 0 19:06 ? 00:00:00 /usr/bin/vmstat -n -S K 30

postfix 18241 17248 0 19:06 ? 00:00:00 tlsmgr -l -t unix -u

postfix 18243 17248 0 19:06 ? 00:00:00 anvil -l -t unix -u

root 21805 5965 0 May08 ? 00:00:21 hald-addon-hid-ups: listening on /dev/usb/hiddev0

postfix 30282 17248 0 20:41 ? 00:00:00 showq -t unix -u

postfix 30976 17248 0 20:42 ? 00:00:00 pickup -l -t fifo -u
copy archive to conecosrv1.conecodepr.com remote directory /media/iomega/iomega-005d42/ConecoSrv1/ZimbraBackup

scp: /media/iomega/iomega-005d42/ConecoSrv1/ZimbraBackup/20_Zimbra_Backup_20100522_FULL.3.dar: truncate: Text file busy Error copying archive and checksum to conecosrv1.conecodepr.com


I do not see why scp has a "Text file busy"
cornbread
Advanced member
Advanced member
Posts: 80
Joined: Fri Sep 12, 2014 10:46 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby cornbread » Mon May 31, 2010 6:48 pm

First of all thx for such a great script.
Getting an rsync error that stops the script and leaves my mail server stopped.
rsync: read errors mapping "/opt/zimbra/store/0/19/msg/2/9582-83735.msg": Input/output error (5)
How can I fix this so I can continue to backup?
LaFong
Outstanding Member
Outstanding Member
Posts: 221
Joined: Fri Sep 12, 2014 11:54 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby LaFong » Mon May 31, 2010 8:43 pm

I would do a filesystem check of both source and destination.
cornbread
Advanced member
Advanced member
Posts: 80
Joined: Fri Sep 12, 2014 10:46 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby cornbread » Tue Jun 01, 2010 12:09 am

filesystem check went fine no issues.
Still getting:

rsync: read errors mapping "/opt/zimbra/store/0/19/msg/2/9582-83735.msg": Input/output error (5)
cornbread
Advanced member
Advanced member
Posts: 80
Joined: Fri Sep 12, 2014 10:46 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby cornbread » Tue Jun 01, 2010 12:59 am

I cp'd the file and rm'd it from the server. backup went fine then cp'd it back to server.
LaFong
Outstanding Member
Outstanding Member
Posts: 221
Joined: Fri Sep 12, 2014 11:54 pm

[SOLVED] Yet Another Backup Script for Community Version

Postby LaFong » Thu Jun 10, 2010 1:29 pm

Since I got a rousing response of 1 inquiry to my post about adding per-user backup to this script, I thought I'd just go ahead and post it.
Here are the changes. In the main backup script, add the lines with the + in them (leaving out the +, of course).


--- OLD/zmbac.0.8.sh.orig 2010-02-14 20:19:22.000000000 -0700

+++ zmbac.sh 2010-05-16 14:43:40.000000000 -0600

@@ -61,25 +61,26 @@

#--- Directories ---#

# Please add the trailing "/" to directories!

ZM_HOME=/opt/zimbra/ # where zimbra lives

SYNC_DIR=/backup/sync/ # intermediate dir for hot/cold syncs. must have at least as much free space as ZM_HOME consumes

ARCHIVEDIR=/backup/current/ # where to store final backups

TO_MEDIA_DIR=/backup/previous/

+ SCRIPT_DIR=/usr/local/sbin/ # where zmbac.sh and zmDBbac.sh reside





@@ -129,6 +130,9 @@

#Hack to start Stats, even run zmlogprocess if needed

STATHACK="yes" # valid answers are "yes" or "no"



+#--- Dump Databases? ---#

+# Adjust zmDBbac.sh script to backup up per-user or raw dbs, or both.

+DUMP_DBS="yes" # valid answers are "yes" or "no"




## ~~~~~!!!! SCRIPT RUNTIME !!!!!~~~~~ ##

# Best you don't change anything from here on,
@@ -695,6 +764,28 @@

mail_log

exit 1

fi

+ # Dump DB's, if enabled

+ if [ $DUMP_DBS = "yes" ] ; then

+ echo

+ # Create db_dumps directory

+ if [ ! -d $SYNC_DIR"db_dumps" ] ; then

+ echo "Creating db_dumps directory..."

+ chmod 755 $SYNC_DIR

+ mkdir $SYNC_DIR"db_dumps"

+ chown $ZM_USER $SYNC_DIR"db_dumps"

+ fi

+ # Run script to dump dbs

+ echo "Backing up Zimbra DBs..."

+ su - $ZM_USER -c $SCRIPT_DIR"zmDBbac.sh $SYNC_DIR"

+ if [ "$?" -ne "0" ] ; then

+ echo "There was an error running DB backup script! Aborting DB backup. Continuing main backup."

+ fi

+ # Check that /opt/zimbra/db_dumps exists, or create empty /opt/zimbra/db_dumps directory tree if necessary

+ if [ ! -d $ZM_HOME"db_dumps" ] && [ -d $SYNC_DIR"db_dumps" ] ; then

+ $RSYNC_BIN -a -f"+ */" -f"- *" $SYNC_DIR"db_dumps" $ZM_HOME

+ fi

+ fi

+ echo


# Starting the Zimbra server again

# Reinstate zimbra user's crontab

echo "Reinstating Zimbra's crontab..."

These changes set the script directory, enable DB backups, then call the DB backup script.
You can download the db backup and restore scripts at: zmDBbac.zip
Backuphttp://bit.ly/rkPbUs>zmDBbac.zip
Backup
> 1. Put the zmDBbac.sh and restore_user.sh scripts in the same directory as zmbac.sh

2. Make them executable:



chmod 755 zmDBbac.sh

chmod 755 restore_user.sh

3. Make any changes to settings in the zmDBbac.sh script. These are



  • Create processed DB dump, with added SQL commands to ease restore. (default)

  • Create RAW db dump, with no processing. (optional. Since you can convert a processed db dump back to raw using the 'makeraw' option in this script, raw dumps should not be needed. Raw db's are handy for individual folder restore.)

  • Specify a domain DN to back up. (optional)

  • Specify a COS to back up (optional)



4. The zmDBbac.sh script will create several folders. It creates a db_dumps directory in your SYNC_DIR directory, with ldap, raw, and mailboxes subdirectories. It creates these same folders in /opt/zimbra. The directories in /opt/zimbra remain empty always, and are merely used by rsync to empty the directories in SYNC_DIR/db_dumps every night.
In your nightly email from the script, it will show the db dump results:



============================

Wed May 5 03:00:02 MDT 2010

Performing DIFF backup

============================



Doing a fast cold sync...
Backing up Zimbra DBs...

Making sure all Zimbra services are stopped

Starting just LDAP and MySQL for db dump...

Starting LDAP...

Started slapd: pid 27636

Starting MySQL...

Starting mysqld...done.

Trying to connect to MySQL...
Dumping and processing per-user dbs and exporting per-user LDAP entries...

E-Mail User ID Database LDAP zimbraId

------ ------- --------- ------ --------

(zimbra) -- zimbra -- 60fa3489-9854-22d9-8f21-000a67a98ef2

Alise.Amadeo@somewhere.com 11 mboxgroup11 11.ldif 71c8a11d-02ec-c29c-9dd9-b49c80600ca5

Bernetta.Verdin@somewhere.com 2 mboxgroup2 2.ldif 2705a240-07fe-cccc-acfd-cc26c489c293

Caitlin.Kellman@somewhere.com 45 mboxgroup45 45.ldif 67a60dca-07ad-c66a-9d6f-9f7832d29440

Cassey.Antczak@somewhere.com 42 mboxgroup42 42.ldif fc4fc12d-b16e-c91d-ae3d-b6b86a72826c

Cecile.Lorenz@somewhere.com 12 mboxgroup12 12.ldif 6d0ec99f-c70e-caa8-afaa-e48ca98cae59

Chau.Gable@somewhere.com 43 mboxgroup43 43.ldif 8dfe7a6f-0b9f-cbff-9b8c-e1f5b860c3b8

Dannie.Weatherman@somewhere.com 103 mboxgroup3 103.ldif f6ddd11c-ce1a-c6b9-a3ad-164762d5c8d3



Total users = 88

Stopping LDAP and MySQL...

Stopping mysqld... done.

Killing slapd with pid 27636 done.
Reinstating Zimbra's crontab...

Starting Zimbra...

Host mail.somewhere.com

Starting ldap...Done.

Starting logger...Done.

Starting mailbox...Done.

Starting antispam...Done.

Starting antivirus...Done.

Starting snmp...Done.

Starting spell...Done.

Starting mta...Done.

Starting stats...Done.

Service down time was - Hr:0 Min:7 Sec:39



::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

diff Zimbra Backup ended at: 03:33

Backup took Hr:0 Min:33 Sec:43 to complete

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

When users are added, only their LDAP entry is created. Their database entry is not created until the account is accessed. There are two ways of dealing with this:


  • If AUTCREATE_DB is set to "yes" (default), the script starts the zmmailboxd service and just queries the user's mailbox size. This prompts Zimbra to create the user's database.

  • If AUTCREATE_DB is set to "no", their entry in the nightly email will look like:
    Some.User@somewhere.com             --          No_DB_Exists!*   --            46fc9b59-74aa-4d31-bfc3-7f66145bf285




The user's LDAP entry will also not get backed up until their database gets created.

A manual way of creating the user's database entry is just to Edit their account from the Admin interface. You don't have to make any changes, just open the Edit window.
Restore
You can find the user's id from the nightly email, or in /mysyncdir/db_dumps/namelist.txt

Restoring user "12" would go like:
Autmatic method, using restore_user.sh script:

1. Be sure you have a good backup.

2. Delete the user, if necessary. You basically always want to delete the user first, if they still exist, for a full user restore. This is to avoid duplicate database entries when you restore their DB data.

3. Restore the user's message store to /opt/zimbra/store/#/12, with "#" being the volume directory (initial volume default is "0"). There may be several volume directories, with a portion of the user's store in each. You need to restore the user's store directory to each volume directory.

4. Restore the user's index to /opt/zimbra/index/#/12, with "#" being the volume directory, in the same manner as the message store.

5. Place the user's .sql and .ldif backup files in the same directory. If the user's database does not exist, also place skeleton.sql into the same directory. The skeleton file is used to create basic table structure before importing the user's SQL data.

6. Run restore_user.sh script, as Zimbra user, giving it the location of the of the backup files and the user's mailbox id, e.g.



/usr/local/sbin/restore_user.sh /tmp/restore 12

The script will ask you whether you want to create the database (if it doesn't already exist), whether to change the email address or mail server host name, and whether to import SQL and LDAP data. It also prompts a Zimbra shutdown: importing data is most safely done when Zimbra services (except mysql and openldap) are not running.
Manual method:

You can manually import data instead of using the restore_user.sh script.

1. Be sure you have a good backup

2. Restore user's store and index, per above.

3. Best practice is to shut down Zimbra, and start only its MySQL and LDAP services.

4. As Zimbra user, restore user's db and zimbra db records:



mysql
Note: the user's database must already exist, or you'll have to create it manually first, including all table structure. This can be done by adding SQL statements above and below the USE mboxgroup* line:

CREATE DATABASE IF NOT EXISTS mboxgroup12;

USE mboxgroup12

SOURCE /tmp/restore/skeleton.sql

Make the changes before importing the .sql file.

5. Restore the user's LDAP entry, if necessary. As Zimbra user (changing paths and host as necessary):



source ~/bin/zmshutil ; zmsetvars

/opt/zimbra/openldap/bin/ldapadd -D "uid=zimbra,cn=admins,cn=zimbra" -f /mysyncdir/db_dumps/mailboxes/12.ldif -w "$zimbra_ldap_password" -x -H ldap://myzimbraserver.somewhere.com

Note: If you want to restore using a different email address, or to a differently-named mailserver, you will need to manually edit the ldif and sql files first.
---------------------

As previously stated:



  • This script must do the db dumps cold, which adds a few, to several, minutes of downtime during backup.

  • This script also does the LDAP user backup cold. Cold LDAP backups add a minute or two to server downtime for small servers. For larger servers, the LDAP backup could be moved to the main backup script, and be done hot.

  • I have done several successful test restores to a secondary server, but have not thoroughly tested every aspect of it. It would be wise to try test restores on a non-production server.



----------------------------------------------------------------------

If you only want to restore a single folder, you can use the DB backup like this:

Restore">http://www.zimbra.com/forums/administrators/15275-solved-yet-another-backup-script-community-version-30.html#post179918>Restore a single folder
liverpoolfcfan
Outstanding Member
Outstanding Member
Posts: 963
Joined: Sat Sep 13, 2014 12:47 am

[SOLVED] Yet Another Backup Script for Community Version

Postby liverpoolfcfan » Thu Jun 10, 2010 3:23 pm

[quote user="LaFong"]Since I got a rousing response of 1 inquiry to my post about adding per-user backup to this script, I thought I'd just go ahead and post it.
[/QUOTE]
Great post.
Will certainly give this a try.

Return to “Administrators”

Who is online

Users browsing this forum: No registered users and 18 guests