Backups - RSYNC & copying files issue

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
sternfan
Advanced member
Advanced member
Posts: 144
Joined: Fri Sep 12, 2014 10:30 pm

Backups - RSYNC & copying files issue

Postby sternfan » Thu Dec 20, 2007 8:14 am

Hi All,
Scenario: I backup my zimbra server using RSYNC to a NFS share (another server named BACKUP next to it, connected using a switch over gigabit). I do the RSYNC and looks good - no errors at the end that I can tell.
I have another server (I call ARCHIVE) that is in another building (off site) but still connected by gigabit network. My standard procedure is to do the backup to the backup server, then copy this backup to the ARCHIVE server (off site backups done the easy way).
OK - so I go to copy the backup folder to ARCHIVE and a whole bunch of files will not copy. I get an ERROR "UNSUPORTED OPERATION..." then the filename. There are over a hundred of these. The first couple are:
rammd5.so

libplain.so

login.so.z

ssapiv2.so
and on and on...
Once done - I look at the properties of both folders:
Backup is 26.1g with 313,273 files.

Archive is 26.1g with 312,990 files.
So I am basically missing about 283 files after the copy operation (they won't copy).
First Question - what can cause this? The fact that ARCHIVE is an SMB share? Something to do with permissions? RSYNC? I've copied larger files to the SMB share before with no problems.
Second Question - I tried to move my Zimbra install to new hardware about a week ago using roughly the same procedure (RSYNC to BACKUP, then on the new server RSYNC from BACKUP etc). I couldn't make this work and am wondering if this could be related to why.
Thanks for any and all help,

Rob


13445raj
Outstanding Member
Outstanding Member
Posts: 743
Joined: Fri Sep 12, 2014 9:59 pm
Location: Canada
Contact:

Backups - RSYNC & copying files issue

Postby 13445raj » Thu Dec 20, 2007 8:40 am

Hi..did you follow the WIKI

Open Source Edition Backup Procedure - Zimbra :: Wiki
you did not mention if you RSYNC'ed while zimbra was runnnig or you stopped all services.

I am assuming you running OSS Version and Yout rying to backup files while zimbra is running..can you please give more info on this.
Basic steps to do COLD backup on same machine (/opt/zimbra_backup ) would be
As zimbra user:

~ su - zimbra

~ zmcontrol stop

~ logout
As root user:

# rsync -avHK /opt/zimbra/ /opt/zimbra_backup/zimbra
As zimbra user:

~ su - zimbra

~ zmcontrol start
Raj
sternfan
Advanced member
Advanced member
Posts: 144
Joined: Fri Sep 12, 2014 10:30 pm

Backups - RSYNC & copying files issue

Postby sternfan » Thu Dec 20, 2007 8:55 am

sorry about that - I stopped the services as normal. Verified with grep.
I did an rysnc -avr; how is that different than a rsync -avHK? I googled for it and can't find any info.
Rob
13445raj
Outstanding Member
Outstanding Member
Posts: 743
Joined: Fri Sep 12, 2014 9:59 pm
Location: Canada
Contact:

Backups - RSYNC & copying files issue

Postby 13445raj » Thu Dec 20, 2007 9:53 am

Hi..i just used from the WIKI links and it works..what exactly it dose is I dont know..may be WIKI author can shed some light.

All I know it backups up files and permissions and any syslinks = backup zimbra correctly for cold backup and restore :)
i use same command to restore from backuped up folder..
Raj
sternfan
Advanced member
Advanced member
Posts: 144
Joined: Fri Sep 12, 2014 10:30 pm

Backups - RSYNC & copying files issue

Postby sternfan » Thu Dec 20, 2007 10:25 am

hi raj,
Thanks for the reply. Just to be clear - I believe rsync is working OK. I can successfully rsync my zimbra box to my backup server. It's when I then try to copy this backup to another server that for some reason the files won't copy. Weird.
Thanks,

Rob
13445raj
Outstanding Member
Outstanding Member
Posts: 743
Joined: Fri Sep 12, 2014 9:59 pm
Location: Canada
Contact:

Backups - RSYNC & copying files issue

Postby 13445raj » Thu Dec 20, 2007 11:01 am

ok lets check this step by step...iam not rsync expert , but lets think logically.
You have ZIMBRA, BACKUP and ARCHIVE servers 3 in total

1) ZIMBRA --> BACKUP - works

2) BACKUP --> ACRCHIVE - dont work
i dont know how RSYNC handels user permissions and syslinks.

What happens if you don’t have SAME original files to syslink and SAME USER on ACRCHIVE..dose RSYNC loose all that info or it saves that into in some metadata.

What iam tying to get to is if ARCHIVE server is NOT of same OS type and don’t have "zimbra" users and group..how will it preserve file permissions by user and other properties..

May be your ZIMBARA and BACKUP server are same but ARCHIVE server is not...
just logically thinking. May be some RSYNC expert can jump in and explain..
Raj
andreychek
Advanced member
Advanced member
Posts: 155
Joined: Fri Sep 12, 2014 9:58 pm

Backups - RSYNC & copying files issue

Postby andreychek » Thu Dec 20, 2007 11:02 am

[quote user="sternfan"]I did an rysnc -avr; how is that different than a rsync -avHK? I googled for it and can't find any info.[/QUOTE]

I'm under the impression that smbfs doesn't handle links very well, there's an excellent chance that's where you're backup is going awry. NFS would handle them better.
As for the difference between the above commands, here's some info from the rsync manpage:
-a, --archive archive mode; same as -rlptgoD (no -H)

-K, --keep-dirlinks treat symlinked dir on receiver as dir

-H, --hard-links preserve hard links
So the "r" you were using is actually redundant with the "a" option. Meaning the only difference between the two commands is the "K" and "H", which describe how to handle links.
But, if the problem is smbfs's link support, that won't help in this case :-)
What you might consider looking into is if you could try using CIFS rather than SMBFS. There is a kernel module for CIFS, you'd just want to use it over the SMBFS module.
I'm not sure if that entails any additional software installs, but poking around on google for your distro and cifs should probably help.
Good luck!

-Eric
sternfan
Advanced member
Advanced member
Posts: 144
Joined: Fri Sep 12, 2014 10:30 pm

Backups - RSYNC & copying files issue

Postby sternfan » Thu Dec 20, 2007 11:54 am

Hi all - thanks for the help.
Before I got back to the board, I setup a new NFS share and am using it now (if it works, SMB was the culprit). It's a 26g file, so it's going to take a while.
Eric - thanks for the info. So in plain language - what's the difference between rsync -avr & rsync -avHK? Just taking notes...
Thanks again,

Rob
sternfan
Advanced member
Advanced member
Posts: 144
Joined: Fri Sep 12, 2014 10:30 pm

Backups - RSYNC & copying files issue

Postby sternfan » Thu Dec 20, 2007 1:51 pm

Learn something new every day... If I copy using SMB, I get errors. If I copy using NFS, no problems.
Thanks,

Rob

Return to “Administrators”

Who is online

Users browsing this forum: Majestic-12 [Bot] and 26 guests