ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
gevezex
Posts: 11
Joined: Sat Sep 13, 2014 1:19 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby gevezex » Thu Dec 19, 2013 1:23 pm

As this is the first hit with google, this is my solution:
If you backup with rsync then exclude the db first like: rsync --exclude 'data/ldap/mdb/db' /folder/to/zimbra /backup/folder/to/zimbra
Then backup the ldap db with the provided tool 'mdb_copy' like:

mdb_copy /folder/to/zimbra/data/ldap/mdb/db /backup/folder/to/zimbra/data/ldap/mdb/db


kkimmell
Posts: 8
Joined: Sat Sep 13, 2014 3:30 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby kkimmell » Thu Jan 30, 2014 2:54 pm

[quote user="gevezex"]As this is the first hit with google, this is my solution:
If you backup with rsync then exclude the db first like: rsync --exclude 'data/ldap/mdb/db' /folder/to/zimbra /backup/folder/to/zimbra
Then backup the ldap db with the provided tool 'mdb_copy' like:

mdb_copy /folder/to/zimbra/data/ldap/mdb/db /backup/folder/to/zimbra/data/ldap/mdb/db[/QUOTE]
Not intending to necro this thread but I'm trying to write a script to take this into account and having a hard time getting the mdb_copy command to work outside of the zimbra user's account. When I try to pass it through via su I get a command not found error.
# su -c 'mdb_copy /opt/zimbra/data/ldap/mdb/db /backup' zimbra

bash: mdb_copy: command not found

I also tried

# su zimbra 'mdb_copy /opt/zimbra/data/ldap/mdb/db /backup'

bash: mdb_copy /opt/zimbra/data/ldap/mdb/db /backup: Permission denied

And the /backup directory was chown'd to zimbra:zimbra


I tried to find a way to run mdb_copy as root but am only finding it inside of a compressed file

# find / -name mdb_copy

/opt/zimbra/openldap-2.4.35.2z/bin/mdb_copy

To goal is a script to:
1. /etc/init.d/zimbra stop

2. rsync all except the db to /backup

3. /etc/init.d/zimbra start

4. mdb_copy the db to it's necessary place in /backup

5. Continue on with the rest of the backup procedure.


Any pointers appreciated.
-KK
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1666
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby quanah » Thu Jan 30, 2014 3:54 pm

This works for me:


[root@ldap01 quanah]# su - zimbra -c 'mdb_copy /opt/zimbra/data/ldap/mdb/db /backup/ldap01.zimbra.com'

--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
kkimmell
Posts: 8
Joined: Sat Sep 13, 2014 3:30 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby kkimmell » Fri Jan 31, 2014 7:16 am

[quote user="quanah"]This works for me:


[root@ldap01 quanah]# su - zimbra -c 'mdb_copy /opt/zimbra/data/ldap/mdb/db /backup/ldap01.zimbra.com'

[/QUOTE]
Thank you for reminding me that no matter how long I've been a tinker-linux-admin, I am still not doing it right!
:D
That did it.
bulletxt
Posts: 30
Joined: Sat Sep 13, 2014 1:08 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby bulletxt » Sat Feb 01, 2014 4:23 pm

Quanah: I'm not understanding the mdb_copy. On Zimbra Wiki it says it will copy the real database size so the generated copied fle will not be a sparse file. If this is true, what If I just run this utility to overwrite the one Zimbra uses so I'll just get rid of the 80GB sparse file and continue using rsync as always??
Is there something wrong with this idea?
bulletxt
Posts: 30
Joined: Sat Sep 13, 2014 1:08 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby bulletxt » Tue Feb 04, 2014 3:43 pm

Hi people, I'm sorry I don't want to necropost but I'de really appreciate if somone has an answer. I don't get why I have to be forced to handle 80GB sparse files on my server that is just "a mess" to handle, we just added complexity in maintenance. If the official way to copy it is as the ZImbra wiki says, then the copy will give a non-sparse file. If that copy is coherent to the original, can't I just replace the copied non-sparse over the original fat 80gb file? If the answer is no, then please specify what the purpose of doing the mdb copy if then I can't restore it..
Thanks!
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1666
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby quanah » Tue Feb 04, 2014 3:46 pm

You can restore the copy made via mdb_copy... And after you start slapd, it'll be transformed back into a sparse file. And there is nothing forcing you to have an 80GB sparse file. How to tune the size of the file is fully documented in the wiki. OpenLDAP Performance Tuning 8.0 - Zimbra :: Wiki
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
bulletxt
Posts: 30
Joined: Sat Sep 13, 2014 1:08 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby bulletxt » Tue Feb 04, 2014 3:49 pm

[quote user="quanah"] And after you start slapd, it'll be transformed back into a sparse file./url][/QUOTE]
Thanks quanah, that was my missing piece in the puzzle! I'll see how to handle this.
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1666
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby quanah » Tue Feb 04, 2014 3:57 pm

you bet . :)
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
minhhoang
Posts: 28
Joined: Sat Sep 13, 2014 3:14 am

ldap data.mdb after upgrade from 8.0.0 -> 8.0.1 has a size of 85GB

Postby minhhoang » Wed Mar 19, 2014 10:34 pm

Dear,
I got a lot of "CRITICAL! LDAP primary MDB database is 90% full". I have read long topic about the actual size of mdb database and this is my output:

[QUOTE][zimbra@mail ~]$ ls /opt/zimbra/data/ldap/mdb/db/ -la

total 13186400

drwxr-xr-x 2 zimbra zimbra 4096 Mar 20 02:20 .

drwxr-xr-x 3 zimbra zimbra 4096 Jul 15 2013 ..

-rw------- 1 zimbra zimbra 13502853120 Mar 20 09:45 data.mdb

-rw------- 1 zimbra zimbra 8192 Mar 20 09:57 lock.mdb

[zimbra@mail ~]$ du -c -h /opt/zimbra/data/ldap/mdb/db/

13G /opt/zimbra/data/ldap/mdb/db/

13G total

[zimbra@mail ~]$ zmlocalconfig -s ldap_db_maxsize

ldap_db_maxsize = 13502853120[/QUOTE]

My server has about 120 mailboxes, 10 domains. Before this warning, I have carried out partition moving.
How can I fix this? By adjust the ldap_db_maxsize to 80Gb?
Thank a lot.

Return to “Administrators”

Who is online

Users browsing this forum: Google [Bot] and 19 guests