Results 1 to 10 of 12

Thread: LDAP MDB back end file huge

Hybrid View

  1. #1
    Join Date
    Dec 2009
    Location
    Netherlands
    Posts
    47
    Rep Power
    5

    Default LDAP MDB back end file huge

    Hi,

    I just upgraded to zcs-8.0.5_GA_5839.UBUNTU10_64.20130910123904 (from 8.0). Ever since the upgrade, the file /opt/zimbra/data/ldap/mdb/db/data.mdb is a 50 GB sparse file, taking 800 kB of space. My backup tool, rdiff-backup, doesn't understand sparse files, and thus backs it up as a 50 GB normal file, making my increments huge and unmanagable.

    What can I do about this?

    I do make slapcat backups daily, so could I just exclude /opt/zimbra/data/ldap/mdb*? My slapcat backup script is:

    Code:
    #!/bin/bash
    
    if [ "`/usr/bin/whoami`" != "zimbra" ]; then
      echo "Not user zimbra, exiting"
      exit 1
    fi
    
    backup_dir="/var/backups/ldap"
    
    /opt/zimbra/libexec/zmslapcat $backup_dir
    /usr/bin/find $backup_dir -type f -mtime +10 -delete
    But I never had to re-load this data. What would happen my server crashes, I restore the backup and use "/opt/zimbra/libexec/zmslapadd < backupfile" to re-load the ldap data? Will that suffice?

    edit: I expanded the script above to also use the mdb_copy command. So, the following script makes slapcat and mdb_copy backups:

    Code:
    #!/bin/bash
    #
    # Author: Wiebe Cazemier <wiebe@halfgaar.net>
    #
    # Since Zimbra 8.0 or 8.0.1 or thereabouts, the LDAP data backend is a 50-80 GB
    # sparse file. This needs to be excluded from the main backup (rdiff-backup in
    # my case), and the LDAP data needs to be backed up another way, which is what
    # this script does.
    #
    # This script backs up LDAP with an ldiff dump (slapcat) and mdb_copy. The
    # ldiff dump should be enough, but better to be safe than sorry.
    #
    # mdb_copy is included since Zimbra 8.0.2.
    
    backup_dir="/var/backups/ldap"
    mailto="admin@example.com"
    
    #############
    
    slapcat="/opt/zimbra/libexec/zmslapcat"
    mdb_copy="/opt/zimbra/openldap/bin/mdb_copy"
    mdb_src="/opt/zimbra/data/ldap/mdb/db"
    
    echo_err()
    {
      echo "$1" 1>&2
      echo "$1" | mail -s "Error in LDAP backup" "$mailto"
    }
    
    if [ "`/usr/bin/whoami`" != "zimbra" ]; then
      echo_err "Not user zimbra, exiting"
      exit 1
    fi
    
    if [ ! -d "$backup_dir" ]; then
      echo_err "$backup_dir doesn't exist"
      exit 1
    fi
    
    if [ ! -w "$backup_dir" ]; then
      echo_err "$backup_dir is not writable."
      exit 1
    fi
    
    if [ ! -x "$slapcat" ]; then
      echo_err "$slapcat not found."
      exit 1
    fi
    
    if [ ! -x "$mdb_copy" ]; then
      echo_err "$mdb_copy not found"
      exit 1
    fi 
    
    $slapcat "$backup_dir"
    
    if [ $? -ne 0 ]; then
      echo_err "slapcat failed"
      exit 1
    fi
    
    mdb_backup_file="$backup_dir/data.mdb"
    datestamp="$(date +%Y.%m.%d-%H.%M.%S)"
    mdb_backup_file_target="${mdb_backup_file}_$datestamp"
    
    $mdb_copy "$mdb_src" "$backup_dir"
    
    if [ $? -ne 0 ] || [ ! -f "$mdb_backup_file" ]; then
      echo_err "mdb_copy failed"
      exit 1
    fi
    
    mv "$mdb_backup_file" "$mdb_backup_file_target"
    
    /usr/bin/find "$backup_dir" -type f -mtime +10 -delete
    Last edited by halfgaar; 12-01-2013 at 03:14 AM. Reason: I pasted in the script that is now my backup of LDAP

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Have you looked at the other thread(s) on this topic and the description on how to backup this file correctly?
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    Join Date
    Dec 2009
    Location
    Netherlands
    Posts
    47
    Rep Power
    5

    Default

    Well, I searched for "MDB", but that yielded 0 results... I guess I will search differently

  4. #4
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

  5. #5
    Join Date
    Dec 2009
    Location
    Netherlands
    Posts
    47
    Rep Power
    5

    Default

    Apparently, the slapcat method works, but I can't really find much info about it in the wiki, as there apparently should be.

    I'm just excluding /opt/zimbra/data/ldap/mdb* and hope for the best, I guess.

  6. #6
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Quote Originally Posted by halfgaar View Post
    Apparently, the slapcat method works, but I can't really find much info about it in the wiki, as there apparently should be.
    The thread link that you've posted gives you full details of how to backup (or reduce the size of) the mdb file, did you read all of the thread and the wiki article?

    Quote Originally Posted by halfgaar View Post
    I'm just excluding /opt/zimbra/data/ldap/mdb* and hope for the best, I guess.
    That's not a good idea and there's no such thing as 'hope for the best' when you've a file to backup - you need to take a valid backup of that file and not leave it to chance.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

Similar Threads

  1. Replies: 3
    Last Post: 10-19-2011, 01:21 PM
  2. Log File Sizes = HUGE
    By samgreco in forum Administrators
    Replies: 12
    Last Post: 06-01-2010, 03:29 PM
  3. ZCB Log File Directory - HUGE
    By derek.rose in forum Zimbra Connector for BlackBerry
    Replies: 0
    Last Post: 04-21-2010, 04:56 AM
  4. Importing tgz file back into Zimbra
    By mrballz in forum General Questions
    Replies: 8
    Last Post: 02-11-2009, 09:37 AM
  5. zmmtaconfig.log file huge
    By msf004 in forum Administrators
    Replies: 1
    Last Post: 06-25-2008, 01:09 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •