Results 1 to 8 of 8

Thread: [SOLVED] Managing /opt/zimbra/redolog/archive Directory

  1. #1
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default [SOLVED] Managing /opt/zimbra/redolog/archive Directory

    We have some 20GB of files in /opt/zimbra/redolog/archive going back several months.

    We switched several months ago to abandoning incremental backups and using only daily full backups.

    What is the process by which Zimbra manages pruning of the files in this folder?

    And, do we even need the files in this directory anymore?

    Thanks!
    Mark

  2. #2
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Hey Mark,
    Quote Originally Posted by LMStone View Post
    We switched several months ago to abandoning incremental backups and using only daily full backups.
    That would contain the answer; every 100mb of the active redolog gets archived, & those archives are usually cleaned up by the incremental which stores/moves them into the /opt/zimbra/backup/ (well wherever your backup target may be).

    However, you should be safe to empty the /opt/zimbra/redolog/archive/ after a full backup has just run. Emphasis on 'just' and be sure to continue reading this before rushing off to do it - because once you get more than 100mb in your current redo.log the archive folder becomes important again for any crashes and restores.

    Empty the contents/don't delete the folder so that you keep permissions on it intact. If that current fills up/rolls into archive too fast; note the time the full started, then delete up to that point rather than emptying the whole folder. Or play it safe by doing a full backup, zmcontrol stop, then empty.

    The Open Source edition behaves the same way as does Network Edition in terms of logging the operations. The difference is in what happens with the redologs once they get to a certain size and roll over. In FOSS, the log is simply truncated. In NE, it is moved to a subdirectory and later backed up during an incremental backup.

    If you're wondering "why doesn't the full delete the redolog/archive?" Well, the fulls don't need to look at the redolog to complete their purpose Plus, as fulls generally take time to complete, you can't just automatically script a rm -rf /opt/zimbra/redolog/archive/*
    after that runs because lots could have happened since the full started (as just discussed above). For cases like yours where people don't run incrementals, maybe the way to solve it would be to have the zmbackup store all the names of the present archive redologs at the start of the full in a temp file, then delete those archives after it completes? Or zmbraRedoLogDeleteOnRollover TRUE would mean you'd never have any in archive, and it sounds like you want them around for a little while.

    Hope the weather in Maine is 'nice'! (I have another house up by the now closing Brunswick NAS which I'm itching to visit...hm, maybe Christmas. Though we gotta get that thing sold before the base is completely empty & there's a surplus of housing in the area!)
    Last edited by mmorse; 10-28-2008 at 09:28 AM.

  3. #3
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    Quote Originally Posted by mmorse View Post
    Hey Mark,

    That would contain the answer; every 100mb of the active redolog gets archived, & those archives are usually cleaned up by the incremental which stores/moves them into the /opt/zimbra/backup/ (well wherever your backup target may be).

    However, you should be safe to empty the /opt/zimbra/redolog/archive/ after a full backup has just run. Emphasis on 'just' and be sure to continue reading this before rushing off to do it - because once you get more than 100mb in your current redo.log the archive folder becomes important again for any crashes and restores.

    Empty the contents/don't delete the folder so that you keep permissions on it intact. If that current fills up/rolls into archive too fast; note the time the full started, then delete up to that point rather than emptying the whole folder. Or play it safe by doing a full backup, zmcontrol stop, then empty.

    If you're wondering "why doesn't the full delete the redolog/archive?" Well, the fulls don't need to look at the redolog to complete their purpose Plus, as fulls generally take time to complete, you can't just automatically script a rm -rf /opt/zimbra/redolog/archive/*
    after that runs because lots could have happened since the full started (as just discussed above). For cases like yours where people don't run incrementals, maybe the way to solve it would be to have the zmbackup store all the names of the present archive redologs at the start of the full in a temp file, then delete those archives after it completes?

    Hope the weather in Maine is 'nice'! (I have another house up by the now closing Brunswick NAS which I'm itching to visit...hm, maybe Christmas. Though we gotta get that thing sold before the base is completely empty & there's a surplus of housing in the area!)
    Thanks Mike!

    So, just to make sure I understand this...

    Our backup strategy is to keep eight days of full backups.

    To be safe, I shouldn't delete anything in /opt/zimbra/redolog/archive/ that is newer than the completion time of the last full backup.

    To be super-safe, I can delete everything in /opt/zimbra/redolog/archive/ nine days old or older.

    Is that right?

    Re Brunswick, I think you missed your near-term selling window, unfortunately. Lots of the jobs there have already gone and there is a lot of property on the market right now (and in Maine in general). Personally, I'd wait a few years.

    There are a lot of redevelopment plans in the works for the NAS, the NAS real estate there is so prime, and waterfront properties are continuing to increase in value past mid-coast now. I can't imagine the State won't seize the opportunity to organize some big comprehensive mixed use development project to stimulate the Brunswick economy, especially with the recent release of a report that showed that Portland alone accounts for some 42% of the State's economic output. If you can hang in there, you should do OK in a few years. Besides, our data center is in Brunswick so you could do on-site support if'n ya wanted to! ;-)

    All the best, and thanks again,
    Mark

  4. #4
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Quote Originally Posted by LMStone View Post
    Our backup strategy is to keep eight days of full backups.

    To be safe, I shouldn't delete anything in /opt/zimbra/redolog/archive/ that is newer than the completion time of the last full backup.
    Start time of the last full, not completion time:
    Quote Originally Posted by mmorse View Post

    - because once you get more than 100mb in your current redo.log the archive folder becomes important again for any crashes and restores.

    note the time the full started, then delete up to that point rather than emptying the whole folder.

    Plus, as fulls generally take time to complete, you can't just automatically script a rm -rf /opt/zimbra/redolog/archive/*
    after that runs because lots could have happened since the full started (as just discussed above).
    just play it super-duper-safe
    Quote Originally Posted by LMStone View Post
    To be super-safe, I can delete everything in /opt/zimbra/redolog/archive/ nine days old or older.
    The:
    Quote Originally Posted by mmorse View Post
    For cases like yours where people don't run incrementals, maybe the way to solve it would be to have the zmbackup store all the names of the present archive redologs at the start of the full in a temp file, then delete those archives after it completes?
    -was really thought discussion for a RFE or something.

    Maine-
    Ya, we've rented it out in the past (empty now) but we'll probably just continue to set it "for rent or sale by owner" rather than pay the agent fees for full sale services. As you can put it in real estate co. listings much cheaper that way.
    lol - sry, I'll probably stay midwest for a while. Though it's a pretty decent house & nice bit of property; I want to say it was built '90ish - Mom was stationed at the hospital and Dad was doing P3 runs.

  5. #5
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    Yes, start time not completion time makes sense now, thanks!

    Just freed up 18.5GB (instead of 20+GB) to play it super-duper safe. :-)

    I'll fill out an RFE on bugzilla re cleaning out the redo log directory.

    Your Mom worked at Mid-Coast? We do a lot of EMR support for practices in Maine and nationwide. Also, FWIW the Maine Medical Association is a client on our hosted Zimbra system and a number of local schools use Zimbra (like Gould Academy).

    All the best,
    Mark

  6. #6
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    I just filed an RFE for automating the pruning of the /opt/zimbra/redolog/archive directory as Bugzilla #21975.

    As they (used to?) say in Chicago, "vote early, vote often".

    Hope that helps,
    Mark

  7. #7
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Bug 21975 - Extend Full Backup Scripts To Prune /opt/zimbra/redolog/archive Directory
    Actually you weren't the first (though it's closed invalid): Bug 1892 - redolog/archive is not purged aggressively enough

    Edumacation:

    Full Backup Process-
    1. Backs up the LDAP directory.
    2. Backs up the global system data including system tables and the local config.xml file.
    3. Iterates through each account to be backed up and backs up the LDAP entries for those accounts.
    4. Places the account’s mailbox in maintenance mode to temporarily block mail delivery and user access to that mailbox.
    5. Backs up the mailbox.
    a. Creates MySQL dump for all entries related to that mailbox.
    b. Creates a backup of the index directory for that mailbox.
    c. Backs up the message directory for that mailbox.
    4. Returns that account’s mailbox to active mode and moves on to the next one.
    (So you now have the message store, the database, the indexes, and the LDAP directory all backed up.)

    Incremental Backup Process-
    1. Backs up the LDAP directory.
    2. Backs up the global system data including system tables and the local config xml.
    3. Iterates through each account to be backed up and backs up the LDAP entries for those accounts.
    4. Looks in the backup target directory to find the latest full backup for the account.
    5. Moves the archive redo logs, created since the last backup, to the <backup target>/redologs directory.

    The 'idea':
    Quote Originally Posted by mmorse View Post
    For cases like yours where people don't run incrementals, maybe the way to solve it would be to have the zmbackup store all the names of the present archive redologs at the start of the full in a temp file, then delete those archives after it completes?
    So like a:
    Code:
     zmbackup -f -s server.domain.com -a all --emptyExistingArchiveRedologs

  8. #8
    Join Date
    Sep 2006
    Location
    477 Congress Street | Portland, ME 04101
    Posts
    1,374
    Rep Power
    11

    Default

    Quote Originally Posted by mmorse View Post
    Bug 21975 - Extend Full Backup Scripts To Prune /opt/zimbra/redolog/archive Directory
    Actually you weren't the first (though it's closed invalid): Bug 1892 - redolog/archive is not purged aggressively enough


    The 'idea':

    So like a:
    Code:
     zmbackup -f -s server.domain.com -a all --emptyExistingArchiveRedologs
    Works for me!

    Mark

Similar Threads

  1. centos 5 zimbra 4.5.6 no statistics
    By rutman286 in forum Installation
    Replies: 9
    Last Post: 08-14-2007, 09:30 AM
  2. Replies: 9
    Last Post: 04-14-2007, 08:31 AM
  3. Replies: 5
    Last Post: 03-01-2007, 02:20 AM
  4. Help: Fullbackup error!
    By victorLeong in forum Administrators
    Replies: 1
    Last Post: 08-02-2006, 01:56 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •