Page 1 of 2 12 LastLast
Results 1 to 10 of 11

Thread: [SOLVED] Free space disappearing

  1. #1
    Join Date
    Sep 2008
    Location
    Chicago, USA
    Posts
    25
    Rep Power
    7

    Default [SOLVED] Free space disappearing

    Hello-

    Last friday my users started receiving warnings on their mail clients (and webmail) that their mail could not be sent due to lack of disk space. Alarming, curious, but it happens. Funny thing was i has just gone thru all my boxes to make sure nothing was short on space probably a week before. there was some stuff i could move onto SAN to make room, and I created about 11G. On Monday the 11G was gone. I made a few G more. Today it is gone again.

    Zimbra is the only application running on the physical disc on this machine. everything else is ln -s linked to the SAN. I cannot for the life of me figure out where the space is going. It is not a constant ticking away of space but an ebb and flow for awhile. no change for awhile and then a giant bite. then ebb and flow, stable, giant bite.

    My questions are these:

    1) any idea what in zimbra could be doing this? Large attachments or messages? some sort of logging feedback loop?

    2) is it a definite no no to move /opt to my SAN and ln -s link? At least this way i have enough space to keep mail flowing while i figure out what is happening. I've read the scuttlebutt about updates not working across the links and im not too worried about this. We are going to build a commercial version of ZCS on a new server in the near future and migrate to it. No updates for our version.

    3) Also, my logger files are pretty huge, and i could use that space. for the time being. I read that this code works
    su - zimbra
    stop logger db (mysql processes)
    rm -rf /opt/zimbra/logger/db/data
    /opt/zimbra/libexec/zmloggerinit
    But i want to be clear.
    stop logger db (mysql processes)
    Is not really code. what i want to do is stop mysql, correct?
    Then,
    rm -rf /opt/zimbra/logger/db/data
    Delete the directory (or in my case move it to the SAN so i can look it over)
    Then,
    /opt/zimbra/libexec/zmloggerinit
    run this script which re-creates the dir structure for logging.
    Start mysql, start logger.

    is this the gist?

    I've already leaked 600MB since this morning, and have about 900MB left on the disk, so please help and thanks in advance. Time is of the essence.

    Mike

  2. #2
    Join Date
    Mar 2006
    Location
    Beaucaire, France
    Posts
    2,322
    Rep Power
    13

    Default

    Go to /opt/zimbra and do a "du -h" with the "one directory level only" parameter (the parameter is different depending on the OS).

    This way you'll find where the free space is going...

  3. #3
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    "I created about 11G." What did you move? (Log files themselves in /opt/zimbra/log I assume)

    Are you still running 4.5.3_GA_733 MAC OS X ?
    Is your zmmtaconfig.log really large?
    If so you can edit line 55ish of /opt/zimbra/libexec/zmmtaconfig from 5 to 3.
    4.5.7 lowered the log intensity level: Bug 18445 - zmmtaconfig default loglevel is set to high
    5.0.0 added log rotation: Bug 10398 - clean up logging for swatch, zmmtaconfig

    To control the logger DB size a bit:
    su - zimbra
    zmprov mcf zimbraLogRawLifetime 1d (31d by default)
    zmprov mcf zimbraLogSummaryLifetime 7d (730d by default)

    If you're curious how this process works examine /opt/zimbra/libexec/zmlogprocess (which is fired off every 10 min from crontab against the zimbra_logger DB.)

    To answer your other questions yes that process would empty the DB.

    As for upgrading it might be time to go to 4.5.11 if you're still at 4.5.3

  4. #4
    Join Date
    Sep 2008
    Location
    Chicago, USA
    Posts
    25
    Rep Power
    7

    Default

    Thanks so much for the quick response...

    Im sorry, I cant find line 55ish of /opt/zimbra/libexec/zmmtaconfig to edit.

    can you tell me how the line reads?

    Working on the other "logger intensity" edits.

    where can i find the upgrade to 4.5.11? i was looking yesterday for OSS upgrades.

    thanks again-
    Mike
    Last edited by moneymonk; 10-16-2008 at 09:45 AM. Reason: clarification

  5. #5
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    my %config = (
    loglevel => 5,

    (We created a zmmtaconfig_log_level attribute later.)

    Halfway down this page: /community/downloads_previous.html
    Old release notes: /community/documentation_previous_releases.html
    Don't forget a backup!

    I wasn't sure if you we're using PPC or not (we don't support that in the 5.0.x series so that migration would be a little more involved if you are).
    You should still do 4.5.11 first before making the jump to 5.0.10 anyways.
    Probably gonna need a some room free first to upgrade - or if you want to risk it comment out the space check in the installer script (I know we check on new installs but don't remember if we check during upgrades).

  6. #6
    Join Date
    Sep 2008
    Location
    Chicago, USA
    Posts
    25
    Rep Power
    7

    Default

    Thanks for all yr help, Mike. Klug too. That du -hd 1 incantation is sweet indeed. (i usually use du -cks * |sort -rn |head -11 which works nice too.) Thanks again.

    Enough chatting. TCB.

    Strange. "pattern not found" using vim search for loglevel in /opt/zimbra/libexec/zmmtaconfig.

    I also ran:

    su - zimbra
    zmprov mcf zimbraLogRawLifetime
    zmprov mcf zimbraLogSummaryLifetime
    Will these logs flush themselves, or should I continue with

    su - zimbra
    /opt/zimbra/mysql/bin mysql stop
    rm -rf /opt/zimbra/logger/db/data
    /opt/zimbra/libexec/zmloggerinit
    /opt/zimbra/mysql/bin mysql stop
    zmloggerctl start
    Thank you too for the info. I will take a close look at the update documents once I put out this fire and have 11 or 12 beers. We are currently using OS X 10.4.xx, but will migrate to a screamer running RHEL (fingers crossed) for our 5.0.xx migration.

    I was hoping to put off the migration until summer '09, but the space issue has become pressing. I wish the guy before me used LVM.

    What is the consensus on moving the /opt dir and creating a ln -s until I can piece together this migration?

    Mike

  7. #7
    Join Date
    Sep 2008
    Location
    Chicago, USA
    Posts
    25
    Rep Power
    7

    Default Update

    Seems I lost 20G to /opt/local/var/amavis/tmp

    any idea what this file does, and can i purge it?

    Mike

  8. #8
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    719
    Rep Power
    9

    Default

    You really mean /opt/local not /opt/zimbra? Is that a Macism or a localism?

    That would be temporary workspace for antispam/antivirus. It's supposed to be cleaned up when the message is either delivered or rejected, but might not be if something crashes. I think we saw growth there back in the 4.5 days, too.

    If ought to be safe to remove anything more than a day old without downtime:

    cd /opt/local/var/amavis/tmp
    find . -maxdepth 1 -mtime +1 -type d | xargs rm -rf

  9. #9
    Join Date
    Sep 2008
    Location
    Chicago, USA
    Posts
    25
    Rep Power
    7

    Default

    It's def /opt/local ... just double checked.

    I didn't set up our Zimbra, so I dont know what the impetus was behind some of the decisions. One of my many predecessors did. I'm just really glad I found where my disk space leak was stemming from.

    is amavisd part of the Zimbra package? what is its purpose?

    everyone has been super helpful. i really appreciate all the feedback.

    Mike

  10. #10
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    719
    Rep Power
    9

    Default

    amavisd-new ("a mail anti-virus system") is one of Zimbra's open source components, but I'd expect the path to be /opt/zimbra/amavisd-(version)/tmp (in 5.0.x, this moved to a subdirectory of data/). It's also possible someone that installed their own custom copy.

    ls -ld /opt/local/var/amavis/tmp

    Is it owned by user zimbra?

    Is /opt/local symlinked to /opt/zimbra or vice versa?

    In any case, old things in "tmp" most certainly can be purged...

Similar Threads

  1. Inconsistent Zimbra install behaviour
    By mkolbjor in forum Installation
    Replies: 2
    Last Post: 02-07-2008, 03:52 AM
  2. ZIMBRA!!!!! Error zmcontrol status and others
    By orison in forum Installation
    Replies: 8
    Last Post: 02-03-2007, 07:52 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •