Results 1 to 7 of 7

Thread: Recovery of tgz mail file fails

  1. #1
    Join Date
    Sep 2008
    Location
    Princeton, NJ
    Posts
    81
    Rep Power
    7

    Default Recovery of tgz mail file fails

    I am trying to recover a very large archived 28GB mail file created with version ZCS 5.0.23. I am now at 7.0.0.

    I have run this command before and had success, but not now.

    $ zmmailbox -z -v -m test@mail.myserver.com postRestURL "//?fmt=tgz&resolve=reset" 2009archivemail.tgz

    ERROR: zclient.IO_ERROR (Broken pipe) (cause: java.net.SocketException Broken pipe)
    com.zimbra.common.zclient.ZClientException: Broken pipe
    ExceptionId:main:1299865829541:0969031b97372ea6
    Code:zclient.IO_ERROR
    at com.zimbra.common.zclient.ZClientException.IO_ERRO R(ZClientException.java:45)
    at com.zimbra.cs.zclient.ZMailbox.postRESTResource(ZM ailbox.java:2499)
    at com.zimbra.cs.zclient.ZMailboxUtil.doPostRestURL(Z MailboxUtil.java:2794)
    at com.zimbra.cs.zclient.ZMailboxUtil.execute(ZMailbo xUtil.java:1137)
    at com.zimbra.cs.zclient.ZMailboxUtil.main(ZMailboxUt il.java:2675)
    Caused by: java.net.SocketException: Broken pipe
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutp utStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStre am.java:136)
    at com.sun.net.ssl.internal.ssl.OutputRecord.writeBuf fer(OutputRecord.java:297)
    at com.sun.net.ssl.internal.ssl.OutputRecord.write(Ou tputRecord.java:286)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRe cordInternal(SSLSocketImpl.java:743)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRe cord(SSLSocketImpl.java:731)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write (AppOutputStream.java:59)
    at java.io.BufferedOutputStream.write(BufferedOutputS tream.java:105)
    at org.apache.commons.httpclient.methods.InputStreamR equestEntity.writeRequest(InputStreamRequestEntity .java:171)
    at org.apache.commons.httpclient.methods.EntityEnclos ingMethod.writeRequestBody(EntityEnclosingMethod.j ava:499)
    at org.apache.commons.httpclient.HttpMethodBase.write Request(HttpMethodBase.java:2114)
    at org.apache.commons.httpclient.HttpMethodBase.execu te(HttpMethodBase.java:1096)
    at org.apache.commons.httpclient.HttpMethodDirector.e xecuteWithRetry(HttpMethodDirector.java:398)
    at org.apache.commons.httpclient.HttpMethodDirector.e xecuteMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMe thod(HttpClient.java:397)
    at com.zimbra.common.httpclient.HttpClientUtil.execut eMethod(HttpClientUtil.java:42)
    at com.zimbra.common.httpclient.HttpClientUtil.execut eMethod(HttpClientUtil.java:38)
    at com.zimbra.cs.zclient.ZMailbox.postRESTResource(ZM ailbox.java:2491)
    ... 3 more

    Any suggested ideas on how to recover this mail file or send it to zimbra a different way? I am going to try some stuff in AJCody's wiki page on migration (Ajcody-Migration-Notes - Zimbra :: Wiki) but thought I would ask in case anyone has resolved this in the past.

    Thanks!
    Tom

  2. #2
    Join Date
    May 2010
    Posts
    46
    Rep Power
    5

    Default

    Hello,

    Try disable Attachent indexing in Administration console on Class of Service in Advanced tab. The same problem I have. And upgrade to 7.0.1

    Tadas

  3. #3
    Join Date
    Sep 2008
    Location
    Princeton, NJ
    Posts
    81
    Rep Power
    7

    Default Still An Issue

    Thanks for the advice.

    Upgraded to 7.0.1 and unchecked Attachment Indexing.

    While the import ran longer, the same "Broken Pipe" error comes out.

    For immediacy I came up with a work-around. I untarred the tgz file and ran a script that sequentially ran through each resulting folder and .eml file and sent them via zmlmtpinject into the desired account. The basic command I used was:

    e.g. If the command recovers archive msgs in Inbox!540.....

    /opt/zimbra/bin/zmlmtpinject /path/to/archive/Inbox\!540/*.eml -r recovery-account\@myserver.com

    This worked well, but took hours to run.

    If anyone knows how this can be done with the original command:
    $ zmmailbox -z -v -m test@mail.myserver.com postRestURL "//?fmt=tgz&resolve=reset" 2009archivemail.tgz

    .... please let me know.

    Thanks!
    Tom
    Last edited by the_griz; 03-14-2011 at 10:56 AM. Reason: Fixed Command

  4. #4
    Join Date
    May 2010
    Posts
    46
    Rep Power
    5

    Default

    Did you restart server?

    su - zimbra -c 'zmcontrol restart'

  5. #5
    Join Date
    Sep 2008
    Location
    Princeton, NJ
    Posts
    81
    Rep Power
    7

    Default

    Server was restarted with the upgrade. As well as twice thereafter.

    Problem still occurs when passing a large .tgz file.

    Thanks!
    Tom

  6. #6
    Join Date
    Sep 2007
    Posts
    11
    Rep Power
    8

    Default

    I ran into the same problems trying to import my 5.1GB tgz file.

    It seems that if you change the resolve= parameter of the command to 'skip' or you just simply remove the &resolve=reset portion of the command that the import will continue from where the error occurred.

    The resolve= parameter has several options:

    * “skip” ignores duplicates of old items, it’s also the default conflict-resolution.
    * “modify” changes old items.
    * “reset” will delete the old sub folder (or entire mailbox if /).
    * “replace” will delete and re-enter them.

    » Zimbra :: Blog

  7. #7
    Join Date
    Sep 2008
    Location
    Princeton, NJ
    Posts
    81
    Rep Power
    7

    Default Effective Solution for Recovering Large TGZ files

    Here is the solution I came up with recovering large archived TGZ files:

    1. Create an account to recover the TGZ file to in ZCS.

    2. Extract the TGZ file to a directory
    - tar xvfz myarchive.tgz

    3. Chown the entire directory to zimbra. e.g. chown -R zimbra.zimbra <directory>

    Note: The structure of the extracted data is <Folder Name>!<Incremented Number>. In my case, I had all mail in the Inbox of this archived account. The TGZ was 44GB. The tar had Inbox!1 through Inbox!1052. You will have to recover each folder and its incremented subfolder into its specific folder in Zimbra.

    Note 2: The contents of each subfolder was each message in eml format and a metadata file. The file names were constructed <sequential number>-<subject text>.eml and <sequential number>-<subject text>.meta. This causes an issue.

    3. Some subject lines have non-ascii characters, spaces, apostrophes, etc. that confuse zmmailbox when it imports them. We need to get rid of them. The best tool I found that worked this issue well was ren-regexp.pl which I downloaded from https://github.com/msabramo/ren-rege.../ren-regexp.pl. It worked great.

    Note: In reality, the filenames of the eml files are can be anything. I just found it easier to keep the sequential leading numbers and clean up the trailing subject-based filenames.

    4. Here's the script I ran over the Inbox files. It's not elegant but it works well to import the email message by message. Run this as zimbra user.

    Specific to my case: I ran this over my Inbox!1 -> Inbox!1052. Substitute folder names as appropriate.

    ==================================================

    #!/bin/bash

    for i in {1..1052};

    do
    echo "******************************";
    echo "* Inbox!$i *";
    echo "******************************";

    cd <path to folders>/Inbox\!$i

    #GET RID OF
    ren-regexp.pl -q "s/\'//g" "s/ /_/g" "s/\!//g" "s/[^[:ascii:]]/_/g"

    #REMOVE UNNECESSARY METADATA FILES
    rm -f *.meta

    #IMPORT THEM INTO YOUR NEW ACCOUNT AND INTO FOLDER /Inbox
    zmmailbox -z -m <account>@<yourmailserver.com> addMessage /Inbox *.eml;

    done
    ==================================================

    5. I hope this helps. It took about 22 hrs to re-import a 40GB TGZ (~580K messages).

Similar Threads

  1. smtp auth failing 'auth_zimbra: not initialized'
    By kkg039 in forum Administrators
    Replies: 7
    Last Post: 09-05-2013, 11:10 AM
  2. Replies: 7
    Last Post: 02-03-2011, 06:01 AM
  3. server dropped connection
    By ferra in forum Installation
    Replies: 20
    Last Post: 10-06-2008, 04:32 PM
  4. DynDNS and Zimbra
    By afterwego in forum Installation
    Replies: 30
    Last Post: 04-01-2007, 03:34 PM
  5. M3 problem with shares
    By titangears in forum Users
    Replies: 4
    Last Post: 01-12-2006, 12:01 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •