Results 1 to 6 of 6

Thread: occasional imapsync invalid date format error

Hybrid View

  1. #1
    Join Date
    Mar 2008
    Posts
    31
    Rep Power
    7

    Default occasional imapsync invalid date format error

    Hello,

    I am testing migration from a Solaris 9/dovecot system to Zimbra 5.0 on RHEL5. imapsync seems to work very well except for some occasional errors like:

    Code:
    Use of uninitialized value in concatenation (.) or string at /usr/bin/imapsync line 1340.
    10791 BAD parse error: invalid date format
     at /usr/bin/imapsync line 1358
    Couldn't append msg #28587 (Subject:[something]) to folder INBOX: 10791 BAD parse error: invalid date format
    By occasional, I mean a few hundred such erroneous e-mails for the entire migration of 300000+ e-mails.

    I tried finding the bad date headers, but can't see anyhting wrong with them. Here are two examples:

    Code:
    Date: Sun, 31 Dec 2006 17:18:22 -0800
    
    Date: Mon, 31 Dec 2007 16:04:21 -0800
    This led me to check the dates associated with more of the errors, and they are all December 31st!

    Any ideas what is going on here?

    Thanks.

  2. #2
    Join Date
    Mar 2008
    Posts
    31
    Rep Power
    7

    Default

    Running with the --debug flag produces these two interesting lines:

    Code:
    internal date from 1: [31-Dec-2007 16:04:27 +4000]
    internal date from 1: [""] (fixed)
    compared to a good e-mail that produces:

    Code:
    internal date from 1: [18-Mar-2008 18:34:21 -0700]
    internal date from 1: ["18-Mar-2008 18:34:21 -0700"] (fixed)
    +4000 doesn't make sense as a time offset, and I still don't know where that comes from.

    In my understanding, the internal dates as derived from an mbox file come from the very first line of an e-mail that begins with "From ". In this example, those lines are as follows:

    broken:
    Code:
    From user@domain.com  Mon Dec 31 16:04:27 2007
    not broken:
    Code:
    From user@domain.com  Tue Mar 18 18:34:21 2008
    On the same system where I'm running imapsync, I have tried this tiny perl program:
    Code:
    #!/usr/bin/perl
    
    require Date::Manip;
    Date::Manip->import(qw(ParseDate Date_Cmp UnixDate));
    
    $a = 'Mon Dec 31 16:04:27 2007';
    
    $d = ParseDate($a);
    
    $dd = UnixDate($d, "%d-%b-%Y %H:%M:%S %z");
    
    print $dd;
    print "\n\n";
    which returns:
    Code:
    31-Dec-2007 16:04:27 -0700
    Which is the correct answer (note -0700 vs +4000). So the date manipulations involved in imapsync seem to be working correctly.

    This is the last roadblock I have come across in the process of migrating to Zimbra. Any help would be greatly appreciated.

    Thanks.

  3. #3
    Join Date
    Jan 2009
    Posts
    17
    Rep Power
    6

    Default

    Hi Oayfer,

    Oddly enough, I just ran into the same thing. Any luck figuring out what was causing it?

    The user in question here is migrating from Dovecot as well, so maybe it's somehow related to Dovecot? Or maybe a bug in imapsync?

    Thanks,
    Brian

  4. #4
    Join Date
    Mar 2008
    Posts
    31
    Rep Power
    7

    Default

    Over 3 years later I'm having trouble remembering the details but I think we ended up simply isolating the handful of messages among hundreds of thousands and dealt with them individually. I will look through my old notes when I get a chance and will post again if I find any further clues.

  5. #5
    Join Date
    Jan 2009
    Posts
    17
    Rep Power
    6

    Default

    Thanks for the quick reply. I also noticed that the troublesome messages are within hour range of the new year, accounting for timezone offset. We're in the Eastern timezone (UTC-5 in December) and all of the troublesome messages are within 5 hours of midnight on Dec 31st. So, I'm assuming it's some sort of new year rollover issue.

    Although our Zimbra server is running Ubuntu 8.04LTS (and that's where I was running imapsync from), as a test I just did a fresh install of Ubuntu 10.04.1LTS on a separate machine and tried to do the imapsync again from there. Same issue. So, if the issue is a bug in imapsync or a bug in PERL's Date::Manip (called by imapsync), it still seems to be broken.

    If you run into any notes, please let me know.

    Thanks again,
    Brian

  6. #6
    Join Date
    Mar 2008
    Posts
    31
    Rep Power
    7

    Default

    Found some notes. It seems we thought this was a Solaris problem at the time, but you have proven this to be wrong. Perhaps the bug is in the Perl libraries. In any case, since this affected so few messages, we fixed this using the following addition (in Perl) to our migration scripts:

    $d = $f_idate;
    $debug and print "internal date from 1: [$d]\n";

    # this is a horrible TERRIBLE workaround for dovecot/solaris9/mbox
    # returning +4000 as the tz offset for dates near a year boundary
    # like dec31 and jan1. don't do this. this really is terrible.
    # it may make some internaldates wrong by 1hr or something.
    #
    if ($d =~ /\+4000$/) {
    $d =~ s/\+4000$/\-0700/;
    $debug and print "internal date from 1: [$d] FIXED solaris9 bug\n";
    }

    require Date::Manip;
    Date::Manip->import(qw(ParseDate Date_Cmp UnixDate));


    Hope this helps. Feel free to reach out with additional questions, I try to help the community as much as I can.

Similar Threads

  1. Zimbra fails after working for 2 weeks
    By Linsys in forum Administrators
    Replies: 10
    Last Post: 10-07-2008, 12:42 AM
  2. [SOLVED] upgrade zimbra 5.0.2 32 bit to 64 bit
    By jwilson in forum Installation
    Replies: 1
    Last Post: 03-11-2008, 12:15 PM
  3. [SOLVED] Debian Etch 32 / 64: MTA not working
    By xflip in forum Installation
    Replies: 2
    Last Post: 01-18-2008, 03:58 AM
  4. 3.0 to 4.5.3 Upgrade failed (mysql error)
    By dealt in forum Installation
    Replies: 35
    Last Post: 03-19-2007, 10:30 PM
  5. M3 problem with shares
    By titangears in forum Users
    Replies: 4
    Last Post: 01-12-2006, 12:01 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •