Results 1 to 8 of 8

Thread: Large folder issues

Hybrid View

  1. #1
    Join Date
    Sep 2006
    Location
    Melbourne, Australia
    Posts
    4
    Rep Power
    9

    Default Large folder issues

    I'm having some issues when testing with really large folders.

    I have imported some 250,000 messages into a folder and both Thunderbird and Evolution only show approx. 8,000 messages and can not do much with the folder. However the messages can all be seen in the web client.

    Another problem comes when trying to delete the folder. If I try via the web client I get the dialogue telling me that the operation is taking too long, but nothing happens and if I leave it long enough the session just times out.

    Using zmmailbox also has some interesting results ...

    Code:
    mbox lists@example.com> gf /Inbox/Tech/LKML
    {
     id: "260",
     name: "LKML",
     path: "/Inbox/Tech/LKML",
     parentId: "258",
     flags: "u",
     color: "orange",
     unreadCount: 262254,
     messageCount: 262254,
     view: "conversation",
     restURL: null,
     url: null,
     effectivePermissions: null
    }
    deleteFolder consistently fails without deleting the folder or even partial contents.
    Code:
    mbox lists@example.com> df /Inbox/Tech/LKML
    [] INFO: I/O exception (org.apache.commons.httpclient.NoHttpResponseException) caught when processing request: The server localhost failed to respond
    [] INFO: Retrying request
    ERROR: zclient.IO_ERROR (invoke Connection refused) (cause: java.net.ConnectException Connection refused)
    Machine and software details are as follows:
    Code:
    [root@mail ~]# yum list 'zim*'
    Loading "installonlyn" plugin
    Setting up repositories
    core                                                                 [1/3]
    updates                                                              [2/3]
    extras                                                               [3/3]
    Reading repository metadata in from local files
    Installed Packages
    zimbra-apache.i386                       4.0.2_GA_362.FC5-20060 installed       
    zimbra-core.i386                         4.0.2_GA_362.FC5-20060 installed       
    zimbra-ldap.i386                         4.0.2_GA_362.FC5-20060 installed       
    zimbra-logger.i386                       4.0.2_GA_362.FC5-20060 installed       
    zimbra-mta.i386                          4.0.2_GA_362.FC5-20060 installed       
    zimbra-spell.i386                        4.0.2_GA_362.FC5-20060 installed       
    zimbra-store.i386                        4.0.2_GA_362.FC5-20060 installed       
    [root@mail ~]# uname -r
    2.6.17-1.2187_FC5smp
    [root@mail ~]# cat /etc/fedora-release 
    Fedora Core release 5 (Bordeaux)
    Is there anyway to do the folder delete at a more bare bones level without nuking the rest of the mailbox?

    Thanks,

    Gawain

  2. #2
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    12

    Default temporary solution:

    Increase the memory allocated to your server process. The "df" zmmailbox operation seems to be causing the server to run out of memory and go down.

    Please file this in bugzilla!
    Bugzilla - Wiki - Downloads - Before posting... Search!

  3. #3
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    12

    Default Filed in bugzilla

    Added to bugzilla as bug 11295.
    Bugzilla - Wiki - Downloads - Before posting... Search!

  4. #4
    Join Date
    Sep 2006
    Location
    Melbourne, Australia
    Posts
    4
    Rep Power
    9

    Default

    Quote Originally Posted by dkarp
    Increase the memory allocated to your server process. The "df" zmmailbox operation seems to be causing the server to run out of memory and go down.
    I have tried various values for memory allocation, the machine is running 1.5GB and I have gone as extreme as:
    mysql_memory_percent = 10
    tomcat_java_heap_memory_percent = 80

    It consistently returns an error like this:
    Code:
    msg - system failure: error decoding message 561893 metadata: d1:f153:My system work well :) after the Oops I got, not frozen, but with the kernel option CONFIG_DEBUG_KERNEL=y CONFIG_MAGIC_SYSRQ=y you can try to recover ...1:r94:Re: [2.6.0-test1] Unable to handle kernel NULL pointer dereference	at virtual address 000003241:s45:Ramón Rey Vicente
    com.zimbra.cs.service.ServiceException: system failure: error decoding message 561893 metadata: d1:f153:My system work well :) after the Oops I got, not frozen, but with the kernel option CONFIG_DEBUG_KERNEL=y CONFIG_MAGIC_SYSRQ=y you can try to recover ...1:r94:Re: [2.6.0-test1] Unable to handle kernel NULL pointer dereference	at virtual address 000003241:s45:Ramón Rey Vicente
    	at com.zimbra.cs.service.ServiceException.FAILURE(ServiceException.java:174)
    	at com.zimbra.cs.mailbox.Metadata.<init>(Metadata.java:130)
    	at com.zimbra.cs.mailbox.MailItem.decodeMetadata(MailItem.java:1649)
    	at com.zimbra.cs.mailbox.MailItem.<init>(MailItem.java:323)
    	at com.zimbra.cs.mailbox.Message.<init>(Message.java:125)
    	at com.zimbra.cs.mailbox.MailItem.constructItem(MailItem.java:795)
    	at com.zimbra.cs.mailbox.Mailbox.getItem(Mailbox.java:2004)
    	at com.zimbra.cs.mailbox.Mailbox.getMessage(Mailbox.java:2522)
    	at com.zimbra.cs.mailbox.Folder.alterUnread(Folder.java:675)
    	at com.zimbra.cs.mailbox.Folder.recursiveAlterUnread(Folder.java:647)
    	at com.zimbra.cs.mailbox.Folder.move(Folder.java:750)
    	at com.zimbra.cs.mailbox.Mailbox.move(Mailbox.java:3729)
    	at com.zimbra.cs.operation.ItemActionOperation.callback(ItemActionOperation.java:293)
    	at com.zimbra.cs.operation.Operation.schedule(Operation.java:190)
    	at com.zimbra.cs.operation.ItemActionOperation.MOVE(ItemActionOperation.java:121)
    	at com.zimbra.cs.service.mail.ItemAction.handleCommon(ItemAction.java:148)
    	at com.zimbra.cs.service.mail.FolderAction.handle(FolderAction.java:110)
    	at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:261)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:162)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:84)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:223)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:173)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:595)
    Caused by: com.zimbra.cs.util.BEncoding$BEncodingException: java.lang.StringIndexOutOfBoundsException: String index out of range: 312
    	at com.zimbra.cs.util.BEncoding.decode(BEncoding.java:62)
    	at com.zimbra.cs.mailbox.Metadata.<init>(Metadata.java:122)
    	... 37 more
    Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: 312
    	at java.lang.String.<init>(String.java:206)
    	at com.zimbra.cs.util.BEncoding.decode(BEncoding.java:130)
    	at com.zimbra.cs.util.BEncoding.decode(BEncoding.java:107)
    	at com.zimbra.cs.util.BEncoding.decode(BEncoding.java:55)
    	... 38 more
    
    code - service.FAILURE
    method - ZmCsfeCommand.prototype.invoke
    detail - soap:Receiver

  5. #5
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    12

    Default *That's* a different issue.

    Different issue, there. Assuming that it's always complaining about message 561893, it looks like the METADATA column for a row in the database somehow got truncated. (No idea how that happened; I doubt it's related to your earlier problem.)

    Since you're deleting the folder anyway, you can probably get away with deleting the relevant row from the database using the mysql command interpreter.
    Bugzilla - Wiki - Downloads - Before posting... Search!

  6. #6
    Join Date
    Sep 2006
    Location
    Melbourne, Australia
    Posts
    4
    Rep Power
    9

    Default

    Quote Originally Posted by dkarp
    Different issue, there. Assuming that it's always complaining about message 561893, it looks like the METADATA column for a row in the database somehow got truncated. (No idea how that happened; I doubt it's related to your earlier problem.)

    Since you're deleting the folder anyway, you can probably get away with deleting the relevant row from the database using the mysql command interpreter.
    They are just coming out of the wood work at me

    My SQL is a whole lot lame, you wouldn't be able to give me an example to work with would you

    Kind thanks,

    Gawain

  7. #7
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    12

    Default Pass at the SQL

    Kids, don't try this at home. Or on a production system. Or while zimbra is running.

    Figure out the mailbox id for your mailbox:
    Code:
    SELECT id FROM zimbra.mailbox WHERE comment LIKE '<username>%';
    Delete the relevant item row:
    Code:
    DELETE FROM mailbox<mailbox id>.mail_item WHERE id = 561893;
    And force a recalculation of folder sizes on next login:
    Code:
    UPDATE zimbra.mailbox SET contact_count = NULL WHERE id = <mailbox id>;
    Bugzilla - Wiki - Downloads - Before posting... Search!

Similar Threads

  1. Exchange 2003 Migration Issues
    By JordanPWilliams in forum Migration
    Replies: 10
    Last Post: 07-27-2007, 10:51 AM
  2. zmmailbox and spaces in folder names
    By badrian in forum Administrators
    Replies: 2
    Last Post: 06-28-2007, 11:26 AM
  3. Replies: 42
    Last Post: 08-11-2006, 09:50 AM
  4. One folder always shows new mail
    By mrballcb in forum Users
    Replies: 2
    Last Post: 02-10-2006, 04:10 PM
  5. Thunderbird large IMAP folder problems
    By adobrin in forum Administrators
    Replies: 7
    Last Post: 10-07-2005, 06:15 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •