Results 1 to 10 of 20

Thread: Zimbra/Tomcat is continually crashing - with no users!

Hybrid View

  1. #1
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default Zimbra/Tomcat is continually crashing - with no users!

    Tomcat is crashing. When it crashes, we have no access to our mailboxes via IMAP etc. So its a fairly important process!

    There is out-of-memory logs in the catalina logs -- but why?

    Data points:

    - Tomcat is started up with 607MB of memory (default setting).
    - Machine has 1.5GB memory (Linux Fedora4)
    - There are NO users on the system when it decides to crash.
    - We have set the max message size to be 10MB
    - Upgraded to the latest/greatest Zimbra release (made no difference)
    - Total mailboxes on the system is a whopping 20!
    - Trouble free system for over a year except for the last week

    What can we do if someone tries to push a bigger email to the system? There is obviously an email in there somewhere that is causing the system to choke. Why did it get in there in the first place though? How do we find out which email is causing the fault?

    Please advise -- as this is making what was a beautifully running system, completely unuseable and a nightmare to manage.

  2. #2
    Join Date
    May 2007
    Location
    San Mateo
    Posts
    68
    Rep Power
    8

    Default

    While you wait for a reply from Zimbra people I had a similar problem with just postfix and IMAP on our current mail server when we implemented it. I needed to up the number of IMAP processes (and possibly the postfix processes, I'd have to check). I think both can be done through the Zimbra admin console.

  3. #3
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    21

    Default

    Yeah. For each connection session, java/tomcat will create a thread. If you have lots of activity: you need more RAM.

    In your case, use the zmlocalconfig option:
    zmlocalconfig -e tomcat_java_heap_memory_percent=(A HIGHER NUMBER)

    Storage capacity has nothing to do with memory requirements. It's all about how much activity you have.

  4. #4
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default

    Thanks for that. However, i don't think that is the problem.

    At the time it crashes, there are NO users logged into the system. No one using the IMAP service, no one using the Webmail

  5. #5
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Quote Originally Posted by AlanWilliamson View Post
    What can we do if someone tries to push a bigger email to the system? There is obviously an email in there somewhere that is causing the system to choke. Why did it get in there in the first place though? How do we find out which email is causing the fault?.
    Why do you think this is the problem? Is there any indication in the logs of a large email trying to be resent? Is this the OSS or NE version?
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  6. #6
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default

    Quote Originally Posted by phoenix View Post
    Why do you think this is the problem? Is there any indication in the logs of a large email trying to be resent? Is this the OSS or NE version?
    This is the OSS version; as for 'why' i think it is this, because it is the only thing that can be introduced into the system. Whether it is large email or one that can crash JavaMail, but it is the only thing that can be introduced.

    I have a lot of experience with JavaMail (written a book on it!) and i know for a fact that it can be easily be tripped up with a poorly formatted MIME message. A rock solid technology it is not! That said, i am not sure how much Zimbra relies on JavaMail, so maybe this is a red herring.

    Our mailbox count has not changed. The server has had no software installed. Infact since most of the company is on holiday's, the actual number of IMAP connections will have dropped dramatically.

    Either way -- Tomcat is the problem, the "mailbox" service is the only one that crashes and all the other processes are running fine. Tomcat complains of "Out of Memory" problems, which is definitely pointing towards a rogue thread attempting to read in a large email that is maybe caught in a loop or has had a problem.

    The real issue is that the error reporting from the java application is very poor -- no indication of what it was really doing at the time of it going belly up.

    Is there any "secret" logging parameters i can throw onto Tomcat that will spark your logging layer to spit out a lot more useful information.

    I really want to get to the bottom of this, because Zimbra has worked for nearly a year with NO PROBLEMS and only in the last week has it started crashing 2 or 3 times a day.

  7. #7
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Are there any errors in /opt/zimbra/log/mailbox.log that indicate that a large message is trying to be resent? It would contain this sort of text 'FileUploadServlet - Exceeded maximum upload size'.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  8. #8
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default

    Quote Originally Posted by phoenix View Post
    Are there any errors in /opt/zimbra/log/mailbox.log that indicate that a large message is trying to be resent? It would contain this sort of text 'FileUploadServlet - Exceeded maximum upload size'.
    no nothing like that, but i have something like this when it does seem to go down. Note the times -- it was restarted at 0945

    Code:
    java.net.SocketException: Broken pipe
            at java.net.SocketOutputStream.socketWrite0(Native Method)
            at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
            at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
            at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
            at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
            at com.zimbra.cs.imap.ImapHandler.sendLine(ImapHandler.java:2099)
            at com.zimbra.cs.imap.ImapHandler.sendResponse(ImapHandler.java:2089)
            at com.zimbra.cs.imap.ImapHandler.sendUntagged(ImapHandler.java:2078)
            at com.zimbra.cs.imap.ImapHandler.dropConnection(ImapHandler.java:2048)
            at com.zimbra.cs.imap.ImapHandler.dropConnection(ImapHandler.java:2034)
            at com.zimbra.cs.tcpserver.ProtocolHandler.run(ProtocolHandler.java:210)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
            at java.lang.Thread.run(Thread.java:595)
    2007-06-28 04:35:45,959 INFO  [ImapServer-83] [] ProtocolHandler - Handler exiting normally
    2007-06-28 04:38:27,352 INFO  [ImapServer-84] [] imap - [81.47.148.165] connected
    2007-06-28 04:38:30,376 INFO  [ImapServer-84] [] ProtocolHandler - Handler exiting normally
    2007-06-28 09:44:25,415 INFO  [main] [] soap - Servlet SoapServlet starting up
    Is there anything else you would like to see or i should watch?

  9. #9
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default

    Well even in the time i am reading this -- the server has gone belly up again, and this time Tomcat was given 900MB. So, whatever memory i give it, it wants to use it up.

    While you chew that over -- i need to go and get someone to reboot the server

Similar Threads

  1. need advice on configuring zimbra to work with fax server
    By pheonix1t in forum Administrators
    Replies: 0
    Last Post: 07-11-2007, 07:46 PM
  2. Testing restore on OSE with missing users upon completion
    By dmmincrjr in forum Administrators
    Replies: 3
    Last Post: 07-05-2007, 07:29 PM
  3. Users' address book
    By alimovz in forum Administrators
    Replies: 3
    Last Post: 03-16-2007, 04:11 PM
  4. Replies: 7
    Last Post: 09-29-2006, 10:07 AM
  5. Installation review
    By npollock in forum Administrators
    Replies: 1
    Last Post: 01-14-2006, 09:34 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •