Page 2 of 2 FirstFirst 12
Results 11 to 20 of 20

Thread: Zimbra/Tomcat is continually crashing - with no users!

  1. #11
    Join Date
    Nov 2005
    Posts
    518
    Rep Power
    10

    Default

    Can you post a big stretch of the mailbox.log (like up to and including an OutOfMemory error)? From that stacktrace file, it doesn't look like anything is happening. Did you mention what version of ZCS this is (zmcontrol -v)?

  2. #12
    Join Date
    Dec 2005
    Location
    Scotland
    Posts
    22
    Rep Power
    9

    Default

    Sure thing -- the maillog i have attached here shows a number of crashes just from last night. Because i am not presently onsite, the machine reboots itself everytime it sees Tomcat has died. This happened a few times last night as you can see.

    I have also attached the catalina.log from Tomcat -- you can see the out of memory problems there. Increasing the memory size makes NO DIFFERENCE. The VM has been started up with 1GB of memory, which should be WAY MORE than enough for an installation of this size. There is definitely a process/thread that is demanding all the memory it can get -- ie caught in an infinite loop.

    As you can see, the machine isn't doing anything of any significant load.

    With respect to the version of Zimbra; you will see it in the log file, but here it is:

    version=4.5.5_GA_838.FC4 release=20070503184457 builddate=20070503-1845 buildhost=build-fc4.liquidsys.com

    Here is the complete access log from tomcat:

    81.2.69.5 - - [29/Jun/2007:01:01:03 +0100] "HEAD / HTTP/1.0" 200 3882 "-" "Wget/1.10.2 (Red Hat modified)"
    127.0.0.1 - - [29/Jun/2007:01:30:11 +0100] "POST /service/admin/soap/ HTTP/1.1" 200 599 "-" "libwww-perl/5.805"
    127.0.0.1 - - [29/Jun/2007:01:30:18 +0100] "POST /service/admin/soap/ HTTP/1.1" 200 279 "-" "libwww-perl/5.805"
    81.2.69.5 - - [29/Jun/2007:02:01:02 +0100] "HEAD / HTTP/1.0" 200 3882 "-" "Wget/1.10.2 (Red Hat modified)"
    127.0.0.1 - - [29/Jun/2007:03:00:04 +0100] "POST /service/admin/soap/ HTTP/1.1" 200 599 "-" "libwww-perl/5.805"
    127.0.0.1 - - [29/Jun/2007:03:00:07 +0100] "POST /service/admin/soap/ HTTP/1.1" 200 807 "-" "libwww-perl/5.805"
    81.2.69.5 - - [29/Jun/2007:03:01:01 +0100] "HEAD / HTTP/1.0" 200 3882 "-" "Wget/1.10.2 (Red Hat modified)"
    81.2.69.5 - - [29/Jun/2007:04:01:01 +0100] "HEAD / HTTP/1.0" 200 3882 "-" "Wget/1.10.2 (Red Hat modified)"
    81.2.69.5 - - [29/Jun/2007:05:01:01 +0100] "HEAD / HTTP/1.0" 200 3882 "-" "Wget/1.10.2 (Red Hat modified)"
    81.2.69.5 - - [29/Jun/2007:07:38:58 +0100] "POST /service/soap/ HTTP/1.1" 200 386 "-" "-"
    81.2.69.5 - - [29/Jun/2007:07:39:12 +0100] "POST /service/soap/ HTTP/1.1" 200 386 "-" "-"
    81.2.69.5 - - [29/Jun/2007:07:39:33 +0100] "POST /service/soap/ HTTP/1.1" 200 386 "-" "-"
    81.2.69.5 - - [29/Jun/2007:07:45:06 +0100] "POST /service/soap/ HTTP/1.1" 200 386 "-" "-"

    The only things hitting it was Zimbra and the watchdog process we have running every hour.

    I hope this gives you some insight as to what is happening, because this problem is really starting to dent my confidence and faith in Zimbra.

    Thank you, and looking forward to some words of wisdom
    Attached Files Attached Files

  3. #13
    Join Date
    Jul 2007
    Posts
    3
    Rep Power
    8

    Smile OutOfMemoryError solved!

    Hi all,
    I had the same OutOfMemoryError in tomcat, after a simple reboot of the server (with normal zimbra shutdown). The symptom for the users was the inability to login using the web client.
    I have tried to solve the problem incrementing the tomcat memory, with no success.
    After half an hour of trials, I discovered the source of the problem: tomcat was not able to process the redo.log file (/opt/zimbra/redolog/redo.log).
    After removing the redo.log file, tomcat started normally
    So the steps are:
    • tomcat stop
    • rm /opt/zimbra/redolog/redo.log
    • tomcat start


    Best Regards,
    Raffaele

  4. #14
    Join Date
    Nov 2005
    Posts
    518
    Rep Power
    10

    Default

    What exactly was wrong with the redolog? Normally it would be better to keep it around.

  5. #15
    Join Date
    Nov 2005
    Posts
    518
    Rep Power
    10

    Default

    Alan, I don't see any explicit OutOfMemory errors in the logs from tomcat, though I do see it disappearing. Is the machine itself running out of memory? Does the system log show anything happening at those times (e.g. oom-killer)?

  6. #16
    Join Date
    Jul 2007
    Posts
    3
    Rep Power
    8

    Default problems with redolog

    The problem (at least for me) with redo.log was the following: tomcat was not able to deal with the redo.log, giving an out of memory error. Initially tomcat was configured to use about 250MB of RAM, but even changing the memory for tomcat (up to 2GB) it was not able to elaborate the redo.log, and users were not able to login (error INVALID_PDU).
    The solution was to remove the redo.log and restart tomcat (and putting back its memory needs to 250MB, changing the tomcat_java_heap_memory_percent).

    Regards,
    Raffaele

  7. #17
    Join Date
    Nov 2005
    Posts
    518
    Rep Power
    10

    Default

    >tomcat was not able to deal with the redo.log, giving an out of memory error.
    Can you post the actual error line from the log?

  8. #18
    Join Date
    Mar 2007
    Location
    Oregon
    Posts
    11
    Rep Power
    8

    Default Crashing Tomcat

    Hello. I got to work today to a crashed email system . I had setup a trial copy of the network zimbra for our business to evaluate and things were going well.

    It seems I am having the same tomcat problem as of this morning. Trying zmcontrol starts the services but checking the status a moment later reports that tomcat is stopped. Looking at the mail log shows a java out of memory error.

    Any suggestions or logfiles you need to see?

    I don't want to high-jack the thread, but figured it better to post here than start another for the same problem.

  9. #19
    Join Date
    Mar 2007
    Location
    Oregon
    Posts
    11
    Rep Power
    8

    Default

    After doing some testing I changed the tomcat-java-memory option with the zmlocalconfig to 95%. Zimbra seems to run ok with java taking 14% of system memory until I try and send a test message (just "Mail test" or something like that) java's memory usage jumps up to 50 then 60 then 75% then crashes and I get an error from the zimbra web-client.


    I hope this helps someone.

    -- Nick

  10. #20
    Join Date
    Jul 2007
    Posts
    3
    Rep Power
    8

    Default redo.log

    Here is my log:
    2007-07-05 10:22:13,828 INFO [main] [] RedoLogManager - Starting pre-startup crash recovery
    2007-07-05 10:22:45,458 INFO [main] [] RedoPlayer - Redoing 1 uncommitted transactions
    2007-07-05 10:22:54,683 FATAL [main] [] soap - Unable to start servlet
    java.lang.OutOfMemoryError: Java heap space
    2007-07-05 10:22:54,685 INFO [main] [] [/service] - Marking servlet SoapServlet as unavailable
    2007-07-05 10:22:54,685 ERROR [main] [] [/service] - Servlet /service threw load() exception
    javax.servlet.UnavailableException: Java heap space
    at com.zimbra.soap.SoapServlet.init(SoapServlet.java: 118)
    at javax.servlet.GenericServlet.init(GenericServlet.j ava:211)
    at org.apache.catalina.core.StandardWrapper.loadServl et(StandardWrapper.java:1105)
    at org.apache.catalina.core.StandardWrapper.load(Stan dardWrapper.java:932)
    at org.apache.catalina.core.StandardContext.loadOnSta rtup(StandardContext.java:3915)
    at org.apache.catalina.core.StandardContext.start(Sta ndardContext.java:4176)
    at org.apache.catalina.core.ContainerBase.addChildInt ernal(ContainerBase.java:759)
    at org.apache.catalina.core.ContainerBase.addChild(Co ntainerBase.java:739)
    at org.apache.catalina.core.StandardHost.addChild(Sta ndardHost.java:524)
    at org.apache.catalina.startup.HostConfig.deployDirec tory(HostConfig.java:910)
    at org.apache.catalina.startup.HostConfig.deployDirec tories(HostConfig.java:873)
    at org.apache.catalina.startup.HostConfig.deployApps( HostConfig.java:474)
    at org.apache.catalina.startup.HostConfig.start(HostC onfig.java:1118)
    at org.apache.catalina.startup.HostConfig.lifecycleEv ent(HostConfig.java:310)
    at org.apache.catalina.util.LifecycleSupport.fireLife cycleEvent(LifecycleSupport.java:119)
    at org.apache.catalina.core.ContainerBase.start(Conta inerBase.java:1020)
    at org.apache.catalina.core.StandardHost.start(Standa rdHost.java:718)
    at org.apache.catalina.core.ContainerBase.start(Conta inerBase.java:1012)
    at org.apache.catalina.core.StandardEngine.start(Stan dardEngine.java:442)
    at org.apache.catalina.core.StandardService.start(Sta ndardService.java:450)
    at org.apache.catalina.core.StandardServer.start(Stan dardServer.java:700)
    at org.apache.catalina.startup.Catalina.start(Catalin a.java:551)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.catalina.startup.Bootstrap.start(Bootst rap.java:275)
    at org.apache.catalina.startup.Bootstrap.main(Bootstr ap.java:406)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at com.zimbra.cs.launcher.TomcatLauncher.start(Tomcat Launcher.java:52)
    at com.zimbra.cs.launcher.TomcatLauncher.main(TomcatL auncher.java:74)
    As i said before, removing the /opt/zimbra/redolog/redo.log and restarting tomcat solved all problems.

Similar Threads

  1. need advice on configuring zimbra to work with fax server
    By pheonix1t in forum Administrators
    Replies: 0
    Last Post: 07-11-2007, 07:46 PM
  2. Testing restore on OSE with missing users upon completion
    By dmmincrjr in forum Administrators
    Replies: 3
    Last Post: 07-05-2007, 07:29 PM
  3. Users' address book
    By alimovz in forum Administrators
    Replies: 3
    Last Post: 03-16-2007, 04:11 PM
  4. Replies: 7
    Last Post: 09-29-2006, 10:07 AM
  5. Installation review
    By npollock in forum Administrators
    Replies: 1
    Last Post: 01-14-2006, 09:34 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •