Page 1 of 2 12 LastLast
Results 1 to 10 of 20

Thread: IMAP goes down 2-3 times a week, tomcat restart necessary

  1. #1
    Join Date
    Mar 2006
    Location
    Hamburg, Germany
    Posts
    44
    Rep Power
    9

    Default IMAP goes down 2-3 times a week, tomcat restart necessary

    (I've submitted this as a support ticket, but this doesn't seem specific to the Network Edition, and maybe someone in the forums is having similar problems :-)

    We've been having problems with ZCS not accepting new IMAP connections 2 or 3 times a week. The existing connections stay up, but no new connections are accepted until we do a "tomcat restart".

    We've started monitoring the service with nagios, and when the problem occurs the imap_check reports a "socket timeout after 10 seconds"

    I haven't been able to find a consistent error message in /opt/zimbra/log/zimbra.log. Today it was:

    Code:
    2006-05-26 09:20:46,776 WARN [ImapServer-1183] [ip=xx.xxx.x.xxx;name=xxxxx@nu2m.de;] imap - error checking account status; dropping connection
    com.zimbra.cs.service.ServiceException: system failure: getDirectContext
    at com.zimbra.cs.service.ServiceException.FAILURE(ServiceException.java:131)
    at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:179)
    at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:163)
    at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountByQuery(LdapProvisioning.java:297)
    at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountById(LdapProvisioning.java:325)
    at com.zimbra.cs.account.ldap.LdapProvisioning.getAccountById(LdapProvisioning.java:336)
    at com.zimbra.cs.mailbox.Mailbox.getAccount(Mailbox.java:359)
    at com.zimbra.cs.mailbox.Mailbox.getAccount(Mailbox.java:346)
    at com.zimbra.cs.imap.ImapHandler.processCommand(ImapHandler.java:521)
    at com.zimbra.cs.tcpserver.ProtocolHandler.processConnection(ProtocolHandler.java:231)
    at com.zimbra.cs.tcpserver.ProtocolHandler.run(ProtocolHandler.java:198)
    at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:595)
    Caused by: javax.naming.CommunicationException: zimbra.nu2m.de:389 [Root exception is java.net.ConnectException: Connection refused]
    at com.sun.jndi.ldap.Connection.<init>(Connection.java:194)
    at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:118)
    at com.sun.jndi.ldap.LdapClientFactory.createPooledConnection(LdapClientFactory.java:44)
    at com.sun.jndi.ldap.pool.Connections.<init>(Connections.java:97)
    at com.sun.jndi.ldap.pool.Pool.getPooledConnection(Pool.java:114)
    at com.sun.jndi.ldap.LdapPoolManager.getLdapClient(LdapPoolManager.java:310)
    at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1572)
    at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2596)
    at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:283)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:175)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:193)
    at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:136)
    at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:66)
    at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
    at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:247)
    at javax.naming.InitialContext.init(InitialContext.java:223)
    at javax.naming.ldap.InitialLdapContext.<init>(InitialLdapContext.java:134)
    at com.zimbra.cs.account.ldap.LdapUtil.getDirContext(LdapUtil.java:174)
    ... 11 more
    Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:507)
    at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at com.sun.jndi.ldap.Connection.createSocket(Connection.java:333)
    at com.sun.jndi.ldap.Connection.<init>(Connection.java:181)
    ... 28 more
    We're running 3.1.1_GA_394 (2006-05-08) under RHEL 4, Dual Xeon with 4GB RAM an 350GB 3ware RAID.
    Last edited by shanson; 05-26-2006 at 10:39 AM.
    Scott Hanson
    Media Ventures GmbH
    Cologne, Germany

  2. #2
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    21

    Default

    hmm,
    Can you post your most recent stacktrace?
    /opt/zimbra/tomcat/logs/(stacktrace with the newest datestamp)

    From here it looks like an LDAP problem:
    Code:
    Caused by: javax.naming.CommunicationException: zimbra.nu2m.de:389 [Root exception is java.net.ConnectException: Connection refused]
    I think your LDAP password may have gotten out of sync, perhaps. When it crashes, try telneting to the machine on port 389, and see what happens.

    john

  3. #3
    Join Date
    Mar 2006
    Location
    Hamburg, Germany
    Posts
    44
    Rep Power
    9

    Default

    The error message I posted is bogus. After cross-checking my /var/logs/zimbra.log, I realized that this error occured while I was running "/etc/init.d/zimbra restart", so of course LDAP was down for about a minute. The IMAP outage had acutally started 10 minutes earlier, and there are no obvious errors in /opt/zimbra/logs/zimbra.log.

    I'm attaching the appropriate tomcat stacktrace.
    Attached Files Attached Files
    Last edited by shanson; 05-27-2006 at 12:07 AM.
    Scott Hanson
    Media Ventures GmbH
    Cologne, Germany

  4. #4
    Join Date
    Aug 2005
    Location
    San Mateo, CA
    Posts
    4,789
    Rep Power
    19

    Default

    Scott,

    Can you try getting a thread dump when IMAP is unavailable.

    kill -QUIT <tomcat pid>
    Looking for new beta users -> Co-Founder of Acompli. Previously worked at Zimbra (and Yahoo! & VMware) since 2005.

  5. #5
    Join Date
    Mar 2006
    Location
    Hamburg, Germany
    Posts
    44
    Rep Power
    9

    Default

    Thanks, I'll do that next time we have problems.

    I've also turned on the imapproxy. Back when I worked with Lotus Notes, I found that using a proxy helped getting various IMAP clients to play nicely with Domino. Maybe it will help here as well. :-)
    Scott Hanson
    Media Ventures GmbH
    Cologne, Germany

  6. #6
    Join Date
    Feb 2006
    Posts
    34
    Rep Power
    9

    Default

    For what its worth- I have the same problem.

    ZCS 3.1.1, open-source, on a RH FC4 system. Imap just stops responding- watching the process via Top, and all the ZCS processes are sleeping and are never woken. I just stop and start every hour.

  7. #7
    Join Date
    Mar 2006
    Location
    Hamburg, Germany
    Posts
    44
    Rep Power
    9

    Default

    Turning on the imapproxy seems to have helped. It's been on now for 5 days, and we haven't had any problems (except that /var/log/zimbra.log is full of perdition entries, how can I turn debug mode off? :-) ).

    It may also be 3.1.1 is better than 3.1.0. We just updated to 3.1.1 last week, a couple of days before the incident I mentioned above.
    Scott Hanson
    Media Ventures GmbH
    Cologne, Germany

  8. #8
    Join Date
    Aug 2005
    Location
    San Mateo, CA
    Posts
    4,789
    Rep Power
    19

    Default

    It's on by default, syslog facility mail:

    from perdition.conf.in:
    C <-- connection logging
    d <--- debug logging
    F mail <--- syslog facility
    Looking for new beta users -> Co-Founder of Acompli. Previously worked at Zimbra (and Yahoo! & VMware) since 2005.

  9. #9
    Join Date
    Mar 2006
    Location
    Hamburg, Germany
    Posts
    44
    Rep Power
    9

    Default Imap just went down again

    The IMAP port was just down again. IMAPS and HTTP were still up. zmcontrol status showed all services running. I took a stacktrace as KevinH requested... the output is in catalina.out, correct? That's what I've attached.

    Additional information: we're running IMAP behind the imapproxy, but not IMAPS. So it could have been a perdition problem.
    Attached Files Attached Files
    Last edited by shanson; 06-06-2006 at 02:43 AM. Reason: Additonal information
    Scott Hanson
    Media Ventures GmbH
    Cologne, Germany

  10. #10
    Join Date
    Aug 2005
    Location
    San Mateo, CA
    Posts
    4,789
    Rep Power
    19

    Default

    I don't see any dead locks in this dump so not sure what caused the outage. Maybe as you said it was perdition, since IMAP/IMAPS should be up/down together.
    Looking for new beta users -> Co-Founder of Acompli. Previously worked at Zimbra (and Yahoo! & VMware) since 2005.

Similar Threads

  1. Replies: 37
    Last Post: 12-28-2010, 06:02 PM
  2. tomcat stop,i can not restart it.
    By nxzwt in forum Administrators
    Replies: 3
    Last Post: 06-12-2007, 01:23 PM
  3. IMAP Enable - Now Tomcat won't startup
    By Oman in forum Installation
    Replies: 4
    Last Post: 05-23-2007, 12:59 PM
  4. Constant crashes, connection refused on Tomcat restart
    By Interkey JeffG in forum Administrators
    Replies: 41
    Last Post: 03-13-2006, 11:47 AM
  5. Error on IMAP - Tomcat crashes!
    By rodrigoccurvo in forum Administrators
    Replies: 5
    Last Post: 11-30-2005, 07:14 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •