Page 1 of 3 123 LastLast
Results 1 to 10 of 27

Thread: [SOLVED] Zimbra hangs with 100% CPU load

  1. #1
    Join Date
    Mar 2009
    Posts
    8
    Rep Power
    6

    Default [SOLVED] Zimbra hangs with 100% CPU load

    Hii

    I have zimbra ldap server which has been run for 2 month. Today, I found that my zimbra ldap suddenly stopped..

    file: /var/log/messages
    Jul 30 08:21:53 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException svr01.forestindo.local:389)

    Notes: the other zimbra service work properly, just zimbra ldap stopped.
    It's happened twice..
    Do you have any idea about the problem and solution about this?

    Thanks before and I appreciate your help..

  2. #2
    Join Date
    Jun 2008
    Posts
    594
    Rep Power
    8

    Default

    > Jul 30 08:21:53 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException svr01.forestindo.local:389)


    What error message do you see before this error message ?

  3. #3
    Join Date
    Mar 2009
    Posts
    8
    Rep Power
    6

    Default

    sorry, it's zimbra log. In /var/log/zimbra.log

    Jul 30 08:20:06 xxx01 zimbramon[19016]: 19016:info: 2009-07-30 08:20:01, STATUS: xxx01.serverku.local: ldap: Running
    Jul 30 08:20:06 xxx01 zimbramon[19016]: 19016:info: 2009-07-30 08:20:01, STATUS: xxx01.serverku.local: snmp: Running
    Jul 30 08:20:06 xxx01 zimbramon[19016]: 19016:info: 2009-07-30 08:20:01, STATUS: xxx01.serverku.local: stats: Running
    Jul 30 08:21:53 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping Global system configuration update.
    Jul 30 08:21:53 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:54 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping All Reverse Proxy URLs update.
    Jul 30 08:21:54 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:55 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping All Reverse Proxy Backends update.
    Jul 30 08:21:55 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:56 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping All Memcached Servers update.
    Jul 30 08:21:56 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:56 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping All MTA Authentication Target URLs update.
    Jul 30 08:21:56 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:57 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Skipping Configuration for server xxx01.serverku.local update.
    Jul 30 08:21:57 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: gs:xxx01.serverku.local ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException xxx01.serverku.local:389)
    Jul 30 08:21:57 xxx01 zimbramon[28791]: 28791:info: zmmtaconfig: Sleeping...Key lookup failed.
    Jul 30 08:22:04 xxx01 zimbramon[19799]: 19799:info: 2009-07-30 08:22:01, STATUS: xxx01.serverku.local: ldap: Stopped
    Jul 30 08:22:04 xxx01 zimbramon[19799]: 19799:info: 2009-07-30 08:22:01, STATUS: xxx01.serverku.local: snmp: Running
    Jul 30 08:22:04 xxx01 zimbramon[19799]: 19799:info: 2009-07-30 08:22:01, STATUS: xxx01.serverku.local: stats: Running
    Last edited by yut4k4; 07-29-2009 at 11:38 PM.

  4. #4
    Join Date
    Jul 2008
    Posts
    17
    Rep Power
    7

    Default Zimbra hangs with 100% CPU load

    I am running Zimbra 5.0.18 and on Sunday the server suddenly went into 100% CPU load, refusing even to respond to a terminal. I have to physically switch it off and on to gain control. Zimbra will start fine, but within two to five minutes will start the 100% CPU/hang again.
    zmmtaconfig.log contains this repeating pattern:
    Code:
    Mon Aug 10 18:54:06 2009  Skipping Global system configuration update.
    Mon Aug 10 18:54:06 2009  gacf ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389) 
    Mon Aug 10 18:54:07 2009  Skipping All Reverse Proxy URLs update.
    Mon Aug 10 18:54:07 2009  Skipping getAllReverseProxyURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389) 
    Mon Aug 10 18:54:07 2009  Skipping All Reverse Proxy Backends update.
    Mon Aug 10 18:54:07 2009  Skipping getAllReverseProxyBackends ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389) 
    Mon Aug 10 18:54:08 2009  Skipping All Memcached Servers update.
    Mon Aug 10 18:54:08 2009  Skipping getAllMemcachedServers ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389) 
    Mon Aug 10 18:54:08 2009  Skipping All MTA Authentication Target URLs update.
    Mon Aug 10 18:54:08 2009  Skipping getAllMtaAuthURLs ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389) 
    Mon Aug 10 18:54:09 2009  Skipping Configuration for server mail.cfaw.info update.
    Mon Aug 10 18:54:09 2009  gs:mail.cfaw.info ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: javax.naming.CommunicationException mail.cfaw.info:389)
    I have spent a day Googling for answers and am at a loss.
    Are there any other logs I should be looking at?
    One possibility: I have an account with Campaign Monitor which was broken into this week, possibly resulting in a massive amount of email being sent to me. I have tried blocking all the mail ports on the router to isolate the server but this doesn't solve the problem.

  5. #5
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,017
    Rep Power
    24

    Default

    Well for some reason it appears not to be able to query LDAP. Anything else in /var/log/zimbra.log or /opt/zimbra/log/* ? Has anything else changed on the server eg. software or patches installed ? Does dmesg or /var/log/messages show anything ?

  6. #6
    Join Date
    Jul 2008
    Posts
    17
    Rep Power
    7

    Default

    There does seem to be a problem with LDAP, but that might be a symptom rather than a cause: if the CPU is running at 100% then LDAP can't function. Here is another clip from /var/log/zimbra.log
    Code:
    Aug 10 13:40:50 mail postfix/trivial-rewrite[16785]: fatal: proxy:ldap:/opt/zimbra/conf/ldap-vad.cf(0,lock|fold_fix): table lookup problem
    Aug 10 13:40:50 mail postfix/proxymap[16788]: error: dict_ldap_connect: Unable to set STARTTLS: -1: Can't contact LDAP server
    Aug 10 13:40:50 mail last message repeated 2 times
    But what is really strange is that zimbra functions fine for up to 5 mins, delivering and receiving email before it dies. It goes out so suddenly that even though I am running top to check which processes are using the CPU, top dies without showing the culprit.

  7. #7
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,017
    Rep Power
    24

    Default

    How much memory does the server have ?

  8. #8
    Join Date
    Jul 2008
    Posts
    17
    Rep Power
    7

    Default

    2GBytes. I have tracked free memory up to the point of the crash and only around 1GB is being used.

  9. #9
    Join Date
    Nov 2006
    Location
    UK
    Posts
    8,017
    Rep Power
    24

    Default

    What process is hogging the CPU ?

  10. #10
    Join Date
    Jul 2008
    Posts
    17
    Rep Power
    7

    Default

    The problem is that the system goes to 100% so fast that I can't see what process is hogging it. I run top, watching it until it freezes and the killer process never shows.
    The machine is running as a VM (XEN) under CentOS, so I can watch the CPU utilization graph from the hypervisor. But the hypervisor won't tell me *why* it has gone to 100%. (b.t.w. the CPU is a pretty powerful AMD6400 dual-core)

Similar Threads

  1. Replies: 8
    Last Post: 01-12-2012, 01:20 AM
  2. postfix relay=none status=bounced for local mails
    By vdd in forum Administrators
    Replies: 1
    Last Post: 08-06-2009, 08:05 AM
  3. slapd message error
    By smoke in forum Administrators
    Replies: 7
    Last Post: 04-27-2008, 03:23 PM
  4. Zimbra shutdowns every n hours.
    By Andrewb in forum Administrators
    Replies: 13
    Last Post: 08-14-2007, 08:55 AM
  5. Unable to start tomcat
    By chanck in forum Administrators
    Replies: 11
    Last Post: 06-11-2006, 12:58 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •