Page 1 of 1

Java at 100% CPU for two days?

Posted: Tue Jun 02, 2009 12:14 pm
by gkra
I've noticed that our Zimbra server has been running at an elevated load average for a couple days now, and have tracked it down to two java threads which are both running at 100% CPU (on our 8-core server).
The two threads in particular are:
 $ ps -efL | grep 3577

UID PID PPID LWP C NLWP STIME TTY TIME CMD

zimbra 26328 26327 3577 98 257 Jun01 ? 1-03:22:46 /opt/zimbra/java/bin/java -Xms4816m -Xmx4816m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml

$ ps -efL | grep 18198

UID PID PPID LWP C NLWP STIME TTY TIME CMD

zimbra 26328 26327 18198 99 250 Jun01 ? 1-03:23:17 /opt/zimbra/java/bin/java -Xms4816m -Xmx4816m -client -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -Xss256k -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml


I can't see anything obvious in the logs about why these two processes are going full-tilt. Is there anything specific I should be looking for?
Is it safe to kill -HUP or -TERM these threads, or should I just let it go until our next maintenance cycle when I can bounce the services?

Java at 100% CPU for two days?

Posted: Mon Jun 08, 2009 3:25 pm
by veronica
Do you have Parallel GC enabled ?
zmlocalconfig -e mailboxd_java_options="-client -XX:NewRatio=2 -XX:MaxPermSize=128m -Djava.awt.headless=true -XX:SoftRefLRUPolicyMSPerMB=1 -XX:+UseParallelGC"
for more performance related refer to Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki

Java at 100% CPU for two days?

Posted: Mon Jun 08, 2009 3:47 pm
by gkra
[quote user="veronica"]Do you have Parallel GC enabled ?
zmlocalconfig -e mailboxd_java_options="-client -XX:NewRatio=2 -XX:MaxPermSize=128m -Djava.awt.headless=true -XX:SoftRefLRUPolicyMSPerMB=1 -XX:+UseParallelGC"
for more performance related refer to Performance Tuning Guidelines for Large Deployments - Zimbra :: Wiki[/QUOTE]
So, then, this is happening for JVM garbage collection?
Also, I'm assuming that this change won't take effect until I restart the zimbra services, too...

Java at 100% CPU for two days?

Posted: Mon Jun 08, 2009 4:00 pm
by uxbod
Yes you will need to restart ZCS.