Java consume 100% CPU after updating to latest 7.2.6

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
1415bhwong
Outstanding Member
Outstanding Member
Posts: 498
Joined: Sat Sep 13, 2014 12:03 am

Java consume 100% CPU after updating to latest 7.2.6

Postby 1415bhwong » Sun Jan 12, 2014 8:46 pm

We previously have already encounter Java hitting 100% once in a while with Zimbra 7.2.0 on Ubuntu 8. Was told to upgrade to the latest 7.2.6 to fix the issue. However, we found that Java hit 100% even more after the upgrade!
We have contacted Support and was instructed to add this Java options:
$ zmlocalconfig -e mailboxd_java_options="-server -Djava.awt.headless=true

-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=2

-XX:PermSize=196m -XX:MaxPermSize=350m -XX:SoftRefLRUPolicyMSPerMB=1

-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

-XX:+PrintGCApplicationStoppedTime

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/zimbra/log

-XX:ErrorFile=/opt/zimbra/log”
However, this does not resolve the issue. Anyone experienced this and manage to overcome it?


czguy
Advanced member
Advanced member
Posts: 86
Joined: Sat Sep 13, 2014 3:29 am

Java consume 100% CPU after updating to latest 7.2.6

Postby czguy » Mon Jan 13, 2014 6:59 pm

Until a few days ago I was on 7.2.0 and experienced Java pegging the CPU at 100% also. I also tried the heap options you tried with no luck. I ended up migrating to a new server with ZCS 8.0.6 and now Java averages 14-20% CPU time. It seems that 8.0.6 is much more efficient and performs better out of the box.
You haven't mentioned what sort of server you're running. Virtual/Cloud/Physical? How much RAM? How many CPUs? etc.
You could also take the route I did. Grab Ubuntu 12.04 64-bit, upgrade to ZCS 8.0.6, and enjoy the ride. If you have a virtual environment or VPS/Cloud setup this could be easily (and economically) by standing up a new server and using ZeXtras to migrate your data.
1415bhwong
Outstanding Member
Outstanding Member
Posts: 498
Joined: Sat Sep 13, 2014 12:03 am

Java consume 100% CPU after updating to latest 7.2.6

Postby 1415bhwong » Mon Jan 13, 2014 7:57 pm

I'm running 7.2.6 on Ubuntu 8 as Virtual with 8GB RAM and 4 vCPU. Our Zimbra CPU cycle never hit more than 40% in the past. Make me wonder what cause Java to suddenly peak 100% recently.
To upgrade, I was told to perform these major steps:

1. Upgrade Ubuntu 8 to 10

2. Reinstall Zimbra 7.2.6 for Ubuntu 10

3. Upgrade Ubuntu 10 to 12

4. Reinstall Zimbra 7.2.6 for ubuntu 12

5. Upgrade Zimbra 7.2.6 to 8.0.6
Was thinking of creating a new VM with Ubuntu 12 directly and install Zimbra 7.2.6 so that I can do a migration from my old VM. Which method is
Btw, my Ubuntu 8 was installed using this guide:

Ubuntu 8.04 LTS Server (Hardy Heron) Install Guide - Zimbra :: Wiki
czguy
Advanced member
Advanced member
Posts: 86
Joined: Sat Sep 13, 2014 3:29 am

Java consume 100% CPU after updating to latest 7.2.6

Postby czguy » Mon Jan 13, 2014 8:13 pm

[quote user="1415bhwong"]I'm running 7.2.6 on Ubuntu 8 as Virtual with 8GB RAM and 4 vCPU. Our Zimbra CPU cycle never hit more than 40% in the past. Make me wonder what cause Java to suddenly peak 100% recently.
To upgrade, I was told to perform these major steps:

1. Upgrade Ubuntu 8 to 10

2. Reinstall Zimbra 7.2.6 for Ubuntu 10

3. Upgrade Ubuntu 10 to 12

4. Reinstall Zimbra 7.2.6 for ubuntu 12

5. Upgrade Zimbra 7.2.6 to 8.0.6
Was thinking of creating a new VM with Ubuntu 12 directly and install Zimbra 7.2.6 so that I can do a migration from my old VM. Which method is
Btw, my Ubuntu 8 was installed using this guide:

Ubuntu 8.04 LTS Server (Hardy Heron) Install Guide - Zimbra :: Wiki[/QUOTE]
Or you could make it easy on yourself and do the following since you're running Virtual.
1.) Create a new VM with Ubuntu 12.04 LTS 64-bit

2.) Install Zimbra 7.2.6 (preferably 8.0.6)

3.) Migrate from the old VM to the new VM using ZeXtras: ZxBackup: Incremental migration with ZeXtras Backup - ZeXtras Suite Wiki

4.) Have some coffee and enjoy! :)
1415bhwong
Outstanding Member
Outstanding Member
Posts: 498
Joined: Sat Sep 13, 2014 12:03 am

Java consume 100% CPU after updating to latest 7.2.6

Postby 1415bhwong » Mon Jan 13, 2014 8:26 pm

Thanks! Have you use ZxBackup? Is it reliable?
I was told that for the migration to be reliable, both Zimbra must be running on the same version!
czguy
Advanced member
Advanced member
Posts: 86
Joined: Sat Sep 13, 2014 3:29 am

Java consume 100% CPU after updating to latest 7.2.6

Postby czguy » Mon Jan 13, 2014 8:41 pm

[quote user="1415bhwong"]Thanks! Have you use ZxBackup? Is it reliable?
I was told that for the migration to be reliable, both Zimbra must be running on the same version![/QUOTE]
It's very reliable. I just went from a ZCS 7.2.0 to ZCS 8.0.6 server using ZeXtras. Definitely read through the ZeXtras Wiki and decide for yourself. I was thoroughly impressed with it.
1415bhwong
Outstanding Member
Outstanding Member
Posts: 498
Joined: Sat Sep 13, 2014 12:03 am

Java consume 100% CPU after updating to latest 7.2.6

Postby 1415bhwong » Mon Jan 13, 2014 8:44 pm

Thanks for the help! I will read thru and give it a try! So you have migrated from ZCS 7.2 to 8.0.6 directly without encounter any issue at all?
btw, I believe my java option contain some errors when I execute ps -aux | grep java
Warning: bad ps syntax, perhaps a bogus '-'? See procps - Frequently Asked Questions (FAQ)

zimbra 1282 25.8 17.0 4144860 1391936 ? Sl 10:32 2:05 /opt/zimbra/java/bin/java -Dfile.encoding=UTF-8 -server -Djava.awt.headless=true -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=2 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/zimbra/log -XX:ErrorFile=/opt/zimbra/log -Xss256k -Dsun.net.inetaddr.ttl=60 -Xms3194m -Xmx3194m -Xmn798m -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml

zimbra 6384 0.0 0.0 3948 664 pts/0 S+ 10:40 0:00 grep java

zimbra 16257 1.7 3.3 619296 277124 pts/0 Sl 09:13 1:30 /opt/zimbra/java/bin/java -XX:ErrorFile=/opt/zimbra/log -client -Xmx256m -Dzimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/ext:/opt/zimbra/lib/jars:/opt/zimbra/lib/ext-common:/opt/zimbra/lib/ext/backup:/opt/zimbra/lib/ext/clamscanner:/opt/zimbra/lib/ext/network:/opt/zimbra/lib/ext/zimbra-license:/opt/zimbra/lib/ext/zimbrahsm:/opt/zimbra/lib/ext/zimbrasync -Djava.io.tmpdir=/opt/zimbra/data/tmp -Dpython.cachedir=/opt/zimbra/data/tmp/jython-cachedir org.python.util.jython /opt/zimbra/libexec/zmconfigd

root 17584 0.0 0.0 3852 464 ? Ss 09:14 0:00 /opt/zimbra/libexec/zmmailboxdmgr start -Dfile.encoding=UTF-8 -server -Djava.awt.headless=true -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:NewRatio=2 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/zimbra/log -XX:ErrorFile=/opt/zimbra/log -Xss256k -Dsun.net.inetaddr.ttl=60 -Xms3194m -Xmx3194m -Xmn798m
czguy
Advanced member
Advanced member
Posts: 86
Joined: Sat Sep 13, 2014 3:29 am

Java consume 100% CPU after updating to latest 7.2.6

Postby czguy » Tue Jan 14, 2014 9:13 am

[quote user="1415bhwong"]Thanks for the help! I will read thru and give it a try! So you have migrated from ZCS 7.2 to 8.0.6 directly without encounter any issue at all?

[/QUOTE]
Yes, that's what I'm saying. We went from ZCS 7.2.0 on Ubuntu 10.04 32-Bit and migrated successfully to ZCS 8.0.6 on Ubuntu 12.04 64-bit. ZeXtras made this migration so easy. Read the Wiki I posted, it says it all. And the best part is that the tools are free for 30 days with no limitations.
Basically here's what I did:
1.) Install ZeXtras Zxmig tool on old server

2.) Install ZeXtras Suite/Zxbackup on new server

3.) Run Zxmig live on old server and exported to /opt/zimbra/backup/migrations

4.) Rsync everything from old server to new server

5.) On new server run ZxBackup and import the /opt/zimbra/backup/migrations data

6.) Wait a while, have a coffee, smoke a cigarette.

7.) Done
The awesome thing about ZeXtras is you can repeat the migration over and over while you're switching the mail flow (MX record) and not have to worry about account/item duplication.
I have to thank Phoenix (Bill) for pointing me towards this, it totally saved my butt!

Return to “Administrators”

Who is online

Users browsing this forum: Baidu [Spider] and 17 guests