Hi!
Last night im upgraded our company zimbra server from 8.0.3 to 8.0.4.
The upgrade was completed without any error in no time.
However i notice that the 'java' process eat the cpu much more like in the previous version.
The zmlogger process also eat 100% cpu time after a while, plus the web UI noticable slower, for example after i click to a mail folder the browser unresponsive for a few seconds.
Im completely disabled the logger feature, this can help a little but the web UI still slower than the 8.0.3 version.
Also our users report that the new version slower than the previous.
Anyone have a new experience with this new zimbra version?
Thanks!
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
[quote user="vadonka"]Anyone have a new experience with this new zimbra version?[/QUOTE]Yes, I installed this earlier this week without problems and it's noticeably faster (the web UI) and with no change in cpu usage or load on my system.
You haven't really given much information about your configuration nor the number of users and whether they're using the web UI or a fat client etc., etc. How about giving some useful diagnostic information about the problem?
You haven't really given much information about your configuration nor the number of users and whether they're using the web UI or a fat client etc., etc. How about giving some useful diagnostic information about the problem?
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
our server have ~2000 user, 300-400 active in the same time.
im using a dl370 g6 server with 2 xeon (hw virt supported) 3.2ghz processor (16 logical cpu), 96GB ram, and a couple of 1TB SAS hdd in raid5 array running with the ESXi 5.1
the zimbra virtual machine have 8 logical cpu and 8GB ram and 2TB vmdk, swappiness is 0.
the iotop said the IO stress is very low, but the cpu load high referring to the top output.
the filesystem is ext4 with noatime mount option.
this setup was working great with the zimbra 8.0.3.
im reverted to the 8.0.3 (from a VM snapshot), the webui faster again, then i try to upgrade again with the 8.0.4
the result is the same, the whole system slowed down.
currently im completely disabled the logger feature, and reduced the amavis max servers to 3.
this can help, its working properly with this modification. however the 8.0.3 version doesn't have such this issue.
im using a dl370 g6 server with 2 xeon (hw virt supported) 3.2ghz processor (16 logical cpu), 96GB ram, and a couple of 1TB SAS hdd in raid5 array running with the ESXi 5.1
the zimbra virtual machine have 8 logical cpu and 8GB ram and 2TB vmdk, swappiness is 0.
the iotop said the IO stress is very low, but the cpu load high referring to the top output.
the filesystem is ext4 with noatime mount option.
this setup was working great with the zimbra 8.0.3.
im reverted to the 8.0.3 (from a VM snapshot), the webui faster again, then i try to upgrade again with the 8.0.4
the result is the same, the whole system slowed down.
currently im completely disabled the logger feature, and reduced the amavis max servers to 3.
this can help, its working properly with this modification. however the 8.0.3 version doesn't have such this issue.
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
So what does it say is taking all the CPU?
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
[quote user="quanah"]So what does it say is taking all the CPU?[/QUOTE]
the 'java' process take a significant cpu time and raise the system load higher than 10.0
the 'java' process take a significant cpu time and raise the system load higher than 10.0
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
ZCS has multiple Java processes. Saying "java" provides no useful information. Please provide the exact java command taking CPU.
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
I installed Zimbra 8.0.4 on Ubuntu 12.04 LTS (64) with 4GB RAM and zero (0) active users. I have experienced the very same issue. I've attempted reign in the memory loss by adjusting the java heap percentage using the zmlocalconfig tool and turned swappiness=0. At first glance these changes seemed to have been positive but it takes a few hours for the server to crash altogether.
Here is what I'm seeing:
20% of memory is crunching on "java", executing this command:
/opt/zimbra/java/bin/java -client -Xmx256m -Djava.net.preferIPv4Stack=true -D
zimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/e
xt:/opt/zimbra/lib/jars:/opt/zimbra/lib/ext-common:/opt/zimbra/lib/ext/clamscanner -Djava.io.tmpdir=/o
pt/zimbra/data/tmp -Dpython.cachedir=/opt/zimbra/data/tmp/jython-cachedir org.python.util.jython /opt/
zimbra/libexec/zmconfigd
Another 21% is crunching away at this command:
/opt/zimbra/java/bin/java -Dfile.encoding=UTF-8 -server -Djava.awt.headless=t
rue -Dsun.net.inetaddr.ttl=60 -Dorg.apache.jasper.compiler.disablejsr199=true -XX:+UseConcMarkSweepGC
-XX:PermSize=128m -XX:MaxPermSize=350m -XX:SoftRefLRUPolicyMSPerMB=1 -verbose:gc -XX:+PrintGCDetails -
XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:-OmitStackTraceInFastThrow -Djava.net.pre
ferIPv4Stack=true -Xss256k -Xms486m -Xmx486m -Xmn121m -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djav
a.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.conf
ig=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc
/start.config -jar /opt/zimbra/mailboxd/start.jar OPTIONS=Server,jsp,jmx,resources,websocket,ext,jta,p
lus,rewrite,setuid /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml
/opt/zimbra/mailboxd/etc/jetty.xml
An aggregate of 24% of memory is working on multiple threads like:
/opt/zimbra/amavisd/sbin/amavisd (master)
OR
/opt/zimbra/amavisd/sbin/amavisd (ch1-avail)
OR
/opt/zimbra/amavisd/sbin/amavisd (virgin child)
The only other software installed on this server is the zExtras suite.
Eventually the server will stop responding and you see various messages like this (OMMing):
Out of memory: Kill process 12222 (java) score 291 or sacrifice child
Killed process 12222 (java) total-vm:5441712kB, anon-rss:669904kB, file-rss:0kB
Out of memory: Kill process 22843 (java) score 148 or sacrifice child
Killed process 22843 (java) total-vm:2724916kB, anon-rss:341628kB, file-rss:0kB
Out of memory: Kill process 8108 (java) score 120 or sacrifice child
Killed process 8108 (java) total-vm:2127264kB, anon-rss:276580kB, file-rss:0kB
Out of memory: Kill process 8254 (java) score 121 or sacrifice child
INFO: rcu_sched self-detected stall on CPU { 0} (t=60000 jiffies g=321307 c=321306 q=5275)
Here is what I'm seeing:
20% of memory is crunching on "java", executing this command:
/opt/zimbra/java/bin/java -client -Xmx256m -Djava.net.preferIPv4Stack=true -D
zimbra.home=/opt/zimbra -Djava.library.path=/opt/zimbra/lib -Djava.ext.dirs=/opt/zimbra/java/jre/lib/e
xt:/opt/zimbra/lib/jars:/opt/zimbra/lib/ext-common:/opt/zimbra/lib/ext/clamscanner -Djava.io.tmpdir=/o
pt/zimbra/data/tmp -Dpython.cachedir=/opt/zimbra/data/tmp/jython-cachedir org.python.util.jython /opt/
zimbra/libexec/zmconfigd
Another 21% is crunching away at this command:
/opt/zimbra/java/bin/java -Dfile.encoding=UTF-8 -server -Djava.awt.headless=t
rue -Dsun.net.inetaddr.ttl=60 -Dorg.apache.jasper.compiler.disablejsr199=true -XX:+UseConcMarkSweepGC
-XX:PermSize=128m -XX:MaxPermSize=350m -XX:SoftRefLRUPolicyMSPerMB=1 -verbose:gc -XX:+PrintGCDetails -
XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:-OmitStackTraceInFastThrow -Djava.net.pre
ferIPv4Stack=true -Xss256k -Xms486m -Xmx486m -Xmn121m -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djav
a.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.conf
ig=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc
/start.config -jar /opt/zimbra/mailboxd/start.jar OPTIONS=Server,jsp,jmx,resources,websocket,ext,jta,p
lus,rewrite,setuid /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml
/opt/zimbra/mailboxd/etc/jetty.xml
An aggregate of 24% of memory is working on multiple threads like:
/opt/zimbra/amavisd/sbin/amavisd (master)
OR
/opt/zimbra/amavisd/sbin/amavisd (ch1-avail)
OR
/opt/zimbra/amavisd/sbin/amavisd (virgin child)
The only other software installed on this server is the zExtras suite.
Eventually the server will stop responding and you see various messages like this (OMMing):
Out of memory: Kill process 12222 (java) score 291 or sacrifice child
Killed process 12222 (java) total-vm:5441712kB, anon-rss:669904kB, file-rss:0kB
Out of memory: Kill process 22843 (java) score 148 or sacrifice child
Killed process 22843 (java) total-vm:2724916kB, anon-rss:341628kB, file-rss:0kB
Out of memory: Kill process 8108 (java) score 120 or sacrifice child
Killed process 8108 (java) total-vm:2127264kB, anon-rss:276580kB, file-rss:0kB
Out of memory: Kill process 8254 (java) score 121 or sacrifice child
INFO: rcu_sched self-detected stall on CPU { 0} (t=60000 jiffies g=321307 c=321306 q=5275)
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
[quote user="gb3"]
The only other software installed on this server is the zExtras suite.
[/QUOTE]
Please confirm whether or not you see this issue if you do not install the zExtras suite. Past issues show that zExtras generally is incompatible with upgrades for some period of time, while they fix their suite to account for changes between releases. I can confirm I do not see any such issue like this on ZCS 8.0.4 without Zextras, running Ubuntu12.
The only other software installed on this server is the zExtras suite.
[/QUOTE]
Please confirm whether or not you see this issue if you do not install the zExtras suite. Past issues show that zExtras generally is incompatible with upgrades for some period of time, while they fix their suite to account for changes between releases. I can confirm I do not see any such issue like this on ZCS 8.0.4 without Zextras, running Ubuntu12.
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
[quote user="quanah"]Please confirm whether or not you see this issue if you do not install the zExtras suite. [/QUOTE]
I can confirm that even before I installed the zExtras Core and Zimlet (1.8.7) the same issue occurred. However I did notice that when I ran:
it ruturned the following warning:
After I ran:
and
This issue went away.
Do you know how Zimbra configures the mailboxd_java_heap_memory_percent by default?
I can confirm that even before I installed the zExtras Core and Zimlet (1.8.7) the same issue occurred. However I did notice that when I ran:
zmlocalconfig mailboxd_java_heap_memory_percent
it ruturned the following warning:
Warning: null valued key 'mailboxd_java_heap_memory_percent'
After I ran:
zmlocalconfig -e mailboxd_java_heap_memory_percent=20
and
sudo sysctl vm.swappiness=0
This issue went away.
Do you know how Zimbra configures the mailboxd_java_heap_memory_percent by default?
Zimbra 8.0.4 processes consumes too much cpu time and web UI slower.
That isn't a valid local config key, thus the error.
The two localconfig errors that relate to the java heap are:
The two localconfig errors that relate to the java heap are:
mailboxd_java_heap_new_size_percent = 25
mailboxd_java_heap_size = 742
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Who is online
Users browsing this forum: Bing [Bot], Google [Bot] and 15 guests