Results 1 to 5 of 5

Thread: CMS: abort preclean due to time/concurrent mode failure

Hybrid View

  1. #1
    Join Date
    Nov 2007
    Location
    Columbus, Ohio
    Posts
    33
    Rep Power
    8

    Default CMS: abort preclean due to time/concurrent mode failure

    Hello.
    With Release 6.0.1_GA_1816.UBUNTU8 UBUNTU8 NETWORK edition.
    we run several days just fine, and then
    the processor time charged to the zimbra java process jumps up.
    User experience with the web interface suffers.

    in zmmailboxd.out I've noticed the incidence of two particular message
    increase: "CMS: abort preclean due to time" and "concurrent mode failure"

    615284.919: [CMS-concurrent-abortable-preclean-start]
    CMS: abort preclean due to time 615289.931: [CMS-concurrent-abortable-preclean: 0.275/5.012 secs] [Times: user=0.30 sys=0.06, real=5.02 secs]
    615289.932: [GC[YG occupancy: 386350 K (554816 K)]615289.933: [Rescan (parallel) , 1.0865790 secs]615291.019: [weak refs processing, 0.0000270 secs] [1 CMS-remark: 1850959K(1851392K)] 2237310K(2406208K), 1.0871770 secs] [Times: user=0.48 sys=0.39, real=1.09 secs]
    Total time for which application threads were stopped: 1.0882570 seconds

    This has occurred 2100 times in the last almost 11+ hours. Yet only
    0-3 times in earlier days after a restart.

    and the second:

    587640.400: [GC 587640.400: [ParNew: 554815K->554815K(554816K), 0.0000810 secs]587640.401: [CMS587640.401: [CMS-concurrent-abortable-preclean: 0.236/4.157 secs] [Times: user=0.31 sys=0.23, real=4.16 secs]
    (concurrent mode failure)[Unloading class sun.reflect.GeneratedSerializationConstructorAcces sor11]
    [Unloading class sun.reflect.GeneratedConstructorAccessor190]
    [Unloading class sun.reflect.GeneratedMethodAccessor469]
    [Unloading class sun.reflect.GeneratedMethodAccessor456]
    [Unloading class sun.reflect.GeneratedMethodAccessor468]
    [Unloading class sun.reflect.GeneratedConstructorAccessor191]
    : 1837879K->1851392K(1851392K), 12.7999000 secs] 2392695K->1899378K(2406208K), [CMS Perm : 48919K->48900K(131072K)], 12.8004090 secs] [Times: user=10.79 sys=0.08, real=12.80 secs]
    Total time for which application threads were stopped: 12.8018620 seconds


    We have 5 GB memory (tuned up as a vm from 3GB after the 6.0 upgrade)
    and are the zimbra java process is running with:
    /opt/zimbra/java/bin/java -server -XX:NewRatio=2 -Djava.awt.headless=true -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=1 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+UseConcMarkSweepGC -XX:PermSize=128m -Xss256k -Xms2409m -Xmx2409m -Xmn602m -Djava.io.tmpdir=/opt/zimbra/mailboxd/work -Djava.library.path=/opt/zimbra/lib -Djava.endorsed.dirs=/opt/zimbra/mailboxd/common/endorsed -Dzimbra.config=/opt/zimbra/conf/localconfig.xml -Djetty.home=/opt/zimbra/mailboxd -DSTART=/opt/zimbra/mailboxd/etc/start.config -jar /opt/zimbra/mailboxd/start.jar /opt/zimbra/mailboxd/etc/jetty.properties /opt/zimbra/mailboxd/etc/jetty-setuid.xml /opt/zimbra/mailboxd/etc/jetty.xml

    Can I restart this process with minimal/no effect to the user's web experience?
    Can someone share insight into what's going on, and what action
    I might take?

    thanks,
    John

  2. #2
    Join Date
    Feb 2007
    Location
    Portland, OR
    Posts
    1,147
    Rep Power
    10

    Default

    With times like [Times: user=0.30 sys=0.06, real=5.02 secs] where it only used .36 seconds of processor time, but 5.02 seconds of total time you either have a different process using a ton of cpu, or this process took forever waiting on disk access.

    You might want to check the disk subsystem this is running on.

    That or something else entirely is wrong...

  3. #3
    Join Date
    Feb 2007
    Location
    Portland, OR
    Posts
    1,147
    Rep Power
    10

    Default

    Also as that process is the main mailbox process it will have an affect on user experience, and restarting just it might cause issues with other services that are depending on it.

    You should be able to restart it with "zmmailboxdctl stop; zmmailboxdctl start" but as I said... I have no idea what the effect of this on the rest of the parts of the server will be.

  4. #4
    Join Date
    Nov 2007
    Location
    Columbus, Ohio
    Posts
    33
    Rep Power
    8

    Default

    Hi.
    It got back to normal after a restart, but I expect it will bog down
    again sometime. Expanded thoughts:
    I can't account for the wall clock delay, and I appreciate the
    consideration of processor bound server or confounding disk subsystem.
    A review of the CPU history shows things are nominal except
    for the java process going heavy into CPU and the disk subsystem
    seems just fine.
    Tonight I scheduled a short outage to zmmailboxdctl stop,
    wait (verify process ends, etc), and a subsequent zmmailboxdctl start.
    zmmailboxd.out looks normal again, without the CMS preclean or concurrent
    mode failures. And the processor load is back to normal. So seems the
    zmmailboxd restart cleared up the issue. I'm expecting to see the
    issue again, as this is now twice I've seen this condition arise.
    I ran zmmailboxdmgr threaddump per
    King0770-Notes - Zimbra :: Wiki
    while it was bogged down, and after the restart. I'll attach the
    log segments for two threaddumps.
    Thanks for any thoughts.
    -John
    Attached Files Attached Files

  5. #5
    Join Date
    Nov 2007
    Location
    Columbus, Ohio
    Posts
    33
    Rep Power
    8

Similar Threads

  1. mysql not starting, sort of.
    By lhutton in forum Installation
    Replies: 6
    Last Post: 06-19-2009, 11:45 AM
  2. Replies: 4
    Last Post: 08-04-2008, 07:17 PM
  3. SLES10: Problem upgrading from Zimbra NE 5.0 to NE 5.0.1
    By trapanator in forum Installation
    Replies: 11
    Last Post: 02-27-2008, 01:51 PM
  4. Offline mode failure
    By gbr in forum General Questions
    Replies: 0
    Last Post: 02-19-2008, 12:06 PM
  5. system failure: getDirectContext
    By avisser in forum Installation
    Replies: 3
    Last Post: 10-12-2005, 06:32 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •