Results 1 to 7 of 7

Thread: Server no longer working, 21/20 thread limit reached?

Threaded View

  1. #1
    Join Date
    Jun 2011
    Posts
    76
    Rep Power
    4

    Default [SOLVED]Server no longer working, 21/20 thread limit reached?

    Hey guys, I'm having some trouble. Over the weekend some complete idiot and/or super genius got in to the server and screwed a lot of stuff up. I believe that I've fixed the bulk of it but there are still a few things I'm having trouble with. As of right now, nothing works no admin console, web e-mail, client or otherwise.

    CentOS 5.6
    Zimbra 7.1.2

    As stated earlier I cannot even get into the admin console to changes things, it just sits at connected, however I was able to telnet in on the server itself and send an email to an external domain, so it feels like the main receiving process / program is no longer working or broken.

    Firewall has been turned off for testing at the moment so its not that.

    One more thing to note, we recently added around 12 new accounts with a new domain name, one account has a 6GB email account with about 100,000 or so emails in the inbox, if this is whats crashing or preventing the server, is there a way to make Zimbra skip an email domain to prevent it from being loaded or something of the sort to see if that's the problem?

    [zimbra@mail root]$ zmcontrol status
    Host mail.birchlake.ath.cx
    antispam Running
    antivirus Running
    ldap Running
    logger Running
    mailbox Running
    memcached Running
    mta Running
    snmp Running
    spell Running
    stats Running
    zmconfigd Running
    [zimbra@mail root]$ zmprov gacf
    ERROR: zclient.IO_ERROR (invoke Read timed out, server: localhost) (cause: java.net.SocketTimeoutException Read timed out)
    I think the problem lies either with postfix or with the thread limit. The postfix error I get comes from the zimbra.log and runs when I'm running the stop commands.

    Dec 19 01:22:03 mail zmmailboxdmgr[12583]: file /opt/zimbra/log/zmmailboxd_manager.pid does not exist
    Dec 19 01:22:03 mail zmmailboxdmgr[12583]: assuming no other instance is running
    Dec 19 01:22:03 mail zmmailboxdmgr[12583]: file /opt/zimbra/log/zmmailboxd.pid does not exist
    Dec 19 01:22:03 mail zmmailboxdmgr[12583]: assuming no other instance is running
    Dec 19 01:22:03 mail zmmailboxdmgr[12583]: no manager process is running
    Dec 19 01:22:19 mail zimbramon[12742]: 12742:info: Stopping services initiated by zmcontrol
    Dec 19 01:22:19 mail zimbramon[12742]: 12742:info: Stopping stats via zmcontrol
    Dec 19 01:22:20 mail zimbramon[12742]: 12742:info: Stopping mta via zmcontrol
    Dec 19 01:22:20 mail postfix/postfix-script[12841]: fatal: the Postfix mail system is not running
    Dec 19 01:22:20 mail zimbramon[12742]: 12742:info: Stopping spell via zmcontrol
    Dec 19 01:22:20 mail zimbramon[12742]: 12742:info: Stopping snmp via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping cbpolicyd via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping archiving via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping antivirus via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping antispam via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping imapproxy via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping memcached via zmcontrol
    Dec 19 01:22:21 mail zimbramon[12742]: 12742:info: Stopping mailbox via zmcontrol

    The thread error comes from the mailbox.log

    2011-12-19 00:08:15,300 WARN [LmtpServer] [] TcpServer/7025 - Thread pool is 105% utilized. 21 out of 20 threads in use.
    2011-12-19 00:08:15,300 ERROR [LmtpServer] [] TcpServer/7025 - cannot handle connection; thread pool exhausted
    java.util.concurrent.RejectedExecutionException
    at java.util.concurrent.ThreadPoolExecutor$AbortPolic y.rejectedExecution(ThreadPoolExecutor.java:1768)
    at java.util.concurrent.ThreadPoolExecutor.reject(Thr eadPoolExecutor.java:767)
    at java.util.concurrent.ThreadPoolExecutor.execute(Th readPoolExecutor.java:658)
    at com.zimbra.cs.tcpserver.TcpServer.run(TcpServer.ja va:202)
    at java.lang.Thread.run(Thread.java:662)

    From the mailbox.log file

    Dec 18 22:24:54 mail postfix/lmtp[8895]: BC7F4BD3D07: to=<webmaster@birchlake.ath.cx>, relay=mail.birchlake.ath.cx[192.168.1.25]:7025, delay=944, delays=944/0.01/0/0, dsn=4.0.0, status=deferred (host mail.birchlake.ath.cx[192.168.1.25] refused to talk to me: 421 mail.birchlake.ath.cx Zimbra LMTP server closing connection; service busy)

    Dec 19 03:01:42 mail postfix/error[2460]: 3F829BD3206: to=<admin@birchlake.ath.cx>, relay=none, delay=0.04, delays=0.03/0/0/0.01, dsn=4.4.2, status=deferred (delivery temporarily suspended: conversation with mail.birchlake.ath.cx[192.168.1.25] timed out while sending end of data -- message may be sent more than once)




    I am unable to get into the zimbra admin console to change the thread limit nor am I able to change it via console command.

    here's some config stuff just in case it might help


    [zimbra@mail root]$ cat /etc/hosts
    # Do not remove the following line, or various programs
    # that require network functionality will fail.
    127.0.0.1 localhost.localdomain localhost
    192.168.1.25 mail.birchlake.ath.cx mail
    ::1 localhost6.localdomain6 localhost6
    [zimbra@mail root]$ cat /etc/resolv.conf
    search mail.birchlake.ath.cx birchlake.ath.cx
    nameserver localhost
    nameserver 192.168.1.25
    nameserver 8.8.8.8
    nameserver 8.8.4.4
    [zimbra@mail root]$ dig mail.birchlake.ath.cx mx
    ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-16.P1.el5_7.1 <<>> mail.birchlake.ath.cx mx
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40288
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;mail.birchlake.ath.cx. IN MX

    ;; AUTHORITY SECTION:
    birchlake.ath.cx. 38400 IN SOA mail. admin.birchlake.ath.cx. 1324277785 10800 3600 604800 38400

    ;; Query time: 0 msec
    ;; SERVER: 192.168.1.25#53(192.168.1.25)
    ;; WHEN: Mon Dec 19 02:57:51 2011
    ;; MSG SIZE rcvd: 85
    [zimbra@mail root]$ dig mail.birchlake.ath.cx any
    ; <<>> DiG 9.3.6-P1-RedHat-9.3.6-16.P1.el5_7.1 <<>> mail.birchlake.ath.cx any
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37242
    ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;mail.birchlake.ath.cx. IN ANY

    ;; ANSWER SECTION:
    mail.birchlake.ath.cx. 38400 IN A 192.168.1.25

    ;; AUTHORITY SECTION:
    birchlake.ath.cx. 38400 IN NS mail.

    ;; Query time: 0 msec
    ;; SERVER: 192.168.1.25#53(192.168.1.25)
    ;; WHEN: Mon Dec 19 02:58:11 2011
    ;; MSG SIZE rcvd: 73
    [zimbra@mail root]$ host 'birchlake.ath.cx'
    birchlake.ath.cx mail is handled by 10 mail.birchlake.ath.cx.
    [zimbra@mail root]$ hostname
    mail.birchlake.ath.cx
    [zimbra@mail init.d]$ postfix check
    postfix/postfix-script: warning: not owned by root: /opt/zimbra/postfix-2.7.4.2z/conf/master.cf.in

    If I do postfix status it comes back with nothing, but postfix start and stop seem like they do something....

    Any help or advice would be greatly appreciated, I'm at the end of my rope and about to take a hammer to the computer.
    thanks!
    Last edited by Eclipse; 12-19-2011 at 04:12 PM.

Similar Threads

  1. HELIX 7.1.1 ported to FreeBSD
    By solko in forum Installation
    Replies: 22
    Last Post: 05-21-2013, 05:37 AM
  2. Backspace causing threads to disappear
    By rhutch in forum General Questions
    Replies: 1
    Last Post: 10-22-2012, 08:03 PM
  3. Zimbra don't start
    By cadinor in forum Migration
    Replies: 1
    Last Post: 08-04-2011, 06:27 PM
  4. zmconfigd exiting: Hung threads detected
    By skyflyer in forum Administrators
    Replies: 3
    Last Post: 06-28-2011, 05:27 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •