Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Thread: Every few days web interface hangs on "Loading...", fixed by restarting Zimbra

  1. #1
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    10

    Default Every few days web interface hangs on "Loading...", fixed by restarting Zimbra

    An instance of Zimbra Release 7.1.2_GA_3268.UBUNTU10_64 UBUNTU10_64 FOSS gets into a state every few days where it stops accepting web logins and just hangs with at the "Loading..." message screen.

    During this time the mailbox.log full of error messages...

    Code:
    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown table engine 'InnoDB'
    Restarting Zimbra always fixes the problem for another few days. Then it breaks again.

    I've tried comparing the zimbra-related processes before and after fault condition and there doesn't seem to be any difference in the process list.

    Would appreciate any clues on how to go about debugging this. Previous references to this error message I've found suggest that the MySQL database needs to be updated during an upgrade. However this makes no sense here as this would imply a permanent error condition, not a recurring intermittent one. I've already run the upgrade script anyway and it didn't fix anything.

  2. #2
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Hardware specification and whether this is a new install or upgrade? What does top (or your favourite monitor software) show at the time this problem occurs?
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  3. #3
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    10

    Default

    Quote Originally Posted by phoenix View Post
    Hardware specification and whether this is a new install or upgrade? What does top (or your favourite monitor software) show at the time this problem occurs?
    Hi Phoenix,

    It's running on VMWare on a Dell PowerEdge 2950 with 8xXenon E5345 CPUs. The VM is allocated 2CPUs and 3GB RAM.

    We have about 40 accounts.

    This is an upgrade. I should have mentioned this in the previous post but it the problem appears to have started since we migrated from CentOS to Ubuntu LTS. This was done by rsyncing the /opt/zimbra directory, fixing the permissions and re-installing the Ubuntu version of Zimbra. That was with version 7.0.0 at both ends. We since upgarded to 7.1.2 in the hope that it would clear the error.

    I will let you know what is in "top" the next time it is in fault condition.


    Also, if it's any help, this is the first error message received in the log printed in full. There are many after this, but the first may give a clue what triggered the condition.

    Code:
    2011-08-20 11:20:21,905 WARN  [mailboxd.csv] [] DbStats - An error occurred while getting current database stats
    com.zimbra.common.service.ServiceException: system failure: SQL: 'SHOW INNODB STATUS'
    ExceptionId:mailboxd.csv:1313835621905:73ea69cded0f6ef7
    Code:service.FAILURE
            at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:248)
            at com.zimbra.cs.db.DbUtil.executeQuery(DbUtil.java:134)
            at com.zimbra.cs.db.DbUtil.executeQuery(DbUtil.java:176)
            at com.zimbra.cs.db.DbUtil.executeQuery(DbUtil.java:202)
            at com.zimbra.cs.db.DbStats.getStatData(DbStats.java:47)
            at com.zimbra.common.stats.RealtimeStats.getData(RealtimeStats.java:67)
            at com.zimbra.cs.stats.ZimbraPerf$MailboxdStats.getDataLines(ZimbraPerf.java:520)
            at com.zimbra.common.stats.StatsDumper.call(StatsDumper.java:110)
            at com.zimbra.common.stats.StatsDumper$1.run(StatsDumper.java:79)
            at java.lang.Thread.run(Thread.java:662)
    Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown table engine 'InnoDB'
    
    Query being executed when exception was thrown:
    SHOW INNODB STATUS
    
    
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
            at com.mysql.jdbc.Util.getInstance(Util.java:384)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3566)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3498)
            at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1959)
            at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2113)
            at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2568)
            at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2113)
            at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2275)
            at com.zimbra.cs.db.DebugPreparedStatement.executeQuery(DebugPreparedStatement.java:156)
            at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
            at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
            at com.zimbra.cs.db.DbUtil.executeQuery(DbUtil.java:130)
            ... 8 more
    Logging into mysql manually as user zimbra and typing "SHOW INNODB STATUS;" seems to work without throwing any error when not in fault condition. Will try again when in fault condition.
    Last edited by simonb; 08-20-2011 at 05:24 AM.

  4. #4
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    A single CPU allocated to that server should do, is there any possibility to allocate more RAM to this server (although that should be enough for 40 users)? Do you have any firewall or AppArmor enabled on this system? I don't think I've ever seen this sort of problem mentioned in the forum, any possibility you have a HD problem? Is this on a RAID system? What else runs on this server apart from Zimbra, is it possibly a general performance problem of the host?

    Just for my own interest, what was the reason you changed from CentOS to Ubuntu?

    [EDIT] I assume that this 'error message' goes away after a restart of Zimbra until the next failure in a couple of days?
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  5. #5
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    10

    Default

    Hi Pheonix,

    I am using UFW to enable the required access ports and haven't disabled AppArmor. Yes, this is running on a local RAID-10.

    I have just noticed that I hadn't allocated any swap file. I've just created a new 10G disk and allocated that as swap. Let's see if that improves things.

    The reason I switched from CentOS to Ubuntu is that Ubuntu has a much smaller basic footprint (great with thin provisioning) and starts a lot faster. We are basing all our VMs on an Ubuntu template now.

  6. #6
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Quote Originally Posted by simonb View Post
    I am using UFW to enable the required access ports and haven't disabled AppArmor.
    I can't believe this would make any difference but could you disable AppArmor to see if anything changes.

    Quote Originally Posted by simonb View Post
    Yes, this is running on a local RAID-10.
    That should be OK.

    Quote Originally Posted by simonb View Post
    The reason I switched from CentOS to Ubuntu is that Ubuntu has a much smaller basic footprint (great with thin provisioning) and starts a lot faster. We are basing all our VMs on an Ubuntu template now.
    CentOS 6 has a minimal install that has a really small footprint (plus a few dependencies you'd need for Zimbra) that should do just as well - CentOS is my distribution of choice but whatever works for you is good.

    I can't really find out much about this error (I'm not too hot on debugging mysql problems) other than the fact that most of the information I've read says or implies that the INNODB engine hasn't been enabled (or built in) for the people that are reporting this problem. That seems to be confirmed by the error messages you've posted, as to what's causing it......

    When it happens again and before you restart Zimbra can you check the following:

    Code:
    ls -l /opt/zimbra/data/tmp
    
    su - zimbra
    mysql
    
    mysql> show variables like 'have_innodb';
    
    mysql> show engines \G
    
    mysql> SHOW INNODB STATUS;
    
    mysql> quit
    
    cat /opt/zimbra/conf/my.cnf
    [EDIT]One final question for today. When this problem occurs, is it possible you're running out of space on the volume that this directory is located: " /opt/zimbra/data/tmp"?
    Last edited by phoenix; 08-20-2011 at 06:34 AM.
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  7. #7
    phoenix is offline Zimbra Consultant & Moderator
    Join Date
    Sep 2005
    Location
    Vannes, France
    Posts
    23,587
    Rep Power
    58

    Default

    Last question for today. Is it possible that when this problem occurs the partition that holds /opt/zimbra/data/tmp is fulll?
    Regards


    Bill


    Acompli: A new adventure for Co-Founder KevinH.

  8. #8
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    10

    Default

    OK. I will check those things next time it goes wrong. The /opt/zimbra (which contains /opt/zimbra/data/tmp) partition certainly isn't full. There's a good 27G left on it.

    Thanks for all your help and advice!

  9. #9
    Join Date
    Apr 2011
    Posts
    3
    Rep Power
    4

    Default I have the same problem

    Hi,

    I'm using latest
    Release 7.1.2_GA_3268.UBUNTU10_64 UBUNTU10_64 FOSS edition

    ...the latest upgrade.

    Absolutely the same problem, i've checked out all the things commented out and it seems my installation is fine.

    Please if someone have a solution besides stop and start zimbra with crontab every night ?

    ..by the way, the system hangs in the night, may be related to swapping or something ? ... in the day the system work smoothly.

  10. #10
    Join Date
    Sep 2005
    Posts
    48
    Rep Power
    10

    Default

    For me it was definitely because I had no swap partition. As soon as I connected to the console I could see all the out of memory errors on the screen.

    I can report that there have been no crashes in the few weeks since I added the swap partition.

Similar Threads

  1. Replies: 31
    Last Post: 12-15-2007, 09:05 PM
  2. Replies: 22
    Last Post: 12-02-2007, 05:05 PM
  3. zmtlsctl give LDAP error
    By sourcehound in forum Administrators
    Replies: 5
    Last Post: 03-11-2007, 04:48 PM
  4. Post instsallation problems
    By Assaf in forum Installation
    Replies: 14
    Last Post: 01-29-2007, 11:38 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •