After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
cerri
Posts: 22
Joined: Sat Sep 13, 2014 3:03 am

After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby cerri » Mon Sep 26, 2016 2:15 pm

After the upgrade, the WebMail is unreliable: after few hours it stops working without any reasons.
The Console is still accessible and other services too, but the webmail is hanged until the proxy is restarted.

In the logs we can see:

Code: Select all

2016-09-26 16:08:30.741:WARN:oejs.HttpChannel:qtp1684106402-1767: https://HOSTNAME/service/home/~/?auth=co&loc=it&id=96359&part=2
java.io.IOException: Broken pipe
        at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
        at sun.nio.ch.IOUtil.write(IOUtil.java:51)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
        at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:175)
        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.flush(SslConnection.java:808)
        at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:419)
        at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:313)
        at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:141)
        at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:752)
        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
        at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:511)
        at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:653)
        at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:702)
        at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor$GzipBufferCB.process(GzipHttpOutputInterceptor.java:364)
        at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
        at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
        at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.gzip(GzipHttpOutputInterceptor.java:143)
        at org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor.write(GzipHttpOutputInterceptor.java:112)
        at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:177)
        at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:163)
        at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:413)
        at com.zimbra.common.util.ByteUtil.copy(ByteUtil.java:726)
        at com.zimbra.common.util.ByteUtil.copy(ByteUtil.java:698)
        at com.zimbra.cs.service.formatter.NativeFormatter.sendbackBinaryData(NativeFormatter.java:630)
        at com.zimbra.cs.service.formatter.NativeFormatter.sendbackBinaryData(NativeFormatter.java:576)
        at com.zimbra.cs.service.formatter.NativeFormatter.sendbackOriginalDoc(NativeFormatter.java:450)
        at com.zimbra.cs.service.formatter.NativeFormatter.handleMessagePart(NativeFormatter.java:275)
        at com.zimbra.cs.service.formatter.NativeFormatter.handleMessage(NativeFormatter.java:162)
        at com.zimbra.cs.service.formatter.NativeFormatter.formatCallback(NativeFormatter.java:129)
        at com.zimbra.cs.service.formatter.Formatter.format(Formatter.java:142)
        at com.zimbra.cs.service.UserServlet.doAuthGet(UserServlet.java:536)
        at com.zimbra.cs.service.UserServlet.doGet(UserServlet.java:314)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:206)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
        at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.common.filters.Base64Filter.doFilter(Base64Filter.java:63)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:116)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
        at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
        at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
        at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
        at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
        at org.eclipse.jetty.server.Server.handle(Server.java:517)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
        at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
 
INFO: [HOSTNAME]:8736 [dev] [3.4.8] processors=4, physical.memory.total=13.7G, physical.memory.free=5.0G, swap.space.total=4.0G, swap.space.free=4.0G, heap.memory.used=763.9M, heap.memory.free=209.3M, heap.memory.total=973.1M, heap.memory.max=973.1M, heap.memory.used/total=78.49%, heap.memory.used/max=78.49%, minor.gc.count=133, minor.gc.time=4060ms, major.gc.count=3, major.gc.time=349ms, load.process=1.00%, load.system=7.00%, load.systemAverage=9.00%, thread.count=130, thread.peakCount=163, event.q.size=0, executor.q.async.size=0, executor.q.client.size=0, executor.q.query.size=0, executor.q.scheduled.size=0, executor.q.io.size=0, executor.q.system.size=0, executor.q.operation.size=0, executor.q.priorityOperation.size=0, executor.q.response.size=0, operations.remote.size=0, operations.running.size=0, proxy.count=1, clientEndpoint.count=0, connection.active.count=0, client.connection.count=0, connection.count=0


Any hint?


cerri
Posts: 22
Joined: Sat Sep 13, 2014 3:03 am

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby cerri » Thu Sep 29, 2016 6:46 am

Waiting for a response...
User avatar
syslint
Posts: 36
Joined: Wed Feb 24, 2016 7:59 am
Contact:

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby syslint » Thu Sep 29, 2016 2:20 pm

What is your server type. I can see the load is little high in your server.
24x7 PROACTIVE ZIMBRA SERVER MANAGEMENT | Sales : sales @ syslint.com | Skype us : SyslintSkype | Call us : (+91) 471-60 - 7799 | visit : https://syslint.com
cerri
Posts: 22
Joined: Sat Sep 13, 2014 3:03 am

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby cerri » Thu Sep 29, 2016 2:24 pm

Thanks for the reply.
The load is normal:

sysadmin@zimbra:~$ top
top - 16:25:21 up 3 days, 5:50, 2 users, load average: 0.16, 0.30, 0.36
Tasks: 195 total, 2 running, 193 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.3%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 14369468k total, 9362696k used, 5006772k free, 305180k buffers
Swap: 4190204k total, 13360k used, 4176844k free, 3512248k cached

And the load cannot "kill" just the web interface (proxy)...
cerri
Posts: 22
Joined: Sat Sep 13, 2014 3:03 am

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby cerri » Mon Oct 03, 2016 8:57 am

Anyone to help?
Anyone from Zimbra?
ccesario
Posts: 12
Joined: Fri Sep 12, 2014 11:44 pm

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby ccesario » Thu Oct 06, 2016 12:12 pm

Hello,

I 'm facing the same problem, I "suspect" that the problem is related to ngix/proxy service. But no solution until now.

best regards,

-Carlos
cerri
Posts: 22
Joined: Sat Sep 13, 2014 3:03 am

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby cerri » Sun Oct 09, 2016 8:18 am

Me too.
The solution is to restart the proxy.
The real solution is to disable it: I've got no problems after disabling it.
ccesario
Posts: 12
Joined: Fri Sep 12, 2014 11:44 pm

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby ccesario » Mon Oct 10, 2016 6:30 pm

Hi @cerri,

Yes, I disabled it for testing and we don't have problems. (thank you for sharing it)
But I 'm looking for the root cause, maybe some special config solve it. But I 'm still looking for it :/


Regards,

-Carlos
ccesario
Posts: 12
Joined: Fri Sep 12, 2014 11:44 pm

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby ccesario » Mon Oct 17, 2016 6:13 pm

Any update about it!?
syed.yakoob
Posts: 7
Joined: Tue Aug 29, 2017 7:56 am

Re: After upgrade from 8.6 to 8.7 on ubuntu 12.04, the webmail hangs after few hours

Postby syed.yakoob » Tue Aug 29, 2017 8:44 am

facing the same issue, does anyone have a permanent fix for this?

Return to “Administrators”

Who is online

Users browsing this forum: guitardood and 6 guests