Results 1 to 5 of 5

Thread: A network service error has occurred

Hybrid View

  1. #1
    Join Date
    Jun 2006
    Location
    Indianapolis, IN
    Posts
    51
    Rep Power
    9

    Default A network service error has occurred

    Long story short: I was in the middle of upgrading from ZCS (NE) 4.0.1 to 4.0.2, when the rpm install process of the install.sh script was working on the first RPM, I lost my ssh connection.

    I logged back in to see that it only had 2 of the 8 RPMs installed. This is when my fun began.

    I had done a "zmbackup -a all" beforehand, so I had data to restore from, but it wasn't going well for the LONGEST time, until I found 'zmrestoreldap'. Now, I'm so close (after 13 hours now) to getting zimbra back and running, but hit another wall.

    Here's the error I'm getting

    Code:
    msg - system failure: java.lang.NullPointerException
    com.zimbra.cs.service.ServiceException: system failure: java.lang.NullPointerException
    	at com.zimbra.cs.service.ServiceException.FAILURE(ServiceException.java:174)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:229)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
    	at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:173)
    	at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
    	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:541)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
    	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
    	at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
    	at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
    	at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
    	at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
    	at java.lang.Thread.run(Thread.java:595)
    Caused by: java.lang.NullPointerException
    	at com.zimbra.cs.service.UserServlet.getRestUrl(UserServlet.java:157)
    	at com.zimbra.cs.service.mail.ToXML.encodeRestUrl(ToXML.java:1355)
    	at com.zimbra.cs.service.mail.ToXML.encodeFolderCommon(ToXML.java:217)
    	at com.zimbra.cs.service.mail.ToXML.encodeFolder(ToXML.java:145)
    	at com.zimbra.cs.service.mail.ToXML.encodeFolder(ToXML.java:136)
    	at com.zimbra.cs.service.mail.GetFolder.encodeFolderNode(GetFolder.java:91)
    	at com.zimbra.cs.service.mail.GetFolder.encodeFolderNode(GetFolder.java:94)
    	at com.zimbra.cs.session.SoapSession.putRefresh(SoapSession.java:273)
    	at com.zimbra.soap.ZimbraSoapContext.generateResponseHeader(ZimbraSoapContext.java:472)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:195)
    	at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:84)
    	at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:223)
    	... 18 more
    
    code - service.FAILURE
    method - ZmCsfeCommand.prototype.invoke
    detail - soap:Receiver
    The steps I took to get here were:

    1. uninstalling all zimbra RPMs
    2. removed /opt/zimbra
    3. installed zcs 4.0.1
    4. zmcontrol stop
    5. zmrestoreldap -lb full-20061011.170501.208 -t /opt/zimbra-backup/backup
    6. zmcontrol start
    7. (one by one, I set each account to "maintenance")
    8. zmlicense -i /opt/zimbra-backup/conf/ZCSLicense.xml
    9. (one by one, I restored each account)
    zmrestore -a user@domain.com -lb full-20061011.170501.208 -t /opt/zimbra-backup/backup
    10. zmcontrol stop
    11. zmrestoreoffline -sys -lb full-20061011.170501.208 -t /opt/zimbra-backup/backup
    12. I set one account to "active" and tried to login. I get "A network service error has occurred."

    The zmrestore -sys helped with mysql and the logger and tomcat not starting, now everything is running, but users can't login. The admin UI works fine, and I get the same "network service" error when the admin clicks on "view mail".

    Any ideas?

  2. #2
    Join Date
    Jun 2006
    Location
    Indianapolis, IN
    Posts
    51
    Rep Power
    9

    Default Sleep helps

    OK, This morning I started working on this again. I decided to poke around the admin UI and check settings, and I noticed the domain for the user I was testing was missing. I didn't delete it, but the restore must not have recovered it properly. I added it, and it complained that it already existed, but a refresh of the domain list showed it there.

    And now... I can login.

    Once I'm happy that it's working correctly, I'm going to try another upgrade to 4.0.2, but this time...I'm running in 'screen'.

    --Brian

  3. #3
    Join Date
    Jan 2006
    Posts
    88
    Rep Power
    9

    Default

    I am very sorry for the issues you are having. Just a tip for the future, use GNU screen when doing important remote admin work.

    It's a bit tricky to learn at first, but it's better than having long scripts crash if your connection gets pulled. If you need any help using it please let me know.

  4. #4
    Join Date
    Jan 2006
    Location
    Indianapolis, IN
    Posts
    3
    Rep Power
    9

    Exclamation Use screen next time

    To prevent a problem like this, use the screen program. If your connection disconnects you can easily just do a 'screen -ls' to see what screen's are available and just re-attach. Then you won't have all the problems you are having due to disconnection and your wife and kid won't be so mad at you for staying out so late.

  5. #5
    Join Date
    Jun 2006
    Location
    Indianapolis, IN
    Posts
    51
    Rep Power
    9

    Default Still broken...

    OK, I had it working in 4.0.1, did a backup with "zmbackup -a all -t /opt/zimbra/mybackup" and then rsynced the entire /opt/zimbra directory to /opt/zimbra-20061012-good by:

    Code:
    mkdir /opt/zimbra-20061012-good
    rsync -av /opt/zimbra/ /opt/zimbra-20061012-good
    I performed an upgrade, and I got an error, but screen kills my screen history, so I ran the install again, but with the "-r" option and luckily got the same error so I could paste it here:

    Code:
    This platform is CentOS4
    Packages found: zimbra-core-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm
    This may or may not work
    
    
    Install anyway? [N] y
    
    The system will be modified.  Continue? [N] y
    
    Shutting down zimbra mail
    
    Backing up ldap
    
    
    Removing existing packages
    
       zimbra-ldap...done
       zimbra-logger...done
       zimbra-mta...done
       zimbra-snmp...done
       zimbra-store...done
       zimbra-spell...done
       zimbra-apache...done
       zimbra-core...done
    
    Removing deployed webapp directories
    Installing packages
    
        zimbra-core......zimbra-core-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-ldap......zimbra-ldap-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-logger......zimbra-logger-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-mta......zimbra-mta-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-snmp......zimbra-snmp-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-store......zimbra-store-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-apache......zimbra-apache-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
        zimbra-spell......zimbra-spell-4.0.2_GA_362.RHEL4-20060922120643.i386.rpm...done
    
    Setting defaults from saved config in /opt/zimbra/.saveconfig/config.save
       HOSTNAME=zimbra1.domain.com
       LDAPHOST=zimbra1.domain.com
       LDAPPORT=389
       SNMPTRAPHOST=zimbra1.domain.com
       SMTPSOURCE=admin@zimbra1.domain.com
       SMTPDEST=admin@zimbra1.domain.com
       SNMPNOTIFY=yes
       SMTPNOTIFY=yes
       LDAPROOTPW=XXXXXXXXX
       LDAPZIMBRAPW=XXXXXXXX
    Restoring existing configuration file from /opt/zimbra/.saveconfig/config.save...Restoring backup schedule...done
    Operations logged to /tmp/zmsetup.log.32544
    Setting defaults...Setting defaults from existing config...Done
    Upgrading from 4.0.1_GA_324 to 4.0.2_GA_362
    Thu Oct 12 12:56:12 2006: Stopping zimbra services
    Thu Oct 12 12:56:51 2006: Starting mysql
    Thu Oct 12 12:57:10 2006: SELECT value FROM config WHERE name = 'db.version'
    /opt/zimbra/bin/mysql: line 30: /root/zcs-4.0.2/${zimbra_home}/mysql/bin/mysql: No such file or directory
    /opt/zimbra/bin/mysql: line 30: exec: /root/zcs-4.0.2/${zimbra_home}/mysql/bin/mysql: cannot execute: No such file or directory
    Error while running '/opt/zimbra/bin/mysql --user=zimbra --password=thisIsNotARealPasswordSoThereYouGo --database=zimbra --batch --skip-column-names'. at /opt/zimbra/libexec/scripts/Migrate.pm line 134.
    Now, my concern is why it's trying to run mysql from "/root/zcs-4.0.2/${zimbra_home}/mysql/bin/mysql"

    /root/zcs-4.0.2 is the unpacked root directory of the 4.0.2. tarball.
    It should be trying to run ${zimbra_home}/mysql/bin/mysql, but it also doesn't appear to be replacing the variable with its contents. Now that I've pasted this here, I'm going to see what I can find. I saw criley's mention of running zmsetup.pl by hand with the "-c" option, so I'm doing that...

    ...and it seems to be doing well! (/em looks for some wood to knock on...)

    Hopefully this is almost done. I didn't see the issue where there were missing "CONFIGURED END" entries in /opt/zimbra/.install_history, so that was nice.

    Well...even though the second time it bombed it showed 2 of 500 accounts (there should have been over 10) they are all there, and everything looks good. Now, I wait for a couple of users (one uses Outlook, other the web interface) and see if it looks good. What a saga. e-mail down for over 24 hours. (only a small percentage of all users have been migrated to zimbra so far) The others are still in Lotus Notes world. (yuck)

    Well, I hope this is helpful to someone. I'm more than happy to share my blood and sweat.

    --Brian

Similar Threads

  1. A network service error has occurred (zclient/io)
    By telescop in forum Installation
    Replies: 17
    Last Post: 12-19-2008, 07:22 AM
  2. Replies: 17
    Last Post: 04-19-2008, 09:27 AM
  3. A network service error has occurred.
    By rlustemberg in forum Administrators
    Replies: 1
    Last Post: 11-29-2006, 03:34 PM
  4. Network Service Error with MIME encoded message
    By Glenham in forum Administrators
    Replies: 2
    Last Post: 10-17-2006, 09:37 AM
  5. Network service error when trying to send email
    By wayne in forum Administrators
    Replies: 4
    Last Post: 09-15-2005, 03:37 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •