Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: server possibly hacked - help getting Zimbra running

  1. #1
    Join Date
    Nov 2007
    Posts
    6
    Rep Power
    8

    Default server possibly hacked - help getting Zimbra running

    Our system admin has moved on to bigger n better things. For now I am watching over the system till his spot is filled, but I am no system admin.

    Today our zimbra mail stopped working. When I looked in the var/log files I noticed somebody from Paris France had been trying for hours to ssh into our linux box and appears they succeeded:

    Nov 3 14:05:37 smtp sshd[6536]: Failed password for invalid user raphael from 193.251.253.164 port 55685 ssh2
    Nov 3 14:06:01 smtp CRON[6543]: (pam_unix) session opened for user zimbra by (uid=0)
    Nov 3 14:06:07 smtp sudo: zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmtomcatmgr status

    Then this happened and the mail account stopped working:

    Nov 4 15:53:25 smtp init: tty1 process (373) killed by signal 15
    Nov 4 15:53:25 smtp init: tty2 process (3257) killed by signal 15
    Nov 4 15:53:25 smtp init: tty3 process (3258) killed by signal 15
    Nov 4 15:53:25 smtp init: svscan process (3259) killed by signal 15
    Nov 4 15:53:27 smtp zimbramon[13151]: 13151:info: Stopping services
    Nov 4 15:53:27 smtp zimbramon[13151]: 13151:info: Stopping mta
    Nov 4 15:53:32 smtp postfix/postfix-script: stopping the Postfix mail system
    Nov 4 15:53:32 smtp postfix/master[4139]: terminating on signal 15
    Nov 4 15:53:32 smtp zimbramon[13151]: 13151:info: Stopping spell
    Nov 4 15:53:40 smtp zimbramon[13151]: 13151:info: Stopping snmp
    Nov 4 15:53:40 smtp zimbramon[13151]: 13151:info: Stopping antivirus
    Nov 4 15:53:40 smtp zimbramon[13151]: 13151:info: Stopping antispam
    Nov 4 15:53:41 smtp amavis[4170]: Net::Server: 2007/11/04-15:53:41 Server closing!
    Nov 4 15:53:43 smtp zimbramon[13151]: 13151:info: Stopping imapproxy
    Nov 4 15:53:45 smtp zimbramon[13151]: 13151:info: Stopping mailbox
    Nov 4 15:53:47 smtp clamd[4090]: Pid file removed.
    Nov 4 15:53:47 smtp clamd[4090]: Exiting (clean)
    Nov 4 15:53:47 smtp clamd[4090]: --- Stopped at Sun Nov 4 15:53:47 2007

    Rest of file is too long to post. But I also noticed this warning in the reboot:

    Nov 4 15:56:00 smtp postfix/postfix-script: warning: not owned by root: /opt/zimbra/postfix-2.2.9/conf/main.cf

    and I have no idea what this is:

    desktop:/home/build/p4/main/ThirdParty/openldap/openldap-2.3.21/servers/slapd


    Can somebody tell me if my system was hacked? Any idea how can I get zimbra back up? Seems logger and snmp are stopped.

    Any help would be greatly appreciated. For now I just shut the server down till I can figure out what is going on.

  2. #2
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    21

    Default Nope.

    Doesn't look to me like it was hacked. It was a sudo from the zimbra user.
    Code:
     Nov  3 14:06:07 smtp sudo:   zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmtomcatmgr status
    Zimbra was asking itself to see if everything was running, and obviously it returned that it's not running.

    If you want to see why it stopped, try taking a look at /opt/zimbra/tomcat/logs/catalina.out

    By the way, a good way to protect against SSH intrusions, is to block access to port 22 from the outside world.

  3. #3
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    21

    Default

    To start things up, try logging into the server, and running
    su - zimbra
    zmcontrol start

  4. #4
    Join Date
    Nov 2007
    Posts
    6
    Rep Power
    8

    Default zimbra not working

    > Doesn't look to me like it was hacked.

    That's a relief! Little paranoid with all those login atempts then zimbra going down. Thanks!!

    OK, I still can't get the mail program to work. Tried zmcontrol start got this:

    zimbra@smtp:~$ zmcontrol start
    Host smtp.(removed for privacy)
    Starting logger...Done
    Starting mailbox...Done.
    Starting antispam...Done.
    Starting antivirus...Done.
    Starting snmp...Done.
    Starting spell...Done.
    Starting mta...Done.

    But I can load the administration console, I can see all the mail accounts, just can't get any of the mail accounts to load.

    I also took a look at logs/catalina.out. File is 414K, not sure what to look for. Did find some warnings though:

    Nov 4, 2007 11:16:59 PM org.apache.catalina.startup.HostConfig deployDescriptor
    WARNING: A docBase /opt/zimbra/apache-tomcat-5.5.15/webapps/zimbra inside the host appBase has been specified, and will be ignored
    log4j:WARN No appenders could be found for logger (org.apache.catalina.session.ManagerBase).
    log4j:WARN Please initialize the log4j system properly.

    Any suggestions? Again thanks for the help!

  5. #5
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    Welcome to the forums,
    (Though unfortunate that you had to get your feet wet this way!)

    Here's a list of all the zimbra related logfiles: /docs/ne/latest/administration_guide/9_Monitoring.12.1.html#1075561
    tail -f /opt/zimbra/log/mailbox.log while you try to login to an account
    crashes will be written to /opt/zimbra/tomcat/logs/catalina.out
    (the mta and other important log is /var/log/zimbra.log)

    (You can either attach them to a post in a .zip, use zimbra private pastebin - collaborative debugging tool, or send via email if you want more privacy - see our profiles for addresses)

    So logger isn't critical, for troubleshooting tips see Logger - Zimbra :: Wiki
    If you can get the admin console gui that's a good thing - what does a 'zmcontrol status' give you?
    Last edited by mmorse; 11-04-2007 at 11:24 PM.

  6. #6
    Join Date
    Nov 2007
    Posts
    6
    Rep Power
    8

    Default zimbra problems

    zmcontrol status:

    antispam Running
    antivirus Running
    ldap Running
    logger Stopped
    zmlogswatchctl is not running
    mailbox Running
    mta Running
    snmp Stopped
    swatch is not running
    spell Running

    Interesting, there is no mailbox.log in /opt/zimbra/log

    Tried to follow catalina.out, but no additional info was wrote to file when I tried to load or reload the page.

    Also tried
    /etc/init.d/zimbra stop
    /etc/init.d/zimbra restart

  7. #7
    Join Date
    Nov 2007
    Posts
    6
    Rep Power
    8

    Default

    This is all I got in the catalina.out tail:

    zimbra@smtp:~$ tail -f /opt/zimbra/tomcat/logs/catalina.out
    Zimbra server reserving server socket port=143 bindaddr=null ssl=false
    Zimbra server reserving server socket port=993 bindaddr=null ssl=true
    Zimbra server process is running as root, changing to user=zimbra uid=1001 gid=1001
    Zimbra server process, after change, is running with uid=1001 euid=1001 gid=1001 egid=1001
    Nov 5, 2007 12:38:09 AM org.apache.coyote.http11.Http11BaseProtocol start
    INFO: Starting Coyote HTTP/1.1 on http-80
    Nov 5, 2007 12:38:09 AM org.apache.coyote.http11.Http11BaseProtocol start
    INFO: Starting Coyote HTTP/1.1 on http-7071
    Nov 5, 2007 12:38:10 AM org.apache.catalina.startup.Catalina start
    INFO: Server startup in 7074 ms

  8. #8
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    ok before we go any further let's get you a backup:
    su zimbra
    zmcontrol stop
    switch back to root
    ps aux | grep zimbra (kill any remaining kill -9 pid)
    mkdir /backup
    rsync -avHK /opt/zimbra/ /backup/zimbra

    and if you want you can tar it up as well:
    mkdir /backuptar
    tar -zcvf /backuptar/zimbra.date.backup.gz -C /backup/zimbra .

    then
    su zimbra
    zmcontrol start

    side note - let's find out if your on NE: zmcontrol -v will give us your zimbra version - if there's 'network' in the string you can open a support ticket as well)
    Plus the network edition has hot, automatic backups
    Last edited by mmorse; 11-05-2007 at 01:09 AM. Reason: NE?

  9. #9
    Join Date
    Nov 2007
    Posts
    6
    Rep Power
    8

    Default

    Release 4.0.4_GA_457.UBUNTU6 UBUNTU6

    Still working on backups...

  10. #10
    Join Date
    May 2006
    Location
    USA
    Posts
    6,242
    Rep Power
    21

    Default

    He let me know that he's completed his rsync, (had trouble with tar & just needed a syntax correction). Reports that he's currently taring and also copying the tar to a safeplace.

    hm...sounds like this prior sys admin is a little behind...4.5's been out for a while and we're rounding on v5
    -In case he needs it, anyone got 4.0.4 Ubuntu FOSS available? (don't see it in sourceforge anymore)
    Last edited by mmorse; 11-05-2007 at 01:10 AM.

Similar Threads

  1. svn version still won't start
    By kinaole in forum Developers
    Replies: 0
    Last Post: 10-04-2006, 07:47 AM
  2. Getting problems in FC4 while instalation
    By kitty_bhoo in forum Installation
    Replies: 13
    Last Post: 09-12-2006, 11:34 PM
  3. Replies: 16
    Last Post: 09-07-2006, 07:39 AM
  4. Trouble sending mail from Outlook
    By czaveri in forum Users
    Replies: 15
    Last Post: 07-24-2006, 12:01 PM
  5. FC3 Install and no zimbra ?
    By aws in forum Installation
    Replies: 10
    Last Post: 10-09-2005, 05:19 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •