Zimbra Admin UI shows services red, and unavailable. Monitor->Mail Queues broken (but got fixed)

[Ed Epstein]

I've recently installed the latest version of ZCS (last few weeks & hours) on CentOS 6, and have a perpetually broken UI.  Right on the home page it warns that 'some services aren't running', and lists service failed under runtime. Command line however shows everything up and running, and the first test server has been running email for a few weeks now. 

Under Monitor, nothing works. Server Status lists everything dead. Mail Queues gave me a soap error, which leads me to this post in the Wiki. I had indeed changed the default SSH port and the following code fixed the Mail Queues:

zmprov ms my.domain zimbraRemoteManagementPort 7777

I've been tailing /var/log/secure, but I don't see any more sshd errors. I do see a lot of this now:

Mar 28 11:22:54 zcs1 sudo:   zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmmailboxdmgr status
Mar 28 11:22:54 zcs1 sudo:   zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmmailboxdmgr status
Mar 28 11:22:54 zcs1 sudo:   zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmmtastatus

At this point the Mail Queues seems to be fixed, but nothing else under Monitor works yet. I'm not sure where else to look to fix this. Any advice is appreciated.

[cayaraa]

Services in red usually means you didn't do the post install steps of setting up [r]syslog for zimbra. For a single server install:
https://www.zimbra.com/docs/ne/8.6.0/single_server_install/wwhelp/wwhimpl/common/html/wwhelp.htm#href=860_SingleServer_Install_NE.Final_Set-Up.html&single=true

[Ed Epstein]

Thanks a bunch. Will try it out and let you know.

[Ed Epstein]

Unfortunately, I can't report yet that it has worked. Services still show up dead. I've rebuilt the entire server from scratch twice just to be sure.



Personally, I don't understand why the installation doesn't set this up and verify it. It seems I need to recreate another server and possibly *not* attempt to secure it until ZCS is installed, and then attempt to lock it down.



I dunno. It doesn't make any sense how a secured SSH setup is incompatible with ZCS out of the box.

[cayaraa]

As far as I know services showing up as read shouldn't have anything to do with ssh. As far as I know thats all controlled by syslog settings. Have you checked to make sure there is data in /var/log/zimbra.log and /var/log/zimbra-stats.log?



ssh is often used for issuing remote commands on the other zimbra servers in a zimbra cluster.

[jorgedlcruz]

Hi Ed Epstein,

Did you change the SSH port? If so, please follow the next steps:

zmprov ms zimbra.domain.com zimbraRemoteManagementPort 1000

Then restart ZCS:

zmmailboxdctl restart

You'll also need to fix what you broke by playing with the authorized_keys file. Hopefully you can just rebuild it:

zmupdateauthkeys

If this doesn't work, you may need to also regenerate your keys:

zmsshkeygen
zmupdateauthkeys

Restart ZCS when you're doing.

Let us know.

[zcs.community.01]

Sorry, it took me awhile for a response. My original account here is completely and utterly locked. Reset the password about a dozen or so times via email, but that didn't work either. It consistently states that either the email address or password is incorrect or the account is not approved, even directly after a password reset procedure on the site. I had to create a new account to get back in. As an aside, there doesn't seem to be any staff or administration email to send support requests too, so if a staff member sees this.....



Nothing listed as a possible answer works. I've created two new CentOS 6 servers from scratch, and nothing. Same problems every single time. I'm currently run on a brand spanking new CentOS 7 install. I was able to locate this for reference (finally):



https://www.zimbra.com/docs/os/8.6.0/single_server_install/wwhelp/wwhimpl/js/html/wwhelp.htm#href=860_SingleServerInstall_OS.Verify_Zimbra_Server_Operation.html



The Mail Queues I can get working, and the SSH problems seem solvable and solved for that matter. What doesn't work at all is the status. I have confirmed that both zimbra.log and zimbra-stats.log are active and being filled with entries.



There's nothing else on the server except base, curl, wget, iftop, and screen. The required dependencies for ZCS were also installed. /etc/sysconfig/syslog doesn't exist, but /etc/sysconfig/rsyslog.conf is the only equivalent. Here's the following after checking it with systemctl:



"rsyslog.service - System Logging Service

Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled)

Active: active (running) since Mon 2015-04-13 15:20:33 PDT; 57s ago

Main PID: 441 (rsyslogd)

CGroup: /system.slice/rsyslog.service

└─441 /usr/sbin/rsyslogd -n -r -m 0"



I added the requested flags, but I still get nothing. Any other ideas? Does ZCS work on CentOS 7? If ZCS doesn't work on CentOS 7, can somebody tell me a distribution that it does work on?



I'm really trying to get a server up into production, but I can't do that with errors all over the place and no management functions.

[Klug]

There are issues with CentOS 7 (Jorge posted about it a couple of weeks ago).

Right now, you should stick to CentOS 6.



One of the issues is about logrotate and selinux.

[jerryboi]

I am having this issue too on CentOS6.6 Minimal Server commissioned through Digital Ocean. I did not have the issue before. ZCS8.6NE.

I did do the post-install steps referenced here above. I am also seeing  these in /var/log/secure:

Mar 28 11:22:54 zcs1 sudo:   zimbra : TTY=unknown ; PWD=/opt/zimbra ; USER=root ; COMMAND=/opt/zimbra/libexec/zmmailboxdmgr status

and I think this is a problem because /opt/zimbra/.ssh/authorized_keys does only allow this one command:  "/opt/zimbra/libexec/zmrcd"

My assumption is that ZimbraAdmin is sending the command 'zmmailbodmgr status' while sshd of zimbra user is expecting something like

echo "HOST:mail1.example.com status" | ssh -i ~/.ssh/zimbra_identity -o strictHostKeyChecking=no zimbra@mail1.example.com "/opt/zimbra/libexec/zmrcd"