Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
bdial
Elite member
Elite member
Posts: 1633
Joined: Fri Sep 12, 2014 10:39 pm

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby bdial » Fri Feb 15, 2008 7:55 am

Thought I'd share my experiences getting logging to work on a multiserver install running openSuSE 10.3 and Zimbra 5.x. Maybe some of these can be integrated into the install script for the next version. These instructions will probably be valid for other 10.x versions of openSuSE but I have not tested it. If anyone has other suggestions, fixes or what not please post them and I will modify to include them.
For the purposes of this guide, we'll use the following 3 servers
mta.domain.com - the zimbra-mta server

ldap.domain.com - the zimbra-ldap server

mailbox.domain.com - the main mailbox server, also running zimbra-logger and will be the central repository for all the servers' logs
The first server we want to setup is the mailbox/logger server. You can pretty much throw out the zmsyslogsetup script here. It attempts to use a syslog-ng.conf.in file which according to the syslog-ng.conf in openSuSE 10.3


# NOTE: The SuSEconfig script and its syslog-ng.conf.in

# configuration template aren't used any more.


So open /etc/syslog-ng/syslog-ng.conf with your favorite text edit. The first thing you'll want to do is uncomment (remote the #) the line that says
udp(ip("0.0.0.0") port(514));

This will allow the other hosts to log to syslog-ng on the logger server. This is equivelant to adding the command line arguments -r -m 0 when you're using the standard syslog. Next, add these lines to the bottom of the file
filter f_local0       { facility(local0); }; # zimbra

destination zmail { file("/var/log/zimbra.log" owner("zimbra") ); }; # zimbra

log { source(src); filter(f_mail); destination(zmail); }; # zimbra

destination local0 { file("/var/log/zimbra.log" owner("zimbra") ); }; # zimbra

log { source(src); filter(f_local0); destination(local0); }; # zimbra

filter f_auth { facility(auth); }; # zimbra

destination zmauth { file("/var/log/zimbra.log" owner("zimbra") ); }; # zimbra

log { source(src); filter(f_auth); destination(zmauth); }; # zimbra


This sets up the necessary logging facilities. Save that file and exit. Now we need to handle the log rotating. Zimbra will have no problem moving the zimbra.log since it has the necessary permissions, but it will not be able to restart the syslog server when it does it and therefor you'll wind up with a blank zimbra.log until root restarts syslog with it's own logrotate process. The first thing you need to do is edit /etc/sudoers down at the bottom you'll find a few entries for zimbra already. Add this one below them
%zimbra ALL=NOPASSWD:/sbin/rcsyslog restart

This allows zimbra to restart the syslog daemon. Now edit the file /opt/zimbra/conf/zmlogrotate and fine the line that says
/sbin/killall -HUP syslogd 2> /dev/null || true

change that line to say
sudo /sbin/rcsyslog restart 2> /dev/null || true

Now as a good test you should su to the zimbra user, and try the command sudo /sbin/rcsyslog restart . If all goes well, it should restart syslog and you should now have a /var/log/zimbra.log with status updates of the mailbox server currently.
Now onto the other hosts mta and ldap. Open /etc/syslog-ng/syslog-ng.conf
comment (put a # in front of) the line that says
log { source(src); filter(f_mail); destination(mail); };

This keeps the system from logging mail stuff from postfix to the local mail log cause you'll want to send it to the logger server. This is only really necessary for the mta server but I guess if it was going to be integrated into the zmsyslogsetup script might as well do it for every machine it won't hurt.
Next, add these lines at the bottom
destination zmlogger { udp("mailbox.domain.com" port(514) ); }; # zimbra

log { source(src); filter(f_mail); destination(zmlogger); }; # zimbra

filter f_local0 { facility(local0); }; # zimbra

log { source(src); filter(f_local0); destination(zmlogger); }; # zimbra

filter f_auth { facility(auth); }; # zimbra

log { source(src); filter(f_auth); destination(zmlogger); }; # zimbra

you'll want to change the destination zmlogger statement to be the address of your logger server. Ultimately the zmsyslogsetup script should populate this with the zmLogHostname from the config like it does for the standard syslog setup. Anyway, save this file now and then restart syslog as root. You don't really need to worry about zimbra's logrotate for the otehr machines as they will not be logging locally anyway and it doesn't matter if it's broke.
You should now see status/smtp logs from the otehr hosts on your mailbox/logger server.


User avatar
L. Mark Stone
Elite member
Elite member
Posts: 2175
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 8.8.15 Network Edition
Contact:

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby L. Mark Stone » Wed Sep 17, 2008 1:52 pm

Thanks for this post!
We can confirm this also works with SUSE Linux Enterprise Server 10 SP2, which also installs syslog-ng by default.
We recently expanded a single-server SLES10 ZCS install to a multi-server setup, and this post is what got our aggregated syslog server working for us.
One suggestion if I may... if you edit /etc/syslog-ng/syslog-ng.conf by hand, SuSEconfig scripts won't modify it at all going forward. Possibly it is safer (or not) to update /etc/syslog-ng/syslog-ng.conf.in and then let SuSEconfig regenerate /etc/syslog-ng/syslog-ng.conf each time SuSEconfig is run.
Hope that helps,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
Zeta Alliance http://www.zetalliance.org/
bsteimel
Posts: 6
Joined: Fri Sep 12, 2014 11:33 pm

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby bsteimel » Thu Mar 19, 2009 9:19 am

We started using syslog-ng on our Ubuntu8 box instead of syslogd. I successfully used this to reconfigure our box to work. I was getting the error "logger service not installed" in the zimbra admin interface even though it said it was the logger service was running.
The only differences were:
in all the syslog-ng config files instead of source(src) i used source(s_all) because that the source i had already configured to send to my network syslog server.
In the logrotate file i used /sbin/syslog-ng instead of rcsyslog because on ubuntu8 the rcsyslog did not exist. This also has to be changed in the sudousers file.
It has been running for a few days now and everything seems to be working properly.
User avatar
L. Mark Stone
Elite member
Elite member
Posts: 2175
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 8.8.15 Network Edition
Contact:

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby L. Mark Stone » Thu Mar 19, 2009 9:24 am

Is syslog-ng the default on a new 8.04 install like it is on SuSE Linux Enterprise Server 10?
Can anyone respond as to the default logging for an RHEL5 install?
If syslog-ng is becoming the new standard, perhaps that's the trigger for Zimbra to update their installation scripts to see which syslogging facility is installed and deploy the correct Zimbra syslog scripts accordingly.
Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
Zeta Alliance http://www.zetalliance.org/
bsteimel
Posts: 6
Joined: Fri Sep 12, 2014 11:33 pm

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby bsteimel » Thu Mar 19, 2009 10:01 am

no syslog-ng is not default on ubuntu8. I just like the interface in webmin and the added features. I run syslog-ng on all my servers for forwarding there logs to our central syslog server that runs splunk and syslog-ng. splunk takes care of all our logs except for snort which puts us over the 500mb/day limit for the free version of splunk. syslog-ng takes care of the snort log.
User avatar
L. Mark Stone
Elite member
Elite member
Posts: 2175
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 8.8.15 Network Edition
Contact:

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby L. Mark Stone » Thu Mar 19, 2009 11:37 am

[quote user="bsteimel"]no syslog-ng is not default on ubuntu8. I just like the interface in webmin and the added features. I run syslog-ng on all my servers for forwarding there logs to our central syslog server that runs splunk and syslog-ng. splunk takes care of all our logs except for snort which puts us over the 500mb/day limit for the free version of splunk. syslog-ng takes care of the snort log.[/QUOTE]
Ah, another Splunk fan! Awesome software...
Thanks for the reply,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
Zeta Alliance http://www.zetalliance.org/
scalper
Advanced member
Advanced member
Posts: 114
Joined: Fri Sep 12, 2014 10:13 pm

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby scalper » Sun Feb 21, 2010 11:28 am

This guide save the day! Upgraded from 5.0.18 to 6.0.5, and the logs died on me for 2 days. Then i found this thread. :)
Verified working on my SuSE 10.2.
User avatar
L. Mark Stone
Elite member
Elite member
Posts: 2175
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 8.8.15 Network Edition
Contact:

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby L. Mark Stone » Wed Feb 24, 2010 9:33 pm

Updating this thread with info to cover ZCS 6.0.x and SLES10...
Now that the old ZCS stats logging system has been replaced and there are now two Zimbra log files in /var/log, we found we needed to update /etc/syslog-ng/syslog.conf as follows.
Also, the newly defined "zimbra_src" didn't work, so we just commented that out and replaced "zimbra_src" in the log lines with the SuSE-defined "src".
Here's tail end of the /etc/syslog-ng/syslog.conf file we now use:
#source zimbra_src {  unix-stream("/dev/log"; keep-alive(yes); max-connections(20);); }; # zimbra

#source zimbra_src { unix-stream("/dev/log" keep-alive(yes) max-connections(20)); }; # zimbra

filter zimbra_local0 { facility(local0); }; # zimbra

filter zimbra_local1 { facility(local1); }; # zimbra

filter zimbra_auth { facility(auth); }; # zimbra

filter zimbra_mail { facility(mail); }; # zimbra

destination zimbra_mail { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra

destination zimbra_local1 { file("/var/log/zimbra-stats.log" owner("zimbra")); }; # zimbra

destination zimbra_local0 { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra

destination zimbra_auth { file("/var/log/zimbra.log" owner("zimbra")); }; # zimbra

log { source(src); filter(zimbra_mail); destination(zimbra_mail); }; # zimbra

log { source(src); filter(zimbra_local0); destination(zimbra_local0); }; # zimbra

log { source(src); filter(zimbra_local1); destination(zimbra_local1); }; # zimbra

log { source(src); filter(zimbra_auth); destination(zimbra_auth); }; # zimbra


Hope that helps,

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
Zeta Alliance http://www.zetalliance.org/
ahenneyza
Posts: 2
Joined: Sat Sep 13, 2014 1:38 am

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby ahenneyza » Tue Oct 26, 2010 5:36 am

Hi
I have read most of the forums, but there is not alot about forwarding the /opt/zimbra/log* through syslog-ng. I have tried most of the possible solutions presented,but none of them work.
How do I configure the audit log to go through syslog-ng to a SIEM tool.
Thanks
phoenix
Ambassador
Ambassador
Posts: 26618
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Guide: Multi-server logging with openSuSE 10.3 syslog-ng

Postby phoenix » Tue Oct 26, 2010 5:54 am

[quote user="ahenneyza"]How do I configure the audit log to go through syslog-ng to a SIEM tool.[/QUOTE]+"how to" +forward +log +"syslog-ng" - Yahoo! Search Results
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra

Return to “Administrators”

Who is online

Users browsing this forum: Google [Bot] and 13 guests