If you do a new installation, the self-signed certificates will already be in place; there is no need to run through the self-signed cert wiki.
I have installed 4.5.6, 4.5.7 and 4.5.9 and out-of-the-box (ie, using the certs that are generated during installation), none work with SMTP Auth over TLS on my Ubuntu 6.06 server. I've checked the AuthURL and other settings per this article and everything is in order.
In answer to some earlier questions:
- Yes, I am a network edition customer, but this is an open-source installation we use as part of our evaluation process in previewing and testing new releases. So support may not be available through the usual channels.
- I need to regenerate the certificates to add some aliases that this server can be contacted on (two different host names on two different domains).
I would really like to increase the verbosity in the postfix logs but I'm a little unsure of the best way to go about this in a persistent manner; any changes I've made get hosed when I restart zimbra or do "postfix reload".
More info would certainly help shed some light on where the problem is.
After many hours of tcpdump, and trawling logs with SASL auth logging everything (cert exchange and cipher negotiation...everything) it appears tehe problem lies in the auth stage of the mail session. Immediately after the CRAM-MD5 exchange, Zimbra terminates the session. See attached session from the logs.
I've verified the username and passowrd are correct, and manually gone through a SSL mail session (TCP/465) and confirmed the same.
I also found the /opt/zimbra/cyrus-sasl/sbin/testsaslauthd command so gave it a shot too running as "zimbra""
zimbra@node:~/cyrus-sasl/sbin$ ./testsaslauthd -u <myuser> -p <mypass>
0: OK "Success."
zimbra@node:~/cyrus-sasl/sbin$ ./testsaslauthd -u <myuser> -p <mypass> -r gray.net.au
0: OK "Success."
So SASL auth is working.
Where to now??
Last edited by Centurion; 11-21-2007 at 04:11 PM. Reason: Added testsaslauthd results
Seems that zimbraMtaAuthURL isn't being hit during the authentication stage of the SMTP transaction. Watching our production (4.5.6) server I can see the relevent entries in the /opt/zimbra/tomcat/logs/access_log.2007-11-22 such as:
10.10.100.25 - - [22/Nov/2007:13:56:59 +1100] "POST /service/soap/ HTTP/1.1" 200 475 "-" "-"
(The production server's IP is 10.10.100.25) - this entry corresponds to a message I sent from same mail client I cannot use with our staging system.
So watching tcpdump on loopback and port 443 confirms traffic for the authentication URL is going back and forth on production. However, we are NOT seeing this behaviour on the staging system.
So it's looking like some weird loopback/IP/name resolution phunkiness.
FWIW, this Ubuntu 6.06LTS server is using the default /etc/hosts file as constructed during installation:
zimbra@node:~$ cat /etc/hosts
127.0.1.1 node.gray.net.au node
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
/etc/nsswitch.conf resolves against files, then dns. Also /etc/resolv.conf is set up correctly to use the staging name server and domains.
<Sigh>...I will keep people posted.
SMTP with Auth but without TLS/SSL results in the same error as my original post. This really shouldn't be so hard
Nov 22 15:41:37 node postfix/smtpd: connect from iceman.gray.net.au[10.0.0.4]
Nov 22 15:41:37 node postfix/master: warning: process /opt/zimbra/postfix-2.2.9/libexec/smtpd pid 19248 killed by signal 11
Again, no hits in tomcat's access log and no traffic over loopback on tcp/443.
So this is looking more like a network problem than a SSL/TLS on. THe question is, where the hell is postfix trying to authenticate?? I've checked the //opt/zimbra/cyrus-sasl/lib/sasl2/smtpd.conf file along with /opt/zimbra/cyrus-saslauthd.conf* and everything matches what is in LDAP.
I'm starting to get really frustrated
Assuming that 10.10.100.25 is the IP of your zimbra server. I also assume you have correct DNS A & MX records point at the server IP address?Code:127.0.0.1 localhost.localdomain localhost 10.10.100.25 node.gray.net.au node
You should also disable ipv6, SElinux and any firewall you have on this server.
Hi Phoenix - thanks for your input!
Ok, fixed the hosts file with the correct IP for the FQDN (10.0.0.5):
Verified the DNS resolution:Code:127.0.0.1 localhost.localdomain localhost 10.0.0.5 node.gray.net.au node
Which was retruned from the DNS server as coconfigured in /etc/resolv.conf.Code:>dig node.gray.net.au ; <<>> DiG 9.3.2 <<>> node.gray.net.au ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47522 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;node.gray.net.au. IN A ;; ANSWER SECTION: node.gray.net.au. 600 IN A 10.0.0.5 ;; AUTHORITY SECTION: gray.net.au. 600 IN NS ns1.gray.net.au. ;; ADDITIONAL SECTION: ns1.gray.net.au. 600 IN A 10.0.0.40
Disabled IPv6 (which broke DCC too along the way - need to recompile it)
Rebooted to make sure IPv6 wasn't coming back, and removed IPv6 entries from /etc/hosts.Code:>cat /etc/modprobe.d/blacklist-ipv6 # Never load IPv6 support blacklist ipv6
Then retried the tests from Thunderbird and Apple Mail:
During this authenticated SMTP attempt, there was no activity on loopback or eth0 on port 443, and there was no auth request sent to tomcat.Code:==> /var/log/zimbra.log <== Nov 22 19:41:15 node postfix/smtpd: connect from iceman.gray.net.au[10.0.0.4] Nov 22 19:41:15 node postfix/smtpd: setting up TLS connection from iceman.gray.net.au[10.0.0.4] Nov 22 19:41:17 node postfix/smtpd: TLS connection established from iceman.gray.net.au[10.0.0.4]: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits) Nov 22 19:41:24 node postfix/master: warning: process /opt/zimbra/postfix-2.2.9/libexec/smtpd pid 29593 killed by signal 11 Nov 22 19:41:24 node postfix/master: warning: /opt/zimbra/postfix-2.2.9/libexec/smtpd: bad command startup -- throttling
FWIW, here's the listening ports on the systtem::
I've just finished downloading 4.5.10_GA and I'll give that a shot.Code:>sudo netstat -tadlp | grep LIST tcp 0 0 *:imaps *:* LISTEN 28819/java tcp 0 0 localhost.localdo:11553 *:* LISTEN 26193/MailWatch SQL tcp 0 0 *:pop3s *:* LISTEN 28819/java tcp 0 0 *:7780 *:* LISTEN 28850/httpd tcp 0 0 node.gray.net.au:ldap *:* LISTEN 28305/slapd tcp 0 0 localhost.localdo:10025 *:* LISTEN 28946/master tcp 0 0 localhost.localdom:7306 *:* LISTEN 28693/mysqld tcp 0 0 localhost.localdom:7307 *:* LISTEN 28550/mysqld tcp 0 0 localhost.l:netbios-ssn *:* LISTEN 4076/smbd tcp 0 0 node.gray.n:netbios-ssn *:* LISTEN 4076/smbd tcp 0 0 *:pop3 *:* LISTEN 28819/java tcp 0 0 *:imap2 *:* LISTEN 28819/java tcp 0 0 *:10000 *:* LISTEN 4268/perl tcp 0 0 *:www *:* LISTEN 4243/apache2 tcp 0 0 *:7025 *:* LISTEN 28819/java tcp 0 0 *:ssmtp *:* LISTEN 28946/master tcp 0 0 ns1.gray.net.au:domain *:* LISTEN 3703/named tcp 0 0 localhost.locald:domain *:* LISTEN 3703/named tcp 0 0 shell.gray.net.au:ssh *:* LISTEN 4097/sshd tcp 0 0 node.gray.net.au:ssh *:* LISTEN 4097/sshd tcp 0 0 127.0.1.1:ssh *:* LISTEN 4097/sshd tcp 0 0 news.gray.net.au:nntp *:* LISTEN 4154/xinetd tcp 0 0 *:ipp *:* LISTEN 3735/cupsd tcp 0 0 *:3128 *:* LISTEN 4176/(squid) tcp 0 0 *:smtp *:* LISTEN 28946/master tcp 0 0 localhost.localdoma:953 *:* LISTEN 3703/named tcp 0 0 *:7035 *:* LISTEN 28819/java tcp 0 0 *:https *:* LISTEN 28819/java tcp 0 0 localhost.:microsoft-ds *:* LISTEN 4076/smbd tcp 0 0 node.gray.:microsoft-ds *:* LISTEN 4076/smbd tcp 0 0 *:7071 *:* LISTEN 28819/java
As per the title. I upgraded to 4.5.10_GA and still no good with SMTP auth. As I said in an earlier post, the (working) production system has a lot of tcp/443 traffic on loopback and you can see the hits on tomcat when SMTP auth is successful.
This staging/test system is dead in the water as far as SMTP auth. There seems to be some sort of disconnect between Postfix and saslauthd, or maybe saslauthd and tomcat(?). As there is nothing in /var/log/zimbra.log from saslauthd I'm leaning towards something wrong between Postfix and saslauthd.
Any other ideas??
After pulling my hair out I decided to use "strace" and map out just what was happenning with the system calls while Postfix was trying to carry out authenticated SMTP. What struck me was Postfix was reading sasl libraries from /usr/lib, not /opt/zimbra...(see attched traces). After further investigation it became obvious salsauthd wasn't loading correctly due to broken library paths. Zimbra runs cyrus-sasl 22.214.171.124 (Zimbra 4.5.10_GA) BUT Ubuntu 6.06LTS system libraries (under /usr/lib) use 2.1.19. This lead to Zimbra's sasl loading a version of the libraries it wsn't linked against (same problem with postfix) and the link between Postfix and saslauthd was broken.
The fix was painfully simple: re-order the library search path in /etc/ld.so.conf so the zimbra directories are searched first:
Modified (working) /etc/ld.so.confCode:/lib /usr/lib /usr/local/lib/opt/zimbra/lib /opt/zimbra/sleepycat/lib /opt/zimbra/openldap/lib /opt/zimbra/cyrus-sasl/lib
After this modification, I simply refreshed the library cache with "sudo ldconfig" then restarted Zimbra. Sure enough SMTP+TLS+Auth works!Code:/opt/zimbra/lib /opt/zimbra/sleepycat/lib /opt/zimbra/openldap/lib /opt/zimbra/cyrus-sasl/lib /lib /usr/lib /usr/local/lib
This thread has highlighted some problems with the installer on Ubuntu 6.06LTS:
- The default /etc/hosts file is not "Zimbra friendly".
- IPv6 is enabled by default during Ubuntu installation - aparently this should be disabled for Zimbra to function correctly.
- The library search path is wrong resulting in sasl auth failure. This needs to be set either globally (as I did) or specicially at run-time by the Zimbra start scripts
Should I raise bugs for these or do one of the Zimbra staff want to handle this?
Either way, this is SOLVED! Unfortunately there was very little Zimbra support could do in this case unless they had direct access to the box I am using, however, I've tried to be as verbose and precise as possible during the fault-finding and resolution stages to allow them, and others, to adequately diagnose similar problems in future.
Changing the library search order as I have described here will very likely break other applications on the system that use libraries with the same names (eg, sasl2 etc). I ran into problems with Apache unable to load the Zimbra sasl libraries. I worked around it by manually exporting the correct "LD_LIBRARY_PATH=..." at the top of the /etc/init.d/apache2 script. Ideally, the Zimbra developers should use this technique (ie, "export LD_LIBRARY_PATH") in Zimbra's startup scripts to ensure the correct library path, rather than forcing admins to break the bundled OS packages (by fudging the /etc/ld/so.conf file)in order to gain proper functionality of Zimbra under Ubuntu 6.06LTS.
Last edited by Centurion; 11-22-2007 at 07:26 PM. Reason: Added warning about modifying the ld.so.conf file.
Logged a bug to get the installer and/or startup scripts to better handle the library search order: Bug ID 21940