Hello,

This is my implementation plan for a major upgrade I'm doing to a company's e-mail server. I haven't gone through with it yet, and plan on doing it tomorrow, but I'd like to know what you think. If it works out well, and contains best practices, I'll see if I can post this somewhere in more detail, maybe on a wiki, for anyone who ever has to go through a scenario anything like this.

Anyways, I've inherited a situation with a company with a POP3 server used for mail, apparently set up on qpopper and running what seems to be inherently open relays, running on an old FreeBSD box, with Vexira doing anti-virus work.

Since this is POP3 only, the user's mailboxes were not centralized -- they're scattered all over the place, unsychronized, on mailboxes on the network, on the workstations, on laptops, at people's houses, and so forth.

Oddly enough for the situation, this place has about 100 users and has somehow managed to work ok, minus numerous inconveniences, drag on the network, and massive use of storage space. After all, a 10MB file, base64 encoded to 20MB, and sent out to 100 different mailboxes, does take up 2GB under this sort of system.

So, I have been assigned to recreate the mail server there to solve these problems and centralize their mail. I was provided a Dell Poweredge SC430 with a Pentium D 2.8, 2GB DDR533, and 2x320GB 7200 16MB cache hard drives that I set up in RAID1.

I was planning on a typical old IMAP/Squirrelmail server, but however, looking around the net for what was new with open source mail, I found Zimbra. It floored me with its features, even though it would meet my original needs perfectly. I was aiming just for centralized mail initially, but it opens up the possibilities of more features and considerations of the commercial version in the future. Plus the install was much easier than the typical install of those servers, which is about as much fun as, say, a by-hand install of gentoo.

Since they had some Debian boxes around for a samba server and some other things, plus they won't pay for RHEL, I set up the open source version on a custom version of Debian Sarge 3.1 with kernel 2.6.14-i386-smp upgraded to 2.6.17-i386-smp. I had to choose i386 because of zimbra support, even though Pentium D's are 64-bit chips.

I could post more details of how I got it running, but if you're in the same situation, google for Poweredge SC430, iso, and Debian.

My additional requirements for this plan were:

* Minimum downtime. Certainly no more than 4 hours.
* High levels of security.
* Remove all the mailboxes from the network file server.
* Their old mail gets copied over to the new server.
* Web interface is optional, but not to be a new requirement.
* They can be able to access their mail by Mozilla, Thunderbird, or in some instances, Eudora.

So, I started reading all of the forums and wikis here deciding how to set this changeover up.

Situation Details

How it's currently set up involves numerous mozilla profiles stored in home folders on a network file server, containing their settings and mail. However, a number of them are in non-standard network folders, and a number of them are also on their hard drives -- or even worse, half and half, meaning profile on the network and mailboxes on the hard drive, or vice-versa. All they know is that they click Mozilla or Thunderbird, and they get their mail, even if in the background 10MB mails are multiplying into 1-2GB worth of data when sent to every user.

So, this means no neat automatic scripts. No imapsync, no nothing. Not all the computers are on the windows domain, so I couldn't rig anything consistently in that manner, either.

However, I was, however, at least able to find some of the locations of their mailboxes on the network. I decided against using mbox utilities like zimbra's inject utility (zmlmptinject?) to split them up, since they didn't seem as reliable, and it would likely take me time to script it properly. Plus people can't be expected to compact their mbox'es, despite my pleas for even basic cleanup, meaning they could end up moving over all kinds of deleted stuff, unless I'd have a way of reindexing them by referencing mozilla's msf indexes.

That means, basically, creating new profiles on their desktop, creating dummy accounts to load up their old mailboxes, then dragging and dropping them from their old folders to the mail server with their mail client. If anyone knows a better way in this situation, please tell me now -- I set aside an entire day on the weekend when the office will be out out to go desktop-to-desktop, though I have 2-3 people to help me.

I hope I didn't miss anything that would prevent that, though I did RTFM as much as possible.

Nevertheless, I wrote a throwaway-code batch file script to set up a new account for them that sets them up to point both to their old account's mail and their local folders for the ones I could find on the network. For mozilla, all you have to do to script a new profile is create a directory, write a prefs.js file in it, and use it as your profile directory. I prototyped it and put in variables based on dos environment variables like %username%. This will allow for some profiles to be auto-created, though I do have procedures for setting up by hand when it fails.

As far as another automation thing goes, I was able to pull all their data from their database system, however, to use that to write a script I could dump in zmprov commands with on the server set up side. This script saves tons of time and makes the install of zimbra practically instantaneous.

I just used php to echo the strings out after reading them from a database to their zimbra-equivalent LDAP formats. The end result was something like this:

Code:
# creating and modifying accounts
ca user@mailserver.com changeme1234
ma user@mailserver.com displayName "Joe User" givenName "Joe" sn "User" ou "Department Name" title "Job Title" telephoneNumber "Phone Number" company "Company Name" physicalDeliveryOfficeName "Office Number" street "Street Name" l "City Name" st "TN" postalcode "12345" co "USA" zimbraPasswordMustChange "TRUE"
# aliases
aaa user@mailserver.com joeuser@mailserver.com
# distribution lists
cdl funnieslist@mailserver.com
adlm funnieslist@mailserver.com joeuser@mailserver.com
adlm funnieslist@mailserver.com janeuser@mailserver.com
I was also able to write mozilla vcards with the same code, which I put in the generated prefs.js files. I used some PHP code that pulled from their data, similar to this, to generate them.

PHP Code:
    $vc[$accountname] = array("begin" => "vcard",
                              
"fn" => $name,
                              
"n" => "$lastname;$firstname",
                              
"org" => $companyname ";" $departmentname,
                              
"adr" => ';;' $address ';' $city ';' $state ';' $zip ';USA',
                              
'email;internet' => $accountname,
                              
'title' => $title,
                              
'tel;work' => $phone " x" $extension,
                              
'url' => $companyurl,
                              
'version' => '2.1',
                              
'end' => 'vcard');

    foreach (
$vc as $k=>$v) {
        echo 
"\n<br>";
        foreach (
$v as $k2=>$v2) {
          echo 
rawurlencode("$k2:$v2\n");
        }
    } 
Currently, I also have this all running on a test server for testing various things. I never found a clean way to change the domain name, so I can't do the porting first and then deploy.

So, my overall implementation plan is something like:

* Put the new mail server on the KVM switch
* Change the network settings of the old mail server to put it aside.
* Change the network settings of the new mail server to replace the old mail server.
* Uninstall the test version of zimbra with
o ./install.sh -u
* Reinstall version of zimbra and set it up in place of the new mail server with
o ./install.sh
* Change all the ports except smtp and 7071 (I couldn't find how to change this) from their default for paranoia purposes.
* (note: someone should not allow the installer to quit until an admin pass is entered. otherwise, it seems like it fails)
* Use a hacked up script from http://easyfwgen.morizot.net/gen/ to make sure no traffic goes to anything but them and ssh (also on a non-default port).
* Run my huge zmprov script to recreate the users, aliases, and distribution lists.
o sudo -u zimbra /opt/zimbra/bin/zmprov < /root/zmprov.txt
* Check to see if it's running and log in as the admin user
* https://mailserver.com:7071/
* Global Settings -> Ban all attachments save for jpg and mov, add zip, enable spam filtering, and do anti-virus updates hourly.
* Turn off logging services to speed up the transfer.
* Restart zimbra:
o sudo -u zimbra /opt/zimbra/bin/zmcontrol stop
# ps ax and kill any zimbra processes
sudo -u zimbra /opt/zimbra/bin/zmcontrol start
* ** Go from workstation to workstation, running my mozilla profile setup script, and once their mozilla profiles are set up, copy by hand from their old mailboxes to the zimbra server.
* Restart logging services
* Restart zimbra as in above.
* Fetchmail or otherwise import the undownloaded mail off the old mail server's IP that might have been sitting there. Worst case scenario, I re-import this by hand.

Sounds like a nightmare scenario, eh? At least in the workstation to workstation part. Well, Zimbra sounds like the cure. I really can't wait to move them off their current setup!

The only problem I've had, by the way, is that it shows the copied-to-server date under Zimbra instead of the Sent Date, though the real sent date remains intact, after copying from Thunderbird. Are there any plans to use the sent date from the message instead in the future, or ways to do this currently? The only thing I ever saw for it was a flag with imapsync, which unfortunately cannot apply here.

If you can make any suggestions, or find anything I'm doing flagrantly wrong, please let me know.

Thanks,

-Peter