Results 1 to 2 of 2

Thread: DSPAM and zcs 7.x HowTo

Threaded View

  1. #1
    Join Date
    May 2010
    Posts
    272
    Rep Power
    5

    Default DSPAM and zcs 7.x HowTo

    Hello Everyone,

    Heres some additionals to the existing Wiki Entries.

    First of all - Zimbra uses a very very very old Version of Dspam (3.9.0)
    Its strongly recommended to use the actual Snapshop from Git
    Its very stable cause only Core members usually Upload there after testing and its way more recent -

    The following is ubuntubased - please use the script with caution because there no fallbacks - just quick and dirty - or do the content manually

    So what we do is
    Get latest dspam from git
    compile it the right way
    install ist parallel to the existing one
    after that you can choose wheter you just wanna use the cssclean up tools from this version or jsut modify the symlink from the 3.9.0 to the recent one

    please run this as root - we also assume you got zimbra in /opt/zimbra

    Code:
    #let go into roots home
    cd ~
    #get the needed tools
    aptitude install git-core libtool automake autoconf 
    #make our git download dir
    mkdir dspam-src
    cd dspam-src
    git init
    #get the snapshot
    git clone git://dspam.git.sourceforge.net/gitroot/dspam/dspam
    cd ~/dspam-src/dspam
    #start the configure process
    ./autogen.sh; \
    ./configure --prefix=/opt/zimbra/dspam-git \
        --sysconfdir=/opt/zimbra/conf \
        --with-logdir=/opt/zimbra/log \
        --enable-daemon \
        --enable-large-scale --with-storage-driver=hash_drv \
        --with-dspam-home-group=root \
        --with-dspam-group=root \
        --with-dspam-mode=0755 \
        --enable-debug --enable-clamav >config.log
    make >make.log
    check the make.log and config.log for errors - if all fine then do a
    make install

    now we got the latest git version in /opt/zimbra/dspam-git
    if you wanna switch to this version just modify the symlink /opt/zimbra/dspam to this directory

    The hole config is the same as zimbra compiles it into the regular distribution with one exception - we do not compile the mysql driver because we do not use it anyway

    So now its time for config heres a example config you may wanna give a try
    i removed all comments - if you wanna see all options refer dspam manual

    Code:
    ## dspam.conf -- DSPAM configuration file
    ####################################################-----SYSTEM-----####################################
    Home /opt/zimbra/data/dspam
    StorageDriver /opt/zimbra/dspam/lib/dspam/libhash_drv.so
    TrustedDeliveryAgent "/usr/bin/procmail"
    OnFail error
    Trust root
    Trust zimbra
    LocalMX 127.0.0.1
    WebStats off
    SystemLog on
    UserLog   on
    Opt out
    Notifications	off
    ####################################################-----ANALYSE-----####################################
    # Acceptable values are: toe, tum, teft, notrain
    TrainingMode toe
    TestConditionalTraining on
    Feature noise
    #Feature tb=5
    Feature whitelist
    Algorithm graham burton
    Tokenizer sbph
    PValue markov
    ProcessorURLContext on
    ProcessorBias on
    #MaxMessageSize 4194304
    #ImprobabilityDrive on
    #TrainPristine on
    #DataSource      document
    #ProcessorWordFrequency  occurrence
    ####################################################-----PREFERENCES-----####################################
    #Preference "spamAction=quarantine"
    Preference "signatureLocation=headers"	# 'message' or 'headers'
    Preference "showFactors=on"
    Preference "spamAction=tag"
    #Preference "spamSubject=SPAM"
    AllowOverride trainingMode
    AllowOverride spamAction spamSubject
    AllowOverride statisticalSedation
    AllowOverride enableBNR
    AllowOverride enableWhitelist
    AllowOverride signatureLocation
    AllowOverride showFactors
    AllowOverride optIn optOut
    AllowOverride whitelistThreshold
    ####################################################-----DATABASE-----####################################
    HashRecMax            6291469 #we use a big file here to prevent to much extents
    HashAutoExtend		on  
    HashMaxExtents		0  #endless extents
    HashExtentSize        3145739 #use half of hasrecmax
    HashPctIncrease 10
    HashMaxSeek		100
    HashConnectionCache	10
    ####################################################-----MAINTENANCE-----####################################
    PurgeSignatures 14          # Stale signatures
    PurgeNeutral    90          # Tokens with neutralish probabilities
    PurgeUnused     90          # Unused tokens
    PurgeHapaxes    30          # Tokens with less than 5 hits (hapaxes)
    PurgeHits1S	15          # Tokens with only 1 spam hit
    PurgeHits1I	15          # Tokens with only 1 innocent hit
    ####################################################-----IGNOREHEADER-----####################################
    IgnoreHeader X-Spam-Status
    IgnoreHeader X-Spam-Scanned
    IgnoreHeader X-Virus-Scanner-Result
    IgnoreHeader Accept-Language
    IgnoreHeader Approved
    IgnoreHeader Archive
    IgnoreHeader Authentication-Results
    IgnoreHeader Cache-Post-Path
    IgnoreHeader Cancel-Key
    IgnoreHeader Cancel-Lock
    IgnoreHeader Complaints-To
    IgnoreHeader Content-Description
    IgnoreHeader Content-Disposition
    IgnoreHeader Content-ID
    IgnoreHeader Content-Language
    IgnoreHeader Content-Return
    IgnoreHeader Content-Transfer-Encoding
    IgnoreHeader Content-Type
    IgnoreHeader DKIM-Signature
    IgnoreHeader Date
    IgnoreHeader Disposition-Notification-To
    IgnoreHeader DomainKey-Signature
    IgnoreHeader Importance
    IgnoreHeader In-Reply-To
    IgnoreHeader Injection-Info
    IgnoreHeader Lines
    IgnoreHeader List-Archive
    IgnoreHeader List-Help
    IgnoreHeader List-Id
    IgnoreHeader List-Post
    IgnoreHeader List-Subscribe
    IgnoreHeader List-Unsubscribe
    IgnoreHeader Message-ID
    IgnoreHeader Message-Id
    IgnoreHeader NNTP-Posting-Date
    IgnoreHeader NNTP-Posting-Host
    IgnoreHeader Newsgroups
    IgnoreHeader OpenPGP
    IgnoreHeader Organization
    IgnoreHeader Originator
    IgnoreHeader PGP-ID
    IgnoreHeader Path
    IgnoreHeader Received
    IgnoreHeader Received-SPF
    IgnoreHeader References
    IgnoreHeader Reply-To
    IgnoreHeader Resent-Date
    IgnoreHeader Resent-From
    IgnoreHeader Resent-Message-ID
    IgnoreHeader Thread-Index
    IgnoreHeader Thread-Topic
    IgnoreHeader User-Agent
    IgnoreHeader X--MailScanner-SpamCheck
    IgnoreHeader X-AV-Scanned
    IgnoreHeader X-AVAS-Spam-Level
    IgnoreHeader X-AVAS-Spam-Score
    IgnoreHeader X-AVAS-Spam-Status
    IgnoreHeader X-AVAS-Spam-Symbols
    IgnoreHeader X-AVAS-Virus-Status
    IgnoreHeader X-AVK-Virus-Check
    IgnoreHeader X-Abuse
    IgnoreHeader X-Abuse-Contact
    IgnoreHeader X-Abuse-Info
    IgnoreHeader X-Abuse-Management
    IgnoreHeader X-Abuse-To
    IgnoreHeader X-Abuse-and-DMCA-Info
    IgnoreHeader X-Accept-Language
    IgnoreHeader X-Admission-MailScanner-SpamCheck
    IgnoreHeader X-Admission-MailScanner-SpamScore
    IgnoreHeader X-Amavis-Alert
    IgnoreHeader X-Amavis-Hold
    IgnoreHeader X-Amavis-Modified
    IgnoreHeader X-Amavis-OS-Fingerprint
    IgnoreHeader X-Amavis-PenPals
    IgnoreHeader X-Amavis-PolicyBank
    IgnoreHeader X-AntiVirus
    IgnoreHeader X-Antispam
    IgnoreHeader X-Antivirus
    IgnoreHeader X-Antivirus-Scanner
    IgnoreHeader X-Antivirus-Status
    IgnoreHeader X-Archive
    IgnoreHeader X-Assp-Spam-Prob
    IgnoreHeader X-Attention
    IgnoreHeader X-BTI-AntiSpam
    IgnoreHeader X-Barracuda
    IgnoreHeader X-Barracuda-Bayes
    IgnoreHeader X-Barracuda-Spam-Flag
    IgnoreHeader X-Barracuda-Spam-Report
    IgnoreHeader X-Barracuda-Spam-Score
    IgnoreHeader X-Barracuda-Spam-Status
    IgnoreHeader X-Barracuda-Virus-Scanned
    IgnoreHeader X-BeenThere
    IgnoreHeader X-Bogosity
    IgnoreHeader X-Brightmail-Tracker
    IgnoreHeader X-CRM114-CacheID
    IgnoreHeader X-CRM114-Status
    IgnoreHeader X-CRM114-Version
    IgnoreHeader X-CTASD-IP
    IgnoreHeader X-CTASD-RefID
    IgnoreHeader X-CTASD-Sender
    IgnoreHeader X-Cache
    IgnoreHeader X-ClamAntiVirus-Scanner
    IgnoreHeader X-Comment-To
    IgnoreHeader X-Comments
    IgnoreHeader X-Complaints
    IgnoreHeader X-Complaints-Info
    IgnoreHeader X-Complaints-To
    IgnoreHeader X-DKIM
    IgnoreHeader X-DMCA-Complaints-To
    IgnoreHeader X-DMCA-Notifications
    IgnoreHeader X-Despammed-Tracer
    IgnoreHeader X-ELTE-SpamCheck
    IgnoreHeader X-ELTE-SpamCheck-Details
    IgnoreHeader X-ELTE-SpamScore
    IgnoreHeader X-ELTE-SpamVersion
    IgnoreHeader X-ELTE-VirusStatus
    IgnoreHeader X-Enigmail-Supports
    IgnoreHeader X-Enigmail-Version
    IgnoreHeader X-Evolution-Source
    IgnoreHeader X-Extra-Info
    IgnoreHeader X-FSFE-MailScanner
    IgnoreHeader X-FSFE-MailScanner-From
    IgnoreHeader X-Face
    IgnoreHeader X-Fellowship-MailScanner
    IgnoreHeader X-Fellowship-MailScanner-From
    IgnoreHeader X-Forwarded
    IgnoreHeader X-GMX-Antispam
    IgnoreHeader X-GMX-Antivirus
    IgnoreHeader X-GPG-Fingerprint
    IgnoreHeader X-GPG-Key-ID
    IgnoreHeader X-GPS-DegDec
    IgnoreHeader X-GPS-MGRS
    IgnoreHeader X-GWSPAM
    IgnoreHeader X-Gateway
    IgnoreHeader X-Greylist
    IgnoreHeader X-HTMLM
    IgnoreHeader X-HTMLM-Info
    IgnoreHeader X-HTMLM-Score
    IgnoreHeader X-HTTP-Posting-Host
    IgnoreHeader X-HTTP-UserAgent
    IgnoreHeader X-HTTP-Via
    IgnoreHeader X-Headers-End
    IgnoreHeader X-ID
    IgnoreHeader X-IMAIL-SPAM-STATISTICS
    IgnoreHeader X-IMAIL-SPAM-URL-DBL
    IgnoreHeader X-IMAIL-SPAM-VALFROM
    IgnoreHeader X-IMAIL-SPAM-VALHELO
    IgnoreHeader X-IMAIL-SPAM-VALREVDNS
    IgnoreHeader X-Info
    IgnoreHeader X-IronPort-Anti-Spam-Filtered
    IgnoreHeader X-IronPort-Anti-Spam-Result
    IgnoreHeader X-KSV-Antispam
    IgnoreHeader X-Kaspersky-Antivirus
    IgnoreHeader X-MDAV-Processed
    IgnoreHeader X-MDRemoteIP
    IgnoreHeader X-MDaemon-Deliver-To
    IgnoreHeader X-MIE-MailScanner-SpamCheck
    IgnoreHeader X-MIMEOLE
    IgnoreHeader X-MIMETrack
    IgnoreHeader X-MMS-Spam-Filter-ID
    IgnoreHeader X-MS-Has-Attach
    IgnoreHeader X-MS-TNEF-Correlator
    IgnoreHeader X-MSMail-Priority
    IgnoreHeader X-MailScanner
    IgnoreHeader X-MailScanner-Information
    IgnoreHeader X-MailScanner-SpamCheck
    IgnoreHeader X-Mailer
    IgnoreHeader X-Mailman-Version
    IgnoreHeader X-Mlf-Spam-Status
    IgnoreHeader X-NAI-Spam-Checker-Version
    IgnoreHeader X-NAI-Spam-Flag
    IgnoreHeader X-NAI-Spam-Level
    IgnoreHeader X-NAI-Spam-Report
    IgnoreHeader X-NAI-Spam-Route
    IgnoreHeader X-NAI-Spam-Rules
    IgnoreHeader X-NAI-Spam-Score
    IgnoreHeader X-NAI-Spam-Threshold
    IgnoreHeader X-NEWT-spamscore
    IgnoreHeader X-NNTP-Posting-Date
    IgnoreHeader X-NNTP-Posting-Host
    IgnoreHeader X-NetcoreISpam1-ECMScanner
    IgnoreHeader X-NetcoreISpam1-ECMScanner-From
    IgnoreHeader X-NetcoreISpam1-ECMScanner-Information
    IgnoreHeader X-NetcoreISpam1-ECMScanner-SpamCheck
    IgnoreHeader X-NetcoreISpam1-ECMScanner-SpamScore
    IgnoreHeader X-Newsreader
    IgnoreHeader X-Newsserver
    IgnoreHeader X-No-Archive
    IgnoreHeader X-No-Spam
    IgnoreHeader X-OSBF-Lua-Score
    IgnoreHeader X-OWM-SpamCheck
    IgnoreHeader X-OWM-VirusCheck
    IgnoreHeader X-Olypen-Virus
    IgnoreHeader X-Orig-Path
    IgnoreHeader X-OriginalArrivalTime
    IgnoreHeader X-Originating-IP
    IgnoreHeader X-PAA-AntiVirus
    IgnoreHeader X-PAA-AntiVirus-Message
    IgnoreHeader X-PGP-Fingerprint
    IgnoreHeader X-PGP-Hash
    IgnoreHeader X-PGP-ID
    IgnoreHeader X-PGP-Key
    IgnoreHeader X-PGP-Key-Fingerprint
    IgnoreHeader X-PGP-KeyID
    IgnoreHeader X-PGP-Sig
    IgnoreHeader X-PIRONET-NDH-MailScanner-SpamCheck
    IgnoreHeader X-PIRONET-NDH-MailScanner-SpamScore
    IgnoreHeader X-PMX
    IgnoreHeader X-PMX-Version
    IgnoreHeader X-PN-SPAMFiltered
    IgnoreHeader X-Posting-Agent
    IgnoreHeader X-Posting-ID
    IgnoreHeader X-Posting-IP
    IgnoreHeader X-Priority
    IgnoreHeader X-Proofpoint-Spam-Details
    IgnoreHeader X-Qmail-Scanner-1.25st
    IgnoreHeader X-Quarantine-ID
    IgnoreHeader X-RAV-AntiVirus
    IgnoreHeader X-RITmySpam
    IgnoreHeader X-RITmySpam-IP
    IgnoreHeader X-RITmySpam-Spam
    IgnoreHeader X-Rc-Spam
    IgnoreHeader X-Rc-Virus
    IgnoreHeader X-Received-Date
    IgnoreHeader X-RedHat-Spam-Score
    IgnoreHeader X-RedHat-Spam-Warning
    IgnoreHeader X-RegEx
    IgnoreHeader X-RegEx-Score
    IgnoreHeader X-Rocket-Spam
    IgnoreHeader X-SA-GROUP
    IgnoreHeader X-SA-RECEIPTSTATUS
    IgnoreHeader X-STA-NotSpam
    IgnoreHeader X-STA-Spam
    IgnoreHeader X-Scam-grey
    IgnoreHeader X-Scanned-By
    IgnoreHeader X-Sender
    IgnoreHeader X-SenderID
    IgnoreHeader X-Sohu-Antivirus
    IgnoreHeader X-Spam
    IgnoreHeader X-Spam-ASN
    IgnoreHeader X-Spam-Check
    IgnoreHeader X-Spam-Checked-By
    IgnoreHeader X-Spam-Checker
    IgnoreHeader X-Spam-Checker-Version
    IgnoreHeader X-Spam-Clean
    IgnoreHeader X-Spam-DCC
    IgnoreHeader X-Spam-Details
    IgnoreHeader X-Spam-Filter
    IgnoreHeader X-Spam-Filtered
    IgnoreHeader X-Spam-Flag
    IgnoreHeader X-Spam-Level
    IgnoreHeader X-Spam-OrigSender
    IgnoreHeader X-Spam-Pct
    IgnoreHeader X-Spam-Prev-Subject
    IgnoreHeader X-Spam-Processed
    IgnoreHeader X-Spam-Pyzor
    IgnoreHeader X-Spam-Rating
    IgnoreHeader X-Spam-Report
    IgnoreHeader X-Spam-Scanned
    IgnoreHeader X-Spam-Score
    IgnoreHeader X-Spam-Status
    IgnoreHeader X-Spam-Tagged
    IgnoreHeader X-Spam-Tests
    IgnoreHeader X-Spam-Tests-Failed
    IgnoreHeader X-Spam-Virus
    IgnoreHeader X-Spam-Warning
    IgnoreHeader X-Spam-detection-level
    IgnoreHeader X-SpamAssassin-Clean
    IgnoreHeader X-SpamAssassin-Warning
    IgnoreHeader X-SpamBouncer
    IgnoreHeader X-SpamCatcher-Score
    IgnoreHeader X-SpamCop-Checked
    IgnoreHeader X-SpamCop-Disposition
    IgnoreHeader X-SpamCop-Whitelisted
    IgnoreHeader X-SpamDetected
    IgnoreHeader X-SpamInfo
    IgnoreHeader X-SpamPal
    IgnoreHeader X-SpamPal-Timeout
    IgnoreHeader X-SpamReason
    IgnoreHeader X-SpamScore
    IgnoreHeader X-SpamTest-Categories
    IgnoreHeader X-SpamTest-Info
    IgnoreHeader X-SpamTest-Method
    IgnoreHeader X-SpamTest-Status
    IgnoreHeader X-SpamTest-Version
    IgnoreHeader X-Spamadvice
    IgnoreHeader X-Spamarrest-noauth
    IgnoreHeader X-Spamarrest-speedcode
    IgnoreHeader X-Spambayes-Classification
    IgnoreHeader X-Spamcount
    IgnoreHeader X-Spamsensitivity
    IgnoreHeader X-TERRACE-SPAMMARK
    IgnoreHeader X-TERRACE-SPAMRATE
    IgnoreHeader X-TM-AS-Category-Info
    IgnoreHeader X-TM-AS-MatchedID
    IgnoreHeader X-TM-AS-Product-Ver
    IgnoreHeader X-TM-AS-Result
    IgnoreHeader X-TMWD-Spam-Summary
    IgnoreHeader X-TNEFEvaluated
    IgnoreHeader X-Text-Classification
    IgnoreHeader X-Text-Classification-Data
    IgnoreHeader X-Trace
    IgnoreHeader X-UCD-Spam-Score
    IgnoreHeader X-User-Agent
    IgnoreHeader X-User-ID
    IgnoreHeader X-User-System
    IgnoreHeader X-Virus-Check
    IgnoreHeader X-Virus-Checked
    IgnoreHeader X-Virus-Checker-Version
    IgnoreHeader X-Virus-Scan
    IgnoreHeader X-Virus-Scanned
    IgnoreHeader X-Virus-Scanner
    IgnoreHeader X-Virus-Scanner-Result
    IgnoreHeader X-Virus-Status
    IgnoreHeader X-VirusChecked
    IgnoreHeader X-Virusscan
    IgnoreHeader X-WSS-ID
    IgnoreHeader X-WinProxy-AntiVirus
    IgnoreHeader X-WinProxy-AntiVirus-Message
    IgnoreHeader X-Yandex-Forward
    IgnoreHeader X-Yandex-Front
    IgnoreHeader X-Yandex-Spam
    IgnoreHeader X-Yandex-TimeMark
    IgnoreHeader X-cid
    IgnoreHeader X-iHateSpam-Checked
    IgnoreHeader X-iHateSpam-Quarantined
    IgnoreHeader X-policyd-weight
    IgnoreHeader X-purgate
    IgnoreHeader X-purgate-Ad
    IgnoreHeader X-purgate-ID
    IgnoreHeader X-sgxh1
    IgnoreHeader X-to-viruscore
    IgnoreHeader Xref
    IgnoreHeader acceptlanguage
    IgnoreHeader thread-index
    IgnoreHeader x-uscspam
    ## EOF

    So now its time to cleanup things - thanks to "la fong" who made a little script in another thread - i made some slight mods
    i assume we have an /opt/admin directory where zimbra got rwx rights
    we place our script there and also use this dir for script logfiles

    Code:
    #!/bin/bash
    # DSPAM maintenance script. It cleans and compresses the DSPAM hash db.
    MAIL="your@example.com"
    
    # Disable dspam, temporarily
    zmlocalconfig -u amavis_dspam_enabled
    zmamavisdctl restart
    
    # Run cssclean and csscompress 
    /opt/zimbra/dspam/bin/cssclean /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css > /opt/admin/dspam_css_maintenance.log
    echo >> /opt/admin/dspam_css_maintenance.log
    echo Compress >> /opt/admin/dspam_css_maintenance.log
    echo -------- >> /opt/admin/dspam_css_maintenance.log
    /opt/zimbra/dspam/bin/csscompress /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css >> /opt/admin/dspam_css_maintenance.log
    
    # Reenable dspam
    zmlocalconfig -e amavis_dspam_enabled=true
    zmamavisdctl restart
    
    # Run cssstats, and mail results
    echo >> /opt/admin/dspam_css_maintenance.log
    echo Stats >> /opt/admin/dspam_css_maintenance.log
    echo ----- >> /opt/admin/dspam_css_maintenance.log
    /opt/zimbra/dspam/bin/cssstat /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css >> /opt/admin/dspam_css_maintenance.log
    mail -s "Monthly dspam maintenance" "$MAIL" </opt/admin/dspam_css_maintenance.log
    now we edit zimbra users crontab - la fong - recommends one sunday a month - so i post it here

    Code:
    # Monthly maintenance of dspam hash db
    0 4 * * 7  [ $(date +\%e) -le 7 ] && /opt/admin/dspam_css_maintenance.sh
    however i recommend a daily usage because of the way cssclean works


    things is i got 5 years exp with dspam but i always used the mysql driver not the hash however in our case with zimbra i would stick to the hashdriver
    even the tools for it are not that perfect and dspam_clean is unuseable (works only with mysql/postgres/litesql)

    at least we can use now markov and sbph which are fare more sophisticated


    Any suggestions on that topic welcome but please no support requests in this thread - if problems occur please refer the manuals first - then goolge - then forums search and then make a new topic thanks
    Last edited by bofh; 12-11-2011 at 09:52 PM.

Similar Threads

  1. Updating Spamassassin In ZCS 7.1.2
    By LMStone in forum Administrators
    Replies: 7
    Last Post: 05-19-2013, 09:09 AM
  2. Dspam
    By kowell in forum Administrators
    Replies: 9
    Last Post: 09-29-2006, 08:32 AM
  3. ZCS 3.1 Released
    By KevinH in forum Announcements
    Replies: 51
    Last Post: 05-09-2006, 08:54 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •