amavis email notification

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
zgokan
Advanced member
Advanced member
Posts: 112
Joined: Sun Apr 17, 2016 8:58 am

amavis email notification

Postby zgokan » Mon Aug 31, 2020 3:35 pm

Hello

Can I get notifications of rejected emails ?

Thanks.


User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 521
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 8.7.11_P14 RHEL6 Network Edition
Contact:

Re: amavis email notification

Postby JDunphy » Mon Aug 31, 2020 5:32 pm

This might help... I use it to look at spam, ham, and discard. I also print the rules that fired to see why classification was working or not working. It was quick and dirty at the time 4-5 years ago so do not know if the log formats have changed. I still use it on 8.7.11 without problems.

Code: Select all

#!/usr/bin/perl

#
# Zimbra Assumptions:
# Amavis at level 3 logging to see spam_scan lines in /var/log/zimbra.log to parse:
#   % zmprov ms `zmhostname` zimbraAmavisLogLevel 3
#   % zmantispamctl restart
#

use Data::Dumper qw(Dumper);
use Getopt::Long;

%Email_list = ();  #ip list
%SA_Rules_list = ();   #failed ip list
$audit_log = 0;   #todays logging

sub usage {

print <<"END";
usage: % check_spam.pl
      [--user=<username>]
      [--ham|h ]
      [--spam|s ]
      [--discard|d ]
      [--rules|r ]
      [--option|o ]
    requires one of
       --ham | --spam | --discard
    where
       --ham will display only ham
       --spam will display only spam
       --discard will display not delivered email due to scoring
       --rules DO NOT display SA rules that fired
       --user will display only email destined for that user
END
  exit 0;
}

#defaults
my $srchuser = '@';
my $ham = 0;
my $spam = 0;
my $discard = 0;
my $rules = 0;
my $help = 0;
my $dcount=0;
my $scount=0;
my $hcount=0;
my $tcount=0;

&GetOptions( "user=s" => \$srchuser,
              "ham" => \$ham,     # display ham
              "discard" => \$discard,  # display discarded not delivered email
              "rules" => \$rules, # display SA rules
              "spam" => \$spam,   # display spam
              "options" => \$help);

print "user is $srchuser rules[$rules] ham[$ham] spam[$spam] discard[$discard]\n";
my $nodisplayoptions=$ham + $spam + $discard;
usage() if($help || !$nodisplayoptions);

chdir "/var/log";

#for (glob 'zimbra.log*') {
for (glob 'zimbra.log') {

  # audit.log is always todays stuff
  #print "***** Opening file $_","\n";
  if ($_ eq 'zimbra.log')
  {
     $audit_log = 1;
     open (IN, sprintf("cat %s |", $_))
       or die("Can't open pipe from command 'zcat $filename' : $!\n");
  }
  else
  {
     $audit_log = 0;
     open (IN, sprintf("zcat %s |", $_))
       or die("Can't open pipe from command 'zcat $filename' : $!\n");
  }

my $score=0;
my $tests="";
my $flag=0;

  while (<IN>)
  {
   # Available when in level 3 logging
   if (m#spam_scan#)
   {
      #print $_;
      ($score,$tests) = m#\s+score=(-?\d+\.?\d*).*tests=\[(.*)\]\s*#i;
      #print " - score is $score, tests is $tests \n";

      # %%% spam_scan can be consequtive given this is multi-threaded writes from the amavisd's.
                #  resulting in lost records.
      #if ($flag) {print " - score is $score, tests is $tests \n";}
      $flag=1;

   }
   # Always available
        # Discarded spam
   elsif (m#DiscardedInbound# && ($flag == 1) && (m#Blocked#))
   {
      #print " - score is $score, tests is $tests \n";
      my($from,$to,$hits,$size) = m#[^<]+<([^>]+)>[^<]+<([^>]+)\>.*Hits:\s*(\d+\.?\d*),\s*size:\s+(.*)$#i;

                #by user
                next if(index($to,$srchuser) == -1);
      next if (!$discard);

      # Sanity check for working on same record
      if ($hits != $score) { next; }

      printf ("Score [%6s] To: %s From: %s\n", $score, $to, $from);
      printf ("      %s\n\n", $tests) if (!$rules);

      # reset, and look for next spam_scan line
      $score=0;
      $tests="";
      $flag=0;
      $dcount++;
   }
        # Ham
   elsif (m#spam-tag# && ($flag == 1) && (m#No#))
   {
      #print " - score is $score, tests is $tests \n";
      my($from,$to,$hits) = m#spam-tag,\s+\<+([^>]+)\>+\s+-\>\s+\<+([^>]+)\>+,\s+No,\s+score=(-?\d+\.?\d*)\s+.*$#i;

                #by user
                next if(index($to,$srchuser) == -1);
      next if (!$ham);

      # Sanity check for working on same record
      if ($hits != $score) { next; }

      #print $_;

      printf ("Score [%6s] To: %s From: %s\n", $score, $to, $from);
      printf ("      %s\n\n", $tests) if (!$rules);

      # reset, and look for next spam_scan line
      $score=0;
      $tests="";
      $flag=0;
      $hcount++;
   }
   # Spam but not discarded
   elsif (m#spam-tag# && ($flag == 1) && (m#Yes#))
   {
      #print " - score is $score, tests is $tests \n";
      my($from,$to,$hits) = m#spam-tag,\s+\<+([^>]+)\>+\s+-\>\s+\<+([^>]+)\>+,\s+Yes,\s+score=(-?\d+\.?\d*)\s+.*$#i;

                #by user
                next if(index($to,$srchuser) == -1);
      next if (!$spam);

      # Sanity check for working on same record
      if ($hits != $score) { next; }

      #print $_;

      printf ("Score [%6s] To: %s From: %s\n", $score, $to, $from);
      printf ("      %s\n\n", $tests) if (!$rules);

      # reset, and look for next spam_scan line
      $score=0;
      $tests="";
      $flag=0;
      $scount++;
   }
  }
  close (IN);

}

$tcount += $dcount if ($discard);
$tcount += $scount if ($spam);
$tcount += $hcount if ($ham);

printf ("\nTotal counts: $tcount");
printf (" Discarded Email: $dcount") if ($discard);
printf (" Spam Email: $scount") if ($spam);
printf (" Ham Email: $hcount") if ($ham);
printf ("\n");

Output looks something like this:

Code: Select all

$ check_rejected_spam.pl --discard
user is @ rules[0] ham[0] spam[0] discard[1]
Score [47.662] To: j@example.com From: leah.freemyer-j=example.com@frizzonaire.monster
      BAYES_99=4,BAYES_999=0.2,BLACKLIST_COUNTRY=2.5,BL_ZEN_SPAMHAUS=1,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FROM_FMBLA_NEWDOM14=0.999,HTML_FONT_LOW_CONTRAST=0.1,HTML_MESSAGE=0.001,HTTP_IN_BODY=0.1,J_BELOW_FOLD=1.5,J_BL_IVMURI=4,J_BL_ZEN_SPAMHAUS=3,J_COUNTRY_URI_SPAM=0.5,J_HIDDEN_WORDS_CNT_400=2.5,J_IBM_BL=0.5,J_IMG_NO_EXTENS=0.1,J_OBFUSCATED_URL=2.5,J_SNEAKY_TRACKER=1.5,J_TRACKING_SPAM=2,J_TRACKING_SPAM0=2,J_TRACKING_SPAM1=0.5,J_URI_DOMAIN_BAD=0.1,J_URI_DOMAIN_TLD=2.5,MIME_HTML_ONLY=0.1,MIME_HTML_ONLY_MULTI=0.001,MIME_QP_LONG_LINE=0.001,MPART_ALT_DIFF=0.79,PDS_RDNS_DYNAMIC_FP=0.001,RCVD_IN_IVMSIP=3,RCVD_IN_IVMSIP24=2,RCVD_IN_SBL_CSS=3.335,RDNS_DYNAMIC=0.982,SPF_HELO_NONE=0.001,URIBL_ABUSE_SURBL=1.25,URIBL_BLACK=1.7,URIBL_DBL_SPAM=2.5,URIBL_IVMURI=0.001

Score [45.134] To: j@example.com From: cxjtbags@126.com
      BAYES_99=4,BLACKLIST_COUNTRY=2.5,BL_BARRACUDA=1,BL_ZEN_SPAMHAUS=1,BODY_8BITS=0.1,CHARSET_FARAWAY=3.2,CHARSET_FARAWAY_HEADER=3.2,DMARC_FAIL_NONE=1.2,FREEMAIL_FROM=0.001,HK_RANDOM_ENVFROM=0.001,HK_RANDOM_FROM=0.999,J_BL_BARRACUDA=3,J_BL_ZEN_SPAMHAUS=3,J_FOREIGN_SORBS_1=2,J_IBM_BL=0.5,J_RCVD_IN_HOSTKARMA_BL=0.002,J_RCVD_IN_TRUNCATE=2,J_SORBS_BL=0.1,J_UNICODE_CHECK_SUBJ=1,KHOP_HELO_FCRDNS=0.399,MAY_BE_FORGED=0.001,MIME_CHARSET_FARAWAY=2,NO_FM_NAME_IP_HOSTN=0.001,PDS_RDNS_DYNAMIC_FP=0.001,RCVD_IN_IVMSIP=3,RCVD_IN_IVMSIP24=2,RCVD_IN_PBL=3.335,RCVD_IN_XBL=0.375,RDNS_DYNAMIC=0.982,SPF_HELO_FAIL=0.001,SPOOFED_FREEMAIL=1.321,TO_NO_BRKTS_DYNIP=1.629,TVD_SPACE_ENCODED=0.845,TVD_SPACE_RATIO_MINFP=0.441

Score [36.513] To: j@example.com From: noreply-J=EXAMPLE.COM@amperrush.eu
      BAYES_99=4,BAYES_999=0.2,BLACKLIST_COUNTRY=2.5,BL_ZEN_SPAMHAUS=1,DKIM_INVALID=0.1,DKIM_SIGNED=0.1,DMARC_FAIL_NONE=1.2,HTML_IMAGE_RATIO_04=0.001,HTML_MESSAGE=0.001,HTTP_IN_BODY=0.1,J_BL_IVMURI=4,J_BL_ZEN_SPAMHAUS=3,J_DOMAIN_SPAM_TLD=2.5,J_IMG_NO_EXTENS=0.1,J_TRACKING_SPAM0=2,J_TRACKING_SPAM1=0.5,J_TRACKING_SPAM2=3,J_URI_DOMAIN_BAD=0.1,J_URI_DOMAIN_TLD=2.5,MIME_BOUND_DIGITS_15=0.1,RCVD_IN_IVMSIP24=2,RCVD_IN_SBL_CSS=3.335,SPF_HELO_NONE=0.001,URIBL_ABUSE_SURBL=1.25,URIBL_DBL_SPAM=2.5,URIBL_GREY=0.424,URIBL_IVMURI=0.001

Score [42.883] To: j@example.com From: 8364-332-183550-2580-j=example.com@mail.assofter.us
      BAYES_99=4,BAYES_999=0.2,BL_ZEN_SPAMHAUS=1,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,HTML_FONT_LOW_CONTRAST=0.1,HTML_MESSAGE=0.001,HTTP_IN_BODY=0.1,J_BELOW_FOLD=1.5,J_BL_IVMURI=4,J_BL_ZEN_SPAMHAUS=3,J_DNSBL_MILTER_META=0.3,J_DOCTYPE_MISSING=0.5,J_HIDDEN_WORDS_CNT_200=2,J_IBM_BL=0.5,J_IMG_NO_EXTENS=0.1,J_RCVD_IN_TRUNCATE=2,J_SPAMMY_TRACKER=1,J_TRACKING_SPAM=2,J_TRACKING_SPAM0=2,KAM_INFOUSMEBIZ=0.5,MIME_HTML_MOSTLY=0.1,MPART_ALT_DIFF=0.79,RCVD_IN_IVMSIP=3,RCVD_IN_IVMSIP24=2,RCVD_IN_MSPIKE_BL=0.001,RCVD_IN_MSPIKE_L5=0.001,RCVD_IN_PSBL=2.7,RCVD_IN_SBL_CSS=3.335,RDNS_NONE=0.793,SPF_HELO_NONE=0.001,T_REMOTE_IMAGE=0.01,URIBL_ABUSE_SURBL=1.25,URIBL_BLACK=1.7,URIBL_DBL_SPAM=2.5,URIBL_IVMURI=0.001

Score [31.127] To: a@example.com From: ethan.foster-a=example.com@mattomer.cyou
      BAYES_99=4,BAYES_999=0.2,BLACKLIST_COUNTRY=2.5,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FROM_FMBLA_NEWDOM14=0.999,HTML_FONT_LOW_CONTRAST=0.1,HTML_FONT_SIZE_LARGE=0.001,HTML_MESSAGE=0.001,HTTP_IN_BODY=0.1,J_BELOW_FOLD=1.5,J_HIDDEN_WORDS_CNT_400=2.5,J_IMG_NO_EXTENS=0.1,J_OBFUSCATED_URL=2.5,J_SNEAKY_TRACKER=1.5,J_TRACKING_SPAM0=2,J_TRACKING_SPAM1=0.5,J_URI_DOMAIN_BAD=0.1,J_URI_DOMAIN_TLD=2.5,MIME_HTML_ONLY=0.1,MIME_HTML_ONLY_MULTI=0.001,MIME_QP_LONG_LINE=0.001,MPART_ALT_DIFF=0.79,PDS_RDNS_DYNAMIC_FP=0.001,RCVD_IN_IVMSIP24=2,RDNS_DYNAMIC=0.982,SPF_HELO_NONE=0.001,UNWANTED_LANGUAGE_BODY=5,URIBL_ABUSE_SURBL=1.25

Score [35.327] To: b@example.com From: anthony_ross-b=example.com@mattomer.cyou
      BAYES_99=4,BAYES_999=0.2,BLACKLIST_COUNTRY=2.5,DKIM_SIGNED=0.1,DKIM_VALID=-0.1,DKIM_VALID_AU=-0.1,FROM_FMBLA_NEWDOM14=0.999,HTML_FONT_LOW_CONTRAST=0.1,HTML_FONT_SIZE_LARGE=0.001,HTML_MESSAGE=0.001,HTTP_IN_BODY=0.1,J_BELOW_FOLD=1.5,J_COUNTRY_URI_SPAM=0.5,J_HIDDEN_WORDS_CNT_400=2.5,J_IMG_NO_EXTENS=0.1,J_OBFUSCATED_URL=2.5,J_SNEAKY_TRACKER=1.5,J_TRACKING_SPAM=2,J_TRACKING_SPAM0=2,J_TRACKING_SPAM1=0.5,J_URI_DOMAIN_BAD=0.1,J_URI_DOMAIN_TLD=2.5,MIME_HTML_ONLY=0.1,MIME_HTML_ONLY_MULTI=0.001,MIME_QP_LONG_LINE=0.001,MPART_ALT_DIFF=0.79,PDS_RDNS_DYNAMIC_FP=0.001,RCVD_IN_IVMSIP24=2,RDNS_DYNAMIC=0.982,SPF_HELO_NONE=0.001,UNWANTED_LANGUAGE_BODY=5,URIBL_ABUSE_SURBL=1.25,URIBL_BLACK=1.7
...
...
Total counts: 55 Discarded Email: 55

If you don't want the rules that fired... add the --rules to the command line. Note: Rules output require that you enable extra logging information. See assumptions and the commands to do this at the top of the perl script. Also note... you can have all the logs or just a single log. See code with glob to find line you want.

HTH,

Jim
zgokan
Advanced member
Advanced member
Posts: 112
Joined: Sun Apr 17, 2016 8:58 am

Re: amavis email notification

Postby zgokan » Sat Sep 05, 2020 2:19 pm

many thanks, very helpful.

Return to “Administrators”

Who is online

Users browsing this forum: No registered users and 20 guests