local mail getting marked as spam?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Tue Aug 12, 2008 4:29 pm

[quote user="dwmtractor"]

[quote user="bjquinn"]

and the IP address of the internet connection they're sending out mail from, which is NOT a mail server, and therefore shouldn't be checked against blacklists.

[/QUOTE]

Not true. In fact an awful lot of spam "out there" comes, not from a normal mail server, but from the SMTP engine of a virus or worm on somebody's DSL connection.

[/QUOTE]

Actually, in the context I'm talking about, this IS true - in my situation, the user is authenticating with my mail server for sending out the email, therefore there are no other email servers in the path, therefore the IP address of the internet connection they're sending out mail from is NOT a mail server.
This, in fact, is my whole point. There's an enormous difference between whether the user has authenticated with my mailserver or not. If not, then the IP of their internet connection is by definition acting as a mailserver; if they have authenticated, then the IP of their internet connection is by definition NOT acting as a mailserver, and therefore should not have been checked against a blacklist. It's not forbidden to send mail from a blacklisted IP (especially the huge swath of swbell's DSL dynamic address space, which as you mentioned I believe the IP in question belongs to), it's only forbidden to run a mailserver on a blacklisted IP.
[quote user="dwmtractor"]

Now as I said before, there is a valid argument in your contention that regardless of IP source, if your user is properly authenticated Zimbra should allow the mail thru. I understand the argument and I'm really not trying to argue against you. But it's obviously not what's happening now

[/QUOTE]
Actually, I'm pretty sure that's what's happening now. What leads you to believe that it isn't? As you suggested, the user's public IP is in a blacklisted dynamic DSL range. Zimbra checks the email, considers that IP as acting as a mailserver (which it is NOT), and tacks several points on to the spam score. Am I missing something?
[quote user="dwmtractor"]

And my previous point remains true--if you are getting blocked internally, you can bet that the same user's emails, under similar circumstances, will be blocked externally by others who use SORBS. So IMHO Zimbra is doing you a favor in helping you to identify and fix the problem before an upset customer doesn't get the communication he's expecting.

[/QUOTE]
No, because the other mailservers out there will see my mail as coming from my mailserver's IP, which is NOT blacklisted. This isn't a problem for outgoing mail, just internal mail.
[quote user="dwmtractor"]

Perhaps what you're really asking is that Zimbra would, in cases of authenticated SMTP, strip out all prior headers and send out the message as though originating from the Zimbra IP itself. I don't know if that's consistent with the SMTP standards, so I can't even tell you if it's permissible or not--others will have to do that. But it would solve the problem.[/QUOTE]
Not really. Since other mailservers other than Zimbra likely don't have this problem (and as far as I can tell it's still a problem only for internal mail within that Zimbra server itself, not even between Zimbra servers as I've tested it). All I want is for Zimbra to indicate to SpamAssassin that an authenticated local user's emails can either be skipped for spam checking or certain rules for certain headers could be skipped. This is such a fundamental issue that I'm going to trust that other MTA software out there doesn't have this problem, or no mobile user could ever send an email to anyone. Their hotel or airport or Starbucks internet connection would always look like a spammer IP and nobody would ever receive any email they ever sent. I imagine that the default action is to put the magnifying glass on the last IP in the chain before the email's destination (except, of course, for authenticated users, who get a pass, at least in theory).
Thanks for your help. This has been a strange one!


bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Thu Aug 21, 2008 4:13 pm

Given my last post in response to your last post, do you think this is worthy of filing a bug or do you have some more questions or suggestions?
I do appreciate your help.
dwmtractor
Outstanding Member
Outstanding Member
Posts: 993
Joined: Fri Sep 12, 2014 10:41 pm

local mail getting marked as spam?

Postby dwmtractor » Mon Aug 25, 2008 10:30 am

[quote]GIVEN MY LAST POST IN RESPONSE TO YOUR LAST POST, DO YOU THINK THIS IS WORTHY OF FILING A BUG OR DO YOU HAVE SOME MORE QUESTIONS OR SUGGESTIONS?
I DO APPRECIATE YOUR HELP.[/QUOTE]

WELL, AS I SAID BEFORE, I DON'T KNOW THE SMTP STANDARDS WELL ENOUGH TO KNOW IF A STANDARDS-COMPLIANT MAILSERVER COULD DO SO, BUT I NOW SEE THE DETAILS OF WHAT YOU DESIRE--EITHER:



  • SIMPLY TELL AMAVISD (OR AT LEAST THE SPAM PORTION OF AMAVISD; MIGHT WANT TO LEAVE ANTIVIRUS ACTIVE) TO SKIP SCANNING FOR AUTHENTICATED USERS ON OUTGOING MAIL (PREFERRED) OR

  • FOR AUTHENTICATED EXTERNAL SMTP USERS ONLY, REMOVE HEADER INFORMATION ON THE CLIENT THAT SENDS THE MAIL TO ZIMBRA (WHICH YOU DO NOT PREFER, BUT WHICH WOULD SOLVE THE PROBLEM)
AGAIN, SINCE I'M NEITHER THE PROGRAMMING NOR STANDARDS-COMPLIANCE EXPERT, I CAN'T RECOMMEND ONE WAY OR THE OTHER WITH ANYTHING APPROXIMATING INTELLIGENCE, BUT I THINK IT'D BE REASONABLE TO FILE A RFE ON BUGZILLA WITH THOSE OPTIONS. BE SURE TO REFERENCE THE DISCUSSION IN THIS THREAD IN THE RFE, AND WHEN YOU'RE DONE, COME BACK AND POST THE BUGZILLA NUMBER SO FOLKS CAN VOTE ON IT.
bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Thu Sep 04, 2008 4:52 pm

I've taken your advice and filed a bug. For anyone who's interested, it's at Bug 31333 – Emails from authenticated users getting marked as spam.
Thanks Dan!
bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Mon Sep 29, 2008 11:51 am

I got a response from Bill Hwang on the bug report saying -
"wiki has detailed discussions"
Does anyone know what wiki article he's talking about? I've searched and I can't find it. I posted a reply in the bug comments asking that question a few weeks ago and didn't get a response.
This bug is really causing us trouble. I'm fine waiting for a true fix, but there's got to be some sort of workaround!
bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Tue Dec 23, 2008 5:16 pm

Well, I never did hear back from Bill, but this problem is really starting to hurt.
Not only am I losing email sent by legitimate local users (come on, Zimbra already knows who they are!), but in order to minimize the false positives I've had to increase my kill/mark spam scores, so ironically I'm also having to deal with higher levels of actual spam.
Has anyone ever found a solution to this?
dwmtractor
Outstanding Member
Outstanding Member
Posts: 993
Joined: Fri Sep 12, 2014 10:41 pm

local mail getting marked as spam?

Postby dwmtractor » Tue Dec 23, 2008 5:51 pm

[quote user="bjquinn"]Well, I never did hear back from Bill, but this problem is really starting to hurt.
Not only am I losing email sent by legitimate local users (come on, Zimbra already knows who they are!), but in order to minimize the false positives I've had to increase my kill/mark spam scores, so ironically I'm also having to deal with higher levels of actual spam.
Has anyone ever found a solution to this?[/quote]

I thought I remembered you saying you had whitelisted mail "from" your domain. We all agree that is only a stopgap measure given the fact that spammers sometimes spoof "from" to be the target domain, but if you did this you shouldn't be losing legitimate internal mail.
You also pointed out some time ago that only through the web client could you train spam/not spam in the Bayesian filter. With the advent of 5.0.11 that is no longer true, dragging mail out of the junk folder to the inbox now trains it as "not spam."
I'm still a little nonplussed that you are having this issue at all, since I routinely send email on the road from my laptop (and Verizon phone) and I find it difficult to believe that I've been lucky enough never to land in a blacklisted ip range such as your users have. But accepting that this is happening to you, have you considered going into your spamassassin scores and simply lowering the score for dynamic IP so that this one filter criterion gets either very little or no spam score? Again, it would wind up being true for ALL mail sent from dynamic IPs--which would include a lot of true spam, but it would certainly beat adjusting your systemwide tag & kill percentages.
I can't speak to the issue of your bug being addressed. I don't know the programming schedule and am no more of an insider than you. But I still think we should be able to work around this and get you a functioning system.
17314rfoster
Posts: 25
Joined: Fri Sep 12, 2014 10:42 pm

local mail getting marked as spam?

Postby 17314rfoster » Fri Dec 26, 2008 11:20 am

How about disabling the Pyzor check for the time being? I am having pretty good results without it. Sounds like it's doing you more harm than good.
Just my quick observation after looking at this thread. Good luck!
Bob
bjquinn
Advanced member
Advanced member
Posts: 179
Joined: Fri Sep 12, 2014 10:00 pm

local mail getting marked as spam?

Postby bjquinn » Mon Dec 29, 2008 11:06 am

Yes, I've tried changing various SA rule values, whitelisting my own domain, etc., but of course that has the effect of increasing the amount of spam I'm getting. I'm getting pressure from both sides - "there's too much spam" and "why does my local mail get marked as spam". Maybe you haven't noticed it because your spam tag percent (mine is 20) isn't quite as tight as ours is, allowing some of the potential situations where this would occur to fly under the radar.
At any rate, I'm not hearing back anything on bugzilla. I considered just trying an upgrade for the heck of it, but I think I'll wait out 6.0.
dwmtractor
Outstanding Member
Outstanding Member
Posts: 993
Joined: Fri Sep 12, 2014 10:41 pm

local mail getting marked as spam?

Postby dwmtractor » Mon Dec 29, 2008 12:00 pm

[quote user="bjquinn"]Maybe you haven't noticed it because your spam tag percent (mine is 20) isn't quite as tight as ours is, allowing some of the potential situations where this would occur to fly under the radar.[/quote]

Actually I tag at 15 so if this were the problem I'd see it worse than you do. . .:D
Rather than adjust the tag, I would suggest specifically overriding ONLY the scores that are creating a problem for you while leaving the rest of SpamAssassin alone (I run a pretty tight filter myself and I'm a strong advocate for the same). In your case, here is what you originally posted as your problem scores:

[QUOTE]X-Spam-Status: Yes, score=6.581 tagged_above=-10 required=4 tests=[AWL=-4.156,

BAYES_05=-1.11, DCC_CHECK=2.17, DIGEST_MULTIPLE=0.765,

DYN_RDNS_SHORT_HELO_HTML=0.499, HTML_90_100=0.113, HTML_MESSAGE=0.001,

MIME_HTML_MOSTLY=1.102, PYZOR_CHECK=3.7, RCVD_IN_SORBS_DUL=2.046,

RDNS_DYNAMIC=0.1, TVD_RCVD_SINGLE=1.351][/QUOTE]

So let's fix the specific filters that are causing you grief. As rfoster mentioned above you could just turn off Pyzor (or SORBS) and that alone would significantly trim your score. If you do not wish to do that, you could at least reduce the scores for the specific hits your mail is getting. For example, you could add the following lines to /opt/zimbra/conf/spamassassin/local.cf:

score RCVD_IN_SORBS_DUL 0.1

score PYZOR_CHECK=1.0



With a tag percent of 20 (score for tag >=4) those two changes would allow that message through as it has a strong AWL balancing code (I actually don't use AWL and I still have virtually no false positives).
The beauty of making your tweaks in local.cf is that you can back up that one file and preserve your tweaks across version upgrades, etc. without screwing around with all the other files. It allows you to target very specific scores and either raise or lower them, all the while preserving the basic configuration supplied with the system.
Again I do not dispute the value of whitelisting authenticated users as you have suggested, but that'll take coding and this should be fixable faster than that. Seems to me to be worth a try. . .

Return to “Administrators”

Who is online

Users browsing this forum: No registered users and 21 guests