This arises out of some bugzilla comments; instead of cluttering up bugzilla, I thought it'd be better to turn it into a forum thread for further discussion, if any.
Basically in Bug 9532 - IMAP/Outlook move to junk doesn't train anti-spam and Bug 37164 - mail filed into Junk by Filters is not used to train anti-spam I raised the question of whether Zimbra ought to train SpamAssassin on messages which have been auto-filed into Junk by a user's Filter.
Currently Zimbra doesn't do this directly, but the capability exists in a sort of roundabout way. E.g., if you have Outlook or an IMAP mail client, you can have it move messages to Junk based on keywords, or on the client's own rules/heuristics, and Zimbra will end up training on that basis. On the face of it this seems to be a valid approach.
But consider this case: a user tells their client to filter based on X-Spam-Level. Dan Martin commented,There are a bunch of ways to go about this, sure. You could re-weight the scores. You could lower the threshold from the default 6.6 to 5.1 or whatever. You could add more detection methods such as DCC, Pyzor, Razor, or image spam detection.it seems to me if a user is trying to do filter-based training on X-Spam scores specifically, something is funky with the overall setup. X-Spam is, after all, supposed to be the automated scoring system. If a user is getting a lot of false negatives (essentially what a "too low" X-Spam score is), then it seems to me you should analyze the overall scoring of those suspect emails and identify, and if necessary refine, the filters accordingly.
But there may still be some information left in the unique combination of recipient + spam score. I mean simply, for certain users, if email is over a fairly low score, then it's guaranteed to be spam, and it may be valid to feed that information back into the system. In theory this resembles some spam training systems that self-train not only on user-sorted false positives/negatives (the way that Zimbra does) but also on their existing corpus. "The spam gets spammier and the ham gets hammier," as it were. If you look at this apache.org page on SA, this is basically item 2 in the section on Effective Training. (ASSP also feeds automatically-identified spam back into its training system, see here.)
The danger here is there could be a chance of "drift" due to unsupervised feedback. Effectively, certain "tokens of legitimacy" could be "poisoned" by being statistically associated with spam, until they become spurious primary indicators of spam. I'm not aware of any real-world exploitations of this concept, but I did find an article discussing it: Does Bayesian Poisoning Exist? (PDF);