[BlueOnyx:25377] Re: Efficient spam filters

[Michael Stauber]

Hi Antoine,

> I'm struggeling with spams... Can I ask you what you guys have found to 
> fight them efficently ?
> 
> Let me explain what I've done till now:
> 
>   * I've enableled milter-greylist (but users complains about delays) in
>     AV-Spam
>   * I used to have geoip milter but I disableled it because users
>     complained about not receiving e-mails from parts of the world I
>     blocked (AV-Spam as well)
>   * I've tried setting blacklists (zen.spamhaus.org /
>     b.barracudacentral.org ...) in the E-mail server settings but it
>     blocked some of my users as well... so maybe it was not so well
>     configured.
>   *  From time to time I run sa-learn on my spam folder where I store
>     not recognized spams... this helps but I first have to collect them...
> 
> Spamassassin tags some spams (the one tagged are spams) but I still have 
> quite a few in my mailbox (most of them I would say).
> 
> Any tips welcome because I'm bored receiving them.


Yeah, the thing is: One man's SPAM might be anothers treasure trove. The 
more diverse the client base is that you have on a server, the more 
difficult it gets to maintain a high SPAM detection rate and a low false 
positive rate. Even if you just run the AV-SPAM in default mode: The 
Bayes database will get populated in a way that the result is suboptimal.

Greylisting is great to stop emails from botnets, as they usually don't 
resend. However, the delays it causes for first time senders is often 
interfering with time critical emails such as one time access codes or 
authentication links.

Milter-GeoIP prime function is to record email volume and report abuse, 
but it also has a lot of features that can reject emails at the MTA if 
the WHOIS record has recent changes, the email is from a certain country 
or belongs to a certain IP address range. This leaves a lot of room for 
individual customization and you can for sure make it too strict for 
some of your email recipients.

RBLs directly tied into the MTA (Sendmail/Postfix)? You don't want to do 
that. The SpamAssassin in the AV-SPAM is already configured to use most 
sensible RBLs and scores the emails accordingly if the sending IP or 
relaying MTA is listed.

Which leaves SpamAssassin. Like said: It has a pretty good set of rules, 
but you can adjust it further to your linking.

If you absolutely MUST get all emails, then configure SpamAssassin not 
to reject high scoring SPAM, but to mark email and move detected SPAM to 
the "caughtspam" IMAP folder(s) of your users.

If emails that are obvious SPAM make it through unmarked, then take a 
look at the raw email or the email headers.

Look for a section like this:

--------------------------------------------------------------------------
X-Spam-Status: No, score=3.3 required=4.5 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RATS_NOPTR,RCVD_IN_DNSWL_BLOCKED,
RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RELAYCOUNTRY_RO,SPF_HELO_NONE,SPF_PASS
	autolearn=no autolearn_force=no version=3.4.0
X-Spam-Level: ***
X-Spam-ASN: AS11377 167.89.0.0/18
--------------------------------------------------------------------------

That's from a SPAM I got the other day. It got a low 3.3 score, which 
was below the 4.5 threshold I had configured for that email account.

You can see which rules were triggered.

What sticks out there?

FREEMAIL_FROM <- Email from a known free email service

RATS_NOPTR <- IP in Dynarats No-PTR RBL

RCVD_IN_DNSWL_BLOCKED <- IP address may not be entered into DNSWL
                          Whitelist due to bad reputation.

RELAYCOUNTRY_RO <- Sent from Romania /me rolls eyes

Now that we identified these, we might perhaps want to adjust the scores 
that SpamAssassin assigns to these rules. And we can easily do so.

Create a new textfile like /etc/mail/spamassassin/myrules.cf and for 
assigning a score of 5.0 to the existing RATS_NOPTR rule, you could put 
this into the textfile:

score RATS_NOPTR 5.0

Then save the changes and restart SpamAssassin:

systemctl restart spamassassin

In my custom config I usually use these:

#----------------------------------------------
score RATS_NOPTR 5.0
score RCVD_IN_BL_SPAMCOP_NET 5.0
score RCVD_IN_PSBL 5.0
score RCVD_IN_MSPIKE_L5 20
score RCVD_IN_DNSWL_HI 0.1
score DCC_CHECK 4.0
score RCVD_IN_PBL 2.5
#----------------------------------------------

And for good measure I also add this extra rule (among many others that 
are tailored specifically against the SPAM I get or used to get):

#--------------------------------------------------------------------
meta     DCC_AND_HTML (DCC_CHECK && HTML_MESSAGE)
score    DCC_AND_HTML 10.00
describe DCC_AND_HTML Listed in DCC and sending HTML messages.

meta     DCC_AND_IMAGE (DCC_CHECK && T_REMOTE_IMAGE)
score    DCC_AND_IMAGE 4.00
describe DCC_AND_IMAGE Listed in DCC and Email contains image.
#--------------------------------------------------------------------

What these two rules do?

First one: If the email has already been reported to DCC and is a HTML 
message? It gets a score of 10.00

Second one: If the email has already been reported to DCC and is using a 
remote image, it gets a 4.00 score tucked onto it.

Do you get frequent emails that bother you with offers for custom 
designed plastic moulds in China? A rule like this deals with that:

#--------------------------------------------------------------------
body   __SMD_CHPM1 /plastic/i
body   __SMD_CHPM2 /moulding/i
body   __SMD_CHPM3 /china/i
meta     SMD_CHPMX ( __SMD_CHPM1 && __SMD_CHPM2 && __SMD_CHPM3 )
describe SMD_CHPMX Plastic Moulding China
score    SMD_CHPMX 100
#--------------------------------------------------------------------

And yes: That one gets score of 100, because I was tired of seeing these. :p

So with some rule adjustments or custom SpamAssassin rules you can gain 
a lot and can tailor SpamAssassin more to your needs.

-- 
With best regards

Michael Stauber