Tux

...making Linux just a little more fun!

John Karns's post tripped some spam filters

Rick Moen [rick at linuxmafia.com]
Mon, 14 Aug 2006 00:56:47 -0700

Hmm, John's post got held by Mailman, claiming that SpamAssassin had marked it as "possible spam". Let's have a look at what got into Mailman and SpamAssassin's tiny little brains:

 Received: from [201.245.212.45] (port=33475 helo=localhost.localdomain)
 	 by linuxmafia.com with esmtp   (Exim 4.61 #1 (EximConfig 2.0))
 	 id 1GCMEs-0005t8-Hg   
 	for <tag at lists.linuxgazette.net>; Sun, 13 Aug 2006 13:07:21 -0700
 Received: by localhost.localdomain (Postfix, from userid 1000)
 	id 371D323055; Sun, 13 Aug 2006 15:07:01 -0500 (COT)
 Received: from localhost (localhost [127.0.0.1])
 	by localhost.localdomain (Postfix) with ESMTP id 31E942303E;
 	Sun, 13 Aug 2006 15:07:01 -0500 (COT)
 Date: Sun, 13 Aug 2006 15:07:01 -0500 (COT)
 From: John Karns <jkarns@etb.net.co>
 To: TAG <tag@lists.linuxgazette.net>
 X-X-Sender: jkarns at localhost.localdomain
 To: jeff at jeffroot.us
 cc: tag at lists.linuxgazette.net
 In-Reply-To: <17630.47578.208478.397536 at localhost.localdomain>
 Message-ID: <Pine.LNX.4.61.0608131345520.21008 at localhost.localdomain>
 References: <17621.16287.466717.206264 at localhost.localdomain>
  <20060806022547.GA3848 at linuxgazette.net> <17621.34053.297464.620391 at localhost.localdomain>
  <20060807030821.GA3903 at linuxgazette.net> <Pine.LNX.4.61.0608091621130.12020 at localhost.localdomain>
  <20060809214806.GA4892 at linuxgazette.net> <Pine.LNX.4.61.0608121407330.836 at localhost.localdomain>
  <17630.47578.208478.397536 at localhost.localdomain>
 MIME-Version: 1.0
 X-SA-Do-Not-Run: Yes
 X-EximConfig: v2.0 on linuxmafia.com (http://www.jcdigita.com/eximconfig)
 X-SA-Exim-Connect-IP: 201.245.212.45
 X-SA-Exim-Mail-From: jkarns at etb.net.co
 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on linuxmafia.com
 X-Spam-Level: *
 X-Spam-Status: No, score=3.5 required=4.0 tests=AWL,BAYES_00,FORGED_RCVD_HELO,
 	RCVD_IN_DSBL,RCVD_IN_DYNABLOCK,RCVD_IN_SORBS,RCVD_IN_SORBS_DUL 
 	autolearn=no version=3.1.1
 Subject: Re: [TAG] Talkback:127/howell.html
 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
 X-SA-Exim-Version: 4.2.1 (built Mon, 27 Mar 2006 13:42:28 +0200)
 X-SA-Exim-Scanned: Yes (on linuxmafia.com)
The weird thing is, it was Mailman that objected to your message and held it for my manual approval, claiming that SpamAssassin had flagged it as "possible spam" -- yet, as you can see, SA's score was 3.5, well below the 4.0 spamicity threshold I set in SpamAssassin. I'm not sure what's going on there.

In any event, spamicity = 3.5 is eyebrow-raising enough in itself, so let's see what all those failed tests in the X-Spam-Status line are:[1]

AWL: Auto-WhiteList. This is a simple "address or IP that has been heard from in the somewhat recent past" database, giving ones not heard from recently a small boost to the maybe-distrust-this spamicity score.

BAYES_00: A "Bayesian" statistical test on the body text. The "BAYES_00" result means that the Bayesian estimate of probability is that there's only a 0-1% likelihood of your post being spam, and that result actually reduces the post's spamicity score.

FORGED_RCVD_HELO: This means that the hosthame your delivering SMTP process reported to mine during the delivery conversation, right at the beginning in the SMTP "HELO" command, has been detected to be provably wrong (which SpamAssassin rather cynically classifies as a forgery). And that, in turn, is because your MTA said (as indicated in the first Received line) that its hostname was "localhost.localdomain". In other words, you badly need to use a valid FQDN (fully qualified domain name) when sending e-mail on the Internetl. Here's an analysis posted to someone else with a similar symptom: http://linuxfromscratch.org/pipermail/lfs-chat/2005-July/026693.html

RCVD_IN_DSBL: Means that the mail was received from an IP address listed in the dsbl.org blocklist "list.dsbl.org" as some sort of open single-stage relay (at least at some point in the past). See: http://dsbl.org/faq

RCVD_IN_DYNABLOCK: Means that the mail was received from an IP address that was detected to be part of a dial-up or dynamic-IP pool. For reasons that would take a while to explain, most of the Internet now attempts to avoid accepting SMTP mail sent directly dynamic IPs. (The reasons are compelling. I just don't want to get into them right now.) People on dynamic IPs should strongly consider relaying their outbound mail via their ISPs' SMTP mail servers.

RCVD_IN_SORBS: A very small boost to spamicity from your IP address being in the SORBS blocklist.

RCVD_IN_SORBS_DUL: A slightly larger boost from your IP address being in the SORS Dial Up List. (See comments about dial-up and dynamic-IP address pools.)

Anyhow, the bottom line is: The rather devil-may-care way you're just hurling mail out onto the 'Net (using dynamic IP for SMTP, and not even using a real hostname) is very, very likely to cause you problems. I've just attempted to whitelist your sending address, which should help you with TAG mail -- but leaves some hundred thousand other SMTP servers not so inclined.

[1] See also: http://spamassassin.apache.org/tests_3_1_x.html


Top    Back


Neil Youngman [ny at youngman.org.uk]
Mon, 14 Aug 2006 09:22:21 +0100

On or around Monday 14 August 2006 08:56, Rick Moen reorganised a bunch of electrons to form the message:

> Hmm, John's post got held by Mailman, claiming that SpamAssassin had
> marked it as "possible spam".  Let's have a look at what got into
> Mailman and SpamAssassin's tiny little brains:
>
>
> Received: from [201.245.212.45] (port=33475 helo=localhost.localdomain)
> 	 by linuxmafia.com with esmtp   (Exim 4.61 #1 (EximConfig 2.0))
> 	 id 1GCMEs-0005t8-Hg
> 	for <tag at lists.linuxgazette.net>; Sun, 13 Aug 2006 13:07:21 -0700
> Received: by localhost.localdomain (Postfix, from userid 1000)
> 	id 371D323055; Sun, 13 Aug 2006 15:07:01 -0500 (COT)
> Received: from localhost (localhost [127.0.0.1])
> 	by localhost.localdomain (Postfix) with ESMTP id 31E942303E;
> 	Sun, 13 Aug 2006 15:07:01 -0500 (COT)
> Date: Sun, 13 Aug 2006 15:07:01 -0500 (COT)
> From: John Karns <jkarns at etb.net.co>
> X-X-Sender: jkarns at localhost.localdomain
> To: jeff at jeffroot.us
> cc: tag at lists.linuxgazette.net
> In-Reply-To: <17630.47578.208478.397536 at localhost.localdomain>
> Message-ID: <Pine.LNX.4.61.0608131345520.21008 at localhost.localdomain>
> References: <17621.16287.466717.206264 at localhost.localdomain>
>  <20060806022547.GA3848 at linuxgazette.net>
> <17621.34053.297464.620391 at localhost.localdomain>
> <20060807030821.GA3903 at linuxgazette.net>
> <Pine.LNX.4.61.0608091621130.12020 at localhost.localdomain>
> <20060809214806.GA4892 at linuxgazette.net>
> <Pine.LNX.4.61.0608121407330.836 at localhost.localdomain>
> <17630.47578.208478.397536 at localhost.localdomain>
> MIME-Version: 1.0
> X-SA-Do-Not-Run: Yes
> X-EximConfig: v2.0 on linuxmafia.com (http://www.jcdigita.com/eximconfig)
> X-SA-Exim-Connect-IP: 201.245.212.45
> X-SA-Exim-Mail-From: jkarns at etb.net.co
> X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on linuxmafia.com
> X-Spam-Level: *
> X-Spam-Status: No, score=3.5 required=4.0
> tests=AWL,BAYES_00,FORGED_RCVD_HELO,
> RCVD_IN_DSBL,RCVD_IN_DYNABLOCK,RCVD_IN_SORBS,RCVD_IN_SORBS_DUL
> 	autolearn=no version=3.1.1
> Subject: Re: [TAG] Talkback:127/howell.html
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
> X-SA-Exim-Version: 4.2.1 (built Mon, 27 Mar 2006 13:42:28 +0200)
> X-SA-Exim-Scanned: Yes (on linuxmafia.com)
>
>
>
> The weird thing is, it was Mailman that objected to your message and
> held it for my manual approval, claiming that SpamAssassin had flagged
> it as "possible spam" -- yet, as you can see, SA's score was 3.5, well
> below the 4.0 spamicity threshold I set in SpamAssassin.  I'm not sure
> what's going on there.

It could be looking at the "X-Spam-Level: ***" and deciding that's a match, rather than looking at "X-Spam-Status: No"?

I have previously seen filter rules that filter on X-Spam-Level rather than X-Spam-Status, which could easily generate this sort of inconsistency.

Neil


Top    Back


Rick Moen [rick at linuxmafia.com]
Mon, 14 Aug 2006 01:49:55 -0700

Quoting Neil Youngman (ny at youngman.org.uk):

> It could be looking at the "X-Spam-Level: ***" and deciding that's a match, 
> rather than looking at "X-Spam-Status: No"?

{shrug} I find nothing in the Mailman config that suggests that. I did try looking at the relevant Mailman settings I'm familiar with.


Top    Back