On 10/2/12 8:30 PM, dar...@chaosreigns.com wrote:
Run the email through spamassassin -D received-header. That'll tell you
how and if the headers got parsed. SA has certainly had bugs where it
failed to parse received headers before, and IPv6 hasn't had a whole lot of
use.
Thanks for pointing that out. That does illuminate the situation.
The debug output shows that SA is (IMO, mis-) interpreting the
x-originating-ip as a Received header.
That's a surprise to me because it is an uninterpreted optional (header)
field (RFC5821, 3.6.8)
which could have any garbage in it and convey any semantics.
There has also been a fair amount of work on IPv6 since the last release,
so it's possible there was a bug, it got fixed, and you don't have the
fix yet.
It turns out it isn't an IPv6 problem. I had wrongly assumed that if a
host with a private address
delivers to a trusted, local host, then that host would be considered
trusted local.
I removed the x-originating-ip header. Then, if I replace the IPv6
link local addresses by
IPv4 10/8 addresses, I get the same behavior as with the IPv6 fe80::/10
addresses.
If the private addresses are in the trusted local addresses, then I
get ALL_TRUSTED
and not RDNS_NONE. If they are not in trusted local, then I get
RDNS_NONE and not
ALL_TRUSTED.
So I have to add things like 10/8 and fe80::/10 to the trusted local
networks.
(When I add back in x-originating-ip, I lose ALL_TRUSTED, which would
be expected if you
treat it like a Received: header.)
On 10/02, Mabry Tyson wrote:
One user complained about a false positive. When I examined the mail,
there appeared to be at least two rules that didn't work as I thought they
should because of a Received line in which IPv6 Link Local addresses were
used. It appears that a patch was previously put in that was thought to
fix these kinds of things.
The sender was apparently using AA.BB.CC.DD (a Comcast address, presumably
his home address).
He logged into the mail system of SRI.COM (independent of our mail system)
and
sent his mail from within it (which is why CCC.SRI.COM is the oldest
Received line).
That should result in a received header clearly indicating that the
connection from comcast was authenticated, and SA should notice that
and use it to skip the tests on that comcast IP.
It mostly sounds like this is what's missing. SRI.com not indicating
the authentication in their received header in the standard way.
You say standard way. Can you point to a standard (eg, RFC) that
indicates
how this is indicated? Maybe it is my lack of knowledge, but I'm not
aware of any
standard way. That mail system is run by an independent business unit.
I have no influence on their operations of their Exchange server. But,
if they're
not following a standard, I can suggest they follow a standard. But they
probably won't listen to a Well, SpamAssassin would recognize things
better if you did ,
RFC5322 (Message Format) doesn't mention authenticate.
RFC5321 (SMTP) does mention authentication (p 73) but gives no
indication this
should be reflected in any header nor any mechanism to do so. It mentions
RFC4409 (Message Submission) as an example of a protocol that causes a
message to be entered
onto the Internet. However, RFC4409 gives no indication of how a conforming
service should (or even might) indicate authentication. Nor does it
indicate the
format of a Received header that it should add such that it indicates
that this
protocol was used.
After reviewing these, I believe that the first host to receive this message
fails to add a Received header as required (which presumably would be
where some
indication of authentication would be found). The oldest Received
header indicates
a transmission from that first host to another host.
(Ref: RFC5821, 3.7.2 When forwarding a message into or out of the
Internet
environment, a gateway MUST prepend a Received: line ...)
4.
[3]http://spamassassin.apache.org/tests_3_3_x.html has
RCVD_IN_PBL = 3.6 (Spamhaus Policy black list)
RCVD_IN_SBL = 2.6 (Spamhaus Spam black list)
RCVD_IN_XBL = 0.7 (Spamhaus Botnet black list)
which seems backward to me. The 3.2 tests scoring seems more reasonable.
Do not attempt to comprehend the depths of the mind of the re-scorer :P
No seriously, it has no concept of this rule means the email is more bad
than another rule, therefore it should have a higher score. Only This
score results in a better approximation of the 1 false positive in 2,500
non-spams goal. Which often results in unexpected things. It comes
up a lot.
I very recently found a case where a rule that hit more non-spam than spam
got a score of something like 3. Which may have been suboptimal.
There may be many ways of assigning scores to the rules to
get nearly equivalent results (of correct assignments, false positives,
misses) on any particular test set of messages. Those
