Re: semi-legit senders in DNSWL and habeas - a hard problem

[Christian Brel]

On Wed, 6 Jan 2010 14:06:23 -0800
"jdow"  wrote:

> From: "Kai Schaetzl" 
> Sent: Wednesday, 2010/January/06 13:03
> 
> 
> > Jdow wrote on Wed, 6 Jan 2010 10:40:14 -0800:
> >
> >> Actually, Charles, this is a VERY good reason I'd use to justify
> >> changing my quote character to something goofy like % or # or
> >> even ; just to annoy the anal retentive types.
> >
> > First, to clarify, it was Charles who sent this to the list, not me.
> > Second, I see, RFC-compliance is "anal-retentive".
> >
> > Kai
> 
> It's a Request For Comment, not a rule or law. Using something
> different would be my comment.
> 
> {^_^} 
> 


Oh dear. *plonk*

Re: ClamAV Plugin Question

[Mark Martinec]

On Tuesday January 5 2010 15:15:58 Art Greenberg wrote:
> > On Tuesday January 5 2010 14:45:30 Art Greenberg wrote:
> >> SA and ClamAV both seem to be scanning email and otherwise working
> >> properly, except that the ClamAV plugin is inserting the X-Spam-Virus
> >> header twice. Messages with a virus found are showing it first with "No"
> >> and the second time with "Yes". In messages where no virus is detected,
> >> it is "No" both times. The second instance immediately follows the
> >> first.
>
> On Tue, 5 Jan 2010, Mark Martinec wrote:
> > The issue is already tracked as:
> >  https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6254
> > Will look into it...
> 
> Thanks for pointing me to that bug report, Mark. I just verified that the
> plugin (clamav.pm) I am using has the fix discussed there.
> 
> What I am seeing differs from what Larry reported in Comment 7, in that I
> am seeing the X-Spam-Virus header inserted twice even when ClamAV does not
> detect a virus.

Please try the new plugin code at:
  http://wiki.apache.org/spamassassin/ClamAVPlugin

not forgetting to add a:
  add_header all Virus _CLAMAVRESULT_
into your local.cf.

See also https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6254

  Mark

Re: ALL_TRUSTED rule no longer working

[Matt Kettler]

On 1/6/2010 3:43 PM, Julian Yap wrote:
>
> On Tue, Jan 5, 2010 at 5:12 PM, Matt Kettler  > wrote:
>
> On 1/5/2010 8:03 PM, Julian Yap wrote:
>> Previously I was running SpamAssassin-3.1.8_1 on FreeBSD.
>>
>> I recently upgraded to 3.2.5_4.
>>
>> It's seems now, I never get any hits on the rule ALL_TRUSTED.
>>
>> Previously it seemed like SA was doing some kind of dynamic
>> evaluation which was working well.
>>
>> - Julian
>>
> is NO_RELAYS or UNPARSEABLE_RELAY also hitting?
>
> In older versions of SA, ALL_TRUSTED was really implemented as "no
> untrusted", so it would fire off if there were no relays, or no
> parseable ones. This caused problems with ALL_TRUSTED matching
> spam when people ran SA on servers with malformed headers.
>
> Later we changed it to fire if there is:
> -at least one trusted relay
> -no untrusted relays
> -no unparseable relays.
>
> Which might be the cause of your problem.
>
>
> NO_RELAYS gets no hits but UNPARSEABLE_RELAY is working.
>
> Should I be getting some hits on NO_RELAYS?
>
> Thanks for the further explanation.
>
> - Julian
>
Neither of these rules should *EVER* fire. They both indicate error
conditions.

Re: semi-legit senders in DNSWL and habeas - a hard problem

[jdow]

From: "Kai Schaetzl" 
Sent: Wednesday, 2010/January/06 13:03



Jdow wrote on Wed, 6 Jan 2010 10:40:14 -0800:


Actually, Charles, this is a VERY good reason I'd use to justify changing
my quote character to something goofy like % or # or even ; just to annoy
the anal retentive types.


First, to clarify, it was Charles who sent this to the list, not me.
Second, I see, RFC-compliance is "anal-retentive".

Kai


It's a Request For Comment, not a rule or law. Using something different
would be my comment.

{^_^} 

Re: [sa] Re: semi-legit senders in DNSWL and habeas - a hard problem

[J.D. Falk]

On Jan 5, 2010, at 3:52 PM, Michael Scheidell wrote:

> or an industry standard, RFC REQUIRED abuse@ address.
> 
> Section 1 of RFC2142

abuse@ works, but it isn't the fastest method for reaching the correct team.

What I think a lot of y'all are missing is that we have more than one product, 
and (unfortunately) a lot of legacy domain names, so anything sent to abuse@ 
goes into a general queue which gets sorted later.  Neil and I have been trying 
to give you the fastest method for resolving issues, but if you'd rather take 
it slow... *shurg*

One of the things I've noticed about the anti-spam community over the years is 
that we'll always heap way more abuse on anyone who is willing to listen than 
we do on the spammers who aren't listening at all.  That's never a good idea, 
because it chases away people who might otherwise be listening -- or even 
helping.

(Oh BTW, take a look at the acknowledgements section of RFC 2142.)

--
J.D. Falk 
Return Path Inc

Re: semi-legit senders in DNSWL and habeas - a hard problem

[J.D. Falk]

On Jan 5, 2010, at 6:01 PM, Greg Troxel wrote:

> Thanks.  A link like "report spam" in the top bar, alongside "marketers

I'll pass all of this along to the appropriate folks.

--
J.D. Falk 
Return Path Inc

Re: lint check of update failed, channel failed

[Justin Mason]

Hmm.  We can use if can() to work around it...

On Wednesday, January 6, 2010, Mark Martinec  wrote:
> jidanni wrote:
>
>> $ sa-update
>> config: failed to parse line, skipping,
>>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>>  clear_originating_ip_headers
>> config: failed to parse line, skipping,
>>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>>  originating_ip_headers X-Yahoo-Post-IP X-Originating-IP X-Apparently-From
>> config: failed to parse line, skipping,
>>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>>  originating_ip_headers X-SenderIP
>> channel: lint check of update failed, channel failed
>
> It's an unfortunate consequence of pushing the Bug 5895 rule change
> into 10_default_prefs.cf, while you are still running 3.3.0-rc1 or
> older 3.3.0 code.
>
> The quickest fix would be to install the current SA code from SVN:
>   http://wiki.apache.org/spamassassin/DevelopmentStuff
>   $ svn checkout https://svn.apache.org/repos/asf/spamassassin/trunk
> with the usual: perl Makefile.PL; make; make install
>
> ...or just not to worry about the failing sa-update,
> until we come up with a better temporary solution.
>
>   Mark
>
>

-- 
--j.

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Kai Schaetzl]

Jdow wrote on Wed, 6 Jan 2010 10:40:14 -0800:

> Actually, Charles, this is a VERY good reason I'd use to justify changing
> my quote character to something goofy like % or # or even ; just to annoy
> the anal retentive types.

First, to clarify, it was Charles who sent this to the list, not me.
Second, I see, RFC-compliance is "anal-retentive".

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Kai Schaetzl]

Charles Gregory wrote on Wed, 6 Jan 2010 12:20:33 -0500 (EST):

> Because I was getting several M$ Outhouse correspondents complaining that 
> my messages (using the 'standard' '>') were 'difficult to read'.
> I could never get them to explain exactly how/why they were difficult to 
> read. It was like they were seeing something completely different with 
> bits of text missing. Not just word-wrapped. They were very insistent 
> that I top post rather than use (to me) standard quote-reply method...

The reason are not the ">", but the fact that these morons are only used to 
the crappy way that Outlook quotes. It makes no difference for them what 
character you use, it's the basic way of quoting.

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Kai Schaetzl]

Jdow wrote on Wed, 6 Jan 2010 10:37:49 -0800:

> I've never received any and I am a member. Every invite has been from
> somebody with a "solid" connection to me.

Well, they seem to provide an option to upload your whole addressbook. And 
some email applications have an option to add every incoming or outgoing 
address to the addressbook. That combination calls for disaster.
I've recently seen quite a few invites sent to mailing lists ...

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

Re: junkemailfilter FP - impressive speed of resolving

[Marc Perkel]

Thanks - however I don't guarantee that kind of response time. :)

Greg Troxel wrote:

Lest people think I object to all whitelists, I'd like to point out that
tonight I got spam from reachmail.net that was listed in HOSTKARMA_WL.
I sent it off to supp...@junkemailfilter.com and SEVEN AND A HALF
MINUTES later Marc told me that the offending sender had been removed
From the whitelist.  This has happened several times over the last few
months, but almost always the white/yellow/black scoring seems
reasonable.  What impresses me is that senders are dropped from
HOSTKARMA_W promptly when there is any evidence of spamming.

  

Re: ALL_TRUSTED rule no longer working

[Julian Yap]

On Tue, Jan 5, 2010 at 5:12 PM, Matt Kettler wrote:

>  On 1/5/2010 8:03 PM, Julian Yap wrote:
>
> Previously I was running SpamAssassin-3.1.8_1 on FreeBSD.
>
> I recently upgraded to 3.2.5_4.
>
> It's seems now, I never get any hits on the rule ALL_TRUSTED.
>
> Previously it seemed like SA was doing some kind of dynamic evaluation
> which was working well.
>
> - Julian
>
>  is NO_RELAYS or UNPARSEABLE_RELAY also hitting?
>
> In older versions of SA, ALL_TRUSTED was really implemented as "no
> untrusted", so it would fire off if there were no relays, or no parseable
> ones. This caused problems with ALL_TRUSTED matching spam when people ran SA
> on servers with malformed headers.
>
> Later we changed it to fire if there is:
> -at least one trusted relay
> -no untrusted relays
> -no unparseable relays.
>
> Which might be the cause of your problem.
>

NO_RELAYS gets no hits but UNPARSEABLE_RELAY is working.

Should I be getting some hits on NO_RELAYS?

Thanks for the further explanation.

- Julian

sa-update failing

[David Chaplin-Loebell]

Hi,

I'm trying to update my rules with sa-update, and it is failing:

deliver3# sa-update -D
[98652] dbg: logger: adding facilities: all
[98652] dbg: logger: logging level is DBG
[98652] dbg: generic: SpamAssassin version 3.2.5
[98652] dbg: config: score set 0 chosen.
[98652] dbg: dns: is Net::DNS::Resolver available? yes
[98652] dbg: dns: Net::DNS version: 0.65
[98652] dbg: generic: sa-update version svn607589
[98652] dbg: generic: using update directory: /var/db/spamassassin/3.002005
[98652] dbg: diag: perl platform: 5.008009 freebsd
make a shorter message>

[98652] dbg: channel: no MIRRORED.BY file available
[98652] dbg: http: GET request, 
http://spamassassin.apache.org/updates/MIRRORED.BY

[98652] dbg: channel: MIRRORED.BY file retrieved
[98652] dbg: channel: reading MIRRORED.BY file
[98652] dbg: channel: found mirror 
http://daryl.dostech.ca/sa-update/asf/ weight=5

[98652] dbg: channel: found mirror http://www.sa-update.pccc.com/ weight=5
[98652] dbg: channel: selected mirror http://daryl.dostech.ca/sa-update/asf
[98652] dbg: http: GET request, 
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz
[98652] dbg: http: GET request, 
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz.sha1
[98652] dbg: http: GET request, 
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz.asc
[98652] dbg: sha1: verification wanted: 
ade9426b8f85bed554604033c71e925e5e597502
[98652] dbg: sha1: verification result: 
b3fa2495f0091f85a8421b39c72b868afef6808f

channel: SHA1 verification failed, channel failed
[98652] dbg: generic: cleaning up temporary directory/files
[98652] dbg: diag: updates complete, exiting with code 4

I searched the list archives for the SHA1 verification error, and found 
nothing recent-- is this problem somehow unique to my system?  Any 
suggestions?


Thanks,
David Chaplin-Loebell

Re: spamassassin or spamd with amavisd-new?

[Terry Carmen]

On 01/06/2010 02:05 PM, Kai Schaetzl wrote:

Terry Carmen wrote on Wed, 06 Jan 2010 13:23:28 -0500:

   

How/where is this turned on in amavisd-new-2.6.4?

I'd be happy to get rid of useless instances of spamd
 

This has nothing to do with amavis. spamd is a separate daemon that *you*
enabled. As you don't seem to know this: are you sure that spamd runs at
all or was it just an assumption?
It had been left enabled from a previous install, however I just 
disabled it, and everything is still working, which is nice, since it 
saves a little RAM.


Terry

sa-update failing

[David Chaplin-Loebell]

Hi,

(apologies if this is a duplicate - I don't think it went through the 
first time)


I'm trying to update my rules with sa-update, and it is failing:

deliver3# sa-update -D
[98652] dbg: logger: adding facilities: all
[98652] dbg: logger: logging level is DBG
[98652] dbg: generic: SpamAssassin version 3.2.5
[98652] dbg: config: score set 0 chosen.
[98652] dbg: dns: is Net::DNS::Resolver available? yes
[98652] dbg: dns: Net::DNS version: 0.65
[98652] dbg: generic: sa-update version svn607589
[98652] dbg: generic: using update directory: /var/db/spamassassin/3.002005
[98652] dbg: diag: perl platform: 5.008009 freebsd

[98652] dbg: channel: no MIRRORED.BY file available
[98652] dbg: http: GET request,
http://spamassassin.apache.org/updates/MIRRORED.BY
[98652] dbg: channel: MIRRORED.BY file retrieved
[98652] dbg: channel: reading MIRRORED.BY file
[98652] dbg: channel: found mirror
http://daryl.dostech.ca/sa-update/asf/ weight=5
[98652] dbg: channel: found mirror http://www.sa-update.pccc.com/ weight=5
[98652] dbg: channel: selected mirror http://daryl.dostech.ca/sa-update/asf
[98652] dbg: http: GET request,
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz
[98652] dbg: http: GET request,
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz.sha1
[98652] dbg: http: GET request,
http://daryl.dostech.ca/sa-update/asf/895075.tar.gz.asc
[98652] dbg: sha1: verification wanted:
ade9426b8f85bed554604033c71e925e5e597502
[98652] dbg: sha1: verification result:
b3fa2495f0091f85a8421b39c72b868afef6808f
channel: SHA1 verification failed, channel failed
[98652] dbg: generic: cleaning up temporary directory/files
[98652] dbg: diag: updates complete, exiting with code 4

I searched the list archives for the SHA1 verification error, and found
nothing recent-- is this problem somehow unique to my system?  Any
suggestions?

Thanks,
David Chaplin-Loebell

Re: lint check of update failed, channel failed

[Mark Martinec]

jidanni wrote:

> $ sa-update
> config: failed to parse line, skipping,
>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>  clear_originating_ip_headers
> config: failed to parse line, skipping,
>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>  originating_ip_headers X-Yahoo-Post-IP X-Originating-IP X-Apparently-From
> config: failed to parse line, skipping,
>  in "/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf":
>  originating_ip_headers X-SenderIP
> channel: lint check of update failed, channel failed

It's an unfortunate consequence of pushing the Bug 5895 rule change
into 10_default_prefs.cf, while you are still running 3.3.0-rc1 or
older 3.3.0 code.

The quickest fix would be to install the current SA code from SVN:
  http://wiki.apache.org/spamassassin/DevelopmentStuff
  $ svn checkout https://svn.apache.org/repos/asf/spamassassin/trunk
with the usual: perl Makefile.PL; make; make install

...or just not to worry about the failing sa-update,
until we come up with a better temporary solution.

  Mark

Re: lint check of update failed, channel failed

[Jason Bertoch]

jida...@jidanni.org wrote:

$ sa-update
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": 
clear_originating_ip_headers
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-Yahoo-Post-IP X-Originating-IP X-Apparently-From
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-SenderIP
channel: lint check of update failed, channel failed



I got the same thing 2 hours ago, and am still getting it now.

/Jason

Re: spamassassin or spamd with amavisd-new?

[Kai Schaetzl]

Terry Carmen wrote on Wed, 06 Jan 2010 13:23:28 -0500:

> How/where is this turned on in amavisd-new-2.6.4?
> 
> I'd be happy to get rid of useless instances of spamd

This has nothing to do with amavis. spamd is a separate daemon that *you* 
enabled. As you don't seem to know this: are you sure that spamd runs at 
all or was it just an assumption?

Kai

-- 
Get your web at Conactive Internet Services: http://www.conactive.com

lint check of update failed, channel failed

[jidanni]

$ sa-update
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": 
clear_originating_ip_headers
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-Yahoo-Post-IP X-Originating-IP X-Apparently-From
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-SenderIP
channel: lint check of update failed, channel failed

Re: semi-legit senders in DNSWL and habeas - a hard problem

[jdow]

From: "Charles Gregory" 
Sent: Wednesday, 2010/January/06 09:20



On Wed, 6 Jan 2010, Kai Schaetzl wrote:
: just wanted to inform you that ">" is the only official quote marker.

Deep sigh. Do you know why I changed it?

Because I was getting several M$ Outhouse correspondents complaining that
my messages (using the 'standard' '>') were 'difficult to read'.
I could never get them to explain exactly how/why they were difficult to
read. It was like they were seeing something completely different with
bits of text missing. Not just word-wrapped. They were very insistent
that I top post rather than use (to me) standard quote-reply method...

I haven't confirmed it yet, but after reading a few notes on the web, I
was testing the possibility that Outhouse was incorrectly attempting to
parse the '>' as an HTML marker and mangling my messages.

But if this is really messing up people (or software) on this list, I'll
put it back to '>'...


Actually, Charles, this is a VERY good reason I'd use to justify changing
my quote character to something goofy like % or # or even ; just to annoy
the anal retentive types.

{^_-} 

Re: semi-legit senders in DNSWL and habeas - a hard problem

[jdow]

From: "Charles Gregory" 
Sent: Wednesday, 2010/January/06 07:11



On Tue, 5 Jan 2010, Greg Troxel wrote:
: Thanks.  A link like "report spam" in the top bar, alongside "marketers
: and senders" would help.  That should link to a page that gives an email
: address where one can forward the full offending message, and a way to
: lookup IP addresses to see if they are still in the database, like other
: DNSBLS.

I agree with both these ideas. The DNSBL lookup could actually be a 'front
end' to the mechanism, so that people don't send a report if an IP has
already been removed from the whitelist(s). Saves time dealing with old
problems


Excellent idea, Charles.


: On the real issue, I find it hard to believe I'm the first one to
: complain about linkedin invitation spam.  Is this really true?

Possibly. Most people accept that this sort of abuse is not a fault WITH
'linkedin', but merely abuse OF 'linkedin' and so they send their abuse
report to linkedin directly.


I've never received any and I am a member. Every invite has been from
somebody with a "solid" connection to me. (Which is easy - I'm not a
very social animal. {^_-})

{^_^} 

Re: spamassassin or spamd with amavisd-new?

[Terry Carmen]

On 01/05/2010 09:49 PM, Matt Kettler wrote:
. . .

Really, all spamd does is create a reusable instance of a 
Mail::SpamAssassin perl object, and keeps it loaded so it can process 
several messages that spamc feeds this. This is exactly what 
amavisd-new is already doing internal to its own code, so it doesn't 
need spamd.


Running "spamassassin" on the command line is really slow, because it 
creates a new Mail::SpamAssassin object, scans a single message, and 
exits. This is great for quick checks of the configuration, but not at 
all efficient in a mailstream. However, amavisd-new does not do this. 
It creates and re-uses a Mail::SpamAssassin object.


Read the main page of the amvavis website which states:

http://www.ijs.si/software/amavisd/

Which will point out:

"when configured to call /Mail::SpamAssassin/ (this is optional), it 
orders SA to pre-load its config files and to precompile the patterns, 
so performance is at least as good as with spamc/spamd setup. All Perl 
modules are pre-loaded by a parent process at startup time, so forked 
children need not re-compile the code, and can hopefully share some 
memory for compiled code;"


How/where is this turned on in amavisd-new-2.6.4?

I'd be happy to get rid of useless instances of spamd

Terry

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Charles Gregory]

On Wed, 6 Jan 2010, Kai Schaetzl wrote:
: just wanted to inform you that ">" is the only official quote marker. 

Deep sigh. Do you know why I changed it?

Because I was getting several M$ Outhouse correspondents complaining that 
my messages (using the 'standard' '>') were 'difficult to read'.
I could never get them to explain exactly how/why they were difficult to 
read. It was like they were seeing something completely different with 
bits of text missing. Not just word-wrapped. They were very insistent 
that I top post rather than use (to me) standard quote-reply method...

I haven't confirmed it yet, but after reading a few notes on the web, I 
was testing the possibility that Outhouse was incorrectly attempting to 
parse the '>' as an HTML marker and mangling my messages.

But if this is really messing up people (or software) on this list, I'll 
put it back to '>'...

- C

Re: [sa] Comparing the envelope-from/sender to the body from to prevent fake local users spams?

[Charles Gregory]

On Wed, 6 Jan 2010, lstep wrote:
: Is there something implemented in Spamassassin to test and prevent mails
: that come from 'outside' that have the header 'From' set to an internal
: user?

And here are YOUR headers on your email, which you would have received on 
your server from an 'outside system' (the apache list server):

: From: lstep 
: Reply-To: users@spamassassin.apache.org

And this is why blocking on a 'forged' From header cannot be done 
as simply as you suggest. If you can check for whether the forged sender 
*exists* you may catch a percentage of spam that has ignorantly used a 
once-valid but now-deleted address. 

- C

Re: Apache SpamAssassin Y2K10 Rule Bug - Update Your Rules Now! (custom sa-update script from howtoforge)

[Bowie Bailey]

Mark Martinec wrote:
> On Tuesday January 5 2010 22:47:42 Bowie Bailey wrote:
>   
>> I patched sa-update to add a verbose option which outputs all the
>> channel names that had changes.  Very simple patch if anyone is
>> interested.  It installs cleanly on 3.2.5, I haven't tried 3.3.
>> 
>
> This looks like an useful small patch.
> Could you please open an enhancement request on the Bugzilla.
>   

Done.

https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6277

-- 
Bowie

Re: SA 3.3.0-rc1 SPF Question

[Jason Bertoch]

Jason Bertoch wrote:


A message passed through my server yesterday from an @allianzlife.com 
address with the following Received header:


Received: from allianzlife.com (securemail.allianzlife.com 
[204.52.250.91] (may be forged))


While investigating other issues with the message, I noted that (unless 
my coffee hasn't kicked in yet) it should have matched on SPF_SOFTFAIL, 
but didn't.  In fact, it didn't match on _any_ SPF rules.  I have a 
number of other messages in my logs that hit all of the various SPF 
rules, so I know that things are at least partially working.  The ugly 
spf trace is included in the pastebin link, but essentially this falls 
back on "~all".  Am I missing something obvious?



http://pastebin.com/m1134a08b




Replying to my own post, I've continued to dig around due to the fact 
that no SPF rules hit.  It occurred to me that the problem may be the 
complex and somewhat malformed SPF record this company has produced.


"v=spf1 mx mx:alfemp01.allianzlife.com. mx:blfemp01.allianzlife.com. 
mx:mail04.allianzlife.com.ppus.azmx.de. 
mx:mail05.allianzlife.com.ppus.azmx.net. 
include:cust-spf.exacttarget.com. ~all"


Notably, they use the mx: prefix where I suspect they needed an a: 
prefix.  (Actually, these are completely unnecessary as these hosts are 
all listed as MX records and they've already specified the "mx" 
parameter).  I would expect that there should be some sanity checks on 
the number of (failed) DNS lookups made in an SPF query.  If that's the 
case, it might explain why I have no match on SPF rules.  Can anyone 
confirm or deny that there are lookup restrictions with regard to SPF 
records?


/Jason

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Charles Gregory]

On Tue, 5 Jan 2010, Greg Troxel wrote:
: Thanks.  A link like "report spam" in the top bar, alongside "marketers
: and senders" would help.  That should link to a page that gives an email
: address where one can forward the full offending message, and a way to
: lookup IP addresses to see if they are still in the database, like other
: DNSBLS.

I agree with both these ideas. The DNSBL lookup could actually be a 'front 
end' to the mechanism, so that people don't send a report if an IP has 
already been removed from the whitelist(s). Saves time dealing with old 
problems

: The 'report spam' link should be blindingly obvious.

This needs empahsizing. The people who use it will not be customers 
familiar with (and willing to navigate) the whole site. If I find that 
returnpath is a problem, it will be quicker and easier to disable the 
whitelist rather than fight my way through those forms, so its in 
returnpath's own interest to make it easy...

: On the real issue, I find it hard to believe I'm the first one to
: complain about linkedin invitation spam.  Is this really true?

Possibly. Most people accept that this sort of abuse is not a fault WITH 
'linkedin', but merely abuse OF 'linkedin' and so they send their abuse 
report to linkedin directly.

: Is my supposition that there is some sort of bulk invitation process 
: correct? Do your whitelist membership criteria permit this kind of 
: misbehavior?

It certainly should not! Any service capable of permitting the submission 
of 'mailing lists' and their use for a 'sneaky' custom e-mail, by FREE
accounts should NEVER be whitelisted, or you are just inviting abuse of 
the mechanism by eager spammers who want the negative score.

: in which case it's wrong to list linkedin, as the postgis-devel
: mailinglist surely did not agree to get invitations.

Actually, what should really happen, and I'm hoping the guys at returnpath 
are listening:

Ask 'linkedin' to setup their servers so that 'invites' are sent by a 
dedicated IP address that is NOT used for any other regular meil. Then 
REMOVE that IP (or range) from the whitelist, so that it does not gain the 
same benefits as personal mail sent from one 'linkedin' member to another.
You still have 'linkedin' as a customer, and their whitelisting is 
meaningful.

- Charles

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Charles Gregory]

On Tue, 5 Jan 2010, Gene Heskett wrote:
: The bottom line is that they are still spammers.  Filter 'em.

About that. 

A principle needs to be discussed here: Prohibition does not work.

The way to gain cooperation from 'big business' that *does* want to 'spam' 
is to find ways to keep them happy and thinking that they are gainnig 
sufficient 'benefit' from this amazing marketing tool called the internet.
And let's be honest with ourselves. Big business is what has *paid* for 
this internet, so that you and I can send e-mails around the globe for a 
ridiculously small price. 

If we 'prohibit' all e-mail advertising, we drive the 'needs' of ALL big 
business into an 'underground' that will *struggle* with all its corporate 
power to get its spew into every mailbox, and most likely in clever 'off 
shore' methods that are even *worse* than what we endure now.

The 'trick' of spam filtering is that there are thousands of people who 
are too stupid or gullible to think spam is 'bad' and *welcome* the 
advertising of 'legitimate' big business. They WANT this crap. They 
willingly and knowingly subscribe to it! And naturally, these are the 
people that bug business MOST wants to reach with their spew. 

So the principle is, if they both want it, we need to find mechanisms that 
HELP that happen, so that they don't fight to get around 'general' 
anti-spam filtering, but are content to be given 'special exception' to 
get their mail to the people who WANT it. Emphasize, absolutely WANT it.

By cooperating, and doing the job well, we can hope to then gain *their* 
cooperation, so that they follow the 'easy path' we give them, and only 
'spam' the people who want it.

Now to be fair, there is always the business that wants to see how far 
they can 'push' the rules. Every business WANTS to spam, and some of them 
try to see if they can do it 'sneakily'. Others try to take advantage of a 
'reputation' service to sneak some stuff out. No system is perfect.

But lamely saying that we should ban every last piece of mail through 
hotmail or returnpath or any other e-mail service is self-defeating. It 
will not result in 'less spam'. It will result in the same spam being 
delivered *more* indiscriminately, to YOU and to ME. 

No thank you. Let the morons who subscribe to this crap have it. Thanks.

- C

Re: semi-legit senders in DNSWL and habeas - a hard problem

[Charles Gregory]

On Tue, 5 Jan 2010, Michael Scheidell wrote:
: > My suggestion: Setup a link/page that provides for rapid reporting by
: > pasting an offending e-mail without a bunch of form-filling. Just use a
: > captcha to avoid poisoning :)
: > - C
: or an industry standard, RFC REQUIRED abuse@ address.

Well, I'm trying to be flexible here, and you have to realize that an 
'abuse@' address can be badly abused at large companies. I can actually 
see the logic where if it is made too *easy* to forward a complaint to 
abuse@ then people adopt the attitude of forwarding *all* the mail 
that they don't like, even if it is not really spam, and with no effort to
determine the actual originating system.

I am lucky enough to get very few complaints here, so I can read them all 
personally. But even so, the majority of them are people complaining 
about spam that merely has *spoofed* our domain name in the From header.
Forcing people to use a web form may be 'non-compliant' to RFC, but it 
puts some thought into the process, and makes people think twice about 
reporting the one 'invite' out of many that they "didn't want". 

But the forms at returnpath are uber unfriendly. They need a quicker 
simpler form. Otherwise, as they sit, it honestly looks like they are 
trying to discourage reports by making them *too* difficult.

- C

SA 3.3.0-rc1 SPF Question

[Jason Bertoch]

A message passed through my server yesterday from an @allianzlife.com 
address with the following Received header:


Received: from allianzlife.com (securemail.allianzlife.com 
[204.52.250.91] (may be forged))


While investigating other issues with the message, I noted that (unless 
my coffee hasn't kicked in yet) it should have matched on SPF_SOFTFAIL, 
but didn't.  In fact, it didn't match on _any_ SPF rules.  I have a 
number of other messages in my logs that hit all of the various SPF 
rules, so I know that things are at least partially working.  The ugly 
spf trace is included in the pastebin link, but essentially this falls 
back on "~all".  Am I missing something obvious?



http://pastebin.com/m1134a08b


/Jason

Speaking of reporting whitelist abusers

[jdow]

How in hell does one report such an abuser of the hostkarma whitelist?

With no method of reporting I am simply turning off their score.

"exprpt.[MUNGE]com" seems to be the culprit reporting themselves as
"ConsumerInfo". It's a credit report scam.

{^_^}

Re: Comparing the envelope-from/sender to the body from to prevent fake local users spams?

[Thomas Harold]

On 1/6/2010 6:47 AM, lstep wrote:


Hello,

I get spams that have an 'Envelope-From' (Sender, or equivalent attribute)
different from the 'From' header contained in the mail. The spam sets the
'From' in the header to an (existing) internal user.

If the spammer would have set the Envelope-From to an internal user as well,
he would have been blocked, not by Spamassassin, but by the MTA (Postfix),
where I set the list of IP allowed to send mail as an internal user.

Is there something implemented in Spamassassin to test and prevent mails
that come from 'outside' that have the header 'From' set to an internal
user?



It's very difficult... checking the MAIL FROM is rather straightforward 
with SPF policy checks.   We test during SMTP session time using 
Postfix's policy service and deny if the SPF record is set to "-all" and 
it doesn't validate.  So nothing with a SPF_FAIL gets as far as SA.


The closest that I got for "From" forgery was to create a few rules and 
then meta them together.  And even then, I don't dare set the score much 
higher then 2.5.


First a rule to see if if the "From" was purporting to be from our domain.

header __LOCAL_FROM_OURDOMAIN
From =~ /\@(domain1|domain2|domain3|domain4)\.com/i

A check to see if it's a mailing list (this needs to be converted into a 
meta-rule that determines a few different ways whether it's a mailing 
list).  On the upside, most mailing lists are listed in one of the DNS 
based whitelists and already come with a negative score when scored by SA.


header __LOCAL_ISMAILINGLIST
List-ID =~ /\<.+\>/

A really braindead test to see whether the Received-SPF line indicated 
that it was from one of our servers.


header LOCAL_SPF_OURSERVER
Recieved-SPF =~ /(domain1|domain2|domain3|domain4)/i
score LOCAL_SPF_OURSERVER -1.0

And finally a meta rule that ties it all together and scores it.

meta LOCAL_FORGED_FROM
(__LOCAL_FROM_OURDOMAIN && !SPF_PASS &&
!LOCAL_SPF_OURSERVER && !__LOCAL_ISMAILINGLIST)
score LOCAL_FORGED_FROM 2.5
describe LOCAL_FORGED_FROM From address is forged as our domain

But I'm not totally convinced that this rule set works properly.  There 
are bound to be corner cases that I haven't considered.  However, on our 
system, we don't delete high-scoring messages - they merely get 
quarantined (moved server-side with a Sieve script) into the user's IMAP 
Junk folder.  And we only quarantine for stuff over ~9.0.  So the cost 
of a high-scoring false-positive is low.

Re: Comparing the envelope-from/sender to the body from to prevent fake local users spams?

[Mike Cardwell]

On 06/01/2010 11:47, lstep wrote:


I get spams that have an 'Envelope-From' (Sender, or equivalent attribute)
different from the 'From' header contained in the mail. The spam sets the
'From' in the header to an (existing) internal user.

If the spammer would have set the Envelope-From to an internal user as well,
he would have been blocked, not by Spamassassin, but by the MTA (Postfix),
where I set the list of IP allowed to send mail as an internal user.

Is there something implemented in Spamassassin to test and prevent mails
that come from 'outside' that have the header 'From' set to an internal
user?


That would break a lot of list mail. Look at the From header compared to 
the envelope sender on this email for example. I *think* you could 
achieve what you're looking for by using DKIM and *requiring* that mail 
from your domain is signed.


--
Mike Cardwell: UK based IT Consultant, LAMP developer, Linux admin
Cardwell IT Ltd. : UK Company - http://cardwellit.com/   #06920226
Technical Blog   : Tech Blog  - https://secure.grepular.com/blog/
Spamalyser   : Spam Tool  - http://spamalyser.com/

Comparing the envelope-from/sender to the body from to prevent fake local users spams?

[lstep]

Hello,

I get spams that have an 'Envelope-From' (Sender, or equivalent attribute)
different from the 'From' header contained in the mail. The spam sets the
'From' in the header to an (existing) internal user.

If the spammer would have set the Envelope-From to an internal user as well,
he would have been blocked, not by Spamassassin, but by the MTA (Postfix),
where I set the list of IP allowed to send mail as an internal user.

Is there something implemented in Spamassassin to test and prevent mails
that come from 'outside' that have the header 'From' set to an internal
user?

Thanks for your help,
Lior
-- 
View this message in context: 
http://old.nabble.com/Comparing-the-envelope-from-sender-to-the-body-from-to-prevent-fake-local-users-spams--tp27026836p27026836.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: [SPAM:9.6] Re: [SPAM:9.6] Re: semi-legit senders in DNSWL and habeas - a hard problem

[Christian Brel]

On Wed, 06 Jan 2010 14:27:25 +0530
ram  wrote:

> On Wed, 2010-01-06 at 07:51 +, Christian Brel wrote:
> > On Tue, 5 Jan 2010 14:18:54 -0800
> > "jdow"  wrote:
> > 
> > > From: "J.D. Falk" 
> > > Sent: Tuesday, 2010/January/05 12:43
> > > 
> > > 
> > > > On Jan 5, 2010, at 10:10 AM, Greg Troxel wrote:
> > > > 
> > > >> Once again I went to returnpath and senderscorecertified's web
> > > >> pages, and found no link to an email address to report being
> > > >> spammed by one of their customers.
> > > > 
> > > > Is the font size for "Contact Us" and "Support" too small?
> > > > 
> > > > I'll forward your report to the appropriate team.
> > > 
> > > J.D., rather than getting snarky it might be a good idea to
> > > suggest to your webmaster that a formal "Report Abuse" link be
> > > placed on your front page? I'd not look to support or contact us
> > > for reporting abuse, myself. So I can understand Greg's problem.
> > > 
> > > {o.o}
> > 
> > I'm jealous, at least you can get a *narky* reply from Return Path.
> > I've been trying for three days
> > 
> > http://www.spampig.org.uk/bbs/showthread.php?tid=31
> > 
> 
> Ebay is definitely a too big spammer. So what if they pay habeas and
> other accreditation lists 
> 
> Their unsubscribe doesnt work.
> I had all notifications off still I used to get their mails. 
> I got fed up of their reminders .. even though I have never purchased
> anything at ebay they keep sending me nonsense
> 
> The only last resort ... I configured a dummy alias on my server and
> changed the ebay notification email address to the dummy alias. 
> After activating the dummy .. now I give a std "450" Try later to all
> mails that come to the dummy.
> 
> 
The point is, if you accredit someone as a email professional, and that
sender fails to act professionally - it's the accreditation that is
brought into question, not the spammy sender. After all, the
accrediation is saying - more or less - that the sender is not a
spammer and will act professionally when complaints are raised.

Just because eBay is a big company does not mean it respects peoples
choices and behaves appropriately.

However, this in *not* the place for that discussion. It just starts a
hissy fit between the 'professional spammers' and those that seek to
stop them.

Sensible folk know people like Return Path will never grow the balls to
stand up to eBay, they will just take the money and smile.

Re: [SPAM:9.6] Re: semi-legit senders in DNSWL and habeas - a hard problem

[ram]

On Wed, 2010-01-06 at 07:51 +, Christian Brel wrote:
> On Tue, 5 Jan 2010 14:18:54 -0800
> "jdow"  wrote:
> 
> > From: "J.D. Falk" 
> > Sent: Tuesday, 2010/January/05 12:43
> > 
> > 
> > > On Jan 5, 2010, at 10:10 AM, Greg Troxel wrote:
> > > 
> > >> Once again I went to returnpath and senderscorecertified's web
> > >> pages, and found no link to an email address to report being
> > >> spammed by one of their customers.
> > > 
> > > Is the font size for "Contact Us" and "Support" too small?
> > > 
> > > I'll forward your report to the appropriate team.
> > 
> > J.D., rather than getting snarky it might be a good idea to suggest to
> > your webmaster that a formal "Report Abuse" link be placed on your
> > front page? I'd not look to support or contact us for reporting
> > abuse, myself. So I can understand Greg's problem.
> > 
> > {o.o}
> 
> I'm jealous, at least you can get a *narky* reply from Return Path.
> I've been trying for three days
> 
> http://www.spampig.org.uk/bbs/showthread.php?tid=31
> 

Ebay is definitely a too big spammer. So what if they pay habeas and
other accreditation lists 

Their unsubscribe doesnt work.
I had all notifications off still I used to get their mails. 
I got fed up of their reminders .. even though I have never purchased
anything at ebay they keep sending me nonsense

The only last resort ... I configured a dummy alias on my server and
changed the ebay notification email address to the dummy alias. 
After activating the dummy .. now I give a std "450" Try later to all
mails that come to the dummy.