Re: Spam slipping through

[Theo Van Dinter]

On Sat, Dec 06, 2008 at 08:00:10PM -0800, John Hardin wrote:
> mechanism for. Devs: there've been wishes for this before; how hard
> would it be to add the ability to match on the substring match captured
> by another rule? Add a flag to say "capture the match for this rule" and
> a syntax for substituting that into the match RE of another rule, and
> dependency enforcement?

Non-trivial.  Write a plugin, where it is trivial.  :)

-- 
Randomly Selected Tagline:
"Advice is kind of like sex. It's not always good, it's not always free
 and you don't always get from the person you want to get it from."
  - Peter Liam Taylor


pgp1xZloiRWbN.pgp
Description: PGP signature

Re: Spam slipping through

[John Hardin]

On Sat, 2008-12-06 at 20:13 +, support wrote:

> Surely, by now, someone has come up with a simple regex rule or
> something that matches if the to & from are the same? Is this too
> obvious?

Unfortunately it's actually not that easy. It involves remembering a
matched substring across *two* rules, which ATM SA does not provide any
mechanism for. Devs: there've been wishes for this before; how hard
would it be to add the ability to match on the substring match captured
by another rule? Add a flag to say "capture the match for this rule" and
a syntax for substituting that into the match RE of another rule, and
dependency enforcement?

In lieu of that, if you're familiar with Perl you could write a plugin
to do what you suggest, or you could do something externally to generate
three rules per known user (or known valid email address) on your
system: one indirect for the To, one indirect for the From, and a meta
to AND them. 

You could do it in sendmail.cf

-- 
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The yardstick you should use when considering whether to support a
  given piece of legislation is "what if my worst enemy is chosen to
  administer this law?"
---
 9 days until Bill of Rights day

Re: Single URI spam not checked against URIBLs

[Theo Van Dinter]

On Sat, Dec 06, 2008 at 11:16:03PM +0100, Wolfgang Zeikat wrote:
> Could you describe more elaborately how you did that?

You may wish to take a look at cpan2rpm, fwiw.

-- 
Randomly Selected Tagline:
"... Either this man is suffering from serious brain damage, or the new 
 vacuum cleaner's arrived..." - Rowan Atkinson


pgpHwSmixaDmg.pgp
Description: PGP signature

Re: Single URI spam not checked against URIBLs

[Ned Slider]

Wolfgang Zeikat wrote:

Ned Slider wrote:


Thanks for the heads up. it indeed works (HTML::Parser 3.59).



For those using RHEL5/CentOS5 and wanting to update,


We use Scientific Linux 5 which is a re-compiled RHEL 5 - with Dag's 
3.56 rpm installed. I installed HTML::Parser 3.59 there from CPAN (with 
local make) without unistalling the rpm. The URI detection behavious 
didn't change, so I am interested in your procedure.


I built a perl-HTML-Parser-3.59 RPM package from Dag's SPEC file 
(v3.56) on RPMForge by dropping in the 3.59 source tarball. It built 
cleanly and is now running on my system :)




Could you describe more elaborately how you did that?

Regards,

wolfgang





Yes, I downloaded the perl-HTML-Parser-3.56 src.rpm package from RPMForge:

http://dag.wieers.com/rpm/packages/perl-HTML-Parser/perl-HTML-Parser-3.56-1.rf.src.rpm

Extract the SPEC file, edit the "Version" and "Release" lines to 3.59 
and 1.el5, respectively.


Download the HTML-Parser-3.59 tarball

http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/HTML-Parser-3.59.tar.gz

Copy the edited SPEC file to the /SPECS dir and the source tarball to 
the /SOURCES dir of your build environment, and build the package with:


rpmbuild -ba --target=`uname -m` perl-HTML-Parser.spec

and install the package with rpm.


Alternatively, I've uploaded my src.rpm here which may be easier:

http://www.pperry.f2s.com/linux/perl-HTML-Parser/

and you can build it with:

rpmbuild --rebuild perl-HTML-Parser-3.59-1.el5.src.rpm

There is a guide on rebuilding source RPMs here:

http://wiki.centos.org/HowTos/RebuildSRPM


I've also uploaded my RPM package there too, but I only built a package 
for x86_64, so if your running a xen kernel or are running on i386 
you'll need to rebuild it yourself.


Hope that helps :)

Re: Single URI spam not checked against URIBLs

[Wolfgang Zeikat]

Ned Slider wrote:


Thanks for the heads up. it indeed works (HTML::Parser 3.59).



For those using RHEL5/CentOS5 and wanting to update,


We use Scientific Linux 5 which is a re-compiled RHEL 5 - with Dag's 
3.56 rpm installed. I installed HTML::Parser 3.59 there from CPAN (with 
local make) without unistalling the rpm. The URI detection behavious 
didn't change, so I am interested in your procedure.


I built a 
perl-HTML-Parser-3.59 RPM package from Dag's SPEC file (v3.56) on 
RPMForge by dropping in the 3.59 source tarball. It built cleanly and is 
now running on my system :)




Could you describe more elaborately how you did that?

Regards,

wolfgang

Re: Single URI spam not checked against URIBLs

[Ned Slider]

mouss wrote:

Bill Landry a écrit :

This issue has been resolved.  Thanks to Justin Mason and Gisle Aas
(HTML::Parser guy) for finding the fix.  The resolution is to update
HTML::Parser to the latest version and then restart SA.



Thanks for the heads up. it indeed works (HTML::Parser 3.59).



For those using RHEL5/CentOS5 and wanting to update, I built a 
perl-HTML-Parser-3.59 RPM package from Dag's SPEC file (v3.56) on 
RPMForge by dropping in the 3.59 source tarball. It built cleanly and is 
now running on my system :)

Re: Spam slipping through

[support]

On Sat, 2008-12-06 at 11:48 -0800, John Hardin wrote:
> On Sat, 6 Dec 2008, Mike Cisar wrote:
> 
> > - the "from" always matches the "to" (so it always looks like its coming
> >   from yourself)
> 
> Silly, basic question: have you whitelist_from'd yourself? Baaad idea.
> 
> SPF checks would catch that if you published SPF records for your domain. 
> If you know that mail from your domain will ever only originate at your 
> MTA, then you might do what I do: use milter-regex to reject at SMTP time 
> any mail inbound from the internet that claims to come from your domain.
> 
> http://www.impsec.org/~jhardin/antispam/
> 
I love these spoofing mails - they are ace. Idea? Well, if you have an
obliging server with NDR's on, it's win win for the spammer. If it's
rejected and generates an NDR, the intended recipient still gets the
spam as an attachment in the NDR. Corking ;-)

Surely, by now, someone has come up with a simple regex rule or
something that matches if the to & from are the same? Is this too
obvious?

Re: Spam slipping through

[John Hardin]

On Sat, 6 Dec 2008, Mike Cisar wrote:


- the "from" always matches the "to" (so it always looks like its coming
  from yourself)


Silly, basic question: have you whitelist_from'd yourself? Baaad idea.

SPF checks would catch that if you published SPF records for your domain. 
If you know that mail from your domain will ever only originate at your 
MTA, then you might do what I do: use milter-regex to reject at SMTP time 
any mail inbound from the internet that claims to come from your domain.


http://www.impsec.org/~jhardin/antispam/

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174 pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  The yardstick you should use when considering whether to support a
  given piece of legislation is "what if my worst enemy is chosen to
  administer this law?"
---
 9 days until Bill of Rights day

Re: Single URI spam not checked against URIBLs

[mouss]

Bill Landry a écrit :
> This issue has been resolved.  Thanks to Justin Mason and Gisle Aas
> (HTML::Parser guy) for finding the fix.  The resolution is to update
> HTML::Parser to the latest version and then restart SA.
> 

Thanks for the heads up. it indeed works (HTML::Parser 3.59).

Re: Spam slipping through

[mouss]

Mike Cisar a écrit :
> Have recently been having 1000's of spam slipping past Spamassassin... they
> all seem to be pretty much identical in format but Spamassassin isn't
> scoring them even high enough to be tagged.
> 
> - they are all flagged as important
> - a single line having so far have one of two common phrases followed by a
> URL (always different) in the format    (angle brackets
> included).
> - the "from" always matches the "to" (so it always looks like its coming
> from yourself)
> 
> I'm sure that at least some of the URL's or messages should be getting
> caught somewhere by SpamAssassin, but they aren't.  So I don't know if
> there's something really crafty about the messages or what.
> 
> Would love to write a custom rule to take care of the problem if I need
> to... but I'm cautious to write a rule based on the two phrases because they
> are guaranteed to trigger a lot of false positive.  I'm thinking it would
> have to be a combination of the phrases, the important flag, the from
> matching to... and something to match that URL format?
> 
> Anybody having problems with this spam, and figured out how to block it?
> 


post a sample on pastebin.com.

I've seen some that have
X-Mailer: %WORD_1 
ratware didn't substitute the variables! SA has rules for these).

if you receive mail via smtp directly (no forwarder, no fetching), then
consider using zen.spamhaus.org.

Re: Spam slipping through

[support]

On Sat, 2008-12-06 at 10:17 -0700, Mike Cisar wrote:
> Have recently been having 1000's of spam slipping past Spamassassin... they
> all seem to be pretty much identical in format but Spamassassin isn't
> scoring them even high enough to be tagged.
> 
> - they are all flagged as important
> - a single line having so far have one of two common phrases followed by a
> URL (always different) in the format    (angle brackets
> included).
> - the "from" always matches the "to" (so it always looks like its coming
> from yourself)
> 
> I'm sure that at least some of the URL's or messages should be getting
> caught somewhere by SpamAssassin, but they aren't.  So I don't know if
> there's something really crafty about the messages or what.
> 
> Would love to write a custom rule to take care of the problem if I need
> to... but I'm cautious to write a rule based on the two phrases because they
> are guaranteed to trigger a lot of false positive.  I'm thinking it would
> have to be a combination of the phrases, the important flag, the from
> matching to... and something to match that URL format?
> 
> Anybody having problems with this spam, and figured out how to block it?
> 
> Thanks much!
> > Mike <
> 
> 
Are the sending IP's on any block lists? Are you doing SPF checks?

Spam slipping through

[Mike Cisar]

Have recently been having 1000's of spam slipping past Spamassassin... they
all seem to be pretty much identical in format but Spamassassin isn't
scoring them even high enough to be tagged.

- they are all flagged as important
- a single line having so far have one of two common phrases followed by a
URL (always different) in the format    (angle brackets
included).
- the "from" always matches the "to" (so it always looks like its coming
from yourself)

I'm sure that at least some of the URL's or messages should be getting
caught somewhere by SpamAssassin, but they aren't.  So I don't know if
there's something really crafty about the messages or what.

Would love to write a custom rule to take care of the problem if I need
to... but I'm cautious to write a rule based on the two phrases because they
are guaranteed to trigger a lot of false positive.  I'm thinking it would
have to be a combination of the phrases, the important flag, the from
matching to... and something to match that URL format?

Anybody having problems with this spam, and figured out how to block it?

Thanks much!
> Mike <

Re: Tagging the mail which already has X-Spam headers

[Bob Proulx]

Nikita Kipriyanov wrote:
> SpamAssassin tags mail with headers X-Spam- But, what if there were  
> some headers like these, as with mail that already passed someones  
> SpamAssassin and has X-Spam-Score, before being recieved by my server?

Those won't matter.

> Will it remove them, replace them or simply add new ones? In the latter  
> case, how do I tell headers, added by my SpamAssassin, from headers,  
> that were there before my mail server?

SpamAssassin will replace existing markup when writing new markup to
the files.  So you will be okay.

Bob

Re: Log

[Theo Van Dinter]

On Sat, Dec 06, 2008 at 01:50:20PM +0100, Jon Essen-Moller wrote:
> So you look in the /var/log/maillog (maybe with grep) and find messages 
> and their id you are interested in. I get you that far.

:)

> Are there a log somewhere where one can find information like the last 
> log entry you pasted below?

The last log entry was from spamd which logs (by default) to syslog's mail
facility, and so typically ends up in maillog with everything else.  If you're
using a different method for calling SA (third party daemon, etc,) then they
may log differently and you'd have to talk to those folks about what they do.

-- 
Randomly Selected Tagline:
"Your next question is 'How does this gate work?'  I don't know.  I
 don't have to know, I'm not an Electrical Engineer, I'm a Computer
 Scientist."  - Prof. Hamel


pgpeHbkD6j6Vx.pgp
Description: PGP signature

Re: Sa-update exit codes

[mouss]

Arthur Dent a écrit :
> Hello All,
> 
> I have the following command running daily in my crontab on my Fedora 9
> box: (excuse the linewrap)
> 
> sa-update --channelfile
> /etc/mail/spamassassin/sare-sa-update-channels.txt --gpgkey 856AA88A
> --gpgkey 6C6191E3 && /sbin/service spamassassin restart
> 
> and every day I get a reassuring email in my root system email showing
> the following:
> 
> Stopping spamd: [  OK  ]
> Starting spamd: [  OK  ]
> 
> The day before yesterday however the spamd restart messages stopped. I
> investigated by running the above command with the -D switch and found -
> unsurprisingly that all of the channels gave results like:
> 
> [19758] dbg: channel: metadata version = 320722979
> [19758] dbg: dns: 5.2.3.sought.rules.yerp.org => 320722979, parsed as
> 320722979
> [19758] dbg: channel: current version is 320722979, new version is
> 320722979, skipping channel
> 
> showing that all the channels were up-to-dateup-to-date.
> 
> The last line however was:
> [19758] dbg: diag: updates complete, exiting with code 1
> 
> Is this an error code?


I use (after sa-update command):


errorcode=$?

if [ $errorcode = 1 ]; then
output "No fresh updates available"
exit 0
fi

if [ $errorcode -gt 1 ]; then
output "Channel update failed (err=$errorcode)"
exit 1
fi

output "channels updated"

output "Compiling rules"
(echo -n ""; date) >  /var/log/sa-compile.out
sa-compile  >>  /var/log/sa-compile.out 2>&1

output "Restarting amavisd"
/usr/local/etc/rc.d/amavisd restart

exit 0


> Does this explain why the "&& /sbin/service spamassassin restart" fails
> to run now?
> 

no. you should be able to restart sa anytime you want!

> Can it be true that there have been NO updates to SA rules, SARE or
> Sought Rules in the last 2 days?...

The only "exception" I see is 90_2tld_cf_sare_sa-update_dostech_net.cf
which was updated on 06-Dec-2008 05:14.

Sought shows a date of: 03-Dec-2008 04:32. This is unusual. maybe it's
related to svn.apache.org certificate renewal?

Re: Log

[Jon Essen-Moller]

Hi,

Thanks again for your answer and sorry for the HTML ;-|

So you look in the /var/log/maillog (maybe with grep) and find messages 
and their id you are interested in. I get you that far.


Are there a log somewhere where one can find information like the last 
log entry you pasted below?


Best regards - Jon

Theo Van Dinter wrote:

On Fri, Dec 05, 2008 at 12:53:20AM +0100, Jon Essen-Moller wrote:

the mail was in HTML, so it's basically unreadable.  text please.

I did get out of it:

"I wish to check a specific mail address and see if many mails are
classified as spam that are sent to that address."

It sounds like you want SA statistics instead of information out of Bayes.  SA
doesn't keep track of this kind of information.  You probably want to take a
look at your mail log (or wherever the appropriate location is for however you
run SA) to get that kind of information.

For example:

Nov 30 04:03:08 eclectic postfix/smtpd[617]: EF297AF143: 
client=p4FCCB2A7.dip.t-dialin.net[79.204.178.167]
Nov 30 04:03:09 eclectic postfix/cleanup[608]: EF297AF143: message-id=<[EMAIL 
PROTECTED]>
Nov 30 04:03:09 eclectic postfix/qmgr[12948]: EF297AF143: from=<[EMAIL 
PROTECTED]>, size=1814, nrcpt=2 (queue active)
Nov 30 04:03:10 eclectic postfix/local[32692]: EF297AF143: to=<[EMAIL PROTECTED]>, 
orig_to=<[EMAIL PROTECTED]>, relay=local, delay=2, status=sent (delivered to command: 
/usr/bin/procmail -a "$EXTENSION")
Nov 30 04:03:17 eclectic postfix/local[917]: EF297AF143: to=<[EMAIL PROTECTED]>, 
orig_to=<[EMAIL PROTECTED]>, relay=local, delay=9, status=sent (delivered to command: 
/usr/bin/procmail -a "$EXTENSION")
Nov 30 04:03:17 eclectic postfix/qmgr[12948]: EF297AF143: removed

So if I was interested in mails to [EMAIL PROTECTED], this would come up, and I
see the message-id in there.  Then, since I use spamd, I can figure out what
the results were:

Nov 30 04:03:10 eclectic spamd[336]: spamd: result: Y 17 - BAYES_99,DCC_CHECK,DIGEST_MULTIPLE,DRUGS_MUSCLE,FB_CIALIS_LEO3,FB_GET_MEDS,FR_ALMOST_VIAG2,FUZZY_MEDICATION,FUZZY_PRICES,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_PBL,RDNS_DYNAMIC,VIA_GAP_GRA scantime=1.5,size=1964,user=felicity,uid=501,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=33858,mid=<[EMAIL PROTECTED]>,bayes=1.00,autolearn=disabled 


Your system may vary entirely.

  

Re: Sa-update exit codes

[Karsten Bräckelmann]

On Sat, 2008-12-06 at 10:17 +, Arthur Dent wrote:
> On Sat, Dec 06, 2008 at 09:38:06AM +, Arthur Dent wrote:
> > Is this an error code?
> > Does this explain why the "&& /sbin/service spamassassin restart" fails
> > to run now?

Yes, it is. Yes, it does.  As documented by 'man sa-update'. And
intended, also as per the logic of your cron job. ;)

> Sigh...
> 
> OK - I've had my morning cup of tea now. I have now actually turned on
> my brain.

Lucky you. Didn't have my coffee yet and barely slept at all. *sigh*

> I realise of course that the question I should have posed is:
> 
> Can it be true that there have been NO updates to SA rules, SARE or
> Sought Rules in the last 2 days?...

Yes, espacially with SARE and the official SA update channel. It's
entirely normal and expected for those to see no update even in weeks.

> Sorry - I'm so used to regular updates from (especially JM Sought) that
> I thought something must be wrong when I saw no restart.

This indeed is unusual for Sought. Noticed this myself already, last
update on Wed. Did happen in the past, though. I wouldn't be alarmed by
it just yet.


> It's Saturday. I think I'll go back to bed...

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Sa-update exit codes

[Arthur Dent]

On Sat, Dec 06, 2008 at 09:38:06AM +, Arthur Dent wrote:
> Is this an error code?
> Does this explain why the "&& /sbin/service spamassassin restart" fails
> to run now?

Sigh...

OK - I've had my morning cup of tea now. I have now actually turned on
my brain.

I realise of course that the question I should have posed is:

Can it be true that there have been NO updates to SA rules, SARE or
Sought Rules in the last 2 days?...

Sorry - I'm so used to regular updates from (especially JM Sought) that
I thought something must be wrong when I saw no restart.

It's Saturday. I think I'll go back to bed...

AD


pgpl2YlKPbU1N.pgp
Description: PGP signature

Sa-update exit codes

[Arthur Dent]

Hello All,

I have the following command running daily in my crontab on my Fedora 9
box: (excuse the linewrap)

sa-update --channelfile
/etc/mail/spamassassin/sare-sa-update-channels.txt --gpgkey 856AA88A
--gpgkey 6C6191E3 && /sbin/service spamassassin restart

and every day I get a reassuring email in my root system email showing
the following:

Stopping spamd: [  OK  ]
Starting spamd: [  OK  ]

The day before yesterday however the spamd restart messages stopped. I
investigated by running the above command with the -D switch and found -
unsurprisingly that all of the channels gave results like:

[19758] dbg: channel: metadata version = 320722979
[19758] dbg: dns: 5.2.3.sought.rules.yerp.org => 320722979, parsed as
320722979
[19758] dbg: channel: current version is 320722979, new version is
320722979, skipping channel

showing that all the channels were up-to-dateup-to-date.

The last line however was:
[19758] dbg: diag: updates complete, exiting with code 1

Is this an error code?
Does this explain why the "&& /sbin/service spamassassin restart" fails
to run now?

If necessary I can post (or paste to pastebin) the whole debug report...

Thanks in advance for any help or suggestions...

AD



pgpxO6exDVEDK.pgp
Description: PGP signature