Re: RulesDuJour and Curl Connect Problem

[Yousef Raffah]

On Sat, 2006-02-11 at 15:01 +0300, Yousef Raffah wrote:
> On Sat, 2006-02-11 at 11:42 +, Shane Kelly wrote:
> > Yousef Raffah wrote:
> > > On Wed, 2006-02-08 at 15:41 +, Shane Kelly wrote:
> > >> Hi Yousef,
> > >>
> > >>
> > >>> I have to connect through a proxy server to get to the internet but I'm
> > >>> not sure how to set the proxy for curl. I tried to change the line
> > >>> in /var/lib/spamassassin/rules_du_jour
> > >>> To this:
> > >>>  [ "${CURL_OPTS}" ] || CURL_OPTS="-w %{http_code} --compressed -O -R -s
> > >>> -S -z -x proxy.nour.net.sa:8080";
> > >>>
> > >>> as I understood from man curl that -x is the parameter to use for a
> > >>> proxy server!
> > >>>
> > >>> The only relevant thread I found so far is this:
> > >>> http://thread.gmane.org/gmane.mail.spam.spamassassin.general/76192
> > >>> but I have the outbound port 80 open and it is confirmed as when I
> > >>> invoke the script manually (not through cron) it works perfectly (at
> > >>> least that is what I see :) )
> > >> I suspect you need to set your proxy settings in the crontab, as most 
> > >> recent crons run with a clean environment for security. (I got bitten by 
> > >> the same thing :-) )
> > >>
> > > Hummm, that's pretty interesting, will test and let you know, because I
> > > though I had it already. Just out of curiousity, should I specify the
> > > proxy (export the variables in crontab) or the environment variable
> > > should just be "there"? Currently I have the variable set for the root
> > > user in the bash profile
> > 
> > I have these as lines above the cron entries in the system crontab (i.e. 
> > /etc/crontab) on a Suse 9.3 system, but below the shell, path and mailto 
> >   vars. Both wget and curl pick them up from there.
> > 
> > HTTP_PROXY='http://wwwcache.xxx.xx.uk:8080'
> > http_proxy='http://wwwcache.xxx.xx.uk:8080'
> > 
> Cool, I just configured it. Will let you know if it worked tonight ;).
> Many thanks.
> 
Unfortunately it didn't work out :/ Could it be the double quotes? I
always have problems whether it is a single or double quote!

Sincerely,
Yousef Raffah
Senior Systems Administrator
SSIS - The Savola Group

--
Aren't you using Firefox? Get it at getfirefox.com
yousef.raffah.com


signature.asc
Description: This is a digitally signed message part

Re: lots of new spam

[Russ B.]

I wrote these rules last week that stop em fast, even before the URIBL's
kick in.

# This will fire if 2 or more are found
rawbody __DRUGS268A /^V$/i
rawbody __DRUGS268B /^I$/i
rawbody __DRUGS268C /^C$/i
rawbody __DRUGS268D /^E$/i
rawbody __DRUGS268E /^33$/i
rawbody __DRUGS268F /^\=20$/i
meta DRUGS268 (( __DRUGS268A + __DRUGS268B + __DRUGS268C + __DRUGS268D +
__DRUGS268E + __DRUGS268F) > 1)
score DRUGS268 105.5
describe DRUGS268 Disguised Drug Message


rawbody URL52 /\.\.org\/(?:..|...)\//i
score URL52 6.5
describe URL52 Short Drug URL

rawbody URL52a /\..\.org\/(?:..|...)\//i
score URL52a 6.5
describe URL52a Short Drug URL

rawbody URL52b /\...\.org\/(?:..|...)\//i
score URL52b 6.5
describe URL52b Short Drug URL

rawbody URL52c /\\.org\/(?:..|...)\//i
score URL52c 6.5
describe URL52c Short Drug URL

Re: General assistance

[Loren Wilton]

> b. Does anyone have any utilities to get statistics from SA?  Such as

Can't help you on your first question, but likely someone else can.

On the second question, there are two different stats scripts.  Confusingly
enough they are BOTH named sa_stats.pl.

One is distributed with SA itself.  I forget the directory where it ends up,
but digging for sa_stats.pl should turn it up.

The other one was written by Dallas, and is available on the rulesemporium
website.

I believe both of these just dig through the log to get their answers.

Loren

Re: getmail?

[Craig White]

On Sat, 2006-02-11 at 23:23 +, Craig McLean wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Gene Heskett wrote:
> [snip sendmail discussion]
> 
> > I've about come to that conclusion myself, so I'm now investigating the 
> > fetchmail->procmail_>dovecot solution right now.  But the dovecot 
> > mailing list might be a problem, I've subbed about an hour ago but have 
> > rx'd no please confirm message yet.
> > 
> > Joanne has me about straight on the fetchmail and procmail stuffs, and I 
> > may even see if I can turn that part on just for grins, but 
> > dovecot's .conf looks like it'll need a philly lawyer to decode it 
> > correctly so it works.
> 
> Heh, yeah. The dovecot config can be pretty daunting, I'll try and
> summarise how I've got it set up here, but many things may not be needed
> where you are.
> The only uncommented lines in my config are:
> 
> - -quote-
> protocols = imap imaps # We don't use POP
> ssl_cert_file = /etc/mail/certs/fukka.co.uk.cert # SSL stuff
> ssl_key_file = /etc/mail/certs/fukka.co.uk.key   # SSL stuff
> disable_plaintext_auth = no# Nasty Squirrelmail
># hack
> login_user = dovecot   # Discrete user for
># processes
> login_processes_count = 1  # Tuning
> login_max_processes_count = 12 # Tuning
> login_max_logging_users = 12   # Tuning
> first_valid_uid = 1000 # Security
> first_valid_gid = 0# Hack for my GID
> mail_extra_groups = mail   # Permissions tweak
> default_mail_env = mbox:/var/mail/%u   # YMMV - check the docs
> lock_method = flock# Multiple things lock
># mail here
> maildir_copy_with_hardlinks = yes  # Dunno. Check docs
> mbox_read_locks = flock# Locking
> mbox_write_locks = flock   # Locking
> mbox_lazy_writes = no  # Tweak
> protocol imap {# IMAP settings in {}
>   login_greeting_capability = no
>   imap_client_workarounds = delay-newmail outlook-idle netscape-eoh
> tb-extra-mailbox-sep
> }
> auth_verbose = yes # Just because
> auth default { # User auth setings in
># {}
>   mechanisms = plain
>   passdb pam {
>   }
>   userdb passwd {
>   }
>   user = root
> }
> - -quote-
> 
> I've found the docs for dovecot to be fairly good, if a little tech-heavy.
> 
> On the other hand, FC also includes both UW-IMAP and Cyrus, more about
> UW at http://www.washington.edu/imap/ and Cyrus at
> http://asg.web.cmu.edu/cyrus/imapd/
> 
> Either of these is likely to be easier to configure that dovecot.
> 
> >> In a configuration where you don't readily run sendmail to accept
> >> mail, I would suggest staying the hell away from it and:
> > 
> > Sendmail does run to collect local mail here, like from amanda and 
> > cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
> > collect mail from the firewall box so I didn't have to log in via ssh 
> > 2-3 times a week and read the chkrootkit reports and such.  Its 
> > normally a mounted samba share from here, so maybe I could get kmail to 
> > do that now that I think about it.  Humm, off to try it by golly.
> > 
> 
> You'll be a whizz at installing IMAP servers soon, you could install one
> on the firewall box and use fetchmail to pull it onto the main server.
> Assuming you felt suitably insane.

he can also ask dovecot questions on fedora list too rather than adding
noise here.

Craig

Re: getmail?

[Craig McLean]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Gene Heskett wrote:
[snip sendmail discussion]

> I've about come to that conclusion myself, so I'm now investigating the 
> fetchmail->procmail_>dovecot solution right now.  But the dovecot 
> mailing list might be a problem, I've subbed about an hour ago but have 
> rx'd no please confirm message yet.
> 
> Joanne has me about straight on the fetchmail and procmail stuffs, and I 
> may even see if I can turn that part on just for grins, but 
> dovecot's .conf looks like it'll need a philly lawyer to decode it 
> correctly so it works.

Heh, yeah. The dovecot config can be pretty daunting, I'll try and
summarise how I've got it set up here, but many things may not be needed
where you are.
The only uncommented lines in my config are:

- -quote-
protocols = imap imaps   # We don't use POP
ssl_cert_file = /etc/mail/certs/fukka.co.uk.cert # SSL stuff
ssl_key_file = /etc/mail/certs/fukka.co.uk.key   # SSL stuff
disable_plaintext_auth = no  # Nasty Squirrelmail
 # hack
login_user = dovecot # Discrete user for
 # processes
login_processes_count = 1# Tuning
login_max_processes_count = 12   # Tuning
login_max_logging_users = 12 # Tuning
first_valid_uid = 1000   # Security
first_valid_gid = 0  # Hack for my GID
mail_extra_groups = mail # Permissions tweak
default_mail_env = mbox:/var/mail/%u # YMMV - check the docs
lock_method = flock  # Multiple things lock
 # mail here
maildir_copy_with_hardlinks = yes# Dunno. Check docs
mbox_read_locks = flock  # Locking
mbox_write_locks = flock # Locking
mbox_lazy_writes = no# Tweak
protocol imap {  # IMAP settings in {}
  login_greeting_capability = no
  imap_client_workarounds = delay-newmail outlook-idle netscape-eoh
tb-extra-mailbox-sep
}
auth_verbose = yes   # Just because
auth default {   # User auth setings in
 # {}
  mechanisms = plain
  passdb pam {
  }
  userdb passwd {
  }
  user = root
}
- -quote-

I've found the docs for dovecot to be fairly good, if a little tech-heavy.

On the other hand, FC also includes both UW-IMAP and Cyrus, more about
UW at http://www.washington.edu/imap/ and Cyrus at
http://asg.web.cmu.edu/cyrus/imapd/

Either of these is likely to be easier to configure that dovecot.

>> In a configuration where you don't readily run sendmail to accept
>> mail, I would suggest staying the hell away from it and:
> 
> Sendmail does run to collect local mail here, like from amanda and 
> cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
> collect mail from the firewall box so I didn't have to log in via ssh 
> 2-3 times a week and read the chkrootkit reports and such.  Its 
> normally a mounted samba share from here, so maybe I could get kmail to 
> do that now that I think about it.  Humm, off to try it by golly.
> 

You'll be a whizz at installing IMAP servers soon, you could install one
on the firewall box and use fetchmail to pull it onto the main server.
Assuming you felt suitably insane.

Regards,
C.
- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD7nHaMDDagS2VwJ4RAklCAJ4yFrD5DTEtx6kY6fM/wdr9ocsESwCfZQfB
wnAhZlgEFECvt98TsXiL5GA=
=8ACT
-END PGP SIGNATURE-

Re: getmail?

[Gene Heskett]

On Saturday 11 February 2006 08:25, Craig McLean wrote:
>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA1
>
>Gene Heskett wrote:
>[snip fetchmail discussion]
>
>> In further reading tonight, sendmail grew the libmilter freature at
>> 8.12, which is the base version running here, and yum won't update
>> it, says its current.
>
>What version of the OS are you running, Gene? FC4 has 8.13.4-2 as the
>latest, not that it necessarily makes any odds.
>
FC2 with lots of tarball installed stuff to replace the originally drain 
bamaged FC2 stuff, like cups, gutenprint (pick a random proggy, its 
possibly a tarball install, or maybe a checkinstalled version.

>> Right now, I'm looking at the
>>  site, trying to see how
>> this is done.
>>
>> But, here is the headache:  At no place in the various files sitting
>> in /etc/mail that serve to configure sendmail, is there an example
>> of how to configure sendmail to make use of these feature
>> facilities.
>
>Basic milter information can be found at:
>http://www.sendmail.org/~ca/email/doc8.12/cf/m4/adding_mailfilters.htm
>l and more in-depth here:
>http://www.milter.org/
>
>An example of how to get sendmail to use spamass-milter (and
>clamav-milter, I use both) looks like this, from sendmail.mc:
>
>- -quote-
>dnl ** Milter Configurations **
>define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
>{if_addr}')
>INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock,
> F=, T=S:4m;R:4m')dnl
>INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
>T=C:15m;S:4m;R:4m;E:10m')
>dnl define(`confINPUT_MAIL_FILTERS', `spamassassin,clmilter')
>- -quote-
>
>Over here I use spamass-milter to pass mail to spamc as it passes
>through the MTA, because I *am* my domain's MX.
>This means sendmail needs to be configured to accept mail via SMTP
> which is fine for me, but might be far more overhead than you need.
>
>> Spamassassin 3.10 contains only very scant references to using it
>> with sendmail, apparently sanctioning only the procmail interface,
>> which in turn then is set to call spamc or spamassassin, adding
>> needless time wasting cpu cycles to what should be a pretty simple
>> job.  I fail to understand why (although it will take smarter people
>> than me what with sendmails configuration complexity) there is no
>> readily published recipe for incorporating spamc into the sendmail
>> processing chain, either by pipeing, or when the libmilter feature
>> is there?
>
>libmilter just provides a mechanism for sendmail to pass the email,
> via a socket, to a small C program, thence to spamc. Talk about
> "needless time wasting CPU cycles"?

I've about come to that conclusion myself, so I'm now investigating the 
fetchmail->procmail_>dovecot solution right now.  But the dovecot 
mailing list might be a problem, I've subbed about an hour ago but have 
rx'd no please confirm message yet.

Joanne has me about straight on the fetchmail and procmail stuffs, and I 
may even see if I can turn that part on just for grins, but 
dovecot's .conf looks like it'll need a philly lawyer to decode it 
correctly so it works.

>In a configuration where you don't readily run sendmail to accept
> mail, I would suggest staying the hell away from it and:

Sendmail does run to collect local mail here, like from amanda and 
cron/logwatch, that sort of stuff.  And I'd like to figure out a way to 
collect mail from the firewall box so I didn't have to log in via ssh 
2-3 times a week and read the chkrootkit reports and such.  Its 
normally a mounted samba share from here, so maybe I could get kmail to 
do that now that I think about it.  Humm, off to try it by golly.

>a) configuring fetchmail to simply use procmail as the MDA. ("--mda
>/usr/bin/procmail" or similar, IIRC)
>b) having procmail run everything handed to it through spamc, and
> filter accordingly.
>
>Peice of cake (relatively speaking) to set up, no sendmail black magic
>and fairly quick to run.
>
>> Or am I simply on the wrong mailing list?  I've sent 3 subscribe
>> messages to the getmail-user list over the last 3 days with no
>> response which is discouraging.  OTOH, now that I know it can't do
>> what I want, who cares.  It might be that if there was a manpage for
>> getmail, it might be possible.  A pox on software that doesn't come
>> with readable manuals.
>
>Or *any* manuals

Yup.  I wonder if the author is reading the traffic.  Obviously not, 
else I'd think the background noise would prompt an attempt at it at 
least. :)

Thanks Craig.

-- 
Cheers, Gene
People having trouble with vz bouncing email to me should add the word
'online' between the 'verizon', and the dot which bypasses vz's
stupid bounce rules.  I do use spamassassin too. :-)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

RE: General assistance

[Ed Russell]

I have to say a heartfelt THANK YOU to everyone who contributed to this
thread.  My filter is working 500% more efficient that it ever was.  I have
done the following:

1.  Installed djbdns and I am using dnscache as I was told.  I have
increased the cache size to 100 Megabytes and completely disabled logging
after determining it was working properly.

2.  I have implemented rbl at the MTA level, I use relays.ordb.org and
sbl-xbl.spamhaus.org.

3.  I have implemented Rules Du Jour.  I selected a subset of the SARE
rules and misc others.

4.  I have turned back on pyzor, razor and dcc.

Scanning times are well within tolerance with a minimal impact on delivery
time.  See below (email addresses removed for privacy):

Feb 11 16:10:18 as spamd[4137]: spamd: identified spam (31.3/4.0) for
[EMAIL PROTECTED] :99 in 4.5 seconds, 1178 bytes. 
Feb 11 16:10:18 as spamd[363]: spamd: clean message (1.2/4.0) for
[EMAIL PROTECTED] :99 in 3.1 seconds, 8939 bytes. 
Feb 11 16:10:19 as spamd[4218]: spamd: clean message (0.0/4.0) for
[EMAIL PROTECTED] :99 in 5.4 seconds, 2245 bytes.

I have some final questions though,

a.  Can I get any statistics from rblsmtpd (I know this isn't a group
devoted to it, but I figured I would ask)?  I would like to know how many
got dropped and from where.

b.  Does anyone have any utilities to get statistics from SA?  Such as
what rules triggered spam etc etc.  I have seen some posts with some
interesting looking reports.  Currently I only use a hacked together script
I wrote to give me the raw amount of spam caught per day which greps
"identified spam" on maillog and then gives me a wc -l.

Once again, thanks so much to everyone.  This group is simply amazing.

Ed




-Original Message-
From: DAve [mailto:[EMAIL PROTECTED] 
Sent: Friday, February 10, 2006 1:19 PM
To: users@spamassassin.apache.org
Subject: Re: General assistance

Ed Russell wrote:
> User validation is going to be tough or all but impossible.  This box
> forwards off the mail to an NT box running SL Mail.  There is no easy way
to
> get a userlist out of this product.  In addition the users change daily
and
> some even use multi-drops.  

You don't need to get a user list, you just need to ask the destination 
server if the user exists before accepting the message. This is what 
milter-ahead does on my MailScanner servers. I process and forward to 
servers running qmail(my toasters) and Exchange, GroupMail, Groupwise, 
Sendmail(my clients servers). All respond correctly to milter-ahead. I 
do not know of a way to duplicate milter-ahead in qmail without 
requiring something like vpopmail or LDAP.

Did you look at using dnscache? That might buy you enough breathing room 
to shop around for a solution to user verification.

DAve



> 
> Ed
> 
> 
> ---
> 
>  Talk is cheap since supply always exceeds demand.
> 
> ---
>  
> 
> -Original Message-
> From: DAve [mailto:[EMAIL PROTECTED] 
> Sent: Friday, February 10, 2006 12:39 PM
> To: users@spamassassin.apache.org
> Subject: Re: General assistance
> 
> Ed Russell wrote:
> 
>>[EMAIL PROTECTED] smtpd]# spamassassin --version
>>SpamAssassin version 3.1.0
>>  running on Perl version 5.8.7
>>
>>
>>Spamd running with:
>>OPTIONS="-L -x -d -u nobody -m 45"
>>
>>No user verification or RBL at the MTA level.
> 
> 
> Absolutely do user verification. I can throw out from 20% to 80% of my 
> traffic depending on the current level of dictionary and Joe-Job 
> attacks. Since you are processing ahead of your clients Exchange boxes 
> I'm not sure how you can do that with qmail. I do it on my gateways 
> running MailScanner via milter-ahead, and on my toasters via checkuser 
> in vpopmail.
> 
> There might be a way to get qmail to check with an Exchange box to 
> validate a user without running vpopmail, but I won't know it.
> 
> DAve
> 
> 
>>
>>12:20pm  up  4:05,  1 user,  load average: 9.49, 9.23, 9.23
>>313 processes: 300 sleeping, 12 running, 1 zombie, 0 stopped
>>CPU states: 18.9% user, 16.6% system,  0.0% nice, 64.4% idle
>>Mem:  2009856K av,  711560K used, 1298296K free,  353776K shrd,  129268K
>>buff
>>Swap: 2097136K av,   0K used, 2097136K free  225380K
>>cached
>>
>>As you can see I have loads of head room as far as memory goes.  I was
>>looking into integrating RBL into Qmail, but with the very high volume I
> 
> am
> 
>>quite concerned that this will introduce a slowdown.  If I increase the
>>inbound concurrent rate I eventually run into qmail-scanner problems with
>>reformime.  Is there anything else I need consider?
>>
>>Ed
>>
>>---
>>
>> Talk is cheap since supply always exceeds demand.
>>
>>---
>> 
>>
>>-Original Message-
>>From: Kristopher Austin [mailto:[EMAIL PROTECTED] 
>>Sent: Friday, February 10, 2006 12:06 PM
>>To: [

RE: General assistance

[Ed Russell]

You are completely correct, qmail-scanner does use spamc to talk to the
already running spamd.  I just had trouble explaining what the setup was
I may indeed look into having procmail be the agent for Spamassassin.  As
for automatic deletion, well that's a decision we made and for the most part
it works.  We just ensure that we are not too aggressive on the rules.

Ed


-Original Message-
From: jdow [mailto:[EMAIL PROTECTED] 
Sent: Saturday, February 11, 2006 12:28 AM
To: users@spamassassin.apache.org
Subject: Re: General assistance

No, Ed, qmail-scanner should not initiate spamd. It should use spamc to
call the already running spamd. I hope that is what you mean. That is
what stood the hairs on end. It made me wonder if you really knew what
was going on. {o.o}

And seriously, if you are using procmail it's perhaps better to fire
off spamc from procmail. That way you can skip SA scanning for some
specific addresses, if you want. Or you can skip SA scanning if the
message size is too big. If you're running procmail anyway it might as
well be the agent for running SpamAssassin. That way you are SURE the
markups are there for when you delegate the spam to /dev/null.

And as a general rule I believe dumping mail to /dev/null is asking for
"I sent you the ebay notifications you needed! I can't help it if your
spam filter deleted them! Why'd you give me a bad review, you [EMAIL PROTECTED]
3-)(#$*&&&!"

{o.o}
- Original Message - 
From: "Ed Russell" <[EMAIL PROTECTED]>
To: 
Sent: 2006 February, 10, Friday 20:11
Subject: RE: General assistance


>I think you are confused as to how I have set this up.  Qmail-scanner is my
> replacement qmail queue.  Qmail simply receives mail from the outside
world,
> then passes it to qmail-scanner for processing.  Qmail-scanner initiates
> spamd which scans the mail and off it goes.  From there procmail will look
> at the mail and determine if the spam status is marked in the header, if
yes
> it kills the mail, if not it passes it along.  Keep in mind no users
> whatsoever live on this box.  It is as I mentioned a pass through filter.

> 
> 
> 
> -Original Message-
> From: jdow [mailto:[EMAIL PROTECTED] 
> Sent: Friday, February 10, 2006 10:55 PM
> To: users@spamassassin.apache.org
> Subject: Re: General assistance
> 
> From: "Ed Russell" <[EMAIL PROTECTED]>
> 
>> If everyone would indulge me I would like to put forth the setup I am
>> utilizing and get some feedback.   I have a box that I have been using
for
>> some time which acts as a pass-through filter for many domains (currently
>> about 100) for spam, this is a fairly high traffic server processing
about
>> 150,000 to 200,000 messages per day.  I use the following method.
>> 
>> Based upon a redhat 6.2 box running kernel 2.2.26, PIV with 2 Gigs of
RAM.
>> 
>> Qmail runs which accepts the email from the world (with a
>> concurrencyincoming of 100) and passes it through qmail-scanner (which
> calls
>> spamd) and spamassassin which checks the email and writes spam status to
> the
>> header.  Each message gets then passed through a procmail filter which
> will
>> delete it if it is spam.  The procmail filter is:
> 
> I note the other answers and thought I'd comment because the above
> description of your mail topology raised the hairs on the back of my
> neck. (And that takes doing considering their length. {^_-})
> 
> First I not you say Qmail (it's own punishment) feets qmail-scanner.
> The qmail-scanner calls spamd? Naw, can't DO that. AND it calls
> spamassassin? That's even stranger. But then it goes to procmail for
> the delivery.
> 
> My topology is somewhat different but useful. If you are using
qmail-scanner
> only to make the spamassassin run and the procmail run then jettison it
> and go to procmail directly. That MAY reduce the machine load a little.
> Also make sure spamd is running, exactly once, from your /etc/init.d
> files or the equivalent on BSDs. You'd then use spamc to get to the
> SpamAssassin run. You show some data below. (I am not sure what the
> EXITCODE is supposed to do for you. I never set it here. But that may
> be because I use procmail alone. It exits and mail is "delivered" either
> to a diversion directory, /dev/null, or the user's mailbox.)
> 
> Anyway, you can call spamc from inside procmail this way:
> 
> :0
> * < 50
> * !^List-Id: .*(spamassassin\.apache.\org)
> | /usr/bin/spamc -t 150 -u $USER
> 
>> :0
>> * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>> {
>>EXITCODE=99
>>:0
>>/dev/null
>> }
>> 
>> :0
>> * ^X-Spam-Status: Yes
>> {
>>EXITCODE=99
>>:0
>>/dev/null
>> }
>> 
>> :0
>> * ^^rom[ ]
>> {
>>  LOG="*** Dropped F off From_ header! Fixing up. "
>>  
>>  :0 fhw
>>  | sed -e '1s/^/F/'
>> }
>> 
>> :0
>> /dev/null
>> 
>> Mail that is clean gets passed off to a second qmail install which then
>> delivers the mail to our servers using smtproutes.
> 
> Ouch. And what is that final redirect of EVERYTHING to /dev/

Stock image woes

[qqqq]

All,

Is anybody having any luck with the Stock spam that consists of an image and 
"noise" to through off bayes?

One example is for (CIVX)

TIA,

Re: [Norton AntiSpam] Re: Question about configuration

[Awie]

> > I want to integrate Spamassassin with qmail + Qmail-Scanner to mark
email
> > that be suspected SPAM, with ***SPAM*** in it's subject.
> >
> >  Should I remove the # sign in local.cf? # rewrite_header Subject
> > *SPAM*
> >
> > Please advise. Thanks alot for your help
> >
> That depends on how you have qmail-scanner set up.. If I recall
> correctly, qmail-scanner does it's own tagging if you have the "fast
> spamassassin" option set. If you want SA to do the tagging you have to
> disable that. Otherwise, configure qmail-scanner's tagging options.
>
> Realistically there is speed difference with fast spamassassin, but it's
> not very large. The big difference is that in fast mode SA doesn't mark
> and pipe the message back, all it gives is a score. (qmail-scanner uses
> spamc with the -c option).

Thanks Matt,

I will do some trials to configure Q-S. My SA installation is successfully
detect GTUBE, so I assume that SA is running well.

Thx & Rgds,

Awie

Re: Spammasssin skips rules?

[Joost Kraaijeveld]

Thanks. Setting the score to 0.001 did the trick.

-- 
Groeten,

Joost Kraaijeveld
Askesis B.V.
Molukkenstraat 14
6524NB Nijmegen
tel: 024-3888063 / 06-51855277
fax: 024-3608416
e-mail: [EMAIL PROTECTED]
web: www.askesis.nl 

Re: Spammasssin skips rules?

[Craig McLean]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Joost Kraaijeveld wrote:
> Hi,
>
> I have two rules in a file "/etc/spamassassin/80_jkr.cf" that seem be to
> skipped by spamassassin:
>
> bodyJKR_TEST_BODY_RULE/e/
> score   JKR_TEST_BODY_RULE0.0
> describeJKR_TEST_BODY_RULEJKR body rule
>
> header  JKR_TEST_HEADER_RULE   Subject =~ /e/
> score   JKR_TEST_HEADER_RULE   0.0
> describeJKR_TEST_HEADER_RULE   JKR header rule
>
>
[snip]

> Is my regex wrong or are there circumstances that Spamassassin skips any
> rules?
>
>

Rules scored at 0 will be skipped by SA. If you want the rule to fire
but with a very low score (for testing, etc) assign a score like 0.01 to it.

C.

- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD7fO+MDDagS2VwJ4RAnPPAJwLX5SKGxES4WxR61ks+rgTyXTGkgCgk1sN
U3ogYpdyrpGGDiPpQuZFMsA=
=ugT8
-END PGP SIGNATURE-

Re: Spammasssin skips rules?

[Matt Kettler]

Joost Kraaijeveld wrote:
> Hi,
>
> I have two rules in a file "/etc/spamassassin/80_jkr.cf" that seem be to
> skipped by spamassassin:
>
> bodyJKR_TEST_BODY_RULE/e/
> score   JKR_TEST_BODY_RULE0.0
> describeJKR_TEST_BODY_RULEJKR body rule
>
> header  JKR_TEST_HEADER_RULE   Subject =~ /e/
> score   JKR_TEST_HEADER_RULE   0.0
> describeJKR_TEST_HEADER_RULE   JKR header rule
>
>
> Running "spamassasin --lint -D" shows:
> 
> debug: config: read file /etc/spamassassin/80_jkr.cf
> ...
>
> so I assume that the file is picked up by spamassassin. I expect all
> mails that have an "e" in either the header or body (which is most
> mails) to have a hit for thos rules. This is however not the case.
> Spamassassin does uses the RulesDuJour rukes, that are stored in the
> same directory.
>
> Is my regex wrong or are there circumstances that Spamassassin skips any
> rules?
>   

Spamassassin never runs any normal rule (one which appears in the hit
list) with a score of 0. This is intended to be a useful way for users
to completely disable rules that are causing too much overhead on their
systems.

Try a score of -0.001 instead.

The only rules that get run with no score are those starting with a
double underscore in the name. By default these rules run but have no
score, but also never appear in the hits list, and are used as sub-parts
of meta rules. Even with these, last I checked, if you explicitly
declared a score of zero for one of these rules it would disable it.


>
>   

Re: Question about configuration

[Matt Kettler]

Awie wrote:
> Hi All,
>
> My apologize for the question of beginner.
>
> I want to integrate Spamassassin with qmail + Qmail-Scanner to mark email
> that be suspected SPAM, with ***SPAM*** in it's subject.
>
>  Should I remove the # sign in local.cf? # rewrite_header Subject
> *SPAM*
>
> Please advise. Thanks alot for your help
>   
That depends on how you have qmail-scanner set up.. If I recall
correctly, qmail-scanner does it's own tagging if you have the "fast
spamassassin" option set. If you want SA to do the tagging you have to
disable that. Otherwise, configure qmail-scanner's tagging options.

Realistically there is speed difference with fast spamassassin, but it's
not very large. The big difference is that in fast mode SA doesn't mark
and pipe the message back, all it gives is a score. (qmail-scanner uses
spamc with the -c option).

Spammasssin skips rules?

[Joost Kraaijeveld]

Hi,

I have two rules in a file "/etc/spamassassin/80_jkr.cf" that seem be to
skipped by spamassassin:

bodyJKR_TEST_BODY_RULE/e/
score   JKR_TEST_BODY_RULE0.0
describeJKR_TEST_BODY_RULEJKR body rule

header  JKR_TEST_HEADER_RULE   Subject =~ /e/
score   JKR_TEST_HEADER_RULE   0.0
describeJKR_TEST_HEADER_RULE   JKR header rule


Running "spamassasin --lint -D" shows:

debug: config: read file /etc/spamassassin/80_jkr.cf
...

so I assume that the file is picked up by spamassassin. I expect all
mails that have an "e" in either the header or body (which is most
mails) to have a hit for thos rules. This is however not the case.
Spamassassin does uses the RulesDuJour rukes, that are stored in the
same directory.

Is my regex wrong or are there circumstances that Spamassassin skips any
rules?


-- 
Groeten,

Joost Kraaijeveld
Askesis B.V.
Molukkenstraat 14
6524NB Nijmegen
tel: 024-3888063 / 06-51855277
fax: 024-3608416
e-mail: [EMAIL PROTECTED]
web: www.askesis.nl 

Re: getmail?

[Craig McLean]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Gene Heskett wrote:
[snip fetchmail discussion]

> 
> In further reading tonight, sendmail grew the libmilter freature at 
> 8.12, which is the base version running here, and yum won't update it, 
> says its current.

What version of the OS are you running, Gene? FC4 has 8.13.4-2 as the
latest, not that it necessarily makes any odds.

> Right now, I'm looking at the  
> site, trying to see how this is done.
> 
> But, here is the headache:  At no place in the various files sitting 
> in /etc/mail that serve to configure sendmail, is there an example of 
> how to configure sendmail to make use of these feature facilities.

Basic milter information can be found at:
http://www.sendmail.org/~ca/email/doc8.12/cf/m4/adding_mailfilters.html
and more in-depth here:
http://www.milter.org/

An example of how to get sendmail to use spamass-milter (and
clamav-milter, I use both) looks like this, from sendmail.mc:

- -quote-
dnl ** Milter Configurations **
define(`confMILTER_MACROS_CONNECT',`b, j, _, {daemon_name}, {if_name},
{if_addr}')
INPUT_MAIL_FILTER(`clmilter',`S=local:/var/run/clamav/clmilter.sock, F=,
T=S:4m;R:4m')dnl
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=,
T=C:15m;S:4m;R:4m;E:10m')
dnl define(`confINPUT_MAIL_FILTERS', `spamassassin,clmilter')
- -quote-

Over here I use spamass-milter to pass mail to spamc as it passes
through the MTA, because I *am* my domain's MX.
This means sendmail needs to be configured to accept mail via SMTP which
is fine for me, but might be far more overhead than you need.

> Spamassassin 3.10 contains only very scant references to using it with 
> sendmail, apparently sanctioning only the procmail interface, which in 
> turn then is set to call spamc or spamassassin, adding needless time 
> wasting cpu cycles to what should be a pretty simple job.  I fail to 
> understand why (although it will take smarter people than me what with 
> sendmails configuration complexity) there is no readily published 
> recipe for incorporating spamc into the sendmail processing chain, 
> either by pipeing, or when the libmilter feature is there?

libmilter just provides a mechanism for sendmail to pass the email, via
a socket, to a small C program, thence to spamc. Talk about "needless
time wasting CPU cycles"?

In a configuration where you don't readily run sendmail to accept mail,
I would suggest staying the hell away from it and:

a) configuring fetchmail to simply use procmail as the MDA. ("--mda
/usr/bin/procmail" or similar, IIRC)
b) having procmail run everything handed to it through spamc, and filter
accordingly.

Peice of cake (relatively speaking) to set up, no sendmail black magic
and fairly quick to run.

> Or am I simply on the wrong mailing list?  I've sent 3 subscribe 
> messages to the getmail-user list over the last 3 days with no response 
> which is discouraging.  OTOH, now that I know it can't do what I want, 
> who cares.  It might be that if there was a manpage for getmail, it 
> might be possible.  A pox on software that doesn't come with readable 
> manuals.

Or *any* manuals

All the best!
C.

- --
Craig McLeanhttp://fukka.co.uk
[EMAIL PROTECTED]   Where the fun never starts
Powered by FreeBSD, and GIN!
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFD7eXCMDDagS2VwJ4RAklQAKDDkbeOOGGfp7I5RuubaSmAAJCjiwCgjwbM
bVGx27+TfZgUG9QwfK6VJU8=
=QDMd
-END PGP SIGNATURE-

Re: Xtracting urls from saved spams & making SA rules - xurl001.p l

[Michael W Cocke]

*Apologies if you've already seen this - I can't find any indication
that it sent when I hit send and I'm fooling with a new mail client*


On Fri, 10 Feb 2006 11:21:25 -0500, you wrote:

>I think I know a bit about extracting URLs from spam ;)  

A bit.  


>It is pretty damn complicated. A lot of tricks they play, like
>www.amazon.com.buy-my-drugs-com.optelnd.net
>
>Then you have hex and decimal links to deal with. And yeah, they do pepper
>the spam with legit urls. What about akami image links? Its was common to
>see 20 links in a spam, and only one was the evil one you wanted. 
>
>Automation without a LOT of checks and balances = FPs. 
>
>You have to have a LOT more autoresearched evidence then just that they are
>contained in a spam. But hey! A+ for effort! Its a start, and it will always
>get better. 

xurl is designed to deal with the rare spam that makes it thru my
SA/amavisd/clam setup...  In the (relatively small) sample of spams I
worked from (not much makes it thru all that), I didn't encounter
anything xurl would create FPs from - mainly what makes it thru my
current setup is one or two lines of glop (bayes poison) and an url.

xurl is only designed to clean up the dust behind SA - in NO WAY is it
supposed to be a front line defense.

Mike-
--
If you're not confused, you're not trying hard enough.
--
Please note - Due to the intense volume of spam, we have installed 
site-wide spam filters at catherders.com.  If email from you bounces,
try non-HTML, non-encoded, non-attachments,

Question about configuration

[Awie]

Hi All,

My apologize for the question of beginner.

I want to integrate Spamassassin with qmail + Qmail-Scanner to mark email
that be suspected SPAM, with ***SPAM*** in it's subject.

 Should I remove the # sign in local.cf? # rewrite_header Subject
*SPAM*

Please advise. Thanks alot for your help

Best Regards,

Awie

Re: RulesDuJour and Curl Connect Problem

[Yousef Raffah]

On Sat, 2006-02-11 at 11:42 +, Shane Kelly wrote:
> Yousef Raffah wrote:
> > On Wed, 2006-02-08 at 15:41 +, Shane Kelly wrote:
> >> Hi Yousef,
> >>
> >>
> >>> I have to connect through a proxy server to get to the internet but I'm
> >>> not sure how to set the proxy for curl. I tried to change the line
> >>> in /var/lib/spamassassin/rules_du_jour
> >>> To this:
> >>>  [ "${CURL_OPTS}" ] || CURL_OPTS="-w %{http_code} --compressed -O -R -s
> >>> -S -z -x proxy.nour.net.sa:8080";
> >>>
> >>> as I understood from man curl that -x is the parameter to use for a
> >>> proxy server!
> >>>
> >>> The only relevant thread I found so far is this:
> >>> http://thread.gmane.org/gmane.mail.spam.spamassassin.general/76192
> >>> but I have the outbound port 80 open and it is confirmed as when I
> >>> invoke the script manually (not through cron) it works perfectly (at
> >>> least that is what I see :) )
> >> I suspect you need to set your proxy settings in the crontab, as most 
> >> recent crons run with a clean environment for security. (I got bitten by 
> >> the same thing :-) )
> >>
> > Hummm, that's pretty interesting, will test and let you know, because I
> > though I had it already. Just out of curiousity, should I specify the
> > proxy (export the variables in crontab) or the environment variable
> > should just be "there"? Currently I have the variable set for the root
> > user in the bash profile
> 
> I have these as lines above the cron entries in the system crontab (i.e. 
> /etc/crontab) on a Suse 9.3 system, but below the shell, path and mailto 
>   vars. Both wget and curl pick them up from there.
> 
> HTTP_PROXY='http://wwwcache.xxx.xx.uk:8080'
> http_proxy='http://wwwcache.xxx.xx.uk:8080'
> 
Cool, I just configured it. Will let you know if it worked tonight ;).
Many thanks.


Sincerely,
Yousef Raffah
Senior Systems Administrator
SSIS - The Savola Group

--
Aren't you using Firefox? Get it at getfirefox.com
yousef.raffah.com


signature.asc
Description: This is a digitally signed message part

Re: RulesDuJour and Curl Connect Problem

[Shane Kelly]

Yousef Raffah wrote:

On Wed, 2006-02-08 at 15:41 +, Shane Kelly wrote:

Hi Yousef,



I have to connect through a proxy server to get to the internet but I'm
not sure how to set the proxy for curl. I tried to change the line
in /var/lib/spamassassin/rules_du_jour
To this:
 [ "${CURL_OPTS}" ] || CURL_OPTS="-w %{http_code} --compressed -O -R -s
-S -z -x proxy.nour.net.sa:8080";

as I understood from man curl that -x is the parameter to use for a
proxy server!

The only relevant thread I found so far is this:
http://thread.gmane.org/gmane.mail.spam.spamassassin.general/76192
but I have the outbound port 80 open and it is confirmed as when I
invoke the script manually (not through cron) it works perfectly (at
least that is what I see :) )
I suspect you need to set your proxy settings in the crontab, as most 
recent crons run with a clean environment for security. (I got bitten by 
the same thing :-) )



Hummm, that's pretty interesting, will test and let you know, because I
though I had it already. Just out of curiousity, should I specify the
proxy (export the variables in crontab) or the environment variable
should just be "there"? Currently I have the variable set for the root
user in the bash profile


I have these as lines above the cron entries in the system crontab (i.e. 
/etc/crontab) on a Suse 9.3 system, but below the shell, path and mailto 
 vars. Both wget and curl pick them up from there.


HTTP_PROXY='http://wwwcache.xxx.xx.uk:8080'
http_proxy='http://wwwcache.xxx.xx.uk:8080'

Regards,
Shane.

Re: I need help .

[Matthias Keller]

Marcos Manhanes wrote:


Friends,

I have a Mailserver with QMAIL and Spamassassin running but it now has a
problem.

Ex.

Feb 10 11:10:34 Mailserver kernel: Out of Memory: Killed process 13335
(spamassassin).
Feb 10 11:12:03 Mailserver kernel: Out of Memory: Killed process 13371
(spamassassin).
Feb 10 11:13:51 Mailserver kernel: Out of Memory: Killed process 13352
(spamassassin).
Feb 10 11:16:03 Mailserver kernel: Out of Memory: Killed process 13379
(spamassassin).
Feb 10 11:17:50 Mailserver kernel: Out of Memory: Killed process 13235
(spamassassin).
Feb 10 11:18:28 Mailserver kernel: Out of Memory: Killed process 13378
(spamassassin).
Feb 10 11:19:07 Mailserver kernel: Out of Memory: Killed process 13350
(spamassassin).
Feb 10 11:26:17 Mailserver kernel: Out of Memory: Killed process 14573
(spamassassin).
Feb 10 11:27:16 Mailserver kernel: Out of Memory: Killed process 15121
(spamassassin).
Feb 10 11:28:20 Mailserver kernel: Out of Memory: Killed process 14947
(spamassassin).
Feb 10 11:29:26 Mailserver kernel: Out of Memory: Killed process 15449
(spamassassin).
Feb 10 11:30:23 Mailserver kernel: Out of Memory: Killed process 15024
(spamassassin).
 


This doesn't necessary have anything to do with spamassassin
It just means all your memory (and swap!) is exhausted and the system 
NEEDED to make some space so it just killed something.. depending on 
your kernel this can be quite random
I'd suggest you have a look at  top  and find the process that takes so 
much memory!
If it's really spamassassin (unlikely?) then you should check which 
rulesets you are using (DONT use bigevil !)
If it's something else then it probably should be stopped and updated 
immediately...


Good luck

Matt

I need help .

[Marcos Manhanes]

Friends,

I have a Mailserver with QMAIL and Spamassassin running but it now has a
problem.

Ex.

Feb 10 11:10:34 Mailserver kernel: Out of Memory: Killed process 13335
(spamassassin).
Feb 10 11:12:03 Mailserver kernel: Out of Memory: Killed process 13371
(spamassassin).
Feb 10 11:13:51 Mailserver kernel: Out of Memory: Killed process 13352
(spamassassin).
Feb 10 11:16:03 Mailserver kernel: Out of Memory: Killed process 13379
(spamassassin).
Feb 10 11:17:50 Mailserver kernel: Out of Memory: Killed process 13235
(spamassassin).
Feb 10 11:18:28 Mailserver kernel: Out of Memory: Killed process 13378
(spamassassin).
Feb 10 11:19:07 Mailserver kernel: Out of Memory: Killed process 13350
(spamassassin).
Feb 10 11:26:17 Mailserver kernel: Out of Memory: Killed process 14573
(spamassassin).
Feb 10 11:27:16 Mailserver kernel: Out of Memory: Killed process 15121
(spamassassin).
Feb 10 11:28:20 Mailserver kernel: Out of Memory: Killed process 14947
(spamassassin).
Feb 10 11:29:26 Mailserver kernel: Out of Memory: Killed process 15449
(spamassassin).
Feb 10 11:30:23 Mailserver kernel: Out of Memory: Killed process 15024
(spamassassin).

I use Procmail with it conf.

<<< .procmailrc >

LOGFILE=/var/log/procmail
:0fw: spamassassin.lock
* < 256000
| /usr/bin/spamassassin
:0e
EXITCODE==$?
* ^X-Spam-Flag: Yes
:0:
./Maildir/

Thank for help.

Re: General assistance

[Michael Monnerie]

On Freitag, 10. Februar 2006 22:42 Ed Russell wrote:
> I was doing some reading and I am beginning to look into Rules Du
> Jour.  I see there are quite a large number of rulesets to choose
> from when utilizing this.  Does anyone have any advice on what ones
> would be safe?

I use those:
SARE_ADULT
SARE_OBFU0 
SARE_OBFU1 
SARE_URI0 
SARE_REDIRECT_POST300 
SARE_HTML0 
SARE_HEADER0 
SARE_SPECIFIC 
SARE_BML 
SARE_FRAUD 
SARE_SPOOF 
SARE_GENLSUBJ0 
SARE_UNSUB 
SARE_WHITELIST_RCVD 
SARE_WHITELIST_SPF 
ZMI_GERMAN

The last one being specific for german language SPAM. 

Additionaly, I use the blacklist by William Stearns for postfix, running 
a cron job: rsync -qL 
rsync.sa-blacklist.stearns.org::wstearns/sa-blacklist/sa-blacklist.current.reject
 /etc/postfix/sender_blacklist ; 
postmap /etc/postfix/sender_blacklist

That's much better than the blacklist by SARE, as it's less memory 
consuming and faster - a drop by MTA is generally faster than handing 
it over to SA.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc  ---   it-management Michael Monnerie
// http://zmi.at   Tel: 0660/4156531  Linux 2.6.11
// PGP Key:   "lynx -source http://zmi.at/zmi2.asc | gpg --import"
// Fingerprint: EB93 ED8A 1DCD BB6C F952  F7F4 3911 B933 7054 5879
// Keyserver: www.keyserver.net Key-ID: 0x70545879


pgp087jPloNCP.pgp
Description: PGP signature

Re: General assistance

[Michael Monnerie]

On Freitag, 10. Februar 2006 19:32 Ed Russell wrote:
> 1.  Does anyone have an opinion as to what RBL to contact?  I
> know there are quite a few.

sbl-xbl.spamhaus.org, multi.surbl.org, safe.dnsbl.sorbs.net, 
dnsbl.njabl.org, bl.spamcop.net, relays.ordb.org

I use those at MTA level. That dropped 62.000 messages, and only 378 
spams were detected by SA during that time. I guess that saved a lot of 
CPU.

Since you seem to have a problem with DNS queries ("if I disable RBL 
checks and razor, pyzor and dcc the delay goes away"), I would suggest:

- make RBL checks at the MTA already
- get permission from RBL maintainers to make a zone transfer to your 
box, and run a local named or whatever. By that, you only have local 
DNS queries, that should help a lot.

> 2.  Once this is in place should I re-activate pzyor, dcc or
> razor?  Is one better than the other?  Are there advantages to
> either?

Each of them are different, altogether they help a lot. I use all of 
them, but I'm not in a situation where I have problems with delay. 
First try RBL at MTA, and possibly you have enough CPU cycles left then 
to reactivate that checks.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc  ---   it-management Michael Monnerie
// http://zmi.at   Tel: 0660/4156531  Linux 2.6.11
// PGP Key:   "lynx -source http://zmi.at/zmi2.asc | gpg --import"
// Fingerprint: EB93 ED8A 1DCD BB6C F952  F7F4 3911 B933 7054 5879
// Keyserver: www.keyserver.net Key-ID: 0x70545879


pgp3dyy87IfpR.pgp
Description: PGP signature