Re: Test for no Subject line field

[Matt Kettler]

Russell P. Sutherland wrote:
Is there a test that one can construct that would
assign a weight to a message that is missing
a certain header, completely? In my case, no Subject
line at all.
From the default ruleset for 3.x:
header __HAS_SUBJECT   exists:Subject
meta MISSING_SUBJECT   !__HAS_SUBJECT

Test for no Subject line field

[Russell P. Sutherland]

Is there a test that one can construct that would
assign a weight to a message that is missing
a certain header, completely? In my case, no Subject
line at all.


-- 
Quist ConsultingEmail: [EMAIL PROTECTED]
219 Donlea DriveVoice: +1.416.696.7600
Toronto ON  M4G 2N1 Cell:  +1.416.803.0080
CANADA  WWW:   http://www.quist.ca

Re: Is this Received header correctly formatted?

[mouss]

List Mail User wrote:
In other words, lowercase is conformant. and your first point is
not correct (though all the examples do show uppercase).  However, you are
completely correct that the "helo=" is flat out wrong,
why? it's inside a comment, no?
 but with a slight
variation, and it becomes something like "(watson1 [4.16.241.28])" which
is not only conformant, but is the the typical behavior or both sendmail
and postfix.
except that here the situation is reversed.
while postfix and sendmail use "from heloname (client_namer 
[client_ip])", others such as qmail prefer "from client_name 
([client_ip]) (helo heloname)" or other variants.

Re: sa-learn hangs -- SOLVED

[Eric Dantan Rzewnicki]

On Wed, Mar 16, 2005 at 05:33:16PM -0500, Eric Dantan Rzewnicki wrote:
> On Wed, Mar 16, 2005 at 01:25:53PM -0500, Eric Dantan Rzewnicki wrote:
> > On Tue, Mar 15, 2005 at 06:11:27PM -0500, Matt Kettler wrote:
> > > Eric Dantan Rzewnicki wrote:
> > > >spamassassin -D -p  --lint doesn't show any problems that I
> > > >can see.
> > > >if I run: 
> > > >sa-learn --showdots --mbox --ham -p 
> 
> I had an old version of sa-learn in /usr/local/bin. The new sa-learn is
> installed in /usr/bin/. The old one was not removed when I upgraded from
> 2.6x to 3.0.2. The old one was in my PATH first. Everything's fine now.

Is there any other residue I should look for from the upgrade?
-- 
Eric Dantan Rzewnicki  |  Systems Administrator
Technical Operations Division  |  Radio Free Asia
2025 M Street, NW  |  Washington, DC 20036  |  202-530-4900
CONFIDENTIAL COMMUNICATION
This e-mail message is intended only for the use of the addressee and
may contain information that is privileged and confidential. Any 
unauthorized dissemination, distribution, or copying is strictly 
prohibited. If you receive this transmission in error, please contact
[EMAIL PROTECTED]

Re: sa-learn hangs -- SOLVED

[Eric Dantan Rzewnicki]

On Wed, Mar 16, 2005 at 01:25:53PM -0500, Eric Dantan Rzewnicki wrote:
> On Tue, Mar 15, 2005 at 06:11:27PM -0500, Matt Kettler wrote:
> > Eric Dantan Rzewnicki wrote:
> > >spamassassin -D -p  --lint doesn't show any problems that I
> > >can see.
> > >if I run: 
> > >sa-learn --showdots --mbox --ham -p 
> > >/opt/MailScanner/etc/spam.assassin.prefs.conf 
> > >sa-learn just hangs. Same happens for --spam.
> > >strace shows it stuck on a read(0,
> > >Any ideas?
> > What about sa-learn -D -p  --dump magic? does that hang? 
> No hang with debug added to --dump. Same as without -D.
> > If 
> > that works, what about sa-learn -D --rebuild?
> rebuild works fine, too.
> > Do you have an alternate bayes_path set in your spam.assassin.prefs.conf?
> I have these:
> bayes_path /var/spool/MailScanner/spamassassin/bayes
> bayes_file_mode 0660
> > Does the user running sa-learn have rw permissions to it, and rwx to the 
> > directory containing it?
> I'm running sa-learn as root. 
> :/var/spool/MailScanner/spamassassin# ls -la
> total 7788
> drwxr-xr-x2 postfix  postfix  4096 Mar 16 13:05 .
> drwxr-xr-x6 root root 4096 Jan 23  2004 ..
> -rw---1 postfix  postfix   390 Mar 16 13:04 bayes.mutex
> -rw-rw1 postfix  postfix320400 Mar 16 13:13 bayes_journal
> -rw-rw1 postfix  postfix   2641920 Mar 15 17:52 bayes_seen
> -rw-rw1 postfix  postfix   5697536 Mar 16 13:05 bayes_toks

I had an old version of sa-learn in /usr/local/bin. The new sa-learn is
installed in /usr/bin/. The old one was not removed when I upgraded from
2.6x to 3.0.2. The old one was in my PATH first. Everything's fine now.
-- 
Eric Dantan Rzewnicki  |  Systems Administrator
Technical Operations Division  |  Radio Free Asia
2025 M Street, NW  |  Washington, DC 20036  |  202-530-4900
CONFIDENTIAL COMMUNICATION
This e-mail message is intended only for the use of the addressee and
may contain information that is privileged and confidential. Any 
unauthorized dissemination, distribution, or copying is strictly 
prohibited. If you receive this transmission in error, please contact
[EMAIL PROTECTED]

Re: Blacklisting embedded URLs

[Kai Schaetzl]

Vicki Brown wrote on Wed, 16 Mar 2005 13:00:59 -0800:

> Okaaay. Help me out here, please? "If network tests are enabled"? 
> I change essentially nothing from the defaults. 
> Mail::SpamAssassin::Plugin::URIDNSBL is loaded in init.pre. 
> Net::DNS is up to date. 
> But as I'm apparently not using URIDNSBL or SURBL... 
> how do I ensure that network tests are enabled?
>

You don't need to have RBL tests enabled if that is what is meant by 
"network tests". You only need to have it configured in init.pre (I 
think's commented out by default). I assume you have to have 
"dns_available yes". Test it with spamassassin -D.

Kai

-- 
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
IE-Center: http://ie5.de & http://msie.winware.org

SPAM/HAM folder

[Norman Zhang]

Hi,
On my SA Gateway, I have no local box except root. Should I forward
HAM/SPAM to local box? Mail are not meant for local delivery here.
Regards,
Norman Zhang

Re: spamassassin build failure on x86_64

[Theo Van Dinter]

On Wed, Mar 16, 2005 at 01:33:21PM -0800, Dan Hollis wrote:
> I'm getting errors building the rpm on x86_64:

Yeah, we haven't quite worked that out yet.  Things are being linked against
things they shouldn't be. :(

For the time being, you can apply the patch attached to bug 4090:
http://bugzilla.spamassassin.org/show_bug.cgi?id=4090

It disables the build of libspamc.so, which shouldn't be an issue for most
people.

-- 
Randomly Generated Tagline:
 Michelle: You expect me to live in a tiny little hole?
  Fry: It'd be deeper, but I'm standing on a gopher.


pgpsr6jdeY1yL.pgp
Description: PGP signature

spamassassin build failure on x86_64

[Dan Hollis]

I'm getting errors building the rpm on x86_64:

Manifying blib/man3/Mail::SpamAssassin::Bayes.3pm
Manifying blib/man3/Mail::SpamAssassin::Plugin::RelayCountry.3pm
+ /usr/bin/make spamc/libspamc.so
/usr/bin/make -f spamc/Makefile spamc/libspamc.so
make[1]: Entering directory `/usr/src/redhat/BUILD/Mail-SpamAssassin-3.0.2'
gcc -Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.5/x86_64-linux-thread-multi/CORE 
spamc/libspamc.c spamc/utils.c \
-o spamc/libspamc.so -shared -ldl 
/usr/bin/ld: /tmp/cc40AdRE.o: relocation R_X86_64_32S against `a local symbol' 
can not be used when making a shared object; recompile with -fPIC
/tmp/cc40AdRE.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[1]: *** [spamc/libspamc.so] Error 1
make[1]: Leaving directory `/usr/src/redhat/BUILD/Mail-SpamAssassin-3.0.2'
make: *** [spamc/libspamc.so] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.41963 (%build)

-Dan

Re: Is this Received header correctly formatted?

[Eric A. Hall]

Daryl C. W. O'Shea wrote:
...and if you can, avoid using running messages to the list through SA 
(easy to do if you're using procmail, not so easy in other cases).
or run them through with "whitelist_from_rcvd *.* apache.org" to pad the 
value so that it doesn't matter

I do wish that postfix would let me add dynamic headers to the message 
before the proxy filter is called, or give me an ACL for no-filter, 
either of which would work to skip well-known message origins

--
Eric A. Hall  http://www.ehsco.com/
Internet Core Protocolshttp://www.oreilly.com/catalog/coreprot/

Re: Blacklisting embedded URLs

[Vicki Brown]

At 20:48 -0800 03/15/2005, Jeff Chan wrote:
>Yes, please see URIDNSBL and SURBL:
>
>
>http://spamassassin.apache.org/full/3.0.x/dist/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
>  http://www.surbl.org/
>
>which are built into SpamAssassin 3 and enabled by default if
>network tests are enabled.


Okaaay. Help me out here, please? "If network tests are enabled"?
I change essentially nothing from the defaults.
Mail::SpamAssassin::Plugin::URIDNSBL is loaded in init.pre.
Net::DNS is up to date.
But as I'm apparently not using URIDNSBL or SURBL...
how do I ensure that network tests are enabled?

I run
   spamd -d -c
at system startup
then, from procmailrc, I push each message through
   | /usr/local/bin/spamc -s 256000 -t 60

What do I need to know/do/read to enable network tests?
-- 
Vicki Brown  ZZZ
Journeyman Sourceror:  zz  |\ _,,,---,,_ Code, Docs, Process,
Scripts & Philtres  zz /,`.-'`'-.  ;-;;,_   Perl, WWW, Mac OS X
http://cfcl.com/vlb   |,4-  ) )-,_. ,\ ( `'-'   SF Bay Area, CA  USA
___  '---''(_/--'  `-'\_)  ___

Re: Re: SA 3.0.2 MASSIVE memory cpu problems

[Wolfgang . Fuertbauer]

Justin!
[EMAIL PROTECTED] (Justin Mason) wrote on 16.03.05 21:46:>-BEGIN PGP SIGNED MESSAGE->Hash: SHA1>>>I would suggest running with -D and monitoring spamd memory size>as it starts up.   Something is causing it to balloon to massive>sizes after startup.
nothing special during startup; it takes some time until the 99% CPU und Memory problem begin>Presumably you are limiting the size of the messages sent in for scanning,>as recommended in the documentation? Scanning messages over 500KB in size>will result in a corresponding increase in resident size for the scanner>process, so a very large message could cause a massive scanner process.>(spamc will do this limit by default.)
yes I do; SA is used with exim/exiscan with the following rules:
  # Always add X-Spam-Score and X-Spam-Report headers, using SA system-wide settings  # (user "exim")    warn  message = X-Spam-Score: $spam_score ($spam_bar)  spam = exim:true  condition = ${if <{$message_size}{250k}{1}{0}}    warn  message = X-Spam-Report: $spam_report  spam = exim:true  condition = ${if >{$spam_score_int}{40}{1}{0}}  # Add X-Spam-Flag if spam is over system-wide threshold    warn message = X-Spam-Flag: YES  spam = exim  condition = ${if <{$message_size}{250k}{1}{0}}
  # Reject spam messages with score over 5, using an extra condition.    deny  message = SPAM This message scored $spam_score SPAM points.  spam = exim:true  condition = ${if >{$spam_score_int}{50}{1}{0}}>>Also you're the first person on linux to note the "select timeout>failed" bug.  could you attach output from strace and "spamd -D">to a new bugzilla bug on this?
i opened bug # 4198
hopefully this helps!
Wolfgang
>>- --j.>>[EMAIL PROTECTED] writes:>> Greg,>> i have use_auto_whitelist 0 in the local.cf But thanks anyway>> Wolfgang "Greg Allen" <[EMAIL PROTECTED]> schrieb am 16.03.2005 13:54:24: > Some users have had problems with corrupt AWL database after upgrade of>> > Spamassassin. Try disabling AWL to see if that is your issue. >>> > -Original Message->> > From: [EMAIL PROTECTED]>> > [mailto:[EMAIL PROTECTED]]>> > Sent: Wednesday, March 16, 2005 5:44 AM>> > To: [EMAIL PROTECTED]>> > Cc: [EMAIL PROTECTED]>> > Subject: SA 3.0.2 MASSIVE memory cpu problems >>> > Dear collegues, > I'm having still extrem problems with memory and cpu consumation of SA>> > 3.0.2 spamd; > PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  Command>> > 19098 exim      30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd>> > 19121 exim      20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd>> > 24591 exim      22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd>> > 25802 exim      21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd>> > 20548 exim      20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd>> > 26479 exim      20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd > I already limited the # of children to 5 and the # of connections to 20>> > my own comment on this problem some month (restart spamd daily) does not>> > work anymore; > I've seen, there is a patch in Bugzilla addressing this problem, which>> > hasn't made his>> > way into 3.0.2; I tried to apply it against 3.0.2 with the following>> > result: > Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!>> > recovering>> > (repeated 20x and more) > the only solution for the moment is to restart spamd on an hourly basis>> ?! > WHAT CAN I DO? > Wolfgang>> > P.S.: some environment > perl -v>> > This is perl, v5.8.0 built for i586-linux-thread-multi > saxophon:~ # spamassassin -V>> > SpamAssassin version 3.0.2>> > running on Perl version>> > saxophon:~ # uname -a>> > Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686>> > unknown unknown GNU/Linux > -->> > Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])>> > EBEWE Pharma>> > Mondseestrasse 11>> > 4866  Unterach, Austria>> > Tel: ++43 7665 8123 315>> > Fax: ++43 7665 8123 11>> > http://www.ebewe.com>-BEGIN PGP SIGNATURE->Version: GnuPG v1.2.5 (GNU/Linux)>Comment: Exmh CVS>>iD8DBQFCOIF2MJF5cimLx9ARApqoAJ46iCjfM13XD0MF08ViuuduVW6ASACfRat9>XaB8642NZWeXoclyDuLXR0g=>=v3ys>-END PGP SIGNATURE->--Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])EBEWE Pharma  Mondseestrasse 11 4866  Unterach, AustriaTel: ++43 7665 8123 315Fax: ++43 7665 8123 11http://www.ebewe.com

RE: URI Tests and Japanese Chars (solved)

[List Mail User]

>
>This is an excerpt that I used in trying to track it down.  No real mailto URI 
>unless there is some translation going on with email addresses embedded in the 
>body by the email client on send.  At first, I just thought it might be a bug 
>since the messages were using ISO-2022-JP character set but if I sent just a 
>plain text message with just the [EMAIL PROTECTED] in the body, then URIBL_SBL 
>was tripped. 
>
>*
>- Original Message -
>From: "user1" <[EMAIL PROTECTED]>
>To: "user2" <[EMAIL PROTECTED]>
>Sent: Friday, March 11, 2005 11:14 AM
>Subject: Re: $BFb;[EMAIL PROTECTED](J 
>
>***
>
>-=B
>
>
>-Original Message-
>From: Jeff Chan [mailto:[EMAIL PROTECTED] 
>Sent: Wednesday, March 16, 2005 7:52 AM
>To: users@spamassassin.apache.org
>Subject: Re: URI Tests and Japanese Chars (solved)
>
>On Wednesday, March 16, 2005, 3:55:52 AM, Bobby Rose wrote:
> 
>> I figured out the problem, it' was the an individuals email address in 
>> the message body (even though not a mailto).  Their email domain isn't 
>> listed at spamhaus.org but it turns out one of their ISPs DNS servers 
>> are which they are using as secondary.  This makes the second time 
>> I've come across this.  The last time it was an ISP's (pipex.net) DNS 
>> server in the U.K. that was tripping the URIBL_SBL rule.
>
>> This time the user is in the med.juntendo.ac.jp (Juntendo Univ Med
>> School) who's ISP is cwidc.net and the DNS server  ns03.cwidc.net
>> (154.33.17.212) is the one in spamhaus.org which they say is hosting a 
>> long time spammer.  
>> http://www.spamhaus.org/sbl/sbl.lasso?query=SBL17240
>
>> Does URI checking really need to be so thorough?  Obviously there must 
>> be some bias at spamhaus if the big named ISPs don't get their name 
>> servers listed because we know that they provide services to spammers.
>> Any idea on how to limit the scope to just the URI at it's face value?
>
>uridnsbl used in the default rule URIBL_SBL does check domain name servers 
>against SBL, but I'm kind of surprised to hear it triggering on email 
>addresses.  It should definitely be checking web sites and the like.  Can you 
>give a sample of the text it hit?  Was it in URI form like:
>
>  mailto://[EMAIL PROTECTED]
>
>That said, I agree that the SBL listings are at times overbroad.
>Name servers for gov.ru and spb.ru for example are listed (ns.rtcomm.ru and 
>ns1.relcom.ru respectively).  Listings like those can cause false positives, 
>and I personally object to deliberately harming innocent bystanders to 
>"pressure" ISPs.
>
>Jeff C.
>--
>Jeff Chan
>mailto:[EMAIL PROTECTED]
>http://www.surbl.org/
>
>
Spamhaus does sometimes "escalate" against companies that ignore
issues for a long time;  But this isn't one of those cases.  Here the listing
is:
http://www.spamhaus.org/SBL/sbl.lasso?query=SBL17240
which covers exactly one IP 154.33.17.212/32 and gives a good reason for it.

This is similar to when I had a friend who bought a cheap hosting
service and was surprised to find out it was blacklisted everywhere - They
hosted spammers on the same machine.

To me it looks like a good case for the people at juntendo.ac.jp to
be looking for another company to do their backup DNS or at least request
that the particular server be changed.  Besides, shouldn't a University
be able to provide their own redundant servers (they do have a legacy class
'B' net to themselves)?

Sorry, we usually agree (I like that SURBLs try for zero FPs, but
every blacklist has a different goal and a different target, and this site
fits Spamhaus' stated objectives exactly).  BTW. Did you notice that the
owner of the SBL'd site is "Cable and Wireless" - so it is not quite true
that Spamhaus lets "big" companies get away with any thing as someone else
implied earlier.

I have no idea why I'm always defending all sorts of people.

Paul Shupak
[EMAIL PROTECTED]

Re: Is this Received header correctly formatted?

[Daryl C. W. O'Shea]

List Mail User wrote:
P.S.  Could whomever maintains this list please try to settle on one format
for the list's name - today's messages are using
SpamAssassin Mailing List <[EMAIL PROTECTED]>
a couple of days ago the format changed to:
"[EMAIL PROTECTED] apache. org" 
and I already have to special case a half dozen variants for when people
put SA output in their messages and my filters "see" high scores (despite
the various whitelistings, special cases and other stuff to handle the list).
Any of the example scripts distributed with the code all fail when people
quote output - So I have additional checks to try to prevent bounces that
add additional tests for the list in either "To:", or "Cc:" lines *and*
"From:" lines, but every time the delivering servers change or the list
description changes, I have to add more cases.
That isn't the list software, that's people sending mail to old 
addresses along with whatever name they've added for it in their 
addresses book.

Just filter based on the List-Id: which is:
List-Id: 
...and if you can, avoid using running messages to the list through SA 
(easy to do if you're using procmail, not so easy in other cases).

Daryl

Re: Upgrade... + other (perl?) problems

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


[EMAIL PROTECTED] writes:
> Am 16.03.2005 um 08:55 Uhr haben Sie geschrieben:
> > Am 16.03.2005 um 00:31 Uhr haben Sie geschrieben:
> > > On Wed, Mar 16, 2005 at 12:27:28AM +0100, [EMAIL PROTECTED]
> > wrote:
> > > > Are there problems with mail header identification?
> > > > Am I in the wrong list with this question?
> > > > > Mar 13 01:16:18 ns spamd[28893]: processing message
> > > > > <[EMAIL PROTECTED]> for web321p1:104.
> > > > > Mar 13 01:16:20 ns spamd[28893]: Use of uninitialized value in
> > > > > concatenation (.) or string at
> > > > >
> /usr/lib/perl5/vendor_perl/5.8.3/Mail/SpamAssassin/NoMailAudit.pm
> > line
> > > > > 184.
> > >
> > > I'm guessing you upgraded to 3.x but are using the 2.x
> > spamassassin/spamd.
> > > NoMailAudit doesn't exist in 3.x.
> > >
> >
> > /var/log/mail tells me
> > spamd[960]: server started on port 783/tcp (running version 3.0.2)
> >
> > :~# spamc  -V
> > SpamAssassin Client version 3.0.2
> >
> > I did my upgrade via CPAN. What did go wrong?
> >
> 
> Are old modules the reason my spam does not get marked as spam ?

I would say so -- I'm surprised anything is working if NoMailAudit
is being used.

- --j.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCOIHeMJF5cimLx9ARAjCyAJ0VPUlAhiT0OY/aCfarRL0TRtkIdQCfeU7f
fat6LlqYi53mJDGbZztdgpo=
=T2CQ
-END PGP SIGNATURE-

Re: URI Tests and Japanese Chars (solved)

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Bobby, could you open a bug in the bugzilla about this?  URI rules
should not be checking mailto links.

- --j.

Jeff Chan writes:
> On Wednesday, March 16, 2005, 5:47:40 AM, Bobby Rose wrote:
> > This is an excerpt that I used in trying to track it down.  No
> > real mailto URI unless there is some translation going on with
> > email addresses embedded in the body by the email client on send.  At 
> > first, I just thought it might be a bug since the messages were
> > using ISO-2022-JP character set but if I sent just a plain text
> > message with just the [EMAIL PROTECTED] in the body, then 
> > URIBL_SBL was tripped. 
> 
> Wow, I didn't think URIBL_SBL would check that.  Hopefully the
> developers (of which I am not one ;-) will speak up about this.
> 
> Jeff C.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCOIGtMJF5cimLx9ARAqlXAJ9iEOrhjJVJzfx+U5I52iz4ifmzPwCfevy6
nGD2j3C3kfGTZGPNINvGh1I=
=btni
-END PGP SIGNATURE-

Re: SA 3.0.2 MASSIVE memory cpu problems

[Justin Mason]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


I would suggest running with -D and monitoring spamd memory size
as it starts up.   Something is causing it to balloon to massive
sizes after startup.

Presumably you are limiting the size of the messages sent in for scanning,
as recommended in the documentation? Scanning messages over 500KB in size
will result in a corresponding increase in resident size for the scanner
process, so a very large message could cause a massive scanner process.
(spamc will do this limit by default.)

Also you're the first person on linux to note the "select timeout
failed" bug.  could you attach output from strace and "spamd -D"
to a new bugzilla bug on this?

- --j.

[EMAIL PROTECTED] writes:
> Greg,
> i have
> 
> use_auto_whitelist 0
> 
> in the local.cf
> 
> But thanks anyway
> Wolfgang
> 
> "Greg Allen" <[EMAIL PROTECTED]> schrieb am 16.03.2005 13:54:24:
> 
> > Some users have had problems with corrupt AWL database after upgrade of
> > Spamassassin. Try disabling AWL to see if that is your issue.
> 
> >
> > -Original Message-
> > From: [EMAIL PROTECTED]
> > [mailto:[EMAIL PROTECTED]
> > Sent: Wednesday, March 16, 2005 5:44 AM
> > To: [EMAIL PROTECTED]
> > Cc: [EMAIL PROTECTED]
> > Subject: SA 3.0.2 MASSIVE memory cpu problems
> 
> >
> > Dear collegues,
> 
> > I'm having still extrem problems with memory and cpu consumation of SA
> > 3.0.2 spamd;
> 
> > PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  Command
> > 19098 exim  30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd
> > 19121 exim  20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd
> > 24591 exim  22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd
> > 25802 exim  21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd
> > 20548 exim  20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd
> > 26479 exim  20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd
> 
> > I already limited the # of children to 5 and the # of connections to 20
> > my own comment on this problem some month (restart spamd daily) does not
> > work anymore;
> 
> > I've seen, there is a patch in Bugzilla addressing this problem, which
> > hasn't made his
> > way into 3.0.2; I tried to apply it against 3.0.2 with the following
> > result:
> 
> > Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!
> > recovering
> > (repeated 20x and more)
> 
> > the only solution for the moment is to restart spamd on an hourly basis
> ?!
> 
> > WHAT CAN I DO?
> 
> > Wolfgang
> > P.S.: some environment
> 
> > perl -v
> > This is perl, v5.8.0 built for i586-linux-thread-multi
> 
> > saxophon:~ # spamassassin -V
> > SpamAssassin version 3.0.2
> > running on Perl version
> > saxophon:~ # uname -a
> > Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686
> > unknown unknown GNU/Linux
> 
> > --
> > Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
> > EBEWE Pharma
> > Mondseestrasse 11
> > 4866  Unterach, Austria
> > Tel: ++43 7665 8123 315
> > Fax: ++43 7665 8123 11
> > http://www.ebewe.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Exmh CVS

iD8DBQFCOIF2MJF5cimLx9ARApqoAJ46iCjfM13XD0MF08ViuuduVW6ASACfRat9
XaB8642NZWeXoclyDuLXR0g=
=v3ys
-END PGP SIGNATURE-

Re: Is this Received header correctly formatted?

[List Mail User]

>To: Loren Wilton <[EMAIL PROTECTED]>
>Cc: SpamAssassin Mailing List <[EMAIL PROTECTED]>
>Subject: Re: Is this Received header correctly formatted?
>
>
>Loren Wilton wrote:
>> Received: from ar39.lsanca2-4.16.241.28.lsanca2.elnk.dsl.genuity.net
>> ([4.16.241.28] helo=watson1)
>>  by pop-a065d23.pas.sa.earthlink.net with smtp (Exim 3.33 #1)
>>  id 1DBKRe-Kp-00; Tue, 15 Mar 2005 14:23:22 -0800
>> 
>> 1) Is "stmp" in lower case valid, or should it have been STMP?
>> 2) Is it valid to have the (Exim etc) stuff between 'stmp' and 'id'?
>> 3) Anything else that may be off the mark?
>
>The robustness principle says that you should be strict in what you send 
>and liberal in what you accept. From that perspective, it's not a 
>strictly conformant header, but its not broken enough for somebody to 
>refuse to parse it.
>
>In answer to your questions:
>
>  1) the spec calls for uppercase
>
>  2) header data in parenthesis is comment data. comments are supposed
> to be ~allowed anywhere that whitespace is allowed (this rule is
> actually documented in RFC2822, which governs header fields). with
> that in mind, yes, it's fine there.
>
>  3) the "helo=" stuff isn't conformant
>
>
>Here's the BNF notation for the Received header as provided in RFC2821:
>
>| Time-stamp-line = "Received:" FWS Stamp 
>|
>| Stamp = From-domain By-domain Opt-info ";"  FWS date-time
>|
>|   ; where "date-time" is as defined in [32]
>|   ; but the "obs-" forms, especially two-digit
>|   ; years, are prohibited in SMTP and MUST NOT be used.
>|
>| From-domain = "FROM" FWS Extended-Domain CFWS
>|
>| By-domain = "BY" FWS Extended-Domain CFWS
>|
>| Extended-Domain = Domain /
>|( Domain FWS "(" TCP-info ")" ) /
>|( Address-literal FWS "(" TCP-info ")" )
>|
>| TCP-info = Address-literal / ( Domain FWS Address-literal )
>|   ; Information derived by server from TCP connection
>|   ; not client EHLO.
>|
>| Opt-info = [Via] [With] [ID] [For]
>|
>| Via = "VIA" FWS Link CFWS
>|
>| With = "WITH" FWS Protocol CFWS
>|
>| ID = "ID" FWS String / msg-id CFWS
>|
>| For = "FOR" FWS 1*( Path / Mailbox ) CFWS
>|
>| Link = "TCP" / Addtl-Link
>| Addtl-Link = Atom
>|   ; Additional standard names for links are registered with the
>|   ; Internet Assigned Numbers Authority (IANA).  "Via" is
>|   ; primarily of value with non-Internet transports.  SMTP
>|   ; servers SHOULD NOT use unregistered names.
>| Protocol = "ESMTP" / "SMTP" / Attdl-Protocol
>| Attdl-Protocol = Atom
>| ; Additional standard names for protocols are registered with the
>| ; Internet Assigned Numbers Authority (IANA).  SMTP servers
>| ; SHOULD NOT use unregistered names.
>
>
>-- 
>Eric A. Hall   http://www.ehsco.com/
>Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
>

Eric, I think you hit all the salient points, but you did miss one
important one. Earlier in RFC2821, before the section you quoted is:

RFC2821 Section 2.4
" ...
The metalinguistic notation used in this document corresponds to the
"Augmented BNF" used in other Internet mail system documents.  The
reader who is not familiar with that syntax should consult the ABNF
specification [8].  Metalanguage terms used in running text are
surrounded by pointed brackets (e.g., ) for clarity."

Where reference [8] is:
"[8]  Crocker, D. and P. Overell, Eds., "Augmented BNF for Syntax
Specifications: ABNF", RFC 2234, November 1997."

and in RFC 2234

RFC2234 Section 2.3
"2.3  Terminal Values

   Rules resolve into a string of terminal values, sometimes called
   characters.  In ABNF a character is merely a non-negative integer.
   In certain contexts a specific mapping (encoding) of values into a
   character set (such as ASCII) will be specified.

   Terminals are specified by one or more numeric characters with the
   base interpretation of those characters indicated explicitly.  The
   following bases are currently defined:

b   =  binary

d   =  decimal

x   =  hexadecimal

   Hence:

CR  =  %d13

CR  =  %x0D

   respectively specify the decimal and hexadecimal representation of
   [US-ASCII] for carriage return.

   A concatenated string of such values is specified compactly, using a
   period (".") to indicate separation of characters within that value.
   Hence:

CRLF=  %d13.10

   ABNF permits specifying literal text string directly, enclosed in
   quotation-marks.  Hence:

command =  "command string"

   Literal text strings are interpreted as a concatenated set of
   printable characters.

NOTE: ABNF strings are case-insensitive and
  the character set for these strings is us-ascii.

   Hence:

rulename = "abc"

   and:

rulename = "aBc"

   will match "abc", "Abc", "aBc", "abC", "ABc", "aBC", "AbC" and "ABC".

To specify a

Re: sa-learn hangs

[Eric Dantan Rzewnicki]

On Tue, Mar 15, 2005 at 06:11:27PM -0500, Matt Kettler wrote:
> Eric Dantan Rzewnicki wrote:
> >spamassassin -D -p  --lint doesn't show any problems that I
> >can see.
> >if I run: 
> >sa-learn --showdots --mbox --ham -p 
> >/opt/MailScanner/etc/spam.assassin.prefs.conf 
> >sa-learn just hangs. Same happens for --spam.
> >strace shows it stuck on a read(0,
> >Any ideas?
> What about sa-learn -D -p  --dump magic? does that hang? 

No hang with debug added to --dump. Same as without -D.

> If 
> that works, what about sa-learn -D --rebuild?

rebuild works fine, too.

> Do you have an alternate bayes_path set in your spam.assassin.prefs.conf?

I have these:

bayes_path /var/spool/MailScanner/spamassassin/bayes
bayes_file_mode 0660

> Does the user running sa-learn have rw permissions to it, and rwx to the 
> directory containing it?

I'm running sa-learn as root. 

:/var/spool/MailScanner/spamassassin# ls -la
total 7788
drwxr-xr-x2 postfix  postfix  4096 Mar 16 13:05 .
drwxr-xr-x6 root root 4096 Jan 23  2004 ..
-rw---1 postfix  postfix   390 Mar 16 13:04 bayes.mutex
-rw-rw1 postfix  postfix320400 Mar 16 13:13 bayes_journal
-rw-rw1 postfix  postfix   2641920 Mar 15 17:52 bayes_seen
-rw-rw1 postfix  postfix   5697536 Mar 16 13:05 bayes_toks


-- 
Eric Dantan Rzewnicki  |  Systems Administrator
Technical Operations Division  |  Radio Free Asia
2025 M Street, NW  |  Washington, DC 20036  |  202-530-4900
CONFIDENTIAL COMMUNICATION
This e-mail message is intended only for the use of the addressee and
may contain information that is privileged and confidential. Any 
unauthorized dissemination, distribution, or copying is strictly 
prohibited. If you receive this transmission in error, please contact
[EMAIL PROTECTED]

Re: Is there such a test?

[List Mail User]

>...
>Date: Wed, 16 Mar 2005 09:38:13 - (GMT)
>Subject: Re: Is there such a test?
>From: "Mike Spamassassin" <[EMAIL PROTECTED]>
>
>I'd take that bet.
>While you are almost certainly correct with the likes of those who
>subscribe to this group, who often have multiple email addresses,
>out there in [EMAIL PROTECTED] land, and hotmail world, most people have a 
>single
>email address strongly related to their name.
>
>Back to the original question:
>Regardless of whether anyone thinks it is a good test or not, has anyone
>yet created such a test?
>
>> Mike Spamassassin wrote:
>>
>>>Point taken, but I still think it would be a valid test.
>>>Like all SpamAssassin tests it should only be one of many indicators.
>>>
>>
>> No, not really. There's a minimum useful S/O ratio for spam rules.
>>
>> I'd bet $5.00 that this rule would have a S/O under 0.80 in the
>> corpus.(ie: no more 80% of it's hits were spam, and at least 20% were ham)
>>
>>
>
>
>
Mike,

I think you are probably correct for exactly the group you describe;
Individual users (i.e. "in [EMAIL PROTECTED] land, and hotmail world").  But, 
you'd
get killed with corporate accounts.  I haven't made it to London is quite
a while, but if you change the bet to a pint, I'll take it too. (BTW. MS
passports, unlike normal Hotmail/MSN accounts would fail very often also,
and "techies" often have "cute" account names, so anyone who does a lot
of technical discussions would might have have problems).

I think the only mail I normally see that wouldn't fail your test
is a few mailing list posters (and many counter examples have already been
listed for you, I could give you another couple of dozen I get nearly every
day) and relatives (and many of them are weird; e.g. "The LastName Household"
with accounts like [EMAIL PROTECTED]/com are common - It used to be
on one of their example pages for "family" accounts with multiple mailboxes).
And it would be hard to pick out the single letter of the account name that
matches the first letter of the "Lastname" - in fact, to me, initials followed
by digits "seem" to be a spammer unless I know the person (but I doubt that
does any better than 50%, or even that well) - now names followed, by a year
might meet the 80% test (ex. [EMAIL PROTECTED] or something like an account
[EMAIL PROTECTED]).  Didn't we have one of those, asking people
to accept mail from invalid domains, on the list a couple of weeks ago?

Paul Shupak
[EMAIL PROTECTED]

Re: Is there such a test?

[Matt Kettler]

Mike Spamassassin wrote:
I'd take that bet.
While you are almost certainly correct with the likes of those who
subscribe to this group, who often have multiple email addresses,
out there in [EMAIL PROTECTED] land, and hotmail world, most people have a single
email address strongly related to their name.
 

Really? My yahoo address is [EMAIL PROTECTED] Not very well 
related to "Matt Kettler".

Most people *try* to have one strongly related to their name, but often 
they fail to be available so they default back to something related to 
their interests. One friend of mine who is a skiing buff uses 
"doubleblack" as an address.  Others intentionally choose something 
unrelated to their name in order to gain privacy. Another friend 
intentionally uses the all-text spelling of his jersey number from high 
school sports as his email. i.e. "thirtyone"

I'd say this is actually much more common in the hotmail world than in 
the admin world. There's too much name collision for easy things like 
"mkettler" to be available on hotmail.

Back to the original question:
Regardless of whether anyone thinks it is a good test or not, has anyone
yet created such a test?
 

I doubt it. It's not exactly a straight-forward thing to do. You'd need 
to have some kind of fuzzy-match algorithm, because there's so many 
different ways to convert a name to an email.

I use descriptors like:
"Matt" "Matt Kettler" "Matthew Kettler" and "Matthew E Kettler"
I use emails like:
matt@ mattk@, mkettler@, mekettler@ mattkettler@ mkettler73@  kettlerm@
Trying to map all of the above combinations to each other is kind of 
tough. Particularly mappings like "Matt" to kettlerm@

Re: Sudden spam to this email address

[Stuart Johnston]

Jeff Chan wrote:
On Tuesday, March 15, 2005, 9:02:44 AM, Stuart Johnston wrote:
SURBLs have them... most of the time... eventually...  Er, yeah.

Just to check, are you using ob.surbl.org and jp.surbl.org
in multi.surbl.org, i.e.:
In the last ~24 hours:
All SA > 5:  32540
*_SURBL:22361 (69%)
JP_SURBL:   20157 (62%)
OB_SURBL:   19900 (61%)
This is after a couple of DNSBLs at SMTP which may skew my stats.

Re: sa-learn hangs

[Joe Zitnik]

I had that happen once before, but it was an earlier version of the
Bayes DB, and it was because my database was hosed.

>>> Eric Dantan Rzewnicki <[EMAIL PROTECTED]> 3/15/2005 6:01 PM >>>
Hello,

I'm using spamassassin 3.0.2 from within MailScanner 4.39.6 on Debian
woody. After upgrading to spamassassin 3.0.2 (installed from source
tarball) I am unable to use sa-learn to train the bayes engine on ham
or
spam. Spamassassin is otherwise working fine. Before upgrading I wiped
out my previous bayes database. It has since grown well beyond the 200
minimum ham and spam and Spamassassin is using bayes to score mail.  

sa-learn --dump magic -p /opt/MailScanner/etc/spam.assassin.prefs.conf
works fine and shows that the bayes database is growing as it should
through autolearning.  

spamassassin -D -p  --lint doesn't show any problems that I
can see.

if I run: 
sa-learn --showdots --mbox --ham -p
/opt/MailScanner/etc/spam.assassin.prefs.conf 

sa-learn just hangs. Same happens for --spam.

strace shows it stuck on a read(0,

Any ideas?

I've tried searching the archives and the wiki, but haven't turned up
anything yet. There doesn't seem to be anything about this in the FAQ,
either. I've reported this on the MailScanner list as well, but so far
have not received a response.
-- 
Eric Dantan Rzewnicki  |  Systems Administrator
Technical Operations Division  |  Radio Free Asia
2025 M Street, NW  |  Washington, DC 20036  |  202-530-4900
CONFIDENTIAL COMMUNICATION
This e-mail message is intended only for the use of the addressee and
may contain information that is privileged and confidential. Any 
unauthorized dissemination, distribution, or copying is strictly 
prohibited. If you receive this transmission in error, please contact
[EMAIL PROTECTED]

Re: Is there such a test?

[Yang Xiao]

Alright, I'm developing such a test.
For American/Anglo-Sexon names, it will do random comparason with the
Webster Dictionary for FLast, FirshL, First.Last Last.F, Last.First
and spell check them all.
For Indian names, it will search the Yahoo movie Database.
For French Names, we will append "Freedom Fries" then do search number 1.
For Chinese names we will address-rewrite everybody to
[EMAIL PROTECTED] and pipe to /dev/null
.


Problem solved!

Yang

Re: Is there such a test?

[Keith Ivey]

Mike Spamassassin wrote:
I'd take that bet.
While you are almost certainly correct with the likes of those who
subscribe to this group, who often have multiple email addresses,
out there in [EMAIL PROTECTED] land, and hotmail world, most people have a 
single
email address strongly related to their name.
I'll kick in another $5 for the bet.  The Hotmail, AOL, Yahoo 
world is if anything more likely to have addresses that have 
nothing to do with names, because the username spaces are so 
fullthat people are forced to be creative and give themselves 
addresses related to their hobbies or favorite sprts teams.

I think the person you're responding to was generous in 
expecting the test might hit 80% spam and 20% ham.  My bet is 
that it would be closer to 50%, assuming you're able to come up 
with a definition of "address is related to name".

--
Keith C. Ivey <[EMAIL PROTECTED]>
Washington, DC

Re: URI Tests and Japanese Chars (solved)

[Jeff Chan]

On Wednesday, March 16, 2005, 5:47:40 AM, Bobby Rose wrote:
> This is an excerpt that I used in trying to track it down.  No
> real mailto URI unless there is some translation going on with
> email addresses embedded in the body by the email client on send.  At 
> first, I just thought it might be a bug since the messages were
> using ISO-2022-JP character set but if I sent just a plain text
> message with just the [EMAIL PROTECTED] in the body, then 
> URIBL_SBL was tripped. 

Wow, I didn't think URIBL_SBL would check that.  Hopefully the
developers (of which I am not one ;-) will speak up about this.

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

RE: URI Tests and Japanese Chars (solved)

[Rose, Bobby]

This is an excerpt that I used in trying to track it down.  No real mailto URI 
(Bunless there is some translation going on with email addresses embedded in the 
(Bbody by the email client on send.  At first, I just thought it might be a bug 
(Bsince the messages were using ISO-2022-JP character set but if I sent just a 
(Bplain text message with just the [EMAIL PROTECTED] in the body, then URIBL_SBL 
(Bwas tripped. 
(B
(B*
(B- Original Message -
(BFrom: "user1" <[EMAIL PROTECTED]>
(BTo: "user2" <[EMAIL PROTECTED]>
(BSent: Friday, March 11, 2005 11:14 AM
(BSubject: Re: $BFb;[EMAIL PROTECTED](J 
(B
(B***
(B
(B-=B
(B
(B
(B-Original Message-
(BFrom: Jeff Chan [mailto:[EMAIL PROTECTED] 
(BSent: Wednesday, March 16, 2005 7:52 AM
(BTo: users@spamassassin.apache.org
(BSubject: Re: URI Tests and Japanese Chars (solved)
(B
(BOn Wednesday, March 16, 2005, 3:55:52 AM, Bobby Rose wrote:
(B 
(B> I figured out the problem, it' was the an individuals email address in 
(B> the message body (even though not a mailto).  Their email domain isn't 
(B> listed at spamhaus.org but it turns out one of their ISPs DNS servers 
(B> are which they are using as secondary.  This makes the second time 
(B> I've come across this.  The last time it was an ISP's (pipex.net) DNS 
(B> server in the U.K. that was tripping the URIBL_SBL rule.
(B
(B> This time the user is in the med.juntendo.ac.jp (Juntendo Univ Med
(B> School) who's ISP is cwidc.net and the DNS server  ns03.cwidc.net
(B> (154.33.17.212) is the one in spamhaus.org which they say is hosting a 
(B> long time spammer.  
(B> http://www.spamhaus.org/sbl/sbl.lasso?query=SBL17240
(B
(B> Does URI checking really need to be so thorough?  Obviously there must 
(B> be some bias at spamhaus if the big named ISPs don't get their name 
(B> servers listed because we know that they provide services to spammers.
(B> Any idea on how to limit the scope to just the URI at it's face value?
(B
(Buridnsbl used in the default rule URIBL_SBL does check domain name servers 
(Bagainst SBL, but I'm kind of surprised to hear it triggering on email 
(Baddresses.  It should definitely be checking web sites and the like.  Can you 
(Bgive a sample of the text it hit?  Was it in URI form like:
(B
(B  mailto://[EMAIL PROTECTED]
(B
(BThat said, I agree that the SBL listings are at times overbroad.
(BName servers for gov.ru and spb.ru for example are listed (ns.rtcomm.ru and 
(Bns1.relcom.ru respectively).  Listings like those can cause false positives, 
(Band I personally object to deliberately harming innocent bystanders to 
(B"pressure" ISPs.
(B
(BJeff C.
(B--
(BJeff Chan
(Bmailto:[EMAIL PROTECTED]
(Bhttp://www.surbl.org/

Re: Re: Re: Re: Upgrade... + other (perl?) problems

[sa-users]

Am 16.03.2005 um 08:55 Uhr haben Sie geschrieben:
> Am 16.03.2005 um 00:31 Uhr haben Sie geschrieben:
> > On Wed, Mar 16, 2005 at 12:27:28AM +0100, [EMAIL PROTECTED]
> wrote:
> > > Are there problems with mail header identification?
> > > Am I in the wrong list with this question?
> > > > Mar 13 01:16:18 ns spamd[28893]: processing message
> > > > <[EMAIL PROTECTED]> for web321p1:104.
> > > > Mar 13 01:16:20 ns spamd[28893]: Use of uninitialized value in
> > > > concatenation (.) or string at
> > > >
/usr/lib/perl5/vendor_perl/5.8.3/Mail/SpamAssassin/NoMailAudit.pm
> line
> > > > 184.
> >
> > I'm guessing you upgraded to 3.x but are using the 2.x
> spamassassin/spamd.
> > NoMailAudit doesn't exist in 3.x.
> >
>
> /var/log/mail tells me
> spamd[960]: server started on port 783/tcp (running version 3.0.2)
>
> :~# spamc  -V
> SpamAssassin Client version 3.0.2
>
> I did my upgrade via CPAN. What did go wrong?
>

Are old modules the reason my spam does not get marked as spam ?

Lars Dierich

Re: RE: SA 3.0.2 MASSIVE memory cpu problems

[Wolfgang . Fuertbauer]

Greg,
i have

use_auto_whitelist 0

in the local.cf

But thanks anyway
Wolfgang

"Greg Allen" <[EMAIL PROTECTED]> schrieb am 16.03.2005 13:54:24:

> Some users have had problems with corrupt AWL database after upgrade of
> Spamassassin. Try disabling AWL to see if that is your issue.

>
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 16, 2005 5:44 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: SA 3.0.2 MASSIVE memory cpu problems

>
> Dear collegues,

> I'm having still extrem problems with memory and cpu consumation of SA
> 3.0.2 spamd;

> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  Command
> 19098 exim  30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd
> 19121 exim  20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd
> 24591 exim  22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd
> 25802 exim  21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd
> 20548 exim  20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd
> 26479 exim  20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd

> I already limited the # of children to 5 and the # of connections to 20
> my own comment on this problem some month (restart spamd daily) does not
> work anymore;

> I've seen, there is a patch in Bugzilla addressing this problem, which
> hasn't made his
> way into 3.0.2; I tried to apply it against 3.0.2 with the following
> result:

> Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!
> recovering
> (repeated 20x and more)

> the only solution for the moment is to restart spamd on an hourly basis
?!

> WHAT CAN I DO?

> Wolfgang
> P.S.: some environment

> perl -v
> This is perl, v5.8.0 built for i586-linux-thread-multi

> saxophon:~ # spamassassin -V
> SpamAssassin version 3.0.2
> running on Perl version
> saxophon:~ # uname -a
> Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686
> unknown unknown GNU/Linux

> --
> Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
> EBEWE Pharma
> Mondseestrasse 11
> 4866  Unterach, Austria
> Tel: ++43 7665 8123 315
> Fax: ++43 7665 8123 11
> http://www.ebewe.com
--
Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
EBEWE Pharma
Mondseestrasse 11
4866  Unterach, Austria
Tel: ++43 7665 8123 315
Fax: ++43 7665 8123 11
http://www.ebewe.com

RE: SA 3.0.2 MASSIVE memory cpu problems

[Peter Tarjan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Dear Greg,

> Some users have had problems with corrupt AWL database after upgrade of
> Spamassassin. Try disabling AWL to see if that is your issue.

I'm totally new to the list and I don't know 3.0.2 so I'm not sure how 
helpful this is going to be, but anyway.

I had a similar problem when upgrading from 2.55 to 2.64. Spamd would
immediately eat all available memory and then some, which resulted in
continuous swapping, bringing the machine to a grinding halt. Best of all,
spamd in the end did not tag the message at all.

For me, what helped was starting spamd in debug mode and realizing that it 
got stuck in the DB_File (or was it File_DB?) perl module. I updated the 
module from CPAN, and voila, problem gone.

Cheers,
Peter
 
> 
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, March 16, 2005 5:44 AM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: SA 3.0.2 MASSIVE memory cpu problems
> 
> 
> Dear collegues,
> 
> I'm having still extrem problems with memory and cpu consumation of SA
> 3.0.2 spamd;
> 
> PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  Command
> 19098 exim  30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd
> 19121 exim  20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd
> 24591 exim  22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd
> 25802 exim  21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd
> 20548 exim  20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd
> 26479 exim  20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd
> 
> I already limited the # of children to 5 and the # of connections to 20
> my own comment on this problem some month (restart spamd daily) does not
> work anymore;
> 
> I've seen, there is a patch in Bugzilla addressing this problem, which
> hasn't made his
> way into 3.0.2; I tried to apply it against 3.0.2 with the following
> result:
> 
> Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!
> recovering
> (repeated 20x and more)
> 
> the only solution for the moment is to restart spamd on an hourly basis ?!
> 
> WHAT CAN I DO?
> 
> Wolfgang
> P.S.: some environment
> 
> perl -v
> This is perl, v5.8.0 built for i586-linux-thread-multi
> 
> saxophon:~ # spamassassin -V
> SpamAssassin version 3.0.2
>   running on Perl version
> saxophon:~ # uname -a
> Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686
> unknown unknown GNU/Linux
> 
> --
> Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
> EBEWE Pharma
> Mondseestrasse 11
> 4866  Unterach, Austria
> Tel: ++43 7665 8123 315
> Fax: ++43 7665 8123 11
> http://www.ebewe.com
> 

- -- 
Before destruction a man's heart is
haughty, but humility goes before honour.
-- Psalms 18:12
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: Made with pgp4pine 1.76

iD8DBQFCODBCofCNq1TXA9IRAm3LAJ9W1rfYY8lMFp1D2P1slx3ITw1taACdESPO
7T6CqyAJAQqRLnHBpsnD508=
=/d5R
-END PGP SIGNATURE-

RE: SA 3.0.2 MASSIVE memory cpu problems

[Greg Allen]

Some users have had problems with corrupt AWL database after upgrade of
Spamassassin. Try disabling AWL to see if that is your issue.


-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]
Sent: Wednesday, March 16, 2005 5:44 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: SA 3.0.2 MASSIVE memory cpu problems


Dear collegues,

I'm having still extrem problems with memory and cpu consumation of SA
3.0.2 spamd;

PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  Command
19098 exim  30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd
19121 exim  20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd
24591 exim  22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd
25802 exim  21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd
20548 exim  20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd
26479 exim  20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd

I already limited the # of children to 5 and the # of connections to 20
my own comment on this problem some month (restart spamd daily) does not
work anymore;

I've seen, there is a patch in Bugzilla addressing this problem, which
hasn't made his
way into 3.0.2; I tried to apply it against 3.0.2 with the following
result:

Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!
recovering
(repeated 20x and more)

the only solution for the moment is to restart spamd on an hourly basis ?!

WHAT CAN I DO?

Wolfgang
P.S.: some environment

perl -v
This is perl, v5.8.0 built for i586-linux-thread-multi

saxophon:~ # spamassassin -V
SpamAssassin version 3.0.2
  running on Perl version
saxophon:~ # uname -a
Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686
unknown unknown GNU/Linux

--
Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
EBEWE Pharma
Mondseestrasse 11
4866  Unterach, Austria
Tel: ++43 7665 8123 315
Fax: ++43 7665 8123 11
http://www.ebewe.com

Re: URI Tests and Japanese Chars (solved)

[Jeff Chan]

On Wednesday, March 16, 2005, 3:55:52 AM, Bobby Rose wrote:
 
> I figured out the problem, it' was the an individuals email address in
> the message body (even though not a mailto).  Their email domain isn't
> listed at spamhaus.org but it turns out one of their ISPs DNS servers
> are which they are using as secondary.  This makes the second time I've
> come across this.  The last time it was an ISP's (pipex.net) DNS server
> in the U.K. that was tripping the URIBL_SBL rule.

> This time the user is in the med.juntendo.ac.jp (Juntendo Univ Med
> School) who's ISP is cwidc.net and the DNS server  ns03.cwidc.net
> (154.33.17.212) is the one in spamhaus.org which they say is hosting a
> long time spammer.  http://www.spamhaus.org/sbl/sbl.lasso?query=SBL17240

> Does URI checking really need to be so thorough?  Obviously there must
> be some bias at spamhaus if the big named ISPs don't get their name
> servers listed because we know that they provide services to spammers.
> Any idea on how to limit the scope to just the URI at it's face value?

uridnsbl used in the default rule URIBL_SBL does check domain
name servers against SBL, but I'm kind of surprised to hear it
triggering on email addresses.  It should definitely be checking
web sites and the like.  Can you give a sample of the text it
hit?  Was it in URI form like:

  mailto://[EMAIL PROTECTED]

That said, I agree that the SBL listings are at times overbroad.
Name servers for gov.ru and spb.ru for example are listed
(ns.rtcomm.ru and ns1.relcom.ru respectively).  Listings like
those can cause false positives, and I personally object to
deliberately harming innocent bystanders to "pressure" ISPs.

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

Re: Header Tagging with # instead of *

[Peter Guhl]

On Sun, 2005-03-13 at 05:09, John Andersen wrote:
> On Saturday 12 March 2005 02:47 pm, jdow wrote:
> > The canonical way to do it is something like:
> >
> > rewrite_header Subject *SPAM* _SCORE(00)_ **
> >
> > That gives headers that look like:
> > Subject: *SPAM* 027.3 ** spoo is best for slow sex
> 
> The OP was interested in header tagging, (Hence the subject of the
> thread), not munging the subject line.

Both are right ;) For me the subject line is part of the header. But, as
I have told, I already tried to escape the # with \# and the result was
that it was writing it including the backslash as \#. Without escaping
it does, again as I said, nothing because anything starting with # is
considered a comment in local.cf

I did use the "+" now. No need to escape anything and (hopefully) no
wildcard.

Thanks for your tips anyway.

Regards
Peter

RE: URI Tests and Japanese Chars (solved)

[Rose, Bobby]

 
I figured out the problem, it' was the an individuals email address in
the message body (even though not a mailto).  Their email domain isn't
listed at spamhaus.org but it turns out one of their ISPs DNS servers
are which they are using as secondary.  This makes the second time I've
come across this.  The last time it was an ISP's (pipex.net) DNS server
in the U.K. that was tripping the URIBL_SBL rule.

This time the user is in the med.juntendo.ac.jp (Juntendo Univ Med
School) who's ISP is cwidc.net and the DNS server  ns03.cwidc.net
(154.33.17.212) is the one in spamhaus.org which they say is hosting a
long time spammer.  http://www.spamhaus.org/sbl/sbl.lasso?query=SBL17240

Does URI checking really need to be so thorough?  Obviously there must
be some bias at spamhaus if the big named ISPs don't get their name
servers listed because we know that they provide services to spammers.
Any idea on how to limit the scope to just the URI at it's face value?

-Original Message-
From: Rose, Bobby [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, March 15, 2005 2:14 PM
To: users@spamassassin.apache.org
Subject: URI Tests and Japanese Chars

I have a user that is of Japanese origin and who converses with other
individuals in Japan in his same field of study.  The messages they send
are in Japanese and trip the URI_SBL rule.  These people are in
different .jp domains and I really don't want to get into the
administrative overhead of whitelisting. I don't see anything in the
message bodies that even looks like a URI.  Has anyone else ran into
this?


Bobby Rose
Wayne State University School of Medicine 

permission problems? SpamAssassin 3.02 with running with sql

[Philipp Snizek]

Hi

when running spamd with 'spamd -D -q' the SQL statement 'SELECT
preference,value FROM sa_prefs WHERE username=_USERNAME_ OR
username='$GLOBAL' OR username=CONCAT('%',_USERNAME_) ORDER BY
username ASC' and testing from shell with 'echo -e "From:
user\nTo:user\Subject: Test\n\n" | spamc -u '$GLOBAL'' all works fine.


When starting spamd with 'spamd -D -q -u filter' and try to do the
same thing I get this:

failed to load user ($GLOBAL) scores from SQL database: SQL Error:
Can't connect to local MySQL server through socket
'/var/lib/mysql/mysql.sock' (13)

logmsg: service unavailable: Error fetching user preferences via SQL


thanks,
Philipp

SA 3.0.2 MASSIVE memory cpu problems

[Wolfgang . Fuertbauer]

Dear collegues,

I'm having still extrem problems with memory and cpu consumation of SA
3.0.2 spamd;

PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  Command
19098 exim  30   5  399m 398m  34m R 99.7 65.8   1:37.47 spamd
19121 exim  20   5  111m 111m  34m S  0.7  4.4   0:20.78 spamd
24591 exim  22   5 75844  74m  35m S  0.0  2.9   0:02.90 spamd
25802 exim  21   5 75040  73m  35m S  0.0  2.9   0:01.46 spamd
20548 exim  20   5 69836  68m  67m S  0.0  2.7   0:03.03 spamd
26479 exim  20   5 69836  68m  67m S  0.0  2.7   0:00.00 spamd

I already limited the # of children to 5 and the # of connections to 20
my own comment on this problem some month (restart spamd daily) does not
work anymore;

I've seen, there is a patch in Bugzilla addressing this problem, which
hasn't made his
way into 3.0.2; I tried to apply it against 3.0.2 with the following
result:

Mar 16 05:47:03 saxophon spamd[5201]: prefork: select timeout failed!
recovering
(repeated 20x and more)

the only solution for the moment is to restart spamd on an hourly basis ?!

WHAT CAN I DO?

Wolfgang
P.S.: some environment

perl -v
This is perl, v5.8.0 built for i586-linux-thread-multi

saxophon:~ # spamassassin -V
SpamAssassin version 3.0.2
  running on Perl version
saxophon:~ # uname -a
Linux saxophon 2.4.20-64GB-SMP #1 SMP Fri Jan 14 15:08:48 UTC 2005 i686
unknown unknown GNU/Linux

--
Wolfgang Fuertbauer (E-Mail: [EMAIL PROTECTED])
EBEWE Pharma
Mondseestrasse 11
4866  Unterach, Austria
Tel: ++43 7665 8123 315
Fax: ++43 7665 8123 11
http://www.ebewe.com

Re: Is this Received header correctly formatted?

[Eric A. Hall]

List Mail User wrote:
the "with" is sometimes also either a "by" or "via" (and probably 
other string values which I haven't noticed). BTW.
"by" "via" and "with" are separate sub-fields with their own meaning
--
Eric A. Hall   http://www.ehsco.com/
Internet Core Protocols http://www.oreilly.com/catalog/coreprot/

Re: Is this Received header correctly formatted?

[Eric A. Hall]

Loren Wilton wrote:
Received: from ar39.lsanca2-4.16.241.28.lsanca2.elnk.dsl.genuity.net
([4.16.241.28] helo=watson1)
 by pop-a065d23.pas.sa.earthlink.net with smtp (Exim 3.33 #1)
 id 1DBKRe-Kp-00; Tue, 15 Mar 2005 14:23:22 -0800
1) Is "stmp" in lower case valid, or should it have been STMP?
2) Is it valid to have the (Exim etc) stuff between 'stmp' and 'id'?
3) Anything else that may be off the mark?
The robustness principle says that you should be strict in what you send 
and liberal in what you accept. From that perspective, it's not a 
strictly conformant header, but its not broken enough for somebody to 
refuse to parse it.

In answer to your questions:
 1) the spec calls for uppercase
 2) header data in parenthesis is comment data. comments are supposed
to be ~allowed anywhere that whitespace is allowed (this rule is
actually documented in RFC2822, which governs header fields). with
that in mind, yes, it's fine there.
 3) the "helo=" stuff isn't conformant
Here's the BNF notation for the Received header as provided in RFC2821:
| Time-stamp-line = "Received:" FWS Stamp 
|
| Stamp = From-domain By-domain Opt-info ";"  FWS date-time
|
|   ; where "date-time" is as defined in [32]
|   ; but the "obs-" forms, especially two-digit
|   ; years, are prohibited in SMTP and MUST NOT be used.
|
| From-domain = "FROM" FWS Extended-Domain CFWS
|
| By-domain = "BY" FWS Extended-Domain CFWS
|
| Extended-Domain = Domain /
|( Domain FWS "(" TCP-info ")" ) /
|( Address-literal FWS "(" TCP-info ")" )
|
| TCP-info = Address-literal / ( Domain FWS Address-literal )
|   ; Information derived by server from TCP connection
|   ; not client EHLO.
|
| Opt-info = [Via] [With] [ID] [For]
|
| Via = "VIA" FWS Link CFWS
|
| With = "WITH" FWS Protocol CFWS
|
| ID = "ID" FWS String / msg-id CFWS
|
| For = "FOR" FWS 1*( Path / Mailbox ) CFWS
|
| Link = "TCP" / Addtl-Link
| Addtl-Link = Atom
|   ; Additional standard names for links are registered with the
|   ; Internet Assigned Numbers Authority (IANA).  "Via" is
|   ; primarily of value with non-Internet transports.  SMTP
|   ; servers SHOULD NOT use unregistered names.
| Protocol = "ESMTP" / "SMTP" / Attdl-Protocol
| Attdl-Protocol = Atom
| ; Additional standard names for protocols are registered with the
| ; Internet Assigned Numbers Authority (IANA).  SMTP servers
| ; SHOULD NOT use unregistered names.
--
Eric A. Hall   http://www.ehsco.com/
Internet Core Protocols http://www.oreilly.com/catalog/coreprot/

Re: Is there such a test?

[Mike Spamassassin]

I'd take that bet.
While you are almost certainly correct with the likes of those who
subscribe to this group, who often have multiple email addresses,
out there in [EMAIL PROTECTED] land, and hotmail world, most people have a 
single
email address strongly related to their name.

Back to the original question:
Regardless of whether anyone thinks it is a good test or not, has anyone
yet created such a test?

> Mike Spamassassin wrote:
>
>>Point taken, but I still think it would be a valid test.
>>Like all SpamAssassin tests it should only be one of many indicators.
>>
>
> No, not really. There's a minimum useful S/O ratio for spam rules.
>
> I'd bet $5.00 that this rule would have a S/O under 0.80 in the
> corpus.(ie: no more 80% of it's hits were spam, and at least 20% were ham)
>
>

Re: Re: Re: Upgrade... + other (perl?) problems

[sa-users]

Am 16.03.2005 um 00:31 Uhr haben Sie geschrieben:
> On Wed, Mar 16, 2005 at 12:27:28AM +0100, [EMAIL PROTECTED]
wrote:
> > Are there problems with mail header identification?
> > Am I in the wrong list with this question?
> > > Mar 13 01:16:18 ns spamd[28893]: processing message
> > > <[EMAIL PROTECTED]> for web321p1:104.
> > > Mar 13 01:16:20 ns spamd[28893]: Use of uninitialized value in
> > > concatenation (.) or string at
> > > /usr/lib/perl5/vendor_perl/5.8.3/Mail/SpamAssassin/NoMailAudit.pm
line
> > > 184.
>
> I'm guessing you upgraded to 3.x but are using the 2.x
spamassassin/spamd.
> NoMailAudit doesn't exist in 3.x.
>

/var/log/mail tells me
spamd[960]: server started on port 783/tcp (running version 3.0.2)

:~# spamc  -V
SpamAssassin Client version 3.0.2

I did my upgrade via CPAN. What did go wrong?

need testers for ldapBlacklist.pm plug-in

[Eric A. Hall]

I got the ldapBlick plug-in pretty much finished, and it just needs some 
polishing I think.

I'd like to get some help testing this for load and latency, so if 
anybody has a local LDAP server running already and is pretty 
comfortable with SA and LDAP, and is willing to poke at this, let me 
know. Be warned that this plugin can really beat the crap out of your 
LDAP server, and will add some measurable latency if the SA system is 
already burdened down. But it works pretty well, and is interesting if 
you're into LDAP.

Responses off-list pls.
Thanks
--
Eric A. Hall   http://www.ehsco.com/
Internet Core Protocols http://www.oreilly.com/catalog/coreprot/

Re: [SPAM-TAG] SpamAssassin, running on "mail.dailyhills.com" ...

[Jeff Chan]

On Tuesday, March 15, 2005, 9:27:50 PM, Vicki Brown wrote:
> Does anyone else find this just too absurdly silly for words?

> Although I guess it surely does prove the point Jeff Chan made for URIDNSBL
> and SURBL - most eloquently in fact :-)

>>SpamAssassin, running on "mail.dailyhills.com", has identified this incoming
>>email as possible spam.  The original message has been attached to this
>>email so you can view it (if it isn't spam).
>>If you have any questions, contact [EMAIL PROTECTED] for details.

Yes, but it's a broken configuration on Dave Hill's mail
server...

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

SpamAssassin, running on "mail.dailyhills.com" ...

[Vicki Brown]

Does anyone else find this just too absurdly silly for words?

Although I guess it surely does prove the point Jeff Chan made for URIDNSBL
and SURBL - most eloquently in fact :-)

>SpamAssassin, running on "mail.dailyhills.com", has identified this incoming
>email as possible spam.  The original message has been attached to this
>email so you can view it (if it isn't spam).
>If you have any questions, contact [EMAIL PROTECTED] for details.
>
>Content preview:  I've been going through a bunch of spam and
>  blacklisting domains. However, some of the more frequent offenders are
>  in the body of the message. For example, today I found about half a
>  dozen porno spams that contained a reference to
>  http://www.a123s.biz/... [...]
>
>Content analysis details:   (6.2 points, 5.0 required)
>
> pts rule name  description
> --
>--
>-0.0 SPF_PASS   SPF: sender matches SPF record
> 2.3 BIZ_TLDURI: Contains an URL in the BIZ top-level domain
> 2.5 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence level above 50%
>[cf: 100]
>-2.6 BAYES_00   BODY: Bayesian spam probability is 0 to 1%
>[score: 0.]
> 2.5 RAZOR2_CHECK   Listed in Razor2 (http://razor.sf.net/)
> 1.0 URIBL_SBL  Contains an URL listed in the SBL blocklist
>[URIs: a123s.biz]
> 0.4 URIBL_AB_SURBL Contains an URL listed in the AB SURBL blocklist
>[URIs: a123s.biz]
> 1.5 URIBL_WS_SURBL Contains an URL listed in the WS SURBL blocklist
>[URIs: a123s.biz]
> 3.2 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist
>[URIs: a123s.biz]
> 4.3 URIBL_SC_SURBL Contains an URL listed in the SC SURBL blocklist
>[URIs: a123s.biz]
>-8.8 AWLAWL: From: address is in the auto white-list
>
> --
>--
>
>
>
>Return-Path:
><[EMAIL PROTECTED]>
>Envelope-To: <[EMAIL PROTECTED]>
>X-Spam-Status: SpamAssassin failed demos
>Received: from mail.apache.org ([209.237.227.199] verified)
>  by daypicnic.com (CommuniGate Pro SMTP 4.2.8)
>  with SMTP id 287354 for [EMAIL PROTECTED]; Tue, 15 Mar 2005 19:25:19
>-0800
>Received: (qmail 13383 invoked by uid 500); 16 Mar 2005 03:25:03 -
>Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
>Precedence: bulk
>list-help: 
>list-unsubscribe: 
>List-Post: 
>List-Id: 
>Delivered-To: mailing list users@spamassassin.apache.org
>Received: (qmail 13369 invoked by uid 99); 16 Mar 2005 03:25:03 -
>X-ASF-Spam-Status: No, hits=9.6 required=10.0
>
>   
> tests=BIZ_TLD,FORGED_RCVD_HELO,URIBL_AB_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC_SURBL,URIBL_WS_SURBL
>X-Spam-Check-By: apache.org
>Received-SPF: pass (hermes.apache.org: local policy)
>Received: from cpe-24-221-172-174.ca.sprintbbd.net (HELO cfcl.com)
>(24.221.172.174)
>  by apache.org (qpsmtpd/0.28) with ESMTP; Tue, 15 Mar 2005 19:25:02 -0800
>Received: from [192.168.254.206] ([192.168.254.206])
>   by cfcl.com (8.12.6/8.12.6) with ESMTP id j2G3SktM066434
>   for ; Tue, 15 Mar 2005 19:28:48 -0800 
> (PST)
>   (envelope-from [EMAIL PROTECTED])
>Mime-Version: 1.0
>Message-Id: <[EMAIL PROTECTED]>
>X-Mailer: Eudora for Macintosh!
>Date: Tue, 15 Mar 2005 19:13:04 -0800
>To: users@spamassassin.apache.org
>From: Vicki Brown <[EMAIL PROTECTED]>
>Subject: Blacklisting embedded URLs
>Content-Type: text/plain; charset="us-ascii"
>X-Virus-Checked: Checked
>
>I've been going through a bunch of spam and blacklisting domains. However,
>some of the more frequent offenders are in the body of the message. For
>example, today I found about half a dozen porno spams that contained a
>reference to
>http://www.a123s.biz/...
>
>I can do a body match rule.
>Is there anything else I can do?
>
>Is there something useful that could be added to SpamAssassin for
>blacklisting URLs within the body of a message?
>
>I have something like this for my weblog; I use Movable Type with
>MT-Blacklist. It goes through a spam comment and grabs all the URLs it finds
>and adds those to the internal blacklist. Very handy for Texas Hold-em Poker
>spamments.
>--
>Vicki Brown  ZZZ
>Journeyman Sourceror:  zz  |\ _,,,---,,_ Code, Docs, Process,
>Scripts & Philtres  zz /,`.-'`'-.  ;-;;,_   Perl, WWW, Mac OS X
>http://cfcl.com/vlb   |,4-  ) )-,_. ,\ ( `'-'   SF Bay Area, CA  USA
>___  '---''(_/--'  `-'\_)  ___

-- 
Vicki Brown  ZZZ
Journeyman Sourceror:  zz  |\ _,,,---,,_ Code, Docs, Process,
Scripts & Philtres  zz /,`.-'`'-.  ;-;;,_   Perl, WWW, Mac OS X
http://cfcl.com/vlb 

Re: [SPAM-TAG] Blacklisting embedded URLs

[Jeff Chan]

On Tuesday, March 15, 2005, 7:13:04 PM, Vicki Brown wrote:
> I've been going through a bunch of spam and blacklisting domains. However,
> some of the more frequent offenders are in the body of the message. For
> example, today I found about half a dozen porno spams that contained a
> reference to
> http://www.a123s.biz/...

> I can do a body match rule.
> Is there anything else I can do?

> Is there something useful that could be added to SpamAssassin for
> blacklisting URLs within the body of a message?

Yes, please see URIDNSBL and SURBL:

  
http://spamassassin.apache.org/full/3.0.x/dist/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
  http://www.surbl.org/

which are built into SpamAssassin 3 and enabled by default if
network tests are enabled.

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

Blacklisting embedded URLs

[Vicki Brown]

I've been going through a bunch of spam and blacklisting domains. However,
some of the more frequent offenders are in the body of the message. For
example, today I found about half a dozen porno spams that contained a
reference to
http://www.a123s.biz/...

I can do a body match rule.
Is there anything else I can do?

Is there something useful that could be added to SpamAssassin for
blacklisting URLs within the body of a message?

I have something like this for my weblog; I use Movable Type with
MT-Blacklist. It goes through a spam comment and grabs all the URLs it finds
and adds those to the internal blacklist. Very handy for Texas Hold-em Poker
spamments.
-- 
Vicki Brown  ZZZ
Journeyman Sourceror:  zz  |\ _,,,---,,_ Code, Docs, Process,
Scripts & Philtres  zz /,`.-'`'-.  ;-;;,_   Perl, WWW, Mac OS X
http://cfcl.com/vlb   |,4-  ) )-,_. ,\ ( `'-'   SF Bay Area, CA  USA
___  '---''(_/--'  `-'\_)  ___

Re: Sudden spam to this email address

[Jeff Chan]

On Tuesday, March 15, 2005, 9:02:44 AM, Stuart Johnston wrote:
> SURBLs have them... most of the time... eventually...  Er, yeah.

Just to check, are you using ob.surbl.org and jp.surbl.org
in multi.surbl.org, i.e.:

urirhssub URIBL_JP_SURBL  multi.surbl.org.A   64
body  URIBL_JP_SURBL  eval:check_uridnsbl('URIBL_JP_SURBL')
describe  URIBL_JP_SURBL  Has URI in JP at http://www.surbl.org/lists.html
tflagsURIBL_JP_SURBL  net

score URIBL_JP_SURBL4.0

They tend to catch new domains pretty quickly.

Jeff C.
-- 
Jeff Chan
mailto:[EMAIL PROTECTED]
http://www.surbl.org/

Re: Is there such a test?

[Daryl C. W. O'Shea]

List Mail User wrote:
Unfortunately even the "quotes", while typical, are optional;  I have
lots of examples of both ham and spam without the quotes.  The rule is that
everything from the ':' up to the '<' is the description (and nearly anything
is "legal").  To show examples, here is an example from RFC2821 appendix D.3
IIRC, the only time quotes are required is when there are funky 
characters, such as some punctuation (as in my last name), in the 
description/name.  The majority of MUAs just put the quotes in to avoid 
having to decide if they are required.

Daryl

Re: Is there such a test?

[List Mail User]

>...
>From: "Loren Wilton" <[EMAIL PROTECTED]>
>To: 
>References: <[EMAIL PROTECTED]>
>Subject: Re: Is there such a test?
>Date: Tue, 15 Mar 2005 15:39:32 -0800
>...
>> I have just received spam from  [EMAIL PROTECTED]
>> Is there a test which identifies that the description (Esmeralada
>> Bouchard) bears no resemblance to the given sender's address?
>
>No.  Because there is no possibly way of knowing that [EMAIL PROTECTED] really
>isn't "Johnny P. Spammer".
>
>> Similarly I sometimes receive spam mail to my email address but with a
>> completely unrecognisable description.
>
>This one can be done on an individual basis, sometimes.  It relies on you
>having a standard format for the stuff in quotes.  You have to allow for
>friends that will put the stuff in quotes in a somewhat different form.  But
>it can't be done (at this time, at least) as a standard test.
>
>Loren
>
Unfortunately even the "quotes", while typical, are optional;  I have
lots of examples of both ham and spam without the quotes.  The rule is that
everything from the ':' up to the '<' is the description (and nearly anything
is "legal").  To show examples, here is an example from RFC2821 appendix D.3

"
D.3 Relayed Mail Scenario
...
C: Date: Thu, 21 May 1998 05:33:29 -0700
C: From: John Q. Public <[EMAIL PROTECTED]>
C: Subject:  The Next Meeting of the Board
C: To: [EMAIL PROTECTED]
..."
Which shows that even the '<' and '>' are optional in headers, though
other sections make clear that they are required in commands.

Though I do believe there is a limit on total line length that
must be allowed, there must be at least some MTAs with bugs regarding this
since a sometimes see both viruses and spam which uses > 300 character
strings for names of both accounts and hosts.

 ... Ah, found it - RFC2821 section 4.5.3.1
"...
local-part
  The maximum total length of a user name or other local-part is 64
  characters.

   domain
  The maximum total length of a domain name or number is 255
  characters.

   path
  The maximum total length of a reverse-path or forward-path is 256
  characters (including the punctuation and element separators).

   command line
  The maximum total length of a command line including the command
  word and the  is 512 characters.  SMTP extensions may be
  used to increase this limit.

   reply line
  The maximum total length of a reply line including the reply code
  and the  is 512 characters.  More information may be
  conveyed through multiple-line replies.

   text line
  The maximum total length of a text line including the  is
  1000 characters (not counting the leading dot duplicated for
  transparency).  This number may be increased by the use of SMTP
  Service Extensions.
... (even more length limits)"


Paul Shupak
[EMAIL PROTECTED]

Re: Is this Received header correctly formatted?

[List Mail User]

>From: "Loren Wilton" <[EMAIL PROTECTED]>
>Subject: Is this Received header correctly formatted?
>Date: Tue, 15 Mar 2005 14:36:36 -0800
>...
>
>Received: from ar39.lsanca2-4.16.241.28.lsanca2.elnk.dsl.genuity.net
>([4.16.241.28] helo=watson1)
> by pop-a065d23.pas.sa.earthlink.net with smtp (Exim 3.33 #1)
> id 1DBKRe-Kp-00; Tue, 15 Mar 2005 14:23:22 -0800
>
>1) Is "stmp" in lower case valid, or should it have been STMP?
>2) Is it valid to have the (Exim etc) stuff between 'stmp' and 'id'?
>3) Anything else that may be off the mark?
>
>Thanks,
>Loren
>
Simply, yes.  All identifiers used in any smtp transaction (commands
and account/host/domain identifiers) are supposed to be case insensitive.
Also there is great inconsistency between different MTAs, some convert to
all lower case (like your example) - Old Novell software converted to all
upper case (about 15-20 years ago).  And finally the "with" is sometimes
also either a "by" or "via" (and probably other string values which I haven't
noticed). BTW. DNS is case insensitive by almost the exact same description
of the syntax (and different versions of Bind/named have acted differently
over time, I like that 9.x preserves whatever is in the zone file).

Paul Shupak
[EMAIL PROTECTED]

Re: Is there such a test?

[List Mail User]

>...
>Point taken, but I still think it would be a valid test.
>Like all SpamAssassin tests it should only be one of many indicators.
>In particular all the ones that I receive I would expect to have "Mike" or
>"Michael" in the description of my email address.
>I would also like to be able to pick out those from "Microsoft Support"
>which are not from microsoft.com and other typical phishing mails.
>...

What I think would be good is something to check the recipient
description against the local known proper one.  Example:  today one
spam trapped useda a line of "To: "03/13ss" <[EMAIL PROTECTED]>", which
I can tell immediately could never be valid.

Obviously, this would have to use either a database of something
like a LDAP (or heaven forbid YP or NIS) lookup for its descisions - Still
I see about 8-12% of incoming spam with obvious mismatches of the recipient's
description.

I feel Matt is correct, there is no good way to match the sender's
description, strange account names completely divorced from the description
are *far* too common (some large corporations I have dealt with generate
meaningless random names then have the employees use Firstname.Lastname@
aliases, but the random names "leak" in replies).

Paul Shupak
[EMAIL PROTECTED]

Re: Is there such a test?

[List Mail User]

>
>At 10:00 AM 3/15/2005, Mike Spamassassin wrote:
>>I have just received spam from  [EMAIL PROTECTED]
>>Is there a test which identifies that the description (Esmeralada
>>Bouchard) bears no resemblance to the given sender's address?
>
>No.. It's quite common for normal people to have that.
>
>For example, take a look at Theo Van Dinter's email address. The only 
>letters in common between his name and his email username are t,i, and e. 
>(The username part is "felicity", and the domain has no resemblance to his 
>name either.. "kludge")
>
>And what about Paul Shupak, who uses "List Mail User" as a description, and 
>"track" as a username?
>
>Or these other combinations from this mailing lists (domains removed to 
>reduce harvesting problems)
>
>"Ben Wylie" sasssin@
>  "Kai Schaetzl"   maillists@
>"Matt Yackley"   sare@
>"Matthias Keller" linux@
>
Actually some organizations do filter on things like this, at
times I "CC:" people at Microsoft from the hostmaster@ account here;  It
identifies itself as "Administrative Account", which cause the internal
MS classifier to always mark it as "BULK".  Several friends have complained
to me about it -- MS does seem to pass "List Mail User" through untouched.
Other accounts which I commonly use have ever "worse" identifiers (Once,
they all said "Paul Shupak", but them I cam across some spamware which
cross references the descriptions to tie accounts together, so they were
mostly changed to reflect what the account is used for).  BTW, several
people on this list (but amazingly no others) have privately complained
about the "ugly" descriptions my accounts use (you know who you are).

Paul Shupak
[EMAIL PROTECTED]

P.S. I would guess that "track@" is identical in use to Kai
Schaetzl's maillists@

Re: URI Tests and Japanese Chars

[alan premselaar]

Rose, Bobby wrote:
I have a user that is of Japanese origin and who converses with other
individuals in Japan in his same field of study.  The messages they send
are in Japanese and trip the URI_SBL rule.  These people are in
different .jp domains and I really don't want to get into the
administrative overhead of whitelisting. I don't see anything in the
message bodies that even looks like a URI.  Has anyone else ran into
this?
Bobby Rose
Wayne State University School of Medicine 


Bobby,
 That seems a little strange, especially if there are no URIs in the 
mail.  I live in Japan and have mail servers local and state-side that 
process Japanese email without this problem.

Can you provide more details about your setup/configuration and possibly 
provide a sample email that triggers the rule?

alan