spit out postprocessed "body" and 'rawbody" so we know what they look like

[jidanni]

Mail::SpamAssassin::Conf:
   body SYMBOLIC_TEST_NAME /pattern/modifiers ...
   The 'body' in this case is the textual parts of the message body;
   any non-text MIME parts are stripped, and the message decoded from
   Quoted-Printable or Base-64-encoded format if necessary.  The
   message Subject header is considered part of the body and becomes
   the first paragraph when running the rules.  All HTML tags and line
   breaks will be removed before matching.

It sure would be nice if spamassassin had a flag that would cause it to
spit out this postprocessed body, so we could know exactly what it is
that we are tying to match against!!

Same for rawbody!

Regarding "All HTML tags and line breaks will be removed before matching,"
painstaking trial and error showed me that at least the line breaks are
replaced by a space (%20), not just "removed". Perhaps someone should do
something about that wording.

Also blanks are compressed into just one... So line breaks and non-text
(whitespace) are compressed into just one blank, one finds. OK.

   rawbody SYMBOLIC_TEST_NAME /pattern/modifiers ...
   The 'raw body' of a message is the raw data inside all textual
   parts. The text will be decoded from base64 or quoted-printable
   encoding, but HTML tags and line breaks will still be present.
   Multiline expressions will need to be used to match strings that
   are broken by line breaks.

Here he forgets to mention if the Subject is also considered part of the
body, perhaps assuming that the reader has just read "body" above it...
My tests show that indeed the Subject is part of rawbody. (Yes I could
look at the source, but let's hope he/they/you will improve the man page.)

Anyway, for "body" at least, even a /SUBJECT.*MESSAGE/s does not help me match
whatever it is that supposedly joins the post processed "first paragraph",

   The message Subject header is considered part of the body and
   becomes the first paragraph when running the rules. All HTML tags
   and line breaks will be removed before matching.

with the rest, despite perlre's

   s   Treat string as single line.  That is, change "." to match any
   character whatsoever, even a newline, which normally it would not
   match.

hence I move that spamassassin should have a flag to spit out this
postprocessed body so we can see if it can be matched against in its
entirety in the first place in any way at all!

By the way, further tests I did show that the Subject is indeed totally
disposed of. Even /.MESSAGE/ won't match.

OK, I think I know what is going on. For the spam message:
Subject: Re: Your Photos

Hello, 
as promised your photos  http://...

These match,
body J_PHO /^Hello, as promised your photos http/
body J_PHO /^Re: Your Photos$/
but the user must remember that _these are still run line by line_ so
there is no way to match across that first "paragraph" boundary
mentioned!

Same for rawbody.

Anyway, still need a flag to spit them out.

Why is the STOX stuff all repeated twice?

[jidanni]

Why is the STOX stuff all repeated twice?
X-Spam-Report:
*  0.2 STOX_REPLY_TYPE STOX_REPLY_TYPE <--- see, twice on the same line!
*  0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail 
provider
*  (m2243.m8715[at]msa.hinet.net)
* -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay 
domain
*  0.1 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends 
in
*  digit (m2243.m8715[at]msa.hinet.net)
* -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, 
no
*  trust
*  [168.95.4.112 listed in list.dnswl.org]
*  1.9 STOX_REPLY_TYPE_WITHOUT_QUOTES STOX_REPLY_TYPE_WITHOUT_QUOTES 
<--twice!

Can't locate object method "check_body_length" via package "Mail::SpamAssassin::PerMsgStatus"

[jidanni]

Help. sa-update is jammed.
What should I do?

rules: failed to run __KAM_BODY_LENGTH_LT_256 test, skipping:
(Can't locate object method "check_body_length" via package 
"Mail::SpamAssassin::PerMsgStatus" at (eval 1114) line 419.

etc. etc.

X-Spam-Checker-Version doesn't reflect what sa-update we are at

[jidanni]

Isn't it bad that X-Spam-Checker-Version doesn't report what sa-update
we are up to so far, and that there is no additional other variable that
we can toggle on in reports to do that.

introducing body J_MAILBOX_FULL

[jidanni]

body J_MAILBOX_FULL /^Your? ((web|E-?) ?mail|mailbox) .*(is|has) 
.*(exceed|over)/i

Got to update it every day to stop those bast*rds.

ERROR: LINT FAILED, suppressing output: rules/70_sandbox.cf

[jidanni]

Seen upon SVN update:

lint: config: failed to parse line, skipping, in "rules/70_sandbox.cf": 
mimeheader  __KAM_BLOCK_UTF7_2  Content-Type =~ 
/charset=(?:unicode-\d+-\d+-)?utf-7/i at build/mkrules line 255.

ERROR: LINT FAILED, suppressing output: rules/70_sandbox.cf

Re: sa-update channel list

[jidanni]

No wonder, the version I had
Jan 15 09:39:29.620 [6248] dbg: dns: 0.4.3.updates.spamassassin.org => 3.4 
1230922, parsed as 3
Jan 15 09:39:29.620 [6248] dbg: channel: current version is 1195375, new 
version is 3, skipping channel
Now after updating from SVN it gives
Jan 15 09:45:34.588 [15283] dbg: dns: 0.4.3.updates.spamassassin.org => 
1230922, parsed as 1230922
Jan 15 09:45:34.588 [15283] dbg: channel: current version is 1230922, new 
version is 1230922, skipping channel
OK, thanks.

Re: sa-update channel list

[jidanni]

> "MS" == Michael Scheidell  writes:
MS> On 1/11/12 9:35 PM, jida...@jidanni.org wrote:

MS> #1 priority:  keep your version of sa updated
MS> Hmmm, taking a look at it, I find the last update was about 2011/10/24.
MS> Too bad sa-update -D doesn't spit out the date.

MS> I meant your version of spamassassin.

MS> 3.3.2 was updated yesterday.

MS> if you don't have the current version of spamassassin then your sa-update 
channel will be older.  (case in point)

All I know is I'm using
Jan 12 11:07:09.394 [21138] dbg: generic: SpamAssassin version 3.4.0-r1102360
which is obviously newer than 3.3.2.

Re: sa-update channel list

[jidanni]

> "MS" == Michael Scheidell  writes:
MS> #1 priority:  keep your version of sa updated
Hmmm, taking a look at it, I find the last update was about 2011/10/24.
Too bad sa-update -D doesn't spit out the date.

spam from a .con ?!

[jidanni]

> From:... <...@1004.con>
".con"? sounds like a con-job.
Ha, I'm not falling for that again!

"Your mailbox has exceeded..."

[jidanni]

Sure a lot of "Your mailbox has exceeded" spam these days. I'll use

body J_MAILBOX_FULL /^Your mailbox has exceeded/
score J_MAILBOX_FULL ...

myself for now.

Re: linkedin messages

[jidanni]

See also https://bugzilla.wikimedia.org/show_bug.cgi?id=29855

Re: linkedin messages

[jidanni]

And even if you are a card carrying member of LinkedIn,
header J_CANT_STOP Subject =~ /^LinkedIn Network Updates/
score J_CANT_STOP 222
is needed, as even LinkedIn staff are unable to stop sending them.

A spam a day keeps the network OK

[jidanni]

> "j" == jdow   writes:

j> I seldom if ever opt out if existing spam. That's what my spam filter
j> is for. It simply leads to nicely segregated spam I can deal with in a
j> trice. If I opt out my spam filter feels lonely. It LIKES its food. It
j> gets better with feeding.

Me too. I OPT-IN to spam. No spam == Link Down.
♫ A spam a day keeps the network OK. ♬
My God, I invented a famous sentence!
You heard it here first. Proof:
http://www.google.com/search?q="A+spam+a+day+keeps+the+network+OK";
And indeed, a day with no spam is a day to dread, as it means your
network link is surely busted. No better testing system for the average Joe!

debugging UNWANTED_LANGUAGE_BODY

[jidanni]

Gentlemen, it turns out it is very hard to debug UNWANTED_LANGUAGE_BODY.

They forgot to put debugging features in the code, apparently TextCat.pm.

Lately spamassassin has decided that several of my mails contain
UNWANTED_LANGUAGE_BODY
X-Spam-Languages: ja.shift-jis
despite that being hogwash.

Well I tried to debug using
spamassassin -D
And got as far as
dbg: rules: ran eval rule UNWANTED_LANGUAGE_BODY ==> got hit (1)

Further grepping led to TextCat.pm.

Now we ask, exactly what caused TextCat.pm decide my message was Japanese?

Do I have to post it here, full of its personal information, or is there
some additional debugging flag I can set?

Re: How to get a fresh start in messy old setup

[jidanni]

All I know is I first run mail through procmail to filter out the big
items before they get to spamassassin.

Never enabled Bayes and don't intend to.

Hmm, I seem to have described it in http://jidanni.org/comp/spam/spamdealer.html

Re: My messages never got to this list, even though I'm subscribed etc.

[jidanni]

>>>>> "j" == jidanni   writes:

j> Well let's see if at least this reply gets through without needing to
j> resort to Nabble.

OK, it got through. ANOTHER_JIDANNI_STUPID_POST did not fire ☺

Re: My messages never got to this list, even though I'm subscribed etc.

[jidanni]

Well let's see if at least this reply gets through without needing to
resort to Nabble.

My messages never got to this list, even though I'm subscribed etc.

[jidanni]

My recent messages never got to this list, even though I'm subscribed etc.
Could it be it thinks they are ... spam?
-- 
View this message in context: 
http://old.nabble.com/My-messages-never-got-to-this-list%2C-even-though-I%27m-subscribed-etc.-tp30962942p30962942.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

whitelist_return_path

[jidanni]

Gentlemen, I wish there was a
whitelist_from *...@facebookmail.com
rule that would use the Return-Path field,

$ egrep '^(From|Return-Path):' a b
a:Return-Path: 
a:From: Facebook 
b:Return-Path: 
b:From: Facebook 

as that is much more reliable.

I see there are other whitelist_ rules, but none that can use what is
already staring one in the face in the headers, and does not need to go
on the network to work.

SUBJ_ALL_CAPS vs. RE:

[jidanni]

SUBJ_ALL_CAPS to you, but not to me:
Subject: RE: 柯小柯
Can't you give the RE: etc. a break?
And also why is the Chinese considered CAPS?
$ unicode P p 柯|grep Category
Category: Lu (Letter, Uppercase)
Category: Ll (Letter, Lowercase)
Category: Lo (Letter, Other)
Sure I can customize this rule for just me, but I think it should be
fixed for everybody.

SpamAssassin Bugzilla hacked 2010.4.4, change your password

[jidanni]

https://issues.apache.org/SpamAssassin/ says "As a result of a security
breach on 4th April 2010, the Apache Infrastructure Team recommends that
all SpamAssassin Bugzilla users change their passwords as a
precautionary measure. Please see the Infrastructure Blog for further
information."

Re: FREEMAIL_ENVFROM_END_DIGIT 2.2 anti-Gmail

[jidanni]

All I know is my friend send me a mail from her FROM_END_DIGIT account
there at Gmail, and there was
  DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
  d=gmail.com; s=gamma;...
  DomainKey-Signature: a=rsa-sha1; c=nofws;...
in the headers too even. And still it got slapped with
FREEMAIL_ENVFROM_END_DIGIT 2.2.
P.S., thanks, I added a note to
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6394 to see this
thread.

FREEMAIL_ENVFROM_END_DIGIT 2.2 anti-Gmail

[jidanni]

Well Gosh,

*  2.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends 
in
*  digit (mb2365[at]gmail.com)

I swear when I recently helped a loved one apply for a Gmail account,
Gmail offered names like "vippenheimer321", "snordsberd123",
nipplovitz246", as the non suffixed versions are usually all taken up
already.

So you are penalizing users for doing what Gmail told them to.

And as Gmail will "do no evil", therefore the evil is being done by
guess who...

Anyway, It's
score FREEMAIL_ENVFROM_END_DIGIT 0.1
for me from now on.

FREEMAIL_REPLY From and body contain different freemails WELL GOSH

[jidanni]

Man you guys blew it.
*  2.5 FREEMAIL_REPLY From and body contain different freemails
All he did was quote my CC.
It is not like he forged anything.

P.S., https://issues.apache.org/SpamAssassin/ doesn't mention when it
itself will someday come back online after maintenance.

FreeMail: RULE (FREEMAIL_FROM) check_freemail_from
FreeMail: all from-addresses: x...@yahoo.com.tw
FreeMail: HIT! x...@yahoo.com.tw is freemail
rules: ran eval rule FREEMAIL_FROM ==> got hit (1)
FreeMail: RULE (__freemail_reply) check_freemail_replyto
FreeMail: From address: x...@yahoo.com.tw
FreeMail: all body freemails: x...@yahoo.com.tw, erliu0...@yahoo.com.tw
FreeMail: comparing x...@yahoo.com.tw to body freemails
FreeMail: HIT! x...@yahoo.com.tw and erliu0...@yahoo.com.tw are 
different freemails

!From: XX 陳 
!Subject: Re: About HF SSB Freq.
!To: jida...@jidanni.org
!Date: Sun, 11 Apr 2010 12:51:21 +0800 (CST)
!
!Thank you very much.
!
!--- 10/4/11 (日)，jida...@jidanni.org  寫道：
!
!寄件者: jida...@jidanni.org 
!主旨: Re: About HF SSB Freq.
!收件者: x...@yahoo.com.tw
!副本: erliu0...@yahoo.com.tw
!日期: 2010年4月11日,日,下午12:44
!
!惟如 
http://radioscanningtw.jidanni.org/index.php?title=%E9%A3%9B%E8%88%AA%E7%AE%A1%E7%90%86
!都盡量不再重複其他站。

SA team lambasted in RISKS Digest

[jidanni]

http://catless.ncl.ac.uk/Risks/25.94.html#subj11
I suggest someone send RISKS a clarification if indeed the issue is resolved.

Re: Why does svn update pull in Mail-SpamAssassin-3.4.0.tar.gz?

[jidanni]

Maybe I accidentally did 'make dist'. OK, never mind.

Re: spamassassin script is v3.003000, but using modules v3.004000

[jidanni]

MM> The usual procedure is:
MM>   perl Makefile.PL; make; make test; make install
Ah, that now works only once! Now only
  set -e; perl Makefile.PL; make clean; perl Makefile.PL; make; make install
will work each time, even if a few months pass between one's upgrades.
Hmmm, they might want to document that.
I chucked your "make test" as it was taking too long.

Re: spamassassin script is v3.003000, but using modules v3.004000

[jidanni]

Maybe they changed something. In the past
perl Makefile.PL PREFIX=$HOME/.spamassassin-tree
also took care of where bin/spamassassin went. Now it seems left behind,
due to this suspicious commented out code?

# needs to be added to MY::install if used
#bin__install: $(INST_SCRIPT)/sa-filter
## $(RM_F) $(B_SCRIPTDIR)/spamassassin
## $(SYMLINK) $(INST_SCRIPT)/sa-filter $(B_SCRIPTDIR)/spamassassin

Why does svn update pull in Mail-SpamAssassin-3.4.0.tar.gz?

[jidanni]

Why does svn update pull in Mail-SpamAssassin-3.4.0.tar.gz?
That's not how things work with Mediawiki.

spamassassin script is v3.003000, but using modules v3.004000

[jidanni]

Help, I dared to update from SVN, and now spamassassin refuses to run:
$ svn update
$ make install
$ sa-update --install http://daryl.dostech.ca/sa-update/asf/>
$ spamassassin
spamassassin: spamassassin script is v3.003000, but using modules v3.004000

sa-update fails: daryl.dostech...404

[jidanni]

$ sa-update
http: GET http://daryl.dostech.ca/sa-update/asf/909775.tar.gz request failed: 
404 Not Found

Re: [Sare-users] painting everybody in Taiwan with the same brush

[jidanni]

Dear sare-users Adam Katz tried to post these to your list. Please read
http://article.gmane.org/gmane.mail.spam.spamassassin.general/126545
http://article.gmane.org/gmane.mail.spam.spamassassin.general/126547

However, as in
http://article.gmane.org/gmane.mail.spam.spamassassin.general/126330
> "MN" == Matija Nalis  writes:

MN> Despite they seemed quite dead for the last several years, at least
MN> one of the SARE Ninjas (or their associate with privileges enough)
MN> is not only alive but had heard your plea, and tried to help you on
MN> 28-Jan-2010 by putting:

MN> score SARE_RECV_SPAM_DOMN0b0.0

MN> it the 70_sare_header1.cf ruleset.

MN> However, that probably would not work too good, because:

MN> - they did not seem to update 70_sare_header1.cf.sig digital signature,
MN>   so automatic update would probably fail even if someone manged to pulled 
it.

MN> - the "Modified" and the "#@@#" history on the top of the Ruleset are not
MN>   updated (they should be)

MN> - the autoupdater (maybe because of previous error(s) ?) does not seem to 
pull
MN>   that change - my sa-update says:

MN> [1016] dbg: channel: attempting channel 
70_sare_header1.cf.sare.sa-update.dostech.net
MN> [1016] dbg: channel: update directory 
/var/lib/spamassassin/3.002005/70_sare_header1_cf_sare_sa-update_dostech_net
MN> [1016] dbg: channel: channel cf file 
/var/lib/spamassassin/3.002005/70_sare_header1_cf_sare_sa-update_dostech_net.cf
MN> [1016] dbg: channel: channel pre file 
/var/lib/spamassassin/3.002005/70_sare_header1_cf_sare_sa-update_dostech_net.pre
MN> [1016] dbg: channel: metadata version = 200605212000
MN> [1016] dbg: dns: 5.2.3.70_sare_header1.cf.sare.sa-update.dostech.net => 
200605212000, parsed as 200605212000
MN> [1016] dbg: channel: current version is 200605212000, new version is 
200605212000, skipping channel

MN> Hopefully someone can fix that issues also

Re: [Sare-users] painting everybody in Taiwan with the same brush

[jidanni]

Anyway, what you are doing here is penalizing all users of that
company's copper wires. No amount of monopoly breakup legislation will
do any good if you penalize based on the wrong part of the physical
infrastructure.
http://en.wikipedia.org/wiki/Common_carrier
http://en.wikipedia.org/wiki/Network_neutrality

> The rule is buggy -- it's looking at all the
> received headers, even the ones before the relay.

Therefore you want to score on who is handing their SMTP etc. Not on who
provides the copper wires to their house... rating on that part of the
infrastructure will spoil your results.

Re: [Sare-users] painting everybody in Taiwan with the same brush

[jidanni]

Long ago, I tried mailing directly direct-to-mx style, but that of
course didn't work, e.g., http://www.spamhaus.org/pbl/query/PBL109625
So only 5% of my mail got through.

So then I tried mailing through The ISP Here, Hinet.Net's SMTP server,
but of course Hinet.Net has a bad name. So only 50% of my mail got through.

So, upon people like you guy's recommendation, I (asked my mom to buy)
me a dreamhost.com account.

However I can't shake off the Original Sin of Being in Taiwan. All
people with Taiwan Colored Skin will have points deducted, no matter
what. We use the Telephone Company's ISP.

> "J" == Jailer-Daemon   writes:
J> On Wed, Jan 27, 2010 at 11:30:28AM -0500,  wrote:
>> 
>> He's using an SMTP relay

J> He is, but it isn't a Hinet relay. At least not in the URL he gave.
J> It should be possible to relay out from your own ISP and not score
J> anything on SARE rules, without having to pay extra for "clean" SMTP
J> relaying (which is what seems to be happening here).

Now you guys are saying I should go back to using Hinet.Net's SMTP, even
though my mom has already paid a 5 year contract for me at Dreamhost.

>> The rule is buggy -- it's looking at all the 
>> received headers, even the ones before the relay.

Yes, and what may seem like a mere 1.6 points is causing me to have to
request the whole spam threshold of that mailing list
http://article.gmane.org/gmane.linux.debian.devel.eeepc/2850/raw be
lowered just for me, just because my mail is being tagged with a stupid
looking "mail Made in Taiwan, penalty 1.666 points" that I can't do
anything about, thanks to you guys and no one else.

Also, I wonder why lots of my mail doesn't seem to get through to
people... and no, I don't want to bother them with various test
messages. Perhaps it is all again due to your sloppy rules?

Actually, I could figure out some underhanded methods to get around
being detected as living in a Undesirable Country, but if ever detected,
I would surely get penalized even more points.

Re: [Sare-users] painting everybody in Taiwan with the same brush

[jidanni]

So what should a Taiwan user (Taiwan~=Hinet)
   HINET: Control of approx 8,476,149 IP addresses 
http://www.fixedorbit.com/AS/3/AS3462.htm
user do. Buy a SMTP account with a US Company?

But that's what I did, as you see from
http://article.gmane.org/gmane.linux.debian.devel.eeepc/2850/raw
headers.

But that's still not good enough.

So what next? Need a ssh tunnel to /usr/lib/sendmail or something on a
US machine to eradicate all traces of Taiwan?

> "KS" == Kai Schaetzl  writes:
KS> The point of discussion was "Email passed through apparent spammer domain"
KS> because of *origination* at a dynamic hinet address. I personally think
KS> this rule is misguided and maybe isn't even doing what it was intended to 
do.

painting everybody in Taiwan with the same brush

[jidanni]

Fellows, I have the highest spam score vs. all my buddies:
http://article.gmane.org/gmane.linux.debian.devel.eeepc/2850/raw

It's all because
http://www.rulesemporium.com/rules/70_sare_header1.cf
headerSARE_RECV_SPAM_DOMN0bReceived =~ 
/\bdynamic.hinet\.(?:com|net|org|info)/
describe  SARE_RECV_SPAM_DOMN0bEmail passed through apparent spammer domain
score SARE_RECV_SPAM_DOMN0b1.666

So how is anybody living in Taiwan supposed to mail things with honor?
They can't get another country, nor cause a revolution. You just paint
them all with one brush. What if you painted everybody in your home
country with one brush until they were supposed to overthrew the
telephone company or whatever?

Re: How to tell if sa-update is actually running

[jidanni]

> "MG" == Martin Gregorie  writes:
MG> I run this script as a weekly cron job:
I just use one line in my crontab:
33 2 * * * sa-update
If something goes wrong I'll get a mail with the errors. Else nothing
will interrupt my leisure yacht cruise.

lint check of update failed, channel failed

[jidanni]

$ sa-update
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": 
clear_originating_ip_headers
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-Yahoo-Post-IP X-Originating-IP X-Apparently-From
config: failed to parse line, skipping, in 
"/tmp/.spamassassin5560GP7SGbtmp/10_default_prefs.cf": originating_ip_headers 
X-SenderIP
channel: lint check of update failed, channel failed

sa-update perhaps should exit 0 if all is good

[jidanni]

Regarding sa-update,
EXIT CODES
   An exit code of 0 means an update was available, and was
   downloaded and installed successfully if --checkonly was
   not specified.

   An exit code of 1 means no fresh updates were available.

I would make this:
  0 means you are all up to date. The connection was made, and if
  something was needed it was download. Anyways you are now all up to
  date.

This would then not stop Makefiles that call it, nor would one need to
do case $? in 0|1)...; esac.

Re: sa-update 403 forbidden

[jidanni]

OK, thanks. I'd put some contact info on top of http://daryl.dostech.ca/,
above "This blog is currently in a static state pending an upgrade
of WordPress", in case something breaks next time.

sa-update 403 forbidden

[jidanni]

Sometimes sa-update works, sometimes one gets
http: GET http://daryl.dostech.ca/sa-update/asf/891585.tar.gz request failed: 
403 Forbidden:
You don't have permission to access /sa-update/asf/891585.tar.gz on this server.
Apache/2.2.3 (Fedora) Server at daryl.dostech.ca Port 80

I recommend that http://daryl.dostech.ca/ have an email address for
contact shown, so I can tell him directly the next time it happens.

streamsend

[jidanni]

http://www.streamsend.com/returnpath_safelist.htm :
"SpamAssassin - Safe List rules help give green light to email"
Not my email though.
header J_STREAMSEND X-Mailer=~/StreamSend/i
 score J_STREAMSEND 10

how to Download From Svn without getting swamped?

[jidanni]

So I thought I would follow
http://wiki.apache.org/spamassassin/DownloadFromSvn instead of getting
snapshots.

So I do
$ svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk 
spamassassin-trunk
and end up many more megabytes of files than one gets with snapshots,
and no, I'm not talking about .svn/* files. I'm talking about files that
you packagers exclude from snapshots.

So how can I download from SVN and get approximately the same number of
files that are in code snapshots? Thanks.

Re: sa-update: rules: failed to run __RCVD_IN_2WEEKS

[jidanni]

> "MM" == Mark Martinec  writes:
MM> Did the rule __RCVD_IN_2WEEKS end up in the current sa-update set?
MM> If so, you can either remove the rule, or install the SA from ... SVN.
If I http://wiki.apache.org/spamassassin/DownloadFromSvn often, does that
mean I don't need to use sa-update anymore, as I will already be up to date?

sa-update: rules: failed to run __RCVD_IN_2WEEKS

[jidanni]

Help, sa-update gives:

rules: failed to run __RCVD_IN_2WEEKS test, skipping:
(Can't locate object method "received_within_months" via package 
"Mail::SpamAssassin::PerMsgStatus" at (eval 755) line 19.
)
channel: lint check of update failed, channel failed

Re: whitelist_from questions

[jidanni]

Actually there should be one or two more whitelists, so one can e.g., score
-100 one's friends
-10  one's schools
-1   one's country

defense against long header lines in reports

[jidanni]

Sometimes first time spammers end up stuffing the entire body of their
message into the Subject: etc. header. I don't see anything on man
Mail::SpamAssassin::Conf to truncate headers after a reasonable length
(but it would also chop multibyte Unicode, or at least RFC 2047
strings, probably). (However I do see here in 3.30:)
POD ERRORS
   Hey! The above document had some coding errors, which are explained 
below:

   Around line 274:
   You forgot a '=back' before '=head2'

Re: ANNOUNCE: Apache SpamAssassin 3.3.0-alpha1 available

[jidanni]

I'd establish a
http://people.apache.org/~jm/devel/README.txt
warning people which one of
http://people.apache.org/~jm/devel/Mail-SpamAssassin-3.3.0-alpha1.tar.bz2
http://people.apache.org/~jm/devel/Mail-SpamAssassin-3.3.0.tar.bz2
they really want. I guessed the former.

Re: Content Preview should use that Charset too

[jidanni]

Never mind. I'll just use report_safe 0.

Re: Content Preview should use that Charset too

[jidanni]

And what does
>normalize_charset ( 0 | 1) (default: 0) Whether to detect
>   character sets and normalize message content to Unicode.
Actually do? It sounds like it will convert all your mail to Unicode,
but it seems not to. Or maybe it will just convert the
Content Preview, as I mentioned in my previous message. But it turns out
that gets converted anyway, sometimes, without normalize_charset 1, nor
with Encode::Detect installed!

Content Preview should use that Charset too

[jidanni]

Gentlemen, why oh why can't the Charset of the Content Preview of the
Report be set to the same as where the Report got it from? E.g.,

$ grep ^Content message
Content-Type: multipart/mixed; boundary="--=_4A446828.7FD08E5A"
Content-Type: text/plain; charset=iso-8859-1 <= Why can't this be big5
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Content preview: ... <=== as here you deposit a few lines of genuine big5
Content analysis details:   (3.1 points, 1.9 required)
Content-Type: message/rfc822; x-spam-type=original
Content-Description: original message before SpamAssassin
Content-Disposition: attachment
Content-Transfer-Encoding: 8bit
Content-Type: multipart/alternative; boundary=0016364c673f034480046d3a4e67
Content-Type: text/plain; charset=Big5 <=== see, you know it is big5
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=Big5 <=== no denying the facts
Content-Transfer-Encoding: quoted-printable

Sure, you will say

"See
report_charset CHARSET(default: unset)
   Set the MIME Content-Type charset used for the text/plain
   report which is attached to spam mail messages.

Holmes. Next."

Well, I would just like to point out
1. the above "(default: unset)" is a lie:
   Content-Type: text/plain; charset=iso-8859-1
2. I don't want to hardwire it, I just want SpamAssassin (3.2.5, or
should I upgrade to 3.30 for this?) to use the same Charset from where
it got those Chars. There is no guessing involved for SpamAssassin, as
the headers where it got the chars mention the Charset.

Sure, you will now say

"See

   normalize_charset ( 0 | 1) (default: 0) Whether to detect
   character sets and normalize message content to Unicode.
   Requires the Encode::Detect module, HTML::Parser version 3.46
   or later, and Perl 5.8.5 or later.

Holmes, Next (as in Next patient waiting in line)."

But I don't want to necessarily use UTF-8 or whatever, I just want you
to use the Charset of where you got the preview. The boilerplate
"Spam detection software, running on the system ..." is all ASCII, so
should work fine with most Charsets... or perhaps the Content Preview
should be isolated into its own MIME section.

Re: Rule to detect same address in sender and receiver

[jidanni]

Ah ha, you can use something like
header FROM_SAME_AS_TO ALL=~/\nFrom: ([^\n]+)\n.*To: \1/sm
> add spf to your domain
But see e.g., http://david.woodhou.se/why-not-spf.html

SUBJ_ALL_CAPS anti-Asian

[jidanni]

How unfair: this triggered SUBJ_ALL_CAPS:
Subject: RE: 請教無線電掃瞄
A little capital E and it gets slammed with SUBJ_ALL_CAPS, no matter how
much Chinese follows.
(source: Subject: =?utf-8?B?UkU6IOiri+aVmeeEoee3mumbu+aOg+eehA==?=)

Re: [Trac] Re: filter own TRAC bug mail

[jidanni]

I see, http://trac-hacks.org/wiki/NeverNotifyUpdaterPlugin is
something one must ask each trac maintainer to install, at each trac
where we have reported bugs.

I will rather just filter the mail myself with my SpamAssassin stanza
version 2:
header __J_TRAC_COMMENT X-Trac-Ticket-URL =~/\#comment/
rawbody __J_TRAC_MY_COMMENT /\+-+\n(Changes|Comment) \(by jidanni\):\n/m
meta   J_TRAC_OTHERS __J_TRAC_COMMENT && ! __J_TRAC_MY_COMMENT
score  J_TRAC_OTHERS -7

Missing To: headers at the trac I used upped SpamAssassin's score so I
didn't need up their threshold to filter.

I hope one day vanilla Trac will offer more than all or none email choices.

filter own TRAC bug mail

[jidanni]

Gentlemen, the TRAC bug tracker is not as smart as bugzilla.

It insists on sending one acknowledgements even for one's own actions.

It was me doing the clicking, no acknowledgement needed.

But if it was someone else changing/commenting on one of my bugs, well
yes, I want mail about it.

So to get mail on others' actions on our bugs, but not our own
actions, we must make fancy SpamAssassin rules:
header __J_TRAC_COMMENT X-Trac-Ticket-URL =~/\#comment/
rawbody __J_TRAC_MY_COMMENT /\+-+\nChanges \(by jidanni\):\n/
meta   J_TRAC_OTHERS __J_TRAC_COMMENT && !__J_TRAC_MY_COMMENT
score  J_TRAC_OTHERS -7

Fortunately TRAC breaks other rules, like missing To: headers, so the
above is all I need, at least for the WordPress TRAC.

access environment variables

[jidanni]

I don't suppose there is a way to use environment variables,
header NO_ME ToCc !~ /${USER}/i
without perhaps a preprocessor to turn them into
header NO_ME ToCc !~ /jidanni/i
Or use procmail's *$...$LOGNAME instead of SpamAssassin here.

SourceForge rules

[jidanni]

Here's what I use to only know about other's SourceForge changes, not my own:
header J_SOURCEFORGE exists:X-SourceForge-Tracker-itemupdate-username
 score J_SOURCEFORGE -5
header J_SOURCEFORGE_ME 
X-SourceForge-Tracker-itemupdate-username=~/^(jidanni|Item Submitter)$/
 score J_SOURCEFORGE_ME 10

Re: 'sought' rules take three times longer to run

[jidanni]

MU> maybe using spamd and spamc is hat you want...
But that would be a http://wiki.dreamhost.com/Persistent_Processes

Re: 'sought' rules take three times longer to run

[jidanni]

I took a look at Mail::SpamAssassin::Plugin::Shortcircuit, but what I
really want to do is "if it is ham, run it through the expensive
'sought' extra tests, to see if it really is ham."

I.e., if the end result is below required_score, continue on into the
"sought" tests.

Probably the only way to do that is via .procmailrc

:0fw
|spamassassin --cf 'Do not run sought-rules'
:0
*^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
Mail/almost-certainly-spam/
:0
*^X-Spam-Status: Yes
Mail/probably-spam/
:0fw
|spamassassin
:0
*^X-Spam-Status: Yes
Mail/probably-spam/

The only problem (besides 'house of cards') is that there is no way to
do --cf 'Do not run sought-rules' on the first spamassassin run. One
must instead tamper with the files sa-update --channel
sought.rules.yerp.org gets, removing sought_rules_yerp_org.cf and
putting its contents into a --cf string on the second spamassassin
run, thus complicating future sa-update runs.

'sought' rules take three times longer to run

[jidanni]

OK, I have just finished
$ sa-update -D --no-gpg --channel sought.rules.yerp.org
And would just like to warn other users that 'sought' rules take three
times longer:
$ time spamassassin --local -t < a_typical_spam_message > /dev/null
real0m14.081s
user0m13.489s
sys 0m0.588s

Up from
real0m4.954s
user0m4.836s
sys 0m0.112s

> you can sa-compile them, perhaps

Well, just figuring out sa-update -D --no-gpg --channel sought.rules.yerp.org
was hard enough.

Re: sa-update damages existing SA installation

[jidanni]

HK> If SVN does not ring a bell,

Oh, you mean like the example on
http://svn.savannah.gnu.org/viewvc/trunk/grub2/docs/grub.texi?root=grub&view=log

$ svn co svn://svn.sv.gnu.org/grub/trunk/grub2/docs/grub.texi
svn: URL 'svn://svn.sv.gnu.org/grub/trunk/grub2/docs/grub.texi' refers to a 
file, not a directory

HK> then better stick with the official version. :)

what's the big risk with sa-update --nogpg?

[jidanni]

So what's the worst thing that could happen to me with sa-update
--nogpg? Just a little more spam getting through? Ha!

> If you would just follow instructions, you wouldn't need --nogpg

Yes, well, let's just say things didn't work out, and we want to use
--nogpg just for that risky feel. Like smoking cigarettes or
something. So what's the worst thing that could happen, our mailbox
getting cancer?

Re: "I have a new email address!" spam

[jidanni]

m> those I looked at triggered JM_SOUGHT_FRAUD_1. so make sure you use the
m> sought channel in your sa-update.

OK, I did all the research to find what it might be that you were
talking about.

I completed the steps (some of them exposing how sa-update fails to
catch a bumbling user):
$ wget http://yerp.org/rules/GPG.KEY
$ sa-update -D --import GPG.KEY
$ sa-update -D sought.rules.yerp.org
$ sa-update -D --no-gpg sought.rules.yerp.org
$ sa-update -D --channel sought.rules.yerp.org

And at long last, finally, of course all was for naught:

403 Forbidden  Forbidden You don't
have permission to access /rules/stage/320729494.tar.gz on this
server.

Re: sought rules updates

[jidanni]

m> http://www.netoyen.net/sa/sa-update.sh.txt
m> http://www.netoyen.net/sa/channel.conf
They give 403 Forbidden.

Re: sa-update damages existing SA installation

[jidanni]

>> DNS seems to have been reporting 709395 as current for about eight weeks

HK> If you want more up-to-date protection, use latest SVN (3.3). That's where
HK> the development happens. It's been working fine here for a long time.

All I know is I have
$ crontab -l
33 3 * * * PATH=$HOME/bin:$PATH sa-update
What should I now put there instead?

"I have a new email address!" spam

[jidanni]

Gentlemen, does one just keep on adding more regexps for each new language
edition of this the spammer makes? Any better way for this particular spam?

body J_NEW_ADDRESS 
/\xA7\xDA\xA6\xB3\xB7s\xAA\xBA\xB9q\xB6l\xA6a\xA7}\xA1I\xA7A\xB2{\xA5i\xB9q\xB6l\xB5\xB9\xA7\xDA|I
 have a new email address!You can now/

do TEST2 only if TEST1 was positive

[jidanni]

Sure we can do
  meta META0 TEST1 && TEST2
but say TEST2 is expensive, and we only want it to be run if TEST1 is
positive. I suppose SpamAssassin's whole train of thought has no ifs
ands or buts, other than a method of quitting early, but that not what
I want to do. I suppose branching is only possible on the procmail level.

I hate one certain language

[jidanni]

Never mind the below, I solved it with
header J_CHSET3 
Subject:raw=~/\s=\?(windows-(125[0125]|874)|koi8-r|GB2312|iso-8859-[28])\?/i

The below:
Here we go again.
How can I filter on
X-Spam-Languages: zh.gb2312
run it through spamassassin a second time?
Use _LANGUAGES_ somehow in a regexp?
Of course the LANGUAGE OPTIONS part of the man page just begs the
question of how to mark one as bad, instead of good. But never mind.
That is a never ending argument that I have forgotten.
Note I love other zh.*, just not zh.gb2312.
Hmm, I see I already do
ifplugin Mail::SpamAssassin::Plugin::TextCat
 # ok_languages en zh.big5
 # http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5697
 ok_languages en zh
 add_header all Languages _LANGUAGES_
 score UNWANTED_LANGUAGE_BODY 5
endif
ok_locales en zh

OK, solved as at top. Thanks. Bye.

overly harsh against Message only has text/html MIME parts

[jidanni]

Gentlemen, it seems spamassassin used full military justice here:

 0.0 HTML_MESSAGE   BODY: HTML included in message
 2.5 MPART_ALT_DIFF BODY: HTML and text parts are different
well of course, because
 2.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
OK, then
 1.7 HTML_MIME_NO_HTML_TAG  HTML-only message, but there is no HTML tag
I dunno, maybe they thought they already said I'm text/html.
 0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME parts
Yes.

Gee, all the message did bad was

|Content-Type:  multipart/alternative;
|   boundary="=_Part_742221_601951471.1223963124030"
|X-Mailer: HiNet WebMail
|
|--=_Part_742221_601951471.1223963124030
|Content-Type: text/html;charset=utf-8
|Content-Transfer-Encoding: quoted-printable
|
|...
|--=_Part_742221_601951471.1223963124030--

Re: user_prefs brilliant indenting mode invented by me

[jidanni]

Oh no oh no, man Mail::SpamAssassin::Conf says
 Whitespace in the files is not significant, but please note that
 starting a line with whitespace is deprecated, as we reserve its use
 for multi-line rule definitions, at some point in the future.
OK, sorry. I regret my previous message.
Wait. Wouldn't it be unfortunate if SpamAssassin goes the python route
where whitespace counts in syntax, vs. perl where one can use the
trusty semicolon and {}, especially as SpamAssassin is perl based...
(Anyway, I hate python as it doesn't fit into wrapped one-liners in
Makefiles, etc. They expect everybody is editing on a terminal with TABs, etc.)

user_prefs brilliant indenting mode invented by me

[jidanni]

Gentlemen, I save wads of space in my user_prefs with

header J_YAHOO_CAL X-Yahoo-Newman-Property=~/calendar-invite/
 score J_YAHOO_CAL 11
header J_MEDIAWIKI_MAILER X-Mailer=~/MediaWiki mailer/
 score J_MEDIAWIKI_MAILER -10

instead of the traditional

header J_YAHOO_CAL X-Yahoo-Newman-Property=~/calendar-invite/
score J_YAHOO_CAL 11

header J_MEDIAWIKI_MAILER X-Mailer=~/MediaWiki mailer/
score J_MEDIAWIKI_MAILER -10

(and as nobody is reading the spam except me, I have disposed of
"describe" entries.)

version now in X-Spam-Checker-Version, so remove from X-Spam-Status

[jidanni]

Gentlemen, I am frustrated by the duplication of information in:
X-Spam-Checker-Version: SpamAssassin
3.2.5-mon_sep__8_23_53_29_2008.jidanni2.jidanni.org (2008-06-10) on
jidanni2.jidanni.org
X-Spam-Status: No, score=0.0 required=1.9 tests=none autolearn=disabled
version=3.2.5-mon_sep__8_23_53_29_2008.jidanni2.jidanni.org

Why not just chuck the newly arrived X-Spam-Checker-Version, I said to
myself. However,

   Note that X-Spam-Checker-Version is not removable because
   the version information is needed by mail administrators

OK, then I tried tinkering with

   version_tag string
   This tag is appended to the SA version in the X-Spam-Status
   header...your last name or your initials

which doesn't yet mention that it ends up in X-Spam-Checker-Version
too... indeed it's either both or nothing.

The obvious solution is to not include version in X-Spam-Status
anymore, as it is not a statusy item, and naturally belongs instead in
X-Spam-Checker-Version, which being a like-it-or-not item, might as
well carry it alone.

Re: score USER_IN_DEF_WHITELIST 0, for me at least

[jidanni]

DCWO> Perhaps you would like to share an example of such a spam

OK, https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5970 thanks.

Re: 1000 times easier to just do sa-update --nogpg

[jidanni]

>> Hello, this is the sa-update program talking to you.
>> We've detected a problem.
>> You need to do
>> $ wget http://spamassassin.apache.org/updates/GPG.KEY
>> $ sa-update --import GPG.KEY
>> and then run sa-update again. Thank you.

DCWO> Patches welcome.  Please keep in mind, when parsing the output of GPG,
DCWO> that the error text may be platform dependent.  For instance, even
DCWO> getting the cross-signed key error is platform dependent.

Well as I am more an expert in breakfast cereals than whatever that is
all about, somebody else please write the patch. Thanks.

score USER_IN_DEF_WHITELIST 0, for me at least

[jidanni]

I set score USER_IN_DEF_WHITELIST 0
as I guess I'm not the well rounded person reflected in the
pre-defined whitelists. Indeed not many people are I bet.

You see one day this spam got through riding high on that -15 point
boost, causing me to notice the existence of these lists. I'm not sure
if my one liner stopped all of them though.

Not sure if --local turns them off too.

Re: 1000 times easier to just do sa-update --nogpg

[jidanni]

> "K" == Kelson  <[EMAIL PROTECTED]> writes:

K> Pardon me for putting words in someone's mouth, but I got the
K> impression that the original poster's point was not to advocate
K> disabling signature checking, but to suggest that the error message
K> should be more useful.

Yes, I'm saying instead of just letting sa-update fail with the generic GNU
message and GNU hyperlink, setting the user off on a PhD Thesis effort
of trying to figure out what to do, instead just detect the problem and print 
out:

Hello, this is the sa-update program talking to you.
We've detected a problem.
You need to do
$ wget http://spamassassin.apache.org/updates/GPG.KEY
$ sa-update --import GPG.KEY
and then run sa-update again. Thank you.

Have that hardwired into the sa-update program, ready and waiting for
the next time it fails. What could be wrong with that? You can even add:

If that doesn't work, use sa-update --nogpg, and consult
http://news.gmane.org/gmane.mail.spam.spamassassin.general/ ...

1000 times easier to just do sa-update --nogpg

[jidanni]

You know, it is a 1000 times easier to just do
$ sa-update --nogpg
than to try to figure our the right way from the messages that
surround "channel: GPG validation failed, channel failed", or the
sa-update man page, or writing this group and asking what to do. So
there, the result is gpg is defeated.

The cure is to have the error message to say
"Do sa-update --import bbblllaaa", with the exact name it wants.

I challenge you to figure it out just from the failure message to
sa-update -D. One ends up lost reading
http://www.gnupg.org/faq/subkey-cross-certify.html.

It is 1000 times easier to just do
$ sa-update --nogpg.

Re: sa-update needs --nogpg

[jidanni]

> "TVD" == Theo Van Dinter <[EMAIL PROTECTED]> writes:

TVD> http://wiki.apache.org/spamassassin/SaUpdateKeyNotCrossCertified

OK, I wish sa-update would mention that step upon detecting that error.

sa-update needs --nogpg

[jidanni]

Just want to mention that
$ sa-update -D
[7581] dbg: gpg: gpg: WARNING: signing subkey 24F434CE is not cross-certified
[7581] dbg: gpg: gpg: please see 
http://www.gnupg.org/faq/subkey-cross-certify.html for more information
The update downloaded successfully, but the GPG signature verification failed.

So need
$ sa-update -D --nogpg
[7612] dbg: http: GET request, 
http://daryl.dostech.ca/sa-update/asf/681717.tar.gz
for it to work.

script to upgrade SpamAssassin (itself, not just rule sets)

[jidanni]

Gentlemen, every few months we must upgrade Spamassassin (the software
itself, not just doing sa-update).

So what script do you use to take the bore out of the process?

Need something like:
set -xeu
set /tmp/$USER.SpamassassinUpgrade
mkdir $1
cd $1
latest=`(fancy code to determine latest version on nearest mirror or
just master)`
wget $latest
bunzip2 *.bz2
cd `ls|sed q`
echo|perl Makefile.PL PREFIX=$HOME #answer the question with RETurn
make
make install

Mail-SpamAssassin-3.2.5 installation went OK

[jidanni]

The following are my (happy) Mail-SpamAssassin-3.2.5 installation
observations.

Seen at untarring:
Please make files dates reflect when they were last changed. Not all
just 2008-06-10.

We see
   checking module dependencies and their versions...
   NOTE: the optional Mail::SPF module is not installed...
Please say if these are Perl modules or SpamAssassin modules or Cpan
Perl modules, etc., even if you say so in README, etc.

After the first of
perl Makefile.PL PREFIX=$HOME && make && make install
we expect cheery messages, "Good boy, looks good", well, at least the
latter two don't bomb out :-)

(Anyway, still accruing debris of older versions and older sa-updates in
the file tree.)

Re: trusted mailing list subscriber spam

[jidanni]

>> All a spam program would have to do is say "[EMAIL PROTECTED] posts lots
>> to that list. His address must be a trusted subscriber. Well, here's
>> one more post from him, muhahaha."

SB> If "Bob" posts a lot to a list(s) and is respected within said
SB> list(s), then the other subs of that list will immediately recognize
SB> by the tone and the writing style of a fake message that it wasn't Bob
SB> that sent it.

Yes, but I'm talking about having spamassassin do the recognizing before
it reaches the humans. OK, that means some training for what each
trusted subscriber's message usually looks like. I have an idea: let's
discuss this complicated question at some other time.

>> OK, I suppose that would be caught by SPF rules etc., if bob likes SPF.

SB> Not all mail systems actually block upon SPF breakage...

BP> what are you talking about ?, to score email addresses found on maillist a 
bit
BP> negative since it looks like none spammy human ?

All I know is that I don't use SPF anymore for my domain as there are
just too many problems... e.g., forwarded messages.

trusted mailing list subscriber spam

[jidanni]

Odd how mailing lists that don't obfuscate addresses don't see more
trusted mailing list subscriber spam.

All a spam program would have to do is say "[EMAIL PROTECTED] posts lots
to that list. His address must be a trusted subscriber. Well, here's
one more post from him, muhahaha."

OK, I suppose that would be caught by SPF rules etc., if bob likes SPF.

unused directories accumulate each update

[jidanni]

I notice that old unused directories accumulate each time one updates
spamassassin:
$ tree -d var
var
`-- spamassassin
|-- 3.002003
|   `-- updates_spamassassin_org
`-- 3.002004
`-- updates_spamassassin_org
This will probably accumulate old perl versions one day too:
lib
`-- perl
`-- 5.8.4
`-- auto
`-- Mail
`-- SpamAssassin
Maybe one should do a clean remove before each install, but that would
wipe out one's customizations.

Re: but he really does use Outlook

[jidanni]

KD> To dig out exactly why, you'll have to dig down through the layers
KD> of meta tests that go into FORGED_MUA_OUTLOOK. :/

20_ratware.cf is apparently not totally aware of the full MicroSoft
scene. Just wanted to let you know. As I am allergic to MicroSoft: OK
thanks, see you later and I hereby drop the ball.

but he really does use Outlook

[jidanni]

Dear Spamassassin: today enjoying my usual chastising of people who
send me inferior mails by showing them what Spamassassin has to say, a
user remarked that he really does use MS Outlook, therefore I ended up
looking goofy. So the test heeds an update beyond anything sa-update
on 3.24 can do.

JB> Return-Path: <[EMAIL PROTECTED]>
JB> X-Original-To: [EMAIL PROTECTED]
JB> Delivered-To: [EMAIL PROTECTED]
JB> Received: from mail-out.m-online.net (mail-out.m-online.net [212.18.0.9])
JB> by blingymail-mx1.g.dreamhost.com (Postfix) with ESMTP id E78B244D3A
JB> for <[EMAIL PROTECTED]>; Tue,  4 Mar 2008 14:12:07 -0800 (PST)
JB> Received: from mail01.m-online.net (mail.m-online.net [192.168.3.149])
JB> by mail-out.m-online.net (Postfix) with ESMTP id 2EF7C21D7C1
JB> for <[EMAIL PROTECTED]>; Tue,  4 Mar 2008 23:12:28 +0100 (CET)
JB> Received: from localhost (unknown [192.168.1.157])
JB> by mail.m-online.net (Postfix) with ESMTP id 957CC9022A
JB> for <[EMAIL PROTECTED]>; Tue,  4 Mar 2008 23:12:05 +0100 (CET)
JB> X-Virus-Scanned: amavisd-new at mnet-online.de
JB> Received: from localhost ([192.168.3.149])
JB> by localhost (scanner1.m-online.net [192.168.1.157]) (amavisd-new, port 
10024)
JB> with ESMTP id 71yUPWSYUvfl for <[EMAIL PROTECTED]>;
JB> Tue,  4 Mar 2008 23:12:04 +0100 (CET)
JB> Received: from granularr (ppp-88-217-90-148.dynamic.mnet-online.de 
[88.217.90.148])
JB> by mail.mnet-online.de (Postfix) with SMTP
JB> for <[EMAIL PROTECTED]>; Tue,  4 Mar 2008 23:12:04 +0100 (CET)
JB> Message-ID: <[EMAIL PROTECTED]>
JB> Reply-To: "Johann Burkard" <[EMAIL PROTECTED]>
JB> From: "Johann Burkard" <[EMAIL PROTECTED]>
JB> To: <[EMAIL PROTECTED]>
JB> References: <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
JB> Subject: Re: not importaint
JB> Date: Tue, 4 Mar 2008 23:12:07 +0100
JB> X-Priority: 3
JB> X-MSMail-Priority: Normal
JB> X-Mailer: Microsoft Outlook Express 5.50.4980.1600
JB> X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4980.1600

>> P.S. your mail caused spamassassin to say
>>> pts rule name  description
>>>  --
>>> --
>>> 4.2 FORGED_MUA_OUTLOOK Forged mail pretending to be from MS
>>> Outlook

JB> Looks like your Spamassassin is wrong. I really use MS Outlook (Express).

JB> Regards,

JB> Johann Burkard

JB> 
JB> 
JB> 

success story: reviewing one's spam by score

[jidanni]

Gentlemen, reviewing one's spam, ordered by score, and threaded, is
definitely the way to go in one's daily check for false positives.
Here's how it looks for me in gnus, using my brilliant
http://jidanni.org/comp/spam/spamdealer.html orderer:

 . 8 080229|2.0|Secretsline In:Best VPN service
 .1 080229|2.1|Secretsline Ad:
 .1 080229|6.8|Secretsline In:
 .1 080229|6.9|Secretsline In:
 .1 080301|8.2|Secretsline Su:
 .1 080301|11.1|Secretsline Ad:
 .1 080301|11.1|Secretsline Ad:
 .1 080301|11.1|Secretsline Ad:
 . 1 080301|2.9|Angie Holliday:Annotation scaling and control of layers
 . 1 080228|3.0|IAC LTD   :WORK AS OUR REPRESENTATIVE
 . 1 080301|3.0|Michael Ez:ATTENTION PLEASE/REPLY FOR MORE DETAIL
 . 2 080302|3.4|Elliot Coulter:Medications that you need.
 .1 080303|16.0|Kurtis Lester :
 . 1 080228|4.0|Universidad de:ncrementa tus VENTAS ahora! Con el UDI
 . 1 080228|4.0|British Nation:Notice!!
 . 1 080229|4.0|UK NATIONAL LO:2008 lucky winner
 . 1 080229|4.1|:~2008/2/29 - 8:24:57 Canon  $100
 . 1 080228|4.4|MR Musa   :job offer
(250 more snipped)

Ah, one's daily spam check has now become a pleasure. A spamassassin
success story.

Notes:
We see on the 29th the VPN guys were scoring low with 2.0, but within
a day, the network tests had caught up with them, up to 11.1.

Musa is a banana genus.
http://en.wikipedia.org/wiki/Musa_%28genus%29
Here in the Taiwan hills I have ones filled with seeds.

Better stop here before the SPAM that CHANGED my LIFE.

I can see it: J. Mason, dedicated spam fighter, one day bored, started
looking at oh just one or two of the spam images and/or messages he
filtered. It was all downhill from there. Soon hypnotized, he began...

Speed gained with (?:) vs. ()

[jidanni]

BP> Using (?: avoids creating backreferences.  It should be slightly
BP> faster if the backreference is not used.

Wonder just how faster, in an actual spamassassin (not just perl)
context. Anybody got some timing statistics?

Re: "Nice girl like to chat" spam

[jidanni]

Say, if we're all getting the same spam, isn't that what we're paying
sa-update to catch? :-)

enjoy the high scores of --local, but still enjoy network tests

[jidanni]

How to enjoy the high scores of --local, but still enjoy network tests?

Man Mail::SpamAssassin::Conf says
  If four valid scores are listed, then the score that is used
  depends on how SpamAssassin is being used. The first score is used
  when both Bayes and network tests are disabled (score set 0). The
  second score is used when Bayes is disabled, but network tests are
  enabled (score set 1).

Too bad there's no way to tell it to always use set 0 without needing to do
$ perl -alpwe 's/^score.*/@F[0..2]/' \
~/var/spamassassin/3.002004/updates_spamassassin_org/50_scores.cf >> user_prefs

How inflexible.

Re: user_prefs: mind the linebreak

[jidanni]

MK> Call me older fashioned.. I consider line-wrapping a bit too "fancy"
MK> for my config editing preferences.

I see, you must have a billion column wide terminal or something.
Oops. You whippersnappers don't call them terminals these days.
Anyway, I swear I am not dreaming:
$ info make
  We split each long line into two lines using backslash-newline; this is
  like using one long line, but is easier to read.
http://en.wikipedia.org/wiki/Backslash#Usage

So OK call me a user with disabilities then, but don't hinder my
accessibility. Please spamassassin implement backslash-newline like
make, sh, etc.

Re: user_prefs: mind the linebreak

[jidanni]

MK> Why would there ever be a problem fitting on one line? Lines aren't
MK> limited to 80 characters or anything silly like that..

MK> That sounds a bit like complaining that a ship must fit in the water..
MK> There's a whole ocean out there, so who cares if you can't put one
MK> boat in 2 rain puddles..

Call me old fashioned, but I still want to be able to keep lines to a
length I prefer.

Yes I imagine your world probably looks like editing a Wikipedia
article with its long lines, but in mine, emacs:
  toggle-truncate-lines is an interactive compiled Lisp function in `simple.el'.
  Toggle whether to fold or truncate long lines for the current buffer.
  With arg, truncate long lines iff arg is positive.
Both are uncomfortable with long lines.
Also how do you print long lines on a line printer? Never mind.

Anyway, maybe
http://en.wikipedia.org/wiki/Carriage_return
http://en.wikipedia.org/wiki/Newline talk about the problem, maybe
not. Don't tell me I'm the only one who is still line-length aware.

user_prefs: mind the linebreak

[jidanni]

RP> Mind the linebreak :-)
That reminds me of this MINOR ITEM,
   Currently, each rule or configuration setting must fit on one-line;
   multi-line settings are not supported yet.

Re: Rule for Russian character sets

[jidanni]

Hmm, let me see. I use the below in user_prefs. Hope that helps.
header J_CHSET3 Subject:raw =~ 
/\s=\?(windows-(125[0125]|874)|koi8-r|iso-8859-[28])\?/i
score J_CHSET3 5
ifplugin Mail::SpamAssassin::Plugin::TextCat
#ok_languages en zh.big5
#http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5697
ok_languages en zh
add_header all Languages _LANGUAGES_
score UNWANTED_LANGUAGE_BODY 5
endif
ok_locales en zh

Re: "Nice girl like to chat" spam

[jidanni]

I just use in user_prefs
body J_GIRL /\bgirl.*\bpic(ture)?s\b/
score J_GIRL 5

Re: Rule for Russian character sets

[jidanni]

KB> If you want to trigger on Russian only, list all but ru.
What if to catch Ms. Ba'loney  Margar'ine, airport security had to keep a
current list of all the other people in the world. So this is the
wrong approach, as we've been thru before. OK, bye.

Re: upgrading is just like installing

[jidanni]

KB> User? SA is for administrators, not for users. Also, there is *nothing*
KB> special about SA version numbers.
Is too or else there wouldn't be a user_prefs file or instructions for
installing non-root. And SA version numbers & aliases often need explaining,
just like Debian package version numbers etc. Not all software version
number systems are the same or else there would be several ways to enter a
decimal password OK never mind.

Re: upgrading is just like installing

[jidanni]

KB> No, you are not. Please note that these are version numbers, not floats.
KB> With respect to minor versions, 24 is massively larger than 2...

It's all a usability (http://www.useit.com/alertbox/) problem. You
might say SpamAssassin is usability exempt, as it is only for computer
pros. But I'm here to give valuable insights of how SpamAssassin looks
to we lesser programmers. Just want to let you know. Can't help. Gotta go.

Anyway, assuming the user guesses correctly the special world of
SpamAssassin version numbers, the UPGRADE file still doesn't mention
anything beyond 3.2.0.

KB> Anyway, the UPGRADE file and its information is targeted at upgrading
KB> minor (or even major versions). Read, when upgrading from 3.1.x (or
KB> older) to 3.2.x for example. It mentions incompatibilities or general
KB> issues that may need attention when doing such an upgrade.

KB> Generally, no such issues exist when upgrading micro versions. Which you
KB> in fact did.

So it should mention all that. Or better yet see how MediaWiki
packages their upgrade instructions.

KB> However, you likely just overwrote your changes to local.cf
Naw, else it wouldn't say
$ head ~/etc/mail/spamassassin/local.cf
# This is the right place to customize your installation of SpamAssassin.

KD> This is why I don't often install from source.

I would but Dreamhost's Debian is too "stable" for me to run my brilliant
http://jidanni.org/comp/spam/spamdealer.html "filter on the server"
solution whilst enjoying the latest SpamAssassin.

upgrading is just like installing

[jidanni]

Let's record my 3.23 to 3.24 upgrade attempt.

I untar Mail-SpamAssassin-3.2.4 and of course read the file entitled
UPGRADE.

In it I find "Note for Users Upgrading to SpamAssassin 3.2.0".
But I am upgrading to 3.24.

Go ahead and call me dumb, but if I mess up, I might endanger my
email.

Now I look in README. No, no instructions there.

OK, now looking at INSTALL
 Upgrading SpamAssassin?
 ---

 Please be sure to read the UPGRADE file for important changes that
 have been made since previous versions.


 Installing or Upgrading SpamAssassin
 
OK, looks like I am on the right track. (I recall upgrading MediaWiki
was more intuitive.)

OK, now after reading the whole INSTALL file, I come to the conclusion
that to upgrade, one just acts like one never installed SpamAssassin
in the first place, and apparently the new SpamAssassin will just
install on top of the old one (with cruft surely accumulating in the
corners, I bet.)

OK... it all worked out. I even remembered to do sa-update, a step
mentioned in here in the newsgroup, not in the highly cluttered
INSTALL file.

OK, see you back here when 3.25 comes around, as I don't remember anything.