Re: AWL doesn't seem to work
At 01:44 AM 8/25/2005, Ilan Aisic wrote: OK, I figured out what my problem was. It's in the way I always restarted SA. This was from the following simple script that I always ran as root: --- echo Running spamassassin --lint and then restarting spamd if OK... spamassassin --lint if [ $? != 0 ] ; then echo "SA discovered errors!" else /etc/init.d/spamassassin restart fi --- Apparently, the command `spamassassin --lint` created the 2 files: -rw-rw-rw-1 root root12288 Aug 25 08:12 auto-whitelist -rw---1 root root6 Aug 25 08:12 auto-whitelist.mutex It should create the two, but the mutex should be deleted when --lint exits. Perhaps this is one of the bugs in SA 3.0.2. I'm not sure, as the DoS vulnerability alone is enough for me to steer clear of running this version of SA on a production box. I know that 3.0.3 fixed some "memory bloat" problems with the AWL, so I wouldn't suggest using the AWL with 3.0.2: http://www.gossamer-threads.com/lists/spamassassin/announce/8
Re: AWL doesn't seem to work
OK, I figured out what my problem was. It's in the way I always restarted SA. This was from the following simple script that I always ran as root: --- echo Running spamassassin --lint and then restarting spamd if OK... spamassassin --lint if [ $? != 0 ] ; then echo "SA discovered errors!" else /etc/init.d/spamassassin restart fi --- Apparently, the command `spamassassin --lint` created the 2 files: -rw-rw-rw-1 root root12288 Aug 25 08:12 auto-whitelist -rw---1 root root6 Aug 25 08:12 auto-whitelist.mutex And spamd who runs as "nobody" could not, I assume because of the mutex which is 600 write into the file. I erased both files and restarted spamd without the lint. This time the files are: -rw-rw-rw-1 nobody nobody 12288 Aug 25 08:33 auto-whitelist -rw---1 nobody nobody 6 Aug 25 08:33 auto-whitelist.mutex And now the auto-whitelist keeps changing whenever email arrives. Thanks for all the help, --ilan On 8/25/05, Ilan Aisic <[EMAIL PROTECTED]> wrote: > BTW, Matt was right in his assumption below. > AWL worked correctly on my test. > I intentionally contrived 2 emails from the same fake address. The > first was inoccent the 2nd was the same text plus few known spammy > words and phrases. > > > On 8/24/05, Matt Kettler <[EMAIL PROTECTED]> wrote: > > jdow wrote: > > > Ilan, you could adopt my strategy and simply turn off auto-whitelist > > > and delete the auto-whitelist file. I've seen too many mis-trained > > > auto-whitelists mentioned on this list to be at all comfortable with > > > it. The same can be said for auto-learn with Bayes. > > > > > > > Are you sure they were mis-trained? > > > > Or were you just seeing cases of negative AWL scores in spam (which is > > perfectly > > normal)? > > > > http://wiki.apache.org/spamassassin/AwlWrongWay > > > > > > > At the VERY least set the thresholds for auto this and that MUCH wider > > > than they come stock. > > > > That won't affect the AWL at all. That only affects bayes. The AWL is an > > averager, so it "learns" every message. > > > > > > > > It appears you have a spam message yet auto-whitelist thinks it is > > > ham. > > > > No. The second message isn't spam. It's a spammy ham, or at least so the > > Ilan > > implies: > > > > "Anyway, I did as advised and ran spamassassin -D < test (instead of the > > --lint option) and I ran it twice on 2 messages from the same address > > (2nd was spammy). This way it does work as advertised" > > > > Note Ilan did not say it was spam, just spammy. Also note that Ilan > > considers > > this part to be *correct* behavior. > > > > So that means his AWL *correctly* deducted points from a message that would > > have > > been a FP otherwise. > > > > It's possible Ilan is using intentionally contrived emails here to force > > the case. > > > > (If it really was ham, you found a reason to sort spam into a > > > spam mailbox and at least glance at the trash before tossing it.) > > > > > > > -- > Ilan Aisic > Registered Linux User 8124 http://counter.li.org >
Re: AWL doesn't seem to work
BTW, Matt was right in his assumption below. AWL worked correctly on my test. I intentionally contrived 2 emails from the same fake address. The first was inoccent the 2nd was the same text plus few known spammy words and phrases. On 8/24/05, Matt Kettler <[EMAIL PROTECTED]> wrote: > jdow wrote: > > Ilan, you could adopt my strategy and simply turn off auto-whitelist > > and delete the auto-whitelist file. I've seen too many mis-trained > > auto-whitelists mentioned on this list to be at all comfortable with > > it. The same can be said for auto-learn with Bayes. > > > > Are you sure they were mis-trained? > > Or were you just seeing cases of negative AWL scores in spam (which is > perfectly > normal)? > > http://wiki.apache.org/spamassassin/AwlWrongWay > > > > At the VERY least set the thresholds for auto this and that MUCH wider > > than they come stock. > > That won't affect the AWL at all. That only affects bayes. The AWL is an > averager, so it "learns" every message. > > > > > It appears you have a spam message yet auto-whitelist thinks it is > > ham. > > No. The second message isn't spam. It's a spammy ham, or at least so the Ilan > implies: > > "Anyway, I did as advised and ran spamassassin -D < test (instead of the > --lint option) and I ran it twice on 2 messages from the same address > (2nd was spammy). This way it does work as advertised" > > Note Ilan did not say it was spam, just spammy. Also note that Ilan considers > this part to be *correct* behavior. > > So that means his AWL *correctly* deducted points from a message that would > have > been a FP otherwise. > > It's possible Ilan is using intentionally contrived emails here to force the > case. > > (If it really was ham, you found a reason to sort spam into a > > spam mailbox and at least glance at the trash before tossing it.) > > -- Ilan Aisic Registered Linux User 8124 http://counter.li.org
Re: AWL doesn't seem to work
Below: On 8/24/05, Matt Kettler <[EMAIL PROTECTED]> wrote: > Ilan Aisic wrote: > > Matt, > > I've modified the permissions on my auto-whitelist file and directory > > to 777 > > I didn't say modify the permissions of the file or directory. I said to modify > your configuration file option in your local.cf to be 0777. The file should be > set to 666 anyway (which is what SA will do if the option is 777, RTFM that I > quoted again, closely this time) That's what I did. I just wasnt' phrasing it right :-) [EMAIL PROTECTED] ~]$ cd /var/spool/spamassassin/ [EMAIL PROTECTED] spamassassin]$ ls -al total 20 drwxrwxrwx2 root root 4096 Aug 25 08:20 . drwxr-xr-x 18 root root 4096 Aug 24 11:17 .. -rw-rw-rw-1 root root12288 Aug 25 08:12 auto-whitelist -rw---1 root root6 Aug 25 08:12 auto-whitelist.mutex Perhaps the problem is that the Mutex is for root only and spamd runs as 'nobody' ? > > > > even though I don't see why this is needed since spamd runs as > > root. > > Spamd will *NEVER*, EVER, scan mail as root. Thus it will not have root > permissions when touching that file. If it finds it's running as root when > mail > is to be scanned, it will setuid itself to nobody as a security measure of > last-resort. > > If you're running as root, take measures to make sure nobody has RWX to the > directory, RW to the file, and your auto_whitelist_file_mode needs to be set > to > 0777. With the exception of very few trusted users, all the mail users can't login to the system. > -- Ilan Aisic Registered Linux User 8124 http://counter.li.org
Re: AWL doesn't seem to work
jdow wrote: > Ilan, you could adopt my strategy and simply turn off auto-whitelist > and delete the auto-whitelist file. I've seen too many mis-trained > auto-whitelists mentioned on this list to be at all comfortable with > it. The same can be said for auto-learn with Bayes. > Are you sure they were mis-trained? Or were you just seeing cases of negative AWL scores in spam (which is perfectly normal)? http://wiki.apache.org/spamassassin/AwlWrongWay > At the VERY least set the thresholds for auto this and that MUCH wider > than they come stock. That won't affect the AWL at all. That only affects bayes. The AWL is an averager, so it "learns" every message. > > It appears you have a spam message yet auto-whitelist thinks it is > ham. No. The second message isn't spam. It's a spammy ham, or at least so the Ilan implies: "Anyway, I did as advised and ran spamassassin -D < test (instead of the --lint option) and I ran it twice on 2 messages from the same address (2nd was spammy). This way it does work as advertised" Note Ilan did not say it was spam, just spammy. Also note that Ilan considers this part to be *correct* behavior. So that means his AWL *correctly* deducted points from a message that would have been a FP otherwise. It's possible Ilan is using intentionally contrived emails here to force the case. (If it really was ham, you found a reason to sort spam into a > spam mailbox and at least glance at the trash before tossing it.)
Re: AWL doesn't seem to work
Ilan, you could adopt my strategy and simply turn off auto-whitelist
and delete the auto-whitelist file. I've seen too many mis-trained
auto-whitelists mentioned on this list to be at all comfortable with
it. The same can be said for auto-learn with Bayes.
At the VERY least set the thresholds for auto this and that MUCH wider
than they come stock.
It appears you have a spam message yet auto-whitelist thinks it is
ham. (If it really was ham, you found a reason to sort spam into a
spam mailbox and at least glance at the trash before tossing it.)
{^_^}
- Original Message -
From: "Ilan Aisic" <[EMAIL PROTECTED]>
Matt,
I've modified the permissions on my auto-whitelist file and directory
to 777 even though I don't see why this is needed since spamd runs as
root.
Anyway, I did as advised and ran spamassassin -D < test (intead of the
--lint option) and I ran it twice on 2 messages from the same address
(2nd was spammy). This way it does work as adveritsed (see excerpt
from output below). I ran an `od -c ` on the file and was able to
see the test email address in there.
However, I then restarted the daemon and it does not touch the
auto-whitelist at all (I sent myself few emails from the outside and
the auto-whitelist file wasn't touched and I didn't see AWL lines in
the X-Spam-Report)
debug: running header regexp tests; score so far=17.627
debug: lock: 31007 created /var/spool/spamassassin/auto-whitelist.mutex
debug: lock: 31007 trying to get lock on
/var/spool/spamassassin/auto-whitelist
with 30 timeoutdebug: lock: 31007 link to
/var/spool/spamassassin/auto-whitelist.mutex: link ok
debug: Tie-ing to DB file R/W in /var/spool/spamassassin/auto-whitelist
debug: auto-whitelist (db-based): [EMAIL PROTECTED]|ip=204.11 scores 1/3.925
debug: AWL active, pre-score: 17.627, autolearn score: 17.627, mean: 3.925,
IP:
204.11.105.95
debug: add_score: New count: 2, new totscore: 21.552
debug: DB addr list: untie-ing and unlocking.
debug: DB addr list: file locked, breaking lock.debug: unlock: 31007
unlocked /var/spool/spamassassin/auto-whitelist.mutex
debug: Post AWL score: 10.776
debug: running body-text per-line regexp tests; score so far=10.776
debug: running uri tests; score so far=10.776
debug: running raw-body-text per-line regexp tests; score so far=10.776
debug: running full-text regexp tests; score so far=10.776
debug: auto-learn: currently using scoreset 1.
debug: auto-learn: message score: 10.776, computed score for autolearn:
18.627
debug: auto-learn? ham=0.1, spam=12, body-points=12.251,
head-points=6.494, learned-points=0
debug: auto-learn? yes, spam (18.627 > 12)
debug: Learning Spam
X-Spam-Report:
* -6.9 AWL AWL: From: address is in the auto white-list
On 8/24/05, Matt Kettler <[EMAIL PROTECTED]> wrote:
At 05:15 AM 8/24/2005, Ilan Aisic wrote:
>Hi,
>I'm running SA 3.0.2 with Exim 4.5. spamd runs as root.
Warning: You are subject to a remote DoS attack on SA's mime parser, it's
triggered by sending you a malformed message. Upgrade to 3.0.4.
http://marc.theaimsgroup.com/?l=spamassassin-announce&m=111886630726077&w=2
>I think I've set my configuration to have a system-wide
>auto-whitelisting.
>However, I've noticed that even though the file "auto-whitelist"
>always remains 12,288 bytes long and unchanged and naturally, scores
>aren't affected.
>Everytime I restart spamd, the file gets a new timestamp though.
>
>I'd appreciate any advice.
>
>My auto-whitelist related comamnds in the local.cf file are:
>auto_whitelist_path/var/spool/spamassassin/auto-whitelist
>auto_whitelist_file_mode 0666
change that mode from 0666 to 0777
auto_whitelist_file_mode(default: 0700)
The file mode bits used for the automatic-whitelist directory or file.
Make sure you specify this using the 'x' mode bits set, as it may also be
used to create directories. However, if a file is created, the resulting
file will not have any execute bits set (the umask is set to 111).
-
What are the permissions on the existing directory and file?
Elsewhere Kevin wrote:
>This isn't terribly helpful.
Why isn't it? --lint does run a message, and you can see that it hasn't
learned it before...
> Please try running with a real email, using the syntax:
>
>spamassassin -t -D < testemail
Drop that -t... in SA 3.0.0 and higher -t disables the AWL and bayes
autolearner.
http://bugzilla.spamassassin.org/show_bug.cgi?id=2632
--
Ilan Aisic
Registered Linux User 8124 http://counter.li.org
Re: AWL doesn't seem to work
Ilan Aisic wrote: > Matt, > I've modified the permissions on my auto-whitelist file and directory > to 777 I didn't say modify the permissions of the file or directory. I said to modify your configuration file option in your local.cf to be 0777. The file should be set to 666 anyway (which is what SA will do if the option is 777, RTFM that I quoted again, closely this time) > even though I don't see why this is needed since spamd runs as > root. Spamd will *NEVER*, EVER, scan mail as root. Thus it will not have root permissions when touching that file. If it finds it's running as root when mail is to be scanned, it will setuid itself to nobody as a security measure of last-resort. If you're running as root, take measures to make sure nobody has RWX to the directory, RW to the file, and your auto_whitelist_file_mode needs to be set to 0777.
Re: AWL doesn't seem to work
Matt, I've modified the permissions on my auto-whitelist file and directory to 777 even though I don't see why this is needed since spamd runs as root. Anyway, I did as advised and ran spamassassin -D < test (intead of the --lint option) and I ran it twice on 2 messages from the same address (2nd was spammy). This way it does work as adveritsed (see excerpt from output below). I ran an `od -c ` on the file and was able to see the test email address in there. However, I then restarted the daemon and it does not touch the auto-whitelist at all (I sent myself few emails from the outside and the auto-whitelist file wasn't touched and I didn't see AWL lines in the X-Spam-Report) debug: running header regexp tests; score so far=17.627 debug: lock: 31007 created /var/spool/spamassassin/auto-whitelist.mutex debug: lock: 31007 trying to get lock on /var/spool/spamassassin/auto-whitelist with 30 timeoutdebug: lock: 31007 link to /var/spool/spamassassin/auto-whitelist.mutex: link ok debug: Tie-ing to DB file R/W in /var/spool/spamassassin/auto-whitelist debug: auto-whitelist (db-based): [EMAIL PROTECTED]|ip=204.11 scores 1/3.925 debug: AWL active, pre-score: 17.627, autolearn score: 17.627, mean: 3.925, IP: 204.11.105.95 debug: add_score: New count: 2, new totscore: 21.552 debug: DB addr list: untie-ing and unlocking. debug: DB addr list: file locked, breaking lock.debug: unlock: 31007 unlocked /var/spool/spamassassin/auto-whitelist.mutex debug: Post AWL score: 10.776 debug: running body-text per-line regexp tests; score so far=10.776 debug: running uri tests; score so far=10.776 debug: running raw-body-text per-line regexp tests; score so far=10.776 debug: running full-text regexp tests; score so far=10.776 debug: auto-learn: currently using scoreset 1. debug: auto-learn: message score: 10.776, computed score for autolearn: 18.627 debug: auto-learn? ham=0.1, spam=12, body-points=12.251, head-points=6.494, learned-points=0 debug: auto-learn? yes, spam (18.627 > 12) debug: Learning Spam X-Spam-Report: * -6.9 AWL AWL: From: address is in the auto white-list On 8/24/05, Matt Kettler <[EMAIL PROTECTED]> wrote: > At 05:15 AM 8/24/2005, Ilan Aisic wrote: > >Hi, > >I'm running SA 3.0.2 with Exim 4.5. spamd runs as root. > > Warning: You are subject to a remote DoS attack on SA's mime parser, it's > triggered by sending you a malformed message. Upgrade to 3.0.4. > > http://marc.theaimsgroup.com/?l=spamassassin-announce&m=111886630726077&w=2 > > > >I think I've set my configuration to have a system-wide auto-whitelisting. > >However, I've noticed that even though the file "auto-whitelist" > >always remains 12,288 bytes long and unchanged and naturally, scores > >aren't affected. > >Everytime I restart spamd, the file gets a new timestamp though. > > > >I'd appreciate any advice. > > > >My auto-whitelist related comamnds in the local.cf file are: > >auto_whitelist_path/var/spool/spamassassin/auto-whitelist > >auto_whitelist_file_mode 0666 > > change that mode from 0666 to 0777 > > > auto_whitelist_file_mode(default: 0700) > The file mode bits used for the automatic-whitelist directory or file. > > Make sure you specify this using the 'x' mode bits set, as it may also be > used to create directories. However, if a file is created, the resulting > file will not have any execute bits set (the umask is set to 111). > - > > What are the permissions on the existing directory and file? > > > Elsewhere Kevin wrote: > >This isn't terribly helpful. > > Why isn't it? --lint does run a message, and you can see that it hasn't > learned it before... > > > Please try running with a real email, using the syntax: > > > >spamassassin -t -D < testemail > > Drop that -t... in SA 3.0.0 and higher -t disables the AWL and bayes > autolearner. > > http://bugzilla.spamassassin.org/show_bug.cgi?id=2632 > > > > > > -- Ilan Aisic Registered Linux User 8124 http://counter.li.org
Re: AWL doesn't seem to work
At 05:15 AM 8/24/2005, Ilan Aisic wrote: Hi, I'm running SA 3.0.2 with Exim 4.5. spamd runs as root. Warning: You are subject to a remote DoS attack on SA's mime parser, it's triggered by sending you a malformed message. Upgrade to 3.0.4. http://marc.theaimsgroup.com/?l=spamassassin-announce&m=111886630726077&w=2 I think I've set my configuration to have a system-wide auto-whitelisting. However, I've noticed that even though the file "auto-whitelist" always remains 12,288 bytes long and unchanged and naturally, scores aren't affected. Everytime I restart spamd, the file gets a new timestamp though. I'd appreciate any advice. My auto-whitelist related comamnds in the local.cf file are: auto_whitelist_path/var/spool/spamassassin/auto-whitelist auto_whitelist_file_mode 0666 change that mode from 0666 to 0777 auto_whitelist_file_mode(default: 0700) The file mode bits used for the automatic-whitelist directory or file. Make sure you specify this using the 'x' mode bits set, as it may also be used to create directories. However, if a file is created, the resulting file will not have any execute bits set (the umask is set to 111). - What are the permissions on the existing directory and file? Elsewhere Kevin wrote: This isn't terribly helpful. Why isn't it? --lint does run a message, and you can see that it hasn't learned it before... Please try running with a real email, using the syntax: spamassassin -t -D < testemail Drop that -t... in SA 3.0.0 and higher -t disables the AWL and bayes autolearner. http://bugzilla.spamassassin.org/show_bug.cgi?id=2632
Re: AWL doesn't seem to work
Ilan Aisic wrote: Related output from running spamassassin -D --lint : [EMAIL PROTECTED]|ip=none scores 0/0debug: AWL active, pre-score: 7.328, autolearn score: 7.328, mean: undef, IP: undef This isn't terribly helpful. Please try running with a real email, using the syntax: spamassassin -t -D < testemail Then post the AWL related output from that debug.
AWL doesn't seem to work
Hi, I'm running SA 3.0.2 with Exim 4.5. spamd runs as root. I think I've set my configuration to have a system-wide auto-whitelisting. However, I've noticed that even though the file "auto-whitelist" always remains 12,288 bytes long and unchanged and naturally, scores aren't affected. Everytime I restart spamd, the file gets a new timestamp though. I'd appreciate any advice. My auto-whitelist related comamnds in the local.cf file are: auto_whitelist_path/var/spool/spamassassin/auto-whitelist auto_whitelist_file_mode 0666 Related output from running spamassassin -D --lint : debug: running meta tests; score so far=7.328 debug: running header regexp tests; score so far=7.328 debug: lock: 29716 created /var/spool/spamassassin/auto-whitelist.mutex debug: lock: 29716 trying to get lock on /var/spool/spamassassin/auto-whitelist with 30 timeout debug: lock: 29716 link to /var/spool/spamassassin/auto-whitelist.mutex: link ok debug: Tie-ing to DB file R/W in /var/spool/spamassassin/auto-whitelist debug: auto-whitelist (db-based): [EMAIL PROTECTED]|ip=none scores 0/0debug: AWL active, pre-score: 7.328, autolearn score: 7.328, mean: undef, IP: undef debug: DB addr list: untie-ing and unlocking. debug: DB addr list: file locked, breaking lock. debug: unlock: 29716 unlocked /var/spool/spamassassin/auto-whitelist.mutexdebug: Post AWL score: 7.328 -- Ilan Aisic Registered Linux User 8124 http://counter.li.org
