Hi, I'm currently being deluged with spam and have been trying to use BAYES filters to try and get rid of some of it. I've made a lot of progress but am now very stuck.
I have go to the point where I have (temporarily) given the amavis user a shell and when I run spamassassin on an email from the command line I get: spamassassin -t </tmp/sample3.txt Content analysis details: (7.4 points, 5.0 required) pts rule name description ---- ---------------------- ----------------------------------------- --------- 3.5 BAYES_99 BODY: Bayes spam probability is 99 to 100% [score: 1.0000] 1.8 REMOVE_BEFORE_LINK BODY: Removal phrase right before a link 0.2 BAYES_999 BODY: Bayes spam probability is 99.9 to 100% [score: 1.0000] 0.0 HTML_MESSAGE BODY: HTML included in message 1.1 DCC_CHECK Detected as bulk mail by DCC (dcc- servers.net) 0.8 RDNS_NONE Delivered to internal network by a host with no rDNS The debugs show: [Tue Nov 22 16:12:01] amavis@mta0 ~ $ spamassassin -D -t </tmp/sample3.txt 2>&1 | grep -i bayes Nov 22 16:12:10.355 [10336] dbg: plugin: loading Mail::SpamAssassin::Plugin::Bayes from @INC Nov 22 16:12:10.603 [10336] dbg: config: fixed relative path: /var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf Nov 22 16:12:10.603 [10336] dbg: config: using "/var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf" for included file Nov 22 16:12:10.603 [10336] dbg: config: read file /var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf Nov 22 16:12:11.594 [10336] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x27d2868) implements 'learner_new', priority 0 Nov 22 16:12:11.595 [10336] dbg: bayes: learner_new self=Mail::SpamAssassin::Plugin::Bayes=HASH(0x27d2868), bayes_store_module=Mail::SpamAssassin::BayesStore::DBM Nov 22 16:12:11.609 [10336] dbg: bayes: learner_new: got store=Mail::SpamAssassin::BayesStore::DBM=HASH(0x2fa76c8) Nov 22 16:12:11.609 [10336] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x27d2868) implements 'learner_is_scan_available', priority 0 Nov 22 16:12:11.613 [10336] dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_toks Nov 22 16:12:11.614 [10336] dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_seen Nov 22 16:12:11.614 [10336] dbg: bayes: found bayes db version 3 Nov 22 16:12:13.528 [10336] dbg: bayes: untie-ing However, when the same email was received through amavsid-new it received the following headers: X-Virus-Scanned: amavisd-new at zednax.com X-Spam-Flag: NO X-Spam-Score: 3.962 X-Spam-Level: *** X-Spam-Status: No, score=3.962 tagged_above=0 required=4 tests=[DCC_CHECK=1.1, HTML_MESSAGE=0.001, RDNS_NONE=1.274, REMOVE_BEFORE_LINK=1.587] autolearn=no autolearn_force=no I start amavisd in screen with debugs on as the same user; the debugs from amavisd show: Nov 22 15:55:46.359 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: plugin: loading Mail::SpamAssassin::Plugin::Bayes from @INC Nov 22 15:55:46.569 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: config: fixed relative path: /var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf Nov 22 15:55:46.569 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: config: using "/var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf" for included file Nov 22 15:55:46.569 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: config: read file /var/lib/spamassassin/3.004000/updates_spamassassin_org/23_bayes.cf Nov 22 15:55:47.565 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'learner_new', priority 0 Nov 22 15:55:47.566 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: learner_new self=Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00), bayes_store_module=Mail::SpamAssassin::BayesStore::DBM Nov 22 15:55:47.566 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: learner_new: got store=Mail::SpamAssassin::BayesStore::DBM=HASH(0x5b102c8) Nov 22 15:55:47.566 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'learner_is_scan_available', priority 0 Nov 22 15:55:47.566 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_toks Nov 22 15:55:47.567 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_seen Nov 22 15:55:47.567 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: found bayes db version 3 Nov 22 15:55:49.089 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'learner_close', priority 0 Nov 22 15:55:49.089 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: bayes: untie-ing Nov 22 15:55:49.089 mta0.zednax.com /usr/sbin/amavisd[7630]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'prefork_init', priority 0 Nov 22 15:55:49.090 mta0.zednax.com /usr/sbin/amavisd[7630]: SpamAssassin loaded plugins: AskDNS, AutoLearnThreshold, Bayes, BodyEval, Check, DCC, DKIM, DNSEval, FreeMail, HTMLEval, HTTPSMismatch, HeaderEval, ImageInfo, MIMEEval, MIMEHeader, Pyzor, Razor2, RelayEval, ReplaceTags, SpamCop, URIDetail, URIEval, VBounce, WLBLEval, WhiteListSubject Nov 22 15:55:49.104 mta0.zednax.com /usr/sbin/amavisd[7649]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'spamd_child_init', priority 0 at startup, then when processing the message: Nov 22 16:08:37.091 mta0.zednax.com /usr/sbin/amavisd[9727]: SA dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x55e7a00) implements 'spamd_child_init', priority 0 Nov 22 16:08:37.223 mta0.zednax.com /usr/sbin/amavisd[9727]: (09727-01) SA dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_toks Nov 22 16:08:37.224 mta0.zednax.com /usr/sbin/amavisd[9727]: (09727-01) SA dbg: bayes: tie-ing to DB file R/O /var/amavis/.spamassassin/bayes_seen Nov 22 16:08:37.224 mta0.zednax.com /usr/sbin/amavisd[9727]: (09727-01) SA dbg: bayes: found bayes db version 3 I'm even seeing debugs that show amavisd learning messages it detects as spam (using non-bayes rules): Nov 22 16:07:40.228 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: locker: safe_lock: created /var/amavis/.spamassassin/bayes.lock.mta0.zednax.com.9064 Nov 22 16:07:40.228 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: locker: safe_lock: trying to get lock on /var/amavis/.spamassassin/bayes with 0 retries Nov 22 16:07:40.228 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: locker: safe_lock: link to /var/amavis/.spamassassin/bayes.lock: link ok Nov 22 16:07:40.229 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: bayes: tie-ing to DB file R/W /var/amavis/.spamassassin/bayes_toks Nov 22 16:07:40.229 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: bayes: tie-ing to DB file R/W /var/amavis/.spamassassin/bayes_seen Nov 22 16:07:40.230 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: bayes: found bayes db version 3 Nov 22 16:07:40.396 mta0.zednax.com /usr/sbin/amavisd[9064]: (09064-11) SA dbg: bayes: learned '3cbcccb5747f8488582ac93a965e6c8590b465c2@sa_gen erated', atime: 1479830854 Having read numerous threads of admins with similar issues I expect it will come down to permissions, but I've tried 0666 and 0777 as the file mode. The options currently set in the spamassassin config are: skip_rbl_checks 0 use_bayes 1 auto_learn 0 bayes_path /var/amavis/.spamassassin/bayes bayes_file_mode 0777 bayes_auto_expire 0 The directory looks like: [Wed Nov 23 09:13:55] mta0 ~ # ls -la /var/amavis/.spamassassin/* -rw-rw-rw- 1 amavis amavis 22 Nov 22 16:19 /var/amavis/.spamassassin/bayes.lock -rw-rw-rw- 1 amavis amavis 2200 Nov 23 09:14 /var/amavis/.spamassassin/bayes.lock.mta0.zednax.com.18174 -rwxrwxrwx 1 amavis amavis 167673856 Nov 22 16:19 /var/amavis/.spamassassin/bayes_seen -rwxrwxrwx 1 amavis amavis 5382144 Nov 22 16:19 /var/amavis/.spamassassin/bayes_toks -rwxrwxrwx 1 amavis amavis 1869 Nov 22 11:29 /var/amavis/.spamassassin/user_prefs And the magic dump looks like: [Wed Nov 23 09:14:14] mta0 ~ # sa-learn --username=amavis --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 387589 0 non-token data: nspam 0.000 0 922763 0 non-token data: nham 0.000 0 175867 0 non-token data: ntokens 0.000 0 1478796541 0 non-token data: oldest atime 0.000 0 1479831533 0 non-token data: newest atime 0.000 0 1479831423 0 non-token data: last journal sync atime 0.000 0 1479802087 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count Any help would be appreciated before I drown in spam... -- Technical Director - Zednax Limited W: http://www.zednax.com T: +44 333 444 0160 F: +44 161 660 8010 Zednax Limited is registered in England and Wales, Company no. 05321754. Registered address: Meadow House, Meadow Lane, Nottingham, NG2 3HS. Zednax Limited is VAT registered, VAT registration no. GB 855 4468 92.