Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 17/06/16 14:49, RW wrote:

On Fri, 17 Jun 2016 14:07:33 +0100
Sebastian Arcus wrote:





Site-wide bayes files are owned
by spamd. Regarding the daemon, it is started with
--socketowner=spamd and socketpath=spamd. Is this enough, or
should it be actually started with "su" as "spamd" user?


If you start it as root with the -u spamd (or --username) it will drop
privileges to spamd. Starting it as root allows it to bind to a low
port should you need that.



"socketpath=spamd" sounds idiotic, hpwever for a site-wide setup
there is no point in start it as root instead directly as the
correct user, see below, can#t say anything about "su" in service
files since i don't touch sysvinit for 5 years now


That is probably so - I've taken another look at my startup scripts,
and I have to say it feels like I've been tying myself in knots with
--socketowner and --socketgroup and --username. I was thinking that
for my setup using:

--username=spamd --socketownder=exim --socketgroup=exim

might be the most suitable. Is it better to run it instead with

--socketmode=666


You should use -u,--username unless you need to access per user data
from unix home directories. You need this even if you start directly as
spamd.


and not bother with setting owner and group for the socket?


Is there any particular reason for even using a socket file?



A good point - if I leave them out, spamd will talk on the default IP 
port, and Exim can do that as well. Thank you for suggesting!

Re: Spamassassin uses bayes, but spamd doesn't

[Matus UHLAR - fantomas]

>> Site-wide bayes files are owned
>> by spamd. Regarding the daemon, it is started with
>> --socketowner=spamd and socketpath=spamd. Is this enough, or
>> should it be actually started with "su" as "spamd" user?


On 17.06.16 14:49, RW wrote:

If you start it as root with the -u spamd (or --username) it will drop
privileges to spamd. Starting it as root allows it to bind to a low
port should you need that.


it also allows spamd to use different system user's preferences and bayes
DBs. Since the bayes DB is site-wide, root is apparently not needed.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Depression is merely anger without enthusiasm. 

Re: Spamassassin uses bayes, but spamd doesn't

[Reindl Harald]

Am 17.06.2016 um 15:49 schrieb RW:

and not bother with setting owner and group for the socket?


Is there any particular reason for even using a socket file?


unix sockets are faster

the only particular reason for *not* usng sockets is when you need to 
access the daemon from other machines


in the best case you are done with as less as possible listening on tcp

Proto Recv-Q Send-Q Local Address   Foreign Address 
State   PID/Program name
tcp0  0 0.0.0.0:10220.0.0.0:* 
LISTEN  924/sshd
tcp0  0 127.0.0.1:530.0.0.0:* 
LISTEN  27164/unbound
tcp0  0 0.0.0.0:25  0.0.0.0:* 
LISTEN  1172/master
udp0  0 127.0.0.1:1053  0.0.0.0:* 
   25906/rbldnsd
udp0  0 127.0.0.1:530.0.0.0:* 
   27164/unbound




signature.asc
Description: OpenPGP digital signature

Re: Spamassassin uses bayes, but spamd doesn't

[RW]

On Fri, 17 Jun 2016 14:07:33 +0100
Sebastian Arcus wrote:


> >  
> >> Site-wide bayes files are owned
> >> by spamd. Regarding the daemon, it is started with
> >> --socketowner=spamd and socketpath=spamd. Is this enough, or
> >> should it be actually started with "su" as "spamd" user?  

If you start it as root with the -u spamd (or --username) it will drop
privileges to spamd. Starting it as root allows it to bind to a low
port should you need that. 


> > "socketpath=spamd" sounds idiotic, hpwever for a site-wide setup
> > there is no point in start it as root instead directly as the
> > correct user, see below, can#t say anything about "su" in service
> > files since i don't touch sysvinit for 5 years now  
> 
> That is probably so - I've taken another look at my startup scripts,
> and I have to say it feels like I've been tying myself in knots with 
> --socketowner and --socketgroup and --username. I was thinking that
> for my setup using:
> 
> --username=spamd --socketownder=exim --socketgroup=exim
> 
> might be the most suitable. Is it better to run it instead with
> 
> --socketmode=666

You should use -u,--username unless you need to access per user data
from unix home directories. You need this even if you start directly as
spamd.
 
> and not bother with setting owner and group for the socket?

Is there any particular reason for even using a socket file? 

Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 16/06/16 18:46, Sebastian Arcus wrote:

I have a particular server running spamd which uses bayes every time I
test it by hand, but apparently never when it goes through exim/spamd.

I run everything (both the spamd daemon and the manual tests) as user
spamd. I checked the permissions on the bayes database. I use a global
bayes database in /var/spool/spamd/bayes/. I ran "spamassassin -D
--lint" - and I get no failures - both as root and as the user spamd.

In spite of all of the above, it looks pretty clear that bayes is only
used when I run an email manually through spamassassin, but not when it
goes from exim through spamd.

Here is the report when ran from the command line:

Content analysis details:   (5.4 points, 5.0 required)

 pts rule name  description
 --
--
 2.0 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5000]
 0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to image
area
 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
background
 0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
 0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
Colors in HTML
 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not
necessarily valid
 0.2 RDNS_NONE  Delivered to internal network by a host with
no rDNS
 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
 0.0 UNPARSEABLE_RELAY  Informational: message has unparseable relay
lines
 0.0 LOTS_OF_MONEY  Huge... sums of money
 1.5 SUBJ_ILLEGAL_CHARS Subject: has too many raw illegal characters
 0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME parts
 0.0 SUBJECT_NEEDS_ENCODING Subject is encoded but does not specify the
encoding


And here is the report included in the same email message when it comes
through exim:

 Content analysis details:   (1.9 points, 5.0 required)

  pts rule name  description
  --
--
  0.7 MPART_ALT_DIFF BODY: HTML and text parts are different
  0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to image
area
  0.0 HTML_MESSAGE   BODY: HTML included in message
  0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
 background
  0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
 Colors in HTML
  1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 -0.1 DKIM_VALID Message has at least one valid DKIM or DK
signature
 -0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature
from author's
 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not
necessarily valid
  0.0 LOTS_OF_MONEY  Huge... sums of money
  0.2 RDNS_NONE  Delivered to internal network by a host
with no rDNS
  0.0 UNPARSEABLE_RELAY  Informational: message has unparseable
relay lines
  0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME
parts


Bayes is clearly not being used when it goes through spamd. Does anybody
know what could be causing this?


OK - thank you to everybody who helped with hints and info. Bayes is 
finally working now. What I initially had in place is:


1. Site-wide bayes db in /var/spool/spamd/bayes/ and owned by spamd.spamd
2. Spamd socket owned by spamd.spamd - which turns out that didn't make 
much sense
3. Spamd ran as root - for some reason I got confused and thought 
setting the owner/group for the socket meant spamd was run as non-root user.


What I have now:
1. Spamd socket owned by exim.exim (as it is the only piece of software 
which needs to talk to spamd) - and mode set to 0660.

2. Spamd runs as "spamd" user.
3. Bayes db still in the same place as above and with the same ownership 
- but I've set them as 0660


In conclusion it would appear that running the spamd as root was the 
cause of the problem - although root should have been able to access the 
bayes database anyway. I'm a little lost on that point I'm afraid. But I 
think it's been a good opportunity to straighten the setup both on the 
server and in my head :-) Thank you again.

Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 17/06/16 04:46, Bill Cole wrote:

On 16 Jun 2016, at 13:46, Sebastian Arcus wrote:


I have a particular server running spamd


Which must run on a particular platform. Since SpamAssassin and Exim can
run on a decade's worth of versions of at least 9 different OSs and one
of those (Linux) has about a half-dozen distinctly different families of
distributions that have become quite divergent, it would help to
identify your OS and version (or if Linux, which distro & its version)
when seeking help from people who don't keep track of what sorts of
systems you run. This helps constrain the scope of sane guessing...

(However, the ability to run arbitrary programs as 'root' implies a
POSIX-y platform with a true-root security model, so I'll assume this
isn't some Windows-Frankenstein abomination or El Capitan)


which uses bayes every time I test it by hand, but apparently never
when it goes through exim/spamd.

I run everything (both the spamd daemon and the manual tests) as user
spamd. I checked the permissions on the bayes database. I use a global
bayes database in /var/spool/spamd/bayes/.


Provide `ls -la /var/spool/spamd/bayes/`, please. Or if the problem that
reveals is obvious, just fix it and you're welcome. :)


I ran "spamassassin -D --lint" - and I get no failures - both as root
and as the user spamd.


And when you run spamassassin as root, you risk having root steal the
Bayes and AWL DBs. Presumably this is why some misguided articles online
documenting SA setup for system-wide use recommend deeply wrong things
like 'chmod -R 777' on your database directory. Don't do that. Ever. On
any directory. Use an ad hoc group, BSD directory setgid semantics or
fileflags, ACLs, a script that runs from cron every minute, or whatever
else can work on your platform to assure that spamd can always read and
write to everything in that directory, but DO NOT 777 it.


In spite of all of the above, it looks pretty clear that bayes is only
used when I run an email manually through spamassassin, but not when
it goes from exim through spamd.


Is spamd configured to do any logging? By default on POSIX platforms it
logs under the mail facility and if it can't open the BayesDB it will
log that fact. If it does so but there's no ownership/permission problem
it could also be due to SELinux, running spamd in a chroot jail (bad
idea,) or maybe AppArmor (about which I know nothing other than that
it's an alternative to SELinux.) These are solvable problems.


Thank you for all the suggestions above - and you are right, I should 
have been more specific about my setup. I'll report back to the list 
with progress or when it is solved.

Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 17/06/16 13:42, Reindl Harald wrote:



Am 17.06.2016 um 14:29 schrieb Sebastian Arcus:

On 17/06/16 00:03, Reindl Harald wrote:



Am 16.06.2016 um 19:46 schrieb Sebastian Arcus:

I have a particular server running spamd which uses bayes every time I
test it by hand, but apparently never when it goes through exim/spamd


then you need to run it as the correct user or train it as the correct


Thank you for the suggestion. There is no training involved, and
auto-learn is switched off in local.cf


how do you imagine bayes working then?


These are bayes databases from another server - any training happens 
there - so training and auto-learning is disabled on this particular server.





Site-wide bayes files are owned
by spamd. Regarding the daemon, it is started with --socketowner=spamd
and socketpath=spamd. Is this enough, or should it be actually started
with "su" as "spamd" user?


"socketpath=spamd" sounds idiotic, hpwever for a site-wide setup there
is no point in start it as root instead directly as the correct user,
see below, can#t say anything about "su" in service files since i don't
touch sysvinit for 5 years now


That is probably so - I've taken another look at my startup scripts, and 
I have to say it feels like I've been tying myself in knots with 
--socketowner and --socketgroup and --username. I was thinking that for 
my setup using:


--username=spamd --socketownder=exim --socketgroup=exim

might be the most suitable. Is it better to run it instead with

--socketmode=666

and not bother with setting owner and group for the socket?

Re: Spamassassin uses bayes, but spamd doesn't

[Reindl Harald]

Am 17.06.2016 um 14:29 schrieb Sebastian Arcus:

On 17/06/16 00:03, Reindl Harald wrote:



Am 16.06.2016 um 19:46 schrieb Sebastian Arcus:

I have a particular server running spamd which uses bayes every time I
test it by hand, but apparently never when it goes through exim/spamd


then you need to run it as the correct user or train it as the correct


Thank you for the suggestion. There is no training involved, and
auto-learn is switched off in local.cf


how do you imagine bayes working then?


Site-wide bayes files are owned
by spamd. Regarding the daemon, it is started with --socketowner=spamd
and socketpath=spamd. Is this enough, or should it be actually started
with "su" as "spamd" user?


"socketpath=spamd" sounds idiotic, hpwever for a site-wide setup there 
is no point in start it as root instead directly as the correct user, 
see below, can#t say anything about "su" in service files since i don't 
touch sysvinit for 5 years now


[root@mail-gw:~]$ cat /etc/systemd/system/spamassassin.service
[Unit]
Description=Spamassassin Daemon
After=network.service systemd-networkd.service network-online.target
Wants=sa-update.timer

[Service]
Environment="TMPDIR=/tmp"
PermissionsStartOnly=true
ExecStartPre=/usr/bin/chown -R sa-cleanup:root /var/lib/spamassassin/
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type d -exec 
/bin/chmod 0755 "{}" \;
ExecStartPre=/usr/bin/find /var/lib/spamassassin/ -type f -exec 
/bin/chmod 0644 "{}" \;
ExecStart=/usr/bin/spamd --max-children=15 --min-children=2 
--min-spare=2 --max-spare=3 --max-conn-per-child=100 
--socketpath=/run/spamassassin/spamassassin.sock --socketmode=0666

ExecReload=/usr/bin/kill -HUP $MAINPID
Environment="LANG=en_GB.UTF-8"
User=sa-milt
Group=sa-milt
RuntimeDirectory=spamassassin
RuntimeDirectoryMode=0775
StandardOutput=null
StandardError=null
SyslogFacility=mail
Restart=always
RestartSec=1

PrivateTmp=yes
PrivateDevices=yes
NoNewPrivileges=yes
CapabilityBoundingSet=CAP_KILL CAP_SYS_CHROOT
RestrictAddressFamilies=~AF_APPLETALK AF_ATMPVC AF_AX25 AF_PACKET AF_X25

ReadOnlyDirectories=/etc
ReadOnlyDirectories=/usr
ReadOnlyDirectories=/var
ReadOnlyDirectories=/var/lib/spamass-milter/.spamassassin
ReadWriteDirectories=/var/lib/spamassassin
InaccessibleDirectories=-/var/lib/spamass-milter/training

InaccessibleDirectories=-/boot
InaccessibleDirectories=-/home
InaccessibleDirectories=-/media
InaccessibleDirectories=-/root
InaccessibleDirectories=-/etc/dbus-1
InaccessibleDirectories=-/etc/modprobe.d
InaccessibleDirectories=-/etc/modules-load.d
InaccessibleDirectories=-/etc/postfix
InaccessibleDirectories=-/etc/ssh
InaccessibleDirectories=-/etc/sysctl.d
InaccessibleDirectories=-/run/console
InaccessibleDirectories=-/run/dbus
InaccessibleDirectories=-/run/lock
InaccessibleDirectories=-/run/mount
InaccessibleDirectories=-/run/systemd/generator
InaccessibleDirectories=-/run/systemd/system
InaccessibleDirectories=-/run/systemd/users
InaccessibleDirectories=-/run/udev
InaccessibleDirectories=-/run/user
InaccessibleDirectories=-/usr/lib64/dbus-1
InaccessibleDirectories=-/usr/lib64/xtables
InaccessibleDirectories=-/usr/lib/dracut
InaccessibleDirectories=-/usr/libexec/iptables
InaccessibleDirectories=-/usr/libexec/openssh
InaccessibleDirectories=-/usr/libexec/postfix
InaccessibleDirectories=-/usr/lib/grub
InaccessibleDirectories=-/usr/lib/kernel
InaccessibleDirectories=-/usr/lib/modprobe.d
InaccessibleDirectories=-/usr/lib/modules
InaccessibleDirectories=-/usr/lib/modules-load.d
InaccessibleDirectories=-/usr/lib/rpm
InaccessibleDirectories=-/usr/lib/sysctl.d
InaccessibleDirectories=-/usr/lib/udev
InaccessibleDirectories=-/usr/local/scripts
InaccessibleDirectories=-/var/db
InaccessibleDirectories=-/var/lib/bayes-persistent
InaccessibleDirectories=-/var/lib/clamav
InaccessibleDirectories=-/var/lib/clamav-spam
InaccessibleDirectories=-/var/lib/dbmail
InaccessibleDirectories=-/var/lib/dbus
InaccessibleDirectories=-/var/lib/dhcpd
InaccessibleDirectories=-/var/lib/dnf
InaccessibleDirectories=-/var/lib/imapproxy
InaccessibleDirectories=-/var/lib/initramfs
InaccessibleDirectories=-/var/lib/mailgraph
InaccessibleDirectories=-/var/lib/mlocate
InaccessibleDirectories=-/var/lib/mysql
InaccessibleDirectories=-/var/lib/ntp
InaccessibleDirectories=-/var/lib/postfix
InaccessibleDirectories=-/var/lib/rkhunter
InaccessibleDirectories=-/var/lib/rpm
InaccessibleDirectories=-/var/lib/systemd
InaccessibleDirectories=-/var/lib/unbound
InaccessibleDirectories=-/var/lib/vnstat
InaccessibleDirectories=-/var/lib/yum
InaccessibleDirectories=-/var/named
InaccessibleDirectories=-/var/spool
InaccessibleDirectories=-/var/www
InaccessibleDirectories=-/Volumes/dune/mysql_data
InaccessibleDirectories=-/Volumes/dune/mysql_tmp
InaccessibleDirectories=-/Volumes/dune/updateservice
InaccessibleDirectories=-/Volumes/dune/www-servers

[Install]
WantedBy=multi-user.target




signature.asc
Description: OpenPGP digital signature

Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 17/06/16 00:03, Reindl Harald wrote:



Am 16.06.2016 um 19:46 schrieb Sebastian Arcus:

I have a particular server running spamd which uses bayes every time I
test it by hand, but apparently never when it goes through exim/spamd


then you need to run it as the correct user or train it as the correct user



Thank you for the suggestion. There is no training involved, and 
auto-learn is switched off in local.cf. Site-wide bayes files are owned 
by spamd. Regarding the daemon, it is started with --socketowner=spamd 
and socketpath=spamd. Is this enough, or should it be actually started 
with "su" as "spamd" user?

Re: Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

On 17/06/16 03:46, Yu Qian wrote:

you can use spamd -D to check the log for exactly what bayes db path
your spamd was using.


Thank Yu. Based on the output below, it appears to find and use the 
sitewide bayes files ok:


# spamd -D 2>&1 | grep -i bayes
Jun 17 13:32:51.719 [4380] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::Bayes from @INC
Jun 17 13:32:52.058 [4380] dbg: config: fixed relative path: 
/var/lib/spamassassin/3.004001/updates_spamassassin_org/23_bayes.cf
Jun 17 13:32:52.058 [4380] dbg: config: using 
"/var/lib/spamassassin/3.004001/updates_spamassassin_org/23_bayes.cf" 
for included file
Jun 17 13:32:52.058 [4380] dbg: config: read file 
/var/lib/spamassassin/3.004001/updates_spamassassin_org/23_bayes.cf
Jun 17 13:32:53.370 [4380] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'learner_new', priority 0
Jun 17 13:32:53.371 [4380] dbg: bayes: learner_new 
self=Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48), 
bayes_store_module=Mail::SpamAssassin::BayesStore::DBM
Jun 17 13:32:53.390 [4380] dbg: bayes: learner_new: got 
store=Mail::SpamAssassin::BayesStore::DBM=HASH(0xab6a6a0)
Jun 17 13:32:53.391 [4380] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'learner_is_scan_available', priority 0
Jun 17 13:32:53.391 [4380] dbg: bayes: tie-ing to DB file R/O 
/var/spool/spamd/bayes/bayes_toks
Jun 17 13:32:53.392 [4380] dbg: bayes: tie-ing to DB file R/O 
/var/spool/spamd/bayes/bayes_seen

Jun 17 13:32:53.393 [4380] dbg: bayes: found bayes db version 3
Jun 17 13:32:53.394 [4380] dbg: bayes: DB journal sync: last sync: 
1466097119
Jun 17 13:32:55.405 [4380] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'learner_close', priority 0

Jun 17 13:32:55.405 [4380] dbg: bayes: untie-ing
Jun 17 13:32:55.487 [4380] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'prefork_init', priority 0
Jun 17 13:32:55.492 [4385] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'spamd_child_init', priority 0
Jun 17 13:32:55.497 [4386] dbg: plugin: 
Mail::SpamAssassin::Plugin::Bayes=HASH(0xa936c48) implements 
'spamd_child_init', priority 0

Re: Spamassassin uses bayes, but spamd doesn't

[Bill Cole]

On 16 Jun 2016, at 13:46, Sebastian Arcus wrote:


I have a particular server running spamd


Which must run on a particular platform. Since SpamAssassin and Exim can 
run on a decade's worth of versions of at least 9 different OSs and one 
of those (Linux) has about a half-dozen distinctly different families of 
distributions that have become quite divergent, it would help to 
identify your OS and version (or if Linux, which distro & its version) 
when seeking help from people who don't keep track of what sorts of 
systems you run. This helps constrain the scope of sane guessing...


(However, the ability to run arbitrary programs as 'root' implies a 
POSIX-y platform with a true-root security model, so I'll assume this 
isn't some Windows-Frankenstein abomination or El Capitan)


which uses bayes every time I test it by hand, but apparently never 
when it goes through exim/spamd.


I run everything (both the spamd daemon and the manual tests) as user 
spamd. I checked the permissions on the bayes database. I use a global 
bayes database in /var/spool/spamd/bayes/.


Provide `ls -la /var/spool/spamd/bayes/`, please. Or if the problem that 
reveals is obvious, just fix it and you're welcome. :)


I ran "spamassassin -D --lint" - and I get no failures - both as root 
and as the user spamd.


And when you run spamassassin as root, you risk having root steal the 
Bayes and AWL DBs. Presumably this is why some misguided articles online 
documenting SA setup for system-wide use recommend deeply wrong things 
like 'chmod -R 777' on your database directory. Don't do that. Ever. On 
any directory. Use an ad hoc group, BSD directory setgid semantics or 
fileflags, ACLs, a script that runs from cron every minute, or whatever 
else can work on your platform to assure that spamd can always read and 
write to everything in that directory, but DO NOT 777 it.


In spite of all of the above, it looks pretty clear that bayes is only 
used when I run an email manually through spamassassin, but not when 
it goes from exim through spamd.


Is spamd configured to do any logging? By default on POSIX platforms it 
logs under the mail facility and if it can't open the BayesDB it will 
log that fact. If it does so but there's no ownership/permission problem 
it could also be due to SELinux, running spamd in a chroot jail (bad 
idea,) or maybe AppArmor (about which I know nothing other than that 
it's an alternative to SELinux.) These are solvable problems.

Re: Spamassassin uses bayes, but spamd doesn't

[Yu Qian]

you can use spamd -D to check the log for exactly what bayes db path your
spamd was using.

---
Yu Qian
Ottawa Ontario
Phone: (514)-553-0198



On Thu, Jun 16, 2016 at 7:03 PM, Reindl Harald 
wrote:

>
>
> Am 16.06.2016 um 19:46 schrieb Sebastian Arcus:
>
>> I have a particular server running spamd which uses bayes every time I
>> test it by hand, but apparently never when it goes through exim/spamd
>>
>
> then you need to run it as the correct user or train it as the correct user
>
>

Re: Spamassassin uses bayes, but spamd doesn't

[Reindl Harald]

Am 16.06.2016 um 19:46 schrieb Sebastian Arcus:

I have a particular server running spamd which uses bayes every time I
test it by hand, but apparently never when it goes through exim/spamd


then you need to run it as the correct user or train it as the correct user



signature.asc
Description: OpenPGP digital signature

Re: Spamassassin uses bayes, but spamd doesn't

[Bowie Bailey]

On 6/16/2016 1:46 PM, Sebastian Arcus wrote:
I have a particular server running spamd which uses bayes every time I 
test it by hand, but apparently never when it goes through exim/spamd.


I run everything (both the spamd daemon and the manual tests) as user 
spamd. I checked the permissions on the bayes database. I use a global 
bayes database in /var/spool/spamd/bayes/. I ran "spamassassin -D 
--lint" - and I get no failures - both as root and as the user spamd.


In spite of all of the above, it looks pretty clear that bayes is only 
used when I run an email manually through spamassassin, but not when 
it goes from exim through spamd.


Here is the report when ran from the command line:

Content analysis details:   (5.4 points, 5.0 required)

 pts rule name  description
 -- 
--

 2.0 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5000]
 0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to 
image area

 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
background
 0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
 0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
Colors in HTML
 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid
 0.2 RDNS_NONE  Delivered to internal network by a host 
with no rDNS

 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
 0.0 UNPARSEABLE_RELAY  Informational: message has unparseable 
relay lines

 0.0 LOTS_OF_MONEY  Huge... sums of money
 1.5 SUBJ_ILLEGAL_CHARS Subject: has too many raw illegal characters
 0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME 
parts

 0.0 SUBJECT_NEEDS_ENCODING Subject is encoded but does not specify the
encoding


And here is the report included in the same email message when it 
comes through exim:


 Content analysis details:   (1.9 points, 5.0 required)

  pts rule name  description
  -- 
--

  0.7 MPART_ALT_DIFF BODY: HTML and text parts are different
  0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to 
image area

  0.0 HTML_MESSAGE   BODY: HTML included in message
  0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or 
identical to

 background
  0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
 Colors in HTML
  1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 -0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature
 -0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from author's

 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid

  0.0 LOTS_OF_MONEY  Huge... sums of money
  0.2 RDNS_NONE  Delivered to internal network by a host 
with no rDNS
  0.0 UNPARSEABLE_RELAY  Informational: message has unparseable 
relay lines
  0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME 
parts



Bayes is clearly not being used when it goes through spamd. Does 
anybody know what could be causing this?


When you test by hand, are you using "spamassassin" or "spamc"?  You 
should be using "spamc" to test as closely as possible to your normal 
mailflow.


Add this to your local.cf to get more detail on Bayes (should be all one 
line):


add_header all Bayes bayes=_BAYES_, 
N=_BAYESTC_(_BAYESTCLEARNED_-_BAYESTCHAMMY_+_BAYESTCSPAMMY_), 
ham=(_HAMMYTOKENS(5,short)_), spam=(_SPAMMYTOKENS(5,short)_)


This will add an X-Spam-Bayes header that may give you more information 
about what Bayes is doing.


--
Bowie

Spamassassin uses bayes, but spamd doesn't

[Sebastian Arcus]

I have a particular server running spamd which uses bayes every time I 
test it by hand, but apparently never when it goes through exim/spamd.


I run everything (both the spamd daemon and the manual tests) as user 
spamd. I checked the permissions on the bayes database. I use a global 
bayes database in /var/spool/spamd/bayes/. I ran "spamassassin -D 
--lint" - and I get no failures - both as root and as the user spamd.


In spite of all of the above, it looks pretty clear that bayes is only 
used when I run an email manually through spamassassin, but not when it 
goes from exim through spamd.


Here is the report when ran from the command line:

Content analysis details:   (5.4 points, 5.0 required)

 pts rule name  description
 -- 
--

 2.0 BAYES_50   BODY: Bayes spam probability is 40 to 60%
[score: 0.5000]
 0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to image 
area

 0.0 HTML_MESSAGE   BODY: HTML included in message
 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
background
 0.8 MPART_ALT_DIFF BODY: HTML and text parts are different
 0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
Colors in HTML
 0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid
 0.2 RDNS_NONE  Delivered to internal network by a host 
with no rDNS

 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid
 0.0 UNPARSEABLE_RELAY  Informational: message has unparseable 
relay lines

 0.0 LOTS_OF_MONEY  Huge... sums of money
 1.5 SUBJ_ILLEGAL_CHARS Subject: has too many raw illegal characters
 0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME parts
 0.0 SUBJECT_NEEDS_ENCODING Subject is encoded but does not specify the
encoding


And here is the report included in the same email message when it comes 
through exim:


 Content analysis details:   (1.9 points, 5.0 required)

  pts rule name  description
  -- 
--

  0.7 MPART_ALT_DIFF BODY: HTML and text parts are different
  0.0 HTML_IMAGE_RATIO_06BODY: HTML has a low ratio of text to 
image area

  0.0 HTML_MESSAGE   BODY: HTML included in message
  0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to
 background
  0.0 T_KAM_HTML_FONT_INVALID BODY: Test for Invalidly Named or Formatted
 Colors in HTML
  1.1 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
 -0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature
 -0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from author's

 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid

  0.0 LOTS_OF_MONEY  Huge... sums of money
  0.2 RDNS_NONE  Delivered to internal network by a host 
with no rDNS
  0.0 UNPARSEABLE_RELAY  Informational: message has unparseable 
relay lines
  0.0 MIME_HTML_ONLY_MULTI   Multipart message only has text/html MIME 
parts



Bayes is clearly not being used when it goes through spamd. Does anybody 
know what could be causing this?