Re: SA doesn't respect my user_prefs

[RW]

On Fri, 11 Sep 2015 16:53:17 +0200
Benny Pedersen wrote:



> but there was a dokument error on what -x do on spamd

What I found confusing is that --virtual-config-dir  doesn't work
without -x. In other words you have to set the nouser-config option to
make spamd read the user config. 


> and -u on spamd is not usefull if user_prefs is needed or enabled

It is if you have  virtual users.


 

Re: SA doesn't respect my user_prefs

[Marc Richter]

Guess this means that I have to run "spamassassin" instead of spamc, 
don't I?


I do not understand the reason for spamc to exist then - but based upon 
the conversation result, it seems like the way to go ... hope my host 
can handle the load.


Am 10.09.2015 um 12:50 schrieb Marc Richter:

Hi @ all,

maybe I'm doing it wrong here - I do not insist on being unfailable.
But what's the correct way to do it then?

Best regards,
Marc

Am 10.09.2015 um 01:48 schrieb RW:

On Wed, 9 Sep 2015 14:48:14 -0700
jdow wrote:


On 2015-09-09 13:51, RW wrote:

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:


Hi RW,


Do you mean that ww is a unix user? The normal way to do this is
to run spamd as root and run spamc as the unix user. Passing -u to
spamc is really intended for virtual users, I'm not sure whether
it works for unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.


Supporting that sounds like a really bad idea. It would mean that
any user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege
escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.


Why don't you try to run spamc -u root as a common user and see what
happens then talk about the results if it is warranted?



Given that it doesn't appear to be currently working with non-root
accounts, what would that prove? And it's still wrong even if root is a
special case.

Re: SA doesn't respect my user_prefs

[Kevin A. McGrail]

Spamc exists to save startup compilation time.

If you have real users and use procmail then spamc will be much faster and pass 
along the username.

If you use a glue or have virtual users, you might need logic to call spamc or 
spamassassin with a desired username.  But for me, I would anticipate switching 
will just make things slower and not solve the issue.

Regards,
KAM

On September 11, 2015 5:35:12 AM AST, Marc Richter  
wrote:
>Guess this means that I have to run "spamassassin" instead of spamc, 
>don't I?
>
>I do not understand the reason for spamc to exist then

Re: SA doesn't respect my user_prefs

[Kevin A. McGrail]

I can't disagree as I was answering the why it exists.  

What are using user prefs to accomplish because I prefer using sql based prefs?
Regards,
KAM

On September 11, 2015 5:50:43 AM AST, Marc Richter  
wrote:
>Hi KAM,
>
>why not - spamassassin seems to respect the user_prefs file. Of course 
>I'd like to stick ti spamc, but if there is no solution for the 
>user_prefs - issue, it fits only half of my needs.
>
>Best regard,
>Marc
>
>Am 11.09.2015 um 11:47 schrieb Kevin A. McGrail:
>> Spamc exists to save startup compilation time.
>>
>> If you have real users and use procmail then spamc will be much
>faster and pass along the username.
>>
>> If you use a glue or have virtual users, you might need logic to call
>spamc or spamassassin with a desired username.  But for me, I would
>anticipate switching will just make things slower and not solve the
>issue.
>>
>> Regards,
>> KAM
>>
>> On September 11, 2015 5:35:12 AM AST, Marc Richter
> wrote:
>>> Guess this means that I have to run "spamassassin" instead of spamc,
>>> don't I?
>>>
>>> I do not understand the reason for spamc to exist then
>>

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi KAM,

why not - spamassassin seems to respect the user_prefs file. Of course 
I'd like to stick ti spamc, but if there is no solution for the 
user_prefs - issue, it fits only half of my needs.


Best regard,
Marc

Am 11.09.2015 um 11:47 schrieb Kevin A. McGrail:

Spamc exists to save startup compilation time.

If you have real users and use procmail then spamc will be much faster and pass 
along the username.

If you use a glue or have virtual users, you might need logic to call spamc or 
spamassassin with a desired username.  But for me, I would anticipate switching 
will just make things slower and not solve the issue.

Regards,
KAM

On September 11, 2015 5:35:12 AM AST, Marc Richter  
wrote:

Guess this means that I have to run "spamassassin" instead of spamc,
don't I?

I do not understand the reason for spamc to exist then

Re: SA doesn't respect my user_prefs

[Reindl Harald]

Am 11.09.2015 um 11:35 schrieb Marc Richter:

Guess this means that I have to run "spamassassin" instead of spamc,
don't I?

I do not understand the reason for spamc to exist then


uhm because it does the real work?

in the case below milter -> spamd -> spamc preforkers

[root@mail-gw:~]$ systemctl status spamassassin.service
● spamassassin.service - Spamassassin Daemon
   Loaded: loaded (/etc/systemd/system/spamassassin.service; enabled)
   Active: active (running) since Fr 2015-09-11 00:59:27 CEST; 10h ago
  Process: 24463 ExecReload=/usr/bin/kill -HUP $MAINPID (code=exited, 
status=0/SUCCESS)
  Process: 10162 ExecStartPre=/usr/bin/find /var/lib/spamassassin/ 
-type f -exec /bin/chmod 0644 {} ; (code=exited, status=0/SUCCESS)
  Process: 10153 ExecStartPre=/usr/bin/find /var/lib/spamassassin/ 
-type d -exec /bin/chmod 0755 {} ; (code=exited, status=0/SUCCESS)

 Main PID: 10235 (spamd)
   CGroup: /system.slice/spamassassin.service
   ├─10235 /usr/bin/perl -T -w /usr/bin/spamd --max-children=20 
--min-children=5 --min-spare=5 --max-spare=10 --max-conn-per-child=200 
--socketpath=/run/spamassassin/spamassassin.sock --socketmode=0666

   ├─24469 spamd chil
   ├─24470 spamd chil
   ├─24471 spamd chil
   ├─24472 spamd chil
   ├─24473 spamd chil
   ├─24474 spamd chil
   ├─24475 spamd chil
   ├─24476 spamd chil
   ├─24477 spamd chil
   └─24478 spamd chil



signature.asc
Description: OpenPGP digital signature

Re: SA doesn't respect my user_prefs

[Olivier Nicole]

Marc Richter  writes:

> Hi KAM,
>
> why not - spamassassin seems to respect the user_prefs file. Of course 
> I'd like to stick ti spamc, but if there is no solution for the 
> user_prefs - issue, it fits only half of my needs.

Sorry for jumping in the conversation, I have not read all the messages,
but if I remember well, un order for spamc -u to work, you need to run
spamd as high priviledged user.

For security reasons, user's .spamassassin directory is readble only by
that user. Spamc -u tells spamd to become that user, but spamd must be
allowed by the system to change user, that means spamd must be running
as root to begin with.

So I would say:

- start spamd as root
- spamc -u user
- or become user and spamassassin 

All this is from memory, because I use SA though amavisd nowdays.

Best regards,

Olivier

> Best regard,
> Marc
>
> Am 11.09.2015 um 11:47 schrieb Kevin A. McGrail:
>> Spamc exists to save startup compilation time.
>>
>> If you have real users and use procmail then spamc will be much faster and 
>> pass along the username.
>>
>> If you use a glue or have virtual users, you might need logic to call spamc 
>> or spamassassin with a desired username.  But for me, I would anticipate 
>> switching will just make things slower and not solve the issue.
>>
>> Regards,
>> KAM
>>
>> On September 11, 2015 5:35:12 AM AST, Marc Richter  
>> wrote:
>>> Guess this means that I have to run "spamassassin" instead of spamc,
>>> don't I?
>>>
>>> I do not understand the reason for spamc to exist then
>>
>

-- 

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi Olivier,

thanks for your ideas, they look reasonable.
But I think it might be not the solution, since

1. my spamd runs as spamd:spamd and my home-dirs/-files have rw 
permissions for at least group spamd:


ww@tango012 ~ $ ls -ald .spamassassin .spamassassin/*
drwxrwx--- 2 wwspamd 4096 11. Sep 11:40 .spamassassin
-rw-rw 1 wwspamd 10387456 27. Aug 14:19 .spamassassin/auto-whitelist
-rw--- 1 spamd spamd6 27. Aug 14:19 
.spamassassin/auto-whitelist.mutex

-rw-rw 1 wwspamd 8667 11. Sep 11:40 .spamassassin/user_prefs
ww@tango012 ~ $

2. I nevertheless tried to run spamd as root and this is what it results in:

spamd: cannot run as nonexistent user or root with -u option

Best regards,
Marc

Am 11.09.2015 um 12:05 schrieb Olivier Nicole:

Marc Richter  writes:


Hi KAM,

why not - spamassassin seems to respect the user_prefs file. Of course
I'd like to stick ti spamc, but if there is no solution for the
user_prefs - issue, it fits only half of my needs.


Sorry for jumping in the conversation, I have not read all the messages,
but if I remember well, un order for spamc -u to work, you need to run
spamd as high priviledged user.

For security reasons, user's .spamassassin directory is readble only by
that user. Spamc -u tells spamd to become that user, but spamd must be
allowed by the system to change user, that means spamd must be running
as root to begin with.

So I would say:

- start spamd as root
- spamc -u user
- or become user and spamassassin

All this is from memory, because I use SA though amavisd nowdays.

Best regards,

Olivier


Best regard,
Marc

Am 11.09.2015 um 11:47 schrieb Kevin A. McGrail:

Spamc exists to save startup compilation time.

If you have real users and use procmail then spamc will be much faster and pass 
along the username.

If you use a glue or have virtual users, you might need logic to call spamc or 
spamassassin with a desired username.  But for me, I would anticipate switching 
will just make things slower and not solve the issue.

Regards,
KAM

On September 11, 2015 5:35:12 AM AST, Marc Richter  
wrote:

Guess this means that I have to run "spamassassin" instead of spamc,
don't I?

I do not understand the reason for spamc to exist then

Re: SA doesn't respect my user_prefs

[Reindl Harald]

Am 11.09.2015 um 16:05 schrieb Marc Richter:

thanks for your ideas, they look reasonable.
But I think it might be not the solution, since

1. my spamd runs as spamd:spamd and my home-dirs/-files have rw
permissions for at least group spamd:

ww@tango012 ~ $ ls -ald .spamassassin .spamassassin/*
drwxrwx--- 2 wwspamd 4096 11. Sep 11:40 .spamassassin
-rw-rw 1 wwspamd 10387456 27. Aug 14:19
.spamassassin/auto-whitelist
-rw--- 1 spamd spamd6 27. Aug 14:19
.spamassassin/auto-whitelist.mutex
-rw-rw 1 wwspamd 8667 11. Sep 11:40 .spamassassin/user_prefs
ww@tango012 ~ $

2. I nevertheless tried to run spamd as root and this is what it results
in:

spamd: cannot run as nonexistent user or root with -u option


spamd must not be startet with the -u option as root, the whole purpose 
is to have the daemon process running as root and then "spamc" is 
invoked with the -u param of the user which is target of the incoming 
message



Am 11.09.2015 um 12:05 schrieb Olivier Nicole:

Marc Richter  writes:


Hi KAM,

why not - spamassassin seems to respect the user_prefs file. Of course
I'd like to stick ti spamc, but if there is no solution for the
user_prefs - issue, it fits only half of my needs.


Sorry for jumping in the conversation, I have not read all the messages,
but if I remember well, un order for spamc -u to work, you need to run
spamd as high priviledged user.

For security reasons, user's .spamassassin directory is readble only by
that user. Spamc -u tells spamd to become that user, but spamd must be
allowed by the system to change user, that means spamd must be running
as root to begin with.

So I would say:

- start spamd as root
- spamc -u user
- or become user and spamassassin

All this is from memory, because I use SA though amavisd nowdays.

Best regards,

Olivier


Best regard,
Marc

Am 11.09.2015 um 11:47 schrieb Kevin A. McGrail:

Spamc exists to save startup compilation time.

If you have real users and use procmail then spamc will be much
faster and pass along the username.

If you use a glue or have virtual users, you might need logic to
call spamc or spamassassin with a desired username.  But for me, I
would anticipate switching will just make things slower and not
solve the issue.

Regards,
KAM

On September 11, 2015 5:35:12 AM AST, Marc Richter
 wrote:

Guess this means that I have to run "spamassassin" instead of spamc,
don't I?

I do not understand the reason for spamc to exist then








--

Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / CISO / Software-Development
m: +43 (676) 40 221 40, p: +43 (1) 595 3999 33
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm



signature.asc
Description: OpenPGP digital signature

Re: SA doesn't respect my user_prefs

[Matus UHLAR - fantomas]

Am 09.09.2015 um 15:01 schrieb Matus UHLAR - fantomas:

how do you plug spamassassin into your mail flow? How do you call
spamassassin? mta, mail client ... ?


On 09.09.15 16:11, Marc Richter wrote:
I'm running postfix as my MTA. In it's master.cf there is configured 
to pipe my mail through a script:


smtp   inet  n  -  n  -  -  smtpd -o content_filter=spamassassin
spamassassin
  unix  -  n  n  -  -  pipe
  flags=Rq user=spamd argv=/var/lib/spamassassin/filter.sh -oi -f 
${sender} ${recipient}


have you tried running spamass-milter?
I haven't tried it with postfix, but runs fine with sendmail, supporting
different users...
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Due to unexpected conditions Windows 2000 will be released
in first quarter of year 1901

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi @ everyone,

GOTCHA !

Finally, I found the solution myself: The issue is in the systemd 
spamassassin.service unit file of Arch Linux! This is how 
/usr/lib/systemd/system/spamassassin.service looks like:


[Unit]
Description=Spamassassin daemon
After=syslog.target network.target

[Service]
ExecStart=/usr/bin/vendor_perl/spamd -x -u spamd -g spamd
StandardOutput=null
StandardError=null
Restart=always

[Install]
WantedBy=multi-user.target

Looking at 
https://spamassassin.apache.org/full/3.0.x/dist/doc/spamd.html , it 
isn't clear what exactly "-x" is doing, since it is listed within one 
single line of two opposite clear-text options:


"""
-x, --nouser-config, --user-config
Turn off(on) reading of per-user configuration files (user_prefs) from 
the user's home directory. The default behaviour is to read per-user 
configuration from the user's home directory.

"""

So, -x could have meant both, to turn this on or off in my reading.

More clearly this is written in the manpage of spamd:

-x, --nouser-config   Disable user config files

Seems as if when -x is set, "allow_user_rules 1" neither has any effect, 
nor is a warning printed anywhere that there are opposite options in 
place or ignored, nor has this apeared in Debuging output.


I have solved this by

1. cp /usr/lib/systemd/system/spamassassin.service /etc/systemd/system/
2. Changed "ExecStart=" from "/usr/bin/vendor_perl/spamd -x -u spamd -g 
spamd" to "/usr/bin/vendor_perl/spamd -u spamd -g spamd"

3. systemctl daemon-reload
4. systemctl restart spamassassin

Now it works again like a charm, running spamd as spamd:spamd, and using 
spamc.


Thanks @ all for trying to help in this case! :)

Best regards,
Marc

Am 09.09.2015 um 08:46 schrieb Marc Richter:

Hi everyone,

I'm running SA 3.4.1 with Perl 5.22.0 .
It works quite well, but since a few weeks, it looks like my user_prefs
isn't taken into account by SA anymore. Let's show this by example:

There are *lots* of blacklist_from entries in there; one of them is:

blacklist_from  *@neuronation.*

Today, I got another mail with the following (relevant) headers:

X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on
 tango012.marc-richter.info
X-Spam-Level: ***
X-Spam-Status: No, score=3.6 required=4.0
tests=BAYES_99,BAYES_999,DKIM_SIGNED,
  DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,
  RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RP_MATCHES_RCVD,SPF_PASS,URIBL_BLOCKED
 autolearn=no autolearn_force=no version=3.4.1
From: NeuroNation 
Date: Wed, 09 Sep 2015 06:05:02 + (UTC)

Thus, this mail should get +100 for matching my blacklist_from entry.
But, as you can see, it isn't.

When I'm running "spamassassin --test-mode < my_maildir_file", I get
expected results:

spamassassin --test-mode < .maildir/cur/msg.SbGC\:2\,S

[...]
Inhaltsanalyse im Detail:   (99.9 Punkte, 3.0 ben�tigt)

Pkte Regelname  Beschreibung
 --
--
  0.0 URIBL_BLOCKED  ADMINISTRATOR NOTICE: The query to URIBL
was blocked.
 See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
  for more information.
 [URIs: neuronation.de]
-0.0 RCVD_IN_MSPIKE_H3  RBL: Good reputation (+3)
 [192.254.116.16 listed in wl.mailspike.net]
  100 USER_IN_BLACKLIST  From: address is in the user's black-list
  0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail
 domains are different
-0.0 SPF_PASS   SPF: Senderechner entspricht SPF-Datensatz
  0.0 RP_MATCHES_RCVDEnvelope sender domain matches handover
relay domain
  0.0 HTML_MESSAGE   BODY: Nachricht enth�lt HTML
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature
from author's
 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not
necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK
signature
-0.0 RCVD_IN_MSPIKE_WL  Mailspike good senders

SA is started by postfix; in the master.cf of postfix there are these
lines:

smtp  inet  n-n--smtpd -o
content_filter=spamassassin
spamassassin
 unix  -nn--pipe
 flags=Rq user=spamfilter argv=/home/spamfilter/filter.sh -oi -f
${sender} ${recipient}

/home/spamfilter/filter.sh contains:

#!/bin/sh
# filter.sh
#
# This script redirects mail flagged as spam to a separate account
# You must first create a user account named "spamvac" to hold the
flagged mail
SENDMAIL="/usr/sbin/sendmail -i"
SPAMASSASSIN=/usr/bin/vendor_perl/spamc
COMMAND="$SENDMAIL $@"
USER=`echo $COMMAND | awk '{ print $NF }' | sed 's/@.*$//'`
NEW_COMMAND=`echo $COMMAND | awk '{ $6 = "spamfilter"; NF = 6; print }'`
# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
umask 077

Re: SA doesn't respect my user_prefs

[Benny Pedersen]

Reindl Harald skrev den 2015-09-11 16:12:


spamd: cannot run as nonexistent user or root with -u option


spamd must not be startet with the -u option as root, the whole
purpose is to have the daemon process running as root and then "spamc"
is invoked with the -u param of the user which is target of the
incoming message


any daemons binding to port below 1024 must be started as root, knowing 
this could solve alot of problems on maillists :=)


but there was a dokument error on what -x do on spamd

note apache or postfix or dovecot droppriveleges to not serve daemonsd 
as root later, this is very easy to see in top, for apache that there is 
ONE apache running as root, but multiple apache not running as root


spamd does the same seen from spamc

and -u on spamd is not usefull if user_prefs is needed or enabled

Re: SA doesn't respect my user_prefs

[Reindl Harald]

Am 11.09.2015 um 16:53 schrieb Benny Pedersen:

Reindl Harald skrev den 2015-09-11 16:12:


spamd: cannot run as nonexistent user or root with -u option


spamd must not be startet with the -u option as root, the whole
purpose is to have the daemon process running as root and then "spamc"
is invoked with the -u param of the user which is target of the
incoming message


any daemons binding to port below 1024 must be started as root, knowing
this could solve alot of problems on maillists :=)

but there was a dokument error on what -x do on spamd

note apache or postfix or dovecot droppriveleges to not serve daemonsd
as root later, this is very easy to see in top, for apache that there is
ONE apache running as root, but multiple apache not running as root

spamd does the same seen from spamc

and -u on spamd is not usefull if user_prefs is needed or enabled


exactly what i said



signature.asc
Description: OpenPGP digital signature

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi RW,


When I issue "spamassassin --test-mode -D" as the user the filter.sh
- runs as, I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is
clear, because it cannot know what user the file "belongs" to, in
test-mode.


you can use -p or alternately set HOME


"spamc -h" brings ...

...
-p, --port port Specify port for connection to spamd.
  [default: 783]
...

What does this help in search of a debug mode?

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi @ all,

maybe I'm doing it wrong here - I do not insist on being unfailable.
But what's the correct way to do it then?

Best regards,
Marc

Am 10.09.2015 um 01:48 schrieb RW:

On Wed, 9 Sep 2015 14:48:14 -0700
jdow wrote:


On 2015-09-09 13:51, RW wrote:

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:


Hi RW,


Do you mean that ww is a unix user? The normal way to do this is
to run spamd as root and run spamc as the unix user. Passing -u to
spamc is really intended for virtual users, I'm not sure whether
it works for unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.


Supporting that sounds like a really bad idea. It would mean that
any user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege
escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.


Why don't you try to run spamc -u root as a common user and see what
happens then talk about the results if it is warranted?



Given that it doesn't appear to be currently working with non-root
accounts, what would that prove? And it's still wrong even if root is a
special case.

Re: SA doesn't respect my user_prefs

[jdow]

I presume you restarted spamd, right?

{^_^}

On 2015-09-08 23:46, Marc Richter wrote:

Hi everyone,

I'm running SA 3.4.1 with Perl 5.22.0 .
It works quite well, but since a few weeks, it looks like my user_prefs isn't
taken into account by SA anymore. Let's show this by example:

There are *lots* of blacklist_from entries in there; one of them is:

blacklist_from  *@neuronation.*

Today, I got another mail with the following (relevant) headers:

X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on
 tango012.marc-richter.info
X-Spam-Level: ***
X-Spam-Status: No, score=3.6 required=4.0 tests=BAYES_99,BAYES_999,DKIM_SIGNED,
  DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,
  RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,RP_MATCHES_RCVD,SPF_PASS,URIBL_BLOCKED
 autolearn=no autolearn_force=no version=3.4.1
From: NeuroNation 
Date: Wed, 09 Sep 2015 06:05:02 + (UTC)

Thus, this mail should get +100 for matching my blacklist_from entry. But, as
you can see, it isn't.

When I'm running "spamassassin --test-mode < my_maildir_file", I get expected
results:

spamassassin --test-mode < .maildir/cur/msg.SbGC\:2\,S

[...]
Inhaltsanalyse im Detail:   (99.9 Punkte, 3.0 ben�tigt)

Pkte Regelname  Beschreibung
 -- --
  0.0 URIBL_BLOCKED  ADMINISTRATOR NOTICE: The query to URIBL was 
blocked.
 See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
  for more information.
 [URIs: neuronation.de]
-0.0 RCVD_IN_MSPIKE_H3  RBL: Good reputation (+3)
 [192.254.116.16 listed in wl.mailspike.net]
  100 USER_IN_BLACKLIST  From: address is in the user's black-list
  0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail
 domains are different
-0.0 SPF_PASS   SPF: Senderechner entspricht SPF-Datensatz
  0.0 RP_MATCHES_RCVDEnvelope sender domain matches handover relay 
domain
  0.0 HTML_MESSAGE   BODY: Nachricht enth�lt HTML
-0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature from 
author's
 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not necessarily
valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
-0.0 RCVD_IN_MSPIKE_WL  Mailspike good senders

SA is started by postfix; in the master.cf of postfix there are these lines:

smtp  inet  n-n--smtpd -o content_filter=spamassassin
spamassassin
 unix  -nn--pipe
 flags=Rq user=spamfilter argv=/home/spamfilter/filter.sh -oi -f ${sender}
${recipient}

/home/spamfilter/filter.sh contains:

#!/bin/sh
# filter.sh
#
# This script redirects mail flagged as spam to a separate account
# You must first create a user account named "spamvac" to hold the flagged mail
SENDMAIL="/usr/sbin/sendmail -i"
SPAMASSASSIN=/usr/bin/vendor_perl/spamc
COMMAND="$SENDMAIL $@"
USER=`echo $COMMAND | awk '{ print $NF }' | sed 's/@.*$//'`
NEW_COMMAND=`echo $COMMAND | awk '{ $6 = "spamfilter"; NF = 6; print }'`
# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69
umask 077
OUTPUT="`mktemp /tmp/mailfilter.XX`"
if [ "$?" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Unable to create
temporary file."
 exit $EX_TEMPFAIL
fi
# Clean up when done or when aborting.
trap "rm -f $OUTPUT" EXIT SIGTERM
$SPAMASSASSIN -x -E -u $USER > $OUTPUT
return="$?"
if [ "$return" == 1 ]; then
 $NEW_COMMAND < $OUTPUT
 exit $?
elif [ "$return" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Temporary SpamAssassin
failure (spamc return $return)"
 exit $EX_TEMPFAIL
fi
$SENDMAIL "$@" < $OUTPUT
exit $?

SA should have access to my user_prefs; these are the groups for the user
"spamfilter":
tango012 ~ # groups spamfilter
users spamd
tango012 ~ #

The full path-permission to my user_prefs are:
ww@tango012 ~ $ ls -ld /home /home/Whitewolf_Fox
/home/Whitewolf_Fox/.spamassassin /home/Whitewolf_Fox/.spamassassin/user_prefs
drwxr-xr-x 13 root root  4096 23. Jul 10:36 /home
drwxr-xr-x 27 ww   users 4096  9. Sep 08:32 /home/Whitewolf_Fox
drwxrwx---  2 ww   spamd 4096  9. Sep 08:32 /home/Whitewolf_Fox/.spamassassin
-rw-rw  1 ww   spamd 8622  4. Sep 15:15
/home/Whitewolf_Fox/.spamassassin/user_prefs
ww@tango012 ~ $

Standing here, I'm out of ideas, since this looks all good to me.

Can somebody imagine what's wrong here?

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[Matus UHLAR - fantomas]

On 09.09.15 01:16, jdow wrote:

I presume you restarted spamd, right?


restarting spamd should not be needed for changes in user_prefs, should it?


On 2015-09-08 23:46, Marc Richter wrote:

I'm running SA 3.4.1 with Perl 5.22.0 .
It works quite well, but since a few weeks, it looks like my user_prefs isn't
taken into account by SA anymore. Let's show this by example:

There are *lots* of blacklist_from entries in there; one of them is:

blacklist_from  *@neuronation.*


have you tried running spamassassin -D ? maybe there's somethign invalid in
SA's configuration or your user_prefs

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Boost your system's speed by 500% - DEL C:\WINDOWS\*.*

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens 
regularly by cron (all 3 hours), along with other jobs like sa-learn, 
sa-update and sa-compile.


> On 09.09.2015 11:12 Matus wrote:
>
> have you tried running spamassassin -D ? maybe there's somethign
> invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh - 
runs as, I get this in the long output:


dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is 
clear, because it cannot know what user the file "belongs" to, in test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs 
to, it works, so no use in that.


How can I make use of the "-D" cmdline option in the normal mail-flow in 
a way it gets logged by journald? Can I simply add "-D" to the filter.sh 
script and it get's caught in journald's database?


How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[Marc Richter]

PS:

I just did the following test:

As the user, filter.sh is executed as, I did test the following:

1. /usr/bin/vendor_perl/spamc -x -E -u ww < /tmp/spam

As the user, who owns the user_prefs, I did test the following:

2. /usr/bin/vendor_perl/spamc -x -E < /tmp/spam
3. spamassassin --test-mode -D < /tmp/spam

Unfortunately, spamc seems to not have a verbosity trigger ... so I can 
only judge on the results:


1. This is, how the filter.sh issues the command. This brings the same 
result like 2.


2. This brings the same result, I see in my Inbox: The mail gets 
processed, but ~ww/.spamassassin/user_prefs seems to be ignored; the 
mail only gets a score of 3.8.


3. When I execute the "spamassassin" - program, everything looks as if 
it gets processed correctly: the user_prefs is read, the mail gets a 
score of 100.1 and is considered spam.


So, does this mean I should switch to use "spamassassin" instead of 
"spamc" in my filter.sh script instead? Manpage of spamc reads:


"""
Spamc is the client half of the spamc/spamd pair.  It should be used in 
place of "spamassassin" in scripts to process mail.

[...]
Spamc has extremely low overhead in loading, so it should be much faster 
to load than the whole spamassassin program.

"""

- What is the "whole spamassassin program"; are there features missing 
in spamc (like respecting user_prefs file)?


- Is it wise to use spamassassin when the developers intend spamc to be 
used for this purpose?


- How do I get spamc to respect user_prefs file?

Best regards,
Marc

Am 09.09.2015 um 13:47 schrieb Marc Richter:

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens
regularly by cron (all 3 hours), along with other jobs like sa-learn,
sa-update and sa-compile.

 > On 09.09.2015 11:12 Matus wrote:
 >
 > have you tried running spamassassin -D ? maybe there's somethign
 > invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh -
runs as, I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is
clear, because it cannot know what user the file "belongs" to, in
test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs
to, it works, so no use in that.

How can I make use of the "-D" cmdline option in the normal mail-flow in
a way it gets logged by journald? Can I simply add "-D" to the filter.sh
script and it get's caught in journald's database?

How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi Matus,

Am 09.09.2015 um 15:01 schrieb Matus UHLAR - fantomas:

On 09.09.15 13:47, Marc Richter wrote:

Regardless if it's necessary or not, I have done so. It also happens
regularly by cron (all 3 hours), along with other jobs like sa-learn,
sa-update and sa-compile.


reload should be enough, restart is rarely necessary.
Also, why do you check oftern than once a day?


You are right, this can be made more clever. I made it that way when I 
started with my server to make sure the frequently changed configs are 
reloaded often automatically. I left it that way since then.
It can be optimized for sure, but I doubt it has something to do with my 
actual troubles.



how do you plug spamassassin into your mail flow? How do you call
spamassassin? mta, mail client ... ?


I'm running postfix as my MTA. In it's master.cf there is configured to 
pipe my mail through a script:


smtp   inet  n  -  n  -  -  smtpd -o content_filter=spamassassin
spamassassin
   unix  -  n  n  -  -  pipe
   flags=Rq user=spamd argv=/var/lib/spamassassin/filter.sh -oi -f 
${sender} ${recipient}


In the script filter.sh spamc is invoked:

#!/bin/sh

SENDMAIL="/usr/sbin/sendmail -i"
SPAMASSASSIN=/usr/bin/vendor_perl/spamc
COMMAND="$SENDMAIL $@"
USER=`echo $COMMAND | awk '{ print $NF }' | sed 's/@.*$//'`
NEW_COMMAND=`echo $COMMAND | awk '{ $6 = "spamfilter"; NF = 6; print }'`

# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69

umask 077

OUTPUT="`mktemp /tmp/mailfilter.XX`"

if [ "$?" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Unable to create 
temporary file."

 exit $EX_TEMPFAIL
fi

# Clean up when done or when aborting.
trap "rm -f $OUTPUT" EXIT SIGTERM

$SPAMASSASSIN -x -E -u $USER > $OUTPUT
return="$?"
if [ "$return" == 1 ]; then
 $NEW_COMMAND < $OUTPUT
 exit $?
elif [ "$return" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Temporary SpamAssassin 
failure (spamc return $return)"

 exit $EX_TEMPFAIL
fi

$SENDMAIL "$@" < $OUTPUT
exit $?


Thus, in effect, mail is filtered as this command would have been executed:

spamc -x -E -u ww < MAILINPUT

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[Matus UHLAR - fantomas]

On 09.09.15 13:47, Marc Richter wrote:
Regardless if it's necessary or not, I have done so. It also happens 
regularly by cron (all 3 hours), along with other jobs like sa-learn, 
sa-update and sa-compile.


reload should be enough, restart is rarely necessary.
Also, why do you check oftern than once a day?


On 09.09.2015 11:12 Matus wrote:

have you tried running spamassassin -D ? maybe there's somethign
invalid in SA's configuration or your user_prefs


When I issue "spamassassin --test-mode -D" as the user the filter.sh 
- runs as, I get this in the long output:


dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is 
clear, because it cannot know what user the file "belongs" to, in 
test-mode.
When I run that as the user (ww) the mail and desired user_prefs 
belongs to, it works, so no use in that.


How can I make use of the "-D" cmdline option in the normal mail-flow 
in a way it gets logged by journald? Can I simply add "-D" to the 
filter.sh script and it get's caught in journald's database?


how do you plug spamassassin into your mail flow? How do you call
spamassassin? mta, mail client ... ?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.

Re: SA doesn't respect my user_prefs

[RW]

On Wed, 9 Sep 2015 13:47:01 +0200
Marc Richter wrote:


>  > On 09.09.2015 11:12 Matus wrote:
>  >
>  > have you tried running spamassassin -D ? maybe there's somethign
>  > invalid in SA's configuration or your user_prefs
> 
> When I issue "spamassassin --test-mode -D" as the user the filter.sh
> - runs as, I get this in the long output:
> 
> dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs
> 
> So, it tries to read the user_prefs from the daemon's home, what is 
> clear, because it cannot know what user the file "belongs" to, in
> test-mode.

you can use -p or alternately set HOME 

>  When I run that as the user (ww) the mail and desired
> user_prefs belongs to, it works, so no use in that.

Do you mean that ww is a unix user? The normal way to do this is to run
spamd as root and run spamc as the unix user. Passing -u to spamc is
really intended for virtual users, I'm not sure whether it works for
unix users.  Are you sure it worked before?

Re: SA doesn't respect my user_prefs

[Marc Richter]

Hi RW,


Do you mean that ww is a unix user? The normal way to do this is to run
spamd as root and run spamc as the unix user. Passing -u to spamc is
really intended for virtual users, I'm not sure whether it works for
unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.

When I issue spamc without the -u trigger as user ww, I get the same 
(wrong) results.


Best regards,
Marc

Re: SA doesn't respect my user_prefs

[jdow]

Is this line in your "local.cf" file? (And is it in the correct place?)

allow_user_rules 1

{^_^}

On 2015-09-09 04:47, Marc Richter wrote:

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens regularly
by cron (all 3 hours), along with other jobs like sa-learn, sa-update and
sa-compile.

 > On 09.09.2015 11:12 Matus wrote:
 >
 > have you tried running spamassassin -D ? maybe there's somethign
 > invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh - runs as,
I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is clear,
because it cannot know what user the file "belongs" to, in test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs to, it
works, so no use in that.

How can I make use of the "-D" cmdline option in the normal mail-flow in a way
it gets logged by journald? Can I simply add "-D" to the filter.sh script and it
get's caught in journald's database?

How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA doesn't respect my user_prefs

[RW]

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:

> Hi RW,
> 
> > Do you mean that ww is a unix user? The normal way to do this is to
> > run spamd as root and run spamc as the unix user. Passing -u to
> > spamc is really intended for virtual users, I'm not sure whether it
> > works for unix users.  Are you sure it worked before?
> 
> ww is a unix user, yes. And it worked before, yes.

Supporting that sounds like a really bad idea. It would mean that any
user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.   

Re: SA doesn't respect my user_prefs

[jdow]

On 2015-09-09 13:51, RW wrote:

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:


Hi RW,


Do you mean that ww is a unix user? The normal way to do this is to
run spamd as root and run spamc as the unix user. Passing -u to
spamc is really intended for virtual users, I'm not sure whether it
works for unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.


Supporting that sounds like a really bad idea. It would mean that any
user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.


Why don't you try to run spamc -u root as a common user and see what happens 
then talk about the results if it is warranted?


{o.o}