date:20150909


Hello,

Can you help me with please?
One of our SA installation has gone mad since a few days:

-it is filtering mails below required points:

Content analysis details: (12.5 points, 15.0 required)
Content analysis details: (10.7 points, 15.0 required)
...

-it has been filtering bad words but with manual regex check there is no 
match


The system was put together years ago, the config was not changed for 
least 6 months, apt-update was not affecting perl or SA, and then it 
suddenly gone mad from one day to another


Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6

KR, Zsolt

Re: SA gone mad



Am 09.09.2015 um 10:56 schrieb Farkas Zsolt:

Can you help me with please?
One of our SA installation has gone mad since a few days:

-it is filtering mails below required points:

Content analysis details: (12.5 points, 15.0 required)
Content analysis details: (10.7 points, 15.0 required)


15.0 required - seriously?

normally thats 5.0 and above 8.0 with a well trained bayes you can 
typically reject without a noticeable FP risk



-it has been filtering bad words but with manual regex check there is no
match


you don't provide any informations


The system was put together years ago, the config was not changed for
least 6 months, apt-update was not affecting perl or SA, and then it
suddenly gone mad from one day to another

Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6


very outdated



signature.asc
Description: OpenPGP digital signature

Re: SA doesn't respect my user_prefs


On 09.09.15 01:16, jdow wrote:

I presume you restarted spamd, right?


restarting spamd should not be needed for changes in user_prefs, should it?


On 2015-09-08 23:46, Marc Richter wrote:

I'm running SA 3.4.1 with Perl 5.22.0 .
It works quite well, but since a few weeks, it looks like my user_prefs isn't
taken into account by SA anymore. Let's show this by example:

There are *lots* of blacklist_from entries in there; one of them is:

blacklist_from  *@neuronation.*


have you tried running spamassassin -D ? maybe there's somethign invalid in
SA's configuration or your user_prefs

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Boost your system's speed by 500% - DEL C:\WINDOWS\*.*

Re: SA gone mad


2015.09.09. 11:02 keltezéssel, Reindl Harald írta:


Am 09.09.2015 um 10:56 schrieb Farkas Zsolt:

Can you help me with please?
One of our SA installation has gone mad since a few days:

-it is filtering mails below required points:

Content analysis details: (12.5 points, 15.0 required)
Content analysis details: (10.7 points, 15.0 required)


15.0 required - seriously?

normally thats 5.0 and above 8.0 with a well trained bayes you can 
typically reject without a noticeable FP risk



it was 4.9 but after the stange behavior it was raised for 15.
if I set back to 4.9 it will give lot of false positive

-it has been filtering bad words but with manual regex check there is no
match


you don't provide any informations


it gave 10 point for badword rule but there was no match

The system was put together years ago, the config was not changed for
least 6 months, apt-update was not affecting perl or SA, and then it
suddenly gone mad from one day to another

Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6


very outdated


this is included in debian 6 LTS

Re: SA gone mad


Am 09.09.2015 um 10:56 schrieb Farkas Zsolt:

One of our SA installation has gone mad since a few days:



Content analysis details: (12.5 points, 15.0 required)
Content analysis details: (10.7 points, 15.0 required)



2015.09.09. 11:02 keltezéssel, Reindl Harald írta:

15.0 required - seriously?


On 09.09.15 11:20, Farkas Zsolt wrote:

it was 4.9 but after the stange behavior it was raised for 15.
if I set back to 4.9 it will give lot of false positive


could you show us which rules hit at some false positives?
playing with required_score is usually not very good idea...


-it has been filtering bad words but with manual regex check there is no
match



you don't provide any informations



it gave 10 point for badword rule but there was no match


... just the same as scoring 10 points for single rules.
They might match where you don't see it.

using properly trained BAYES and network rules should make SA behave much
better than banned words rules.


Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6


very outdated


this is included in debian 6 LTS


it should still be better to upgrade to more current versions.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"The box said 'Requires Windows 95 or better', so I bought a Macintosh".

Re: SA gone mad




Am 09.09.2015 um 11:20 schrieb Farkas Zsolt:

2015.09.09. 11:02 keltezéssel, Reindl Harald írta:


Am 09.09.2015 um 10:56 schrieb Farkas Zsolt:

Can you help me with please?
One of our SA installation has gone mad since a few days:

-it is filtering mails below required points:

Content analysis details: (12.5 points, 15.0 required)
Content analysis details: (10.7 points, 15.0 required)


15.0 required - seriously?

normally thats 5.0 and above 8.0 with a well trained bayes you can
typically reject without a noticeable FP risk


it was 4.9 but after the stange behavior it was raised for 15.
if I set back to 4.9 it will give lot of false positive


your first post sounded like spam is coming through and not FP's


-it has been filtering bad words but with manual regex check there is no
match


you don't provide any informations


it gave 10 point for badword rule but there was no match


that are *not* useful informations

http://www.catb.org/esr/faqs/smart-questions.html#beprecise


The system was put together years ago, the config was not changed for
least 6 months, apt-update was not affecting perl or SA, and then it
suddenly gone mad from one day to another

Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6


very outdated


this is included in debian 6 LTS


nayways, outdated



signature.asc
Description: OpenPGP digital signature

Re: SA gone mad




your first post sounded like spam is coming through and not FP's

I mean: lot of emails were  marked as SPAM when the limit was at 4.9, 
and it started from one day to another without config or system change

now the limit is at 15 but it sill filters mais below 15 without a reason
so arte they FPs or what? or mistreated emails?

-it has been filtering bad words but with manual regex check there 
is no

match


you don't provide any informations


it gave 10 point for badword rule but there was no match



shell I send you the badwords rule and the false marked mails?


that are *not* useful informations

http://www.catb.org/esr/faqs/smart-questions.html#beprecise


The system was put together years ago, the config was not changed for
least 6 months, apt-update was not affecting perl or SA, and then it
suddenly gone mad from one day to another

Debian 6.0.10
Samassassin 3.3.2-5+deb7u2
perl 5.10.1-17squeeze6


very outdated


this is included in debian 6 LTS


nayways, outdated

Re: SA gone mad




Am 09.09.2015 um 12:15 schrieb Farkas Zsolt:



your first post sounded like spam is coming through and not FP's


I mean: lot of emails were  marked as SPAM when the limit was at 4.9,
and it started from one day to another without config or system change
now the limit is at 15 but it sill filters mais below 15 without a reason
so arte they FPs or what? or mistreated emails?


-it has been filtering bad words but with manual regex check there
is no
match


you don't provide any informations


it gave 10 point for badword rule but there was no match



shell I send you the badwords rule and the false marked mails?


what about starting with the log-entries showing the matched rules from 
some of that messages from the very first start? "it don't work as i 
expect" is not enough and the only correct answer would be "i am sorry 
for you"



that are *not* useful informations

http://www.catb.org/esr/faqs/smart-questions.html#beprecise




signature.asc
Description: OpenPGP digital signature

Re: SA gone mad



what about starting with the log-entries showing the matched rules 
from some of that messages from the very first start? "it don't work 
as i expect" is not enough and the only correct answer would be "i am 
sorry for you"




it has been done at the start, but there is no visible reason for the 
abnormal behavior: limit is 15 but it is filtered with 13.1


Content type: Spam
Internal reference code for the message is 12522-01/XdmY-Z7h6a4o

According to a 'Received:' trace, the message apparently originated at:
  [96.100.228.198], unknown HELO localhost
  mh...@lnainsurance.com@96.100.228.198

Return-Path: <>
From:mh...@lnainsurance.com
Subject: Are you ready to impress your beloved one at night?
The message has been quarantined as: spam-XdmY-Z7h6a4o.gz

The message WAS NOT relayed to:
:
   554 5.7.0 Bounce, id=12522-01 - spam

Spam scanner report:
Spam detection software, running on the system "server5.xx", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see
@@CONTACT_ADDRESS@@ for details.

Content preview:  Please your wife like never 
beforehttp://anger.xn--d1ahtf1cujf.xn--p1ai/
   [...]

Content analysis details:   (13.1 points, 15.0 required)

 pts rule name  description
 -- --
 3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
[score: 1.]
 2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
 0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
 0.0 TVD_RCVD_IPTVD_RCVD_IP
 1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for HELO
 1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
[113.167.11.149 listed in bb.barracudacentral.org]
 0.5 MISSING_MIDMissing Message-Id: header
 0.8 RDNS_NONE  Delivered to internal network by a host with no rDNS
 1.4 MISSING_DATE   Missing Date: header
 2.0 FROM_12LTRDOM  From a 12-letter domain
 0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS


header


Return-Path: <>
X-Greylist: delayed 281 seconds by postgrey-1.34 at server5; Wed, 09 Sep 2015 
11:43:16 CEST
Received: from 113.167.11.149 (unknown [113.167.11.149])
by server5.xxx (Postfix) with SMTP id 849BA264029
for; Wed,  9 Sep 2015 11:43:16 +0200 (CEST)
Received: from unknown (HELO localhost) (mh...@lnainsurance.com@96.100.228.198)
by 113.167.11.149 with ESMTPA; Wed, 9 Sep 2015 16:41:05 +0700
From:mh...@lnainsurance.com
To:sgauvin@xx
Subject: Are you ready to impress your beloved one at night?

Re: SA gone mad

On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:

> Content analysis details:   (13.1 points, 15.0 required)
> 
>   pts rule name  description
>  --
> -- 3.5 BAYES_99   
>BODY: Bayes spam probability is 99 to 100% [score: 1.]
>   2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
>   0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
>   0.0 TVD_RCVD_IPTVD_RCVD_IP
>   1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for HELO
>   1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
>  [113.167.11.149 listed in
> bb.barracudacentral.org] 0.5 MISSING_MIDMissing Message-Id:
> header
>   0.8 RDNS_NONE  Delivered to internal network by a host with
> no rDNS 1.4 MISSING_DATE   Missing Date: header
>   2.0 FROM_12LTRDOM  From a 12-letter domain
>   0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS
> header

You said earlier "it gave 10 point for badword rule but there was no match".

I see no badword rule match getting 10 points in what you posted above.

Are you telling us about the same message each time you post bits of 
information?


Antoy.

-- 
"A person lives in the UK, but commutes to France daily for work.
He belongs in the UK."

 - From UK Revenue & Customs notice 741, page 13, paragraph 3.5.1
 - http://tinyurl.com/o7gnm4

   Please reply to the list;
 please *don't* CC me.

Re: SA gone mad


Hello,

what about starting with the log-entries showing the matched rules 
from some of that messages from the very first start? "it don't 
work as i expect" is not enough and the only correct answer would 
be "i am sorry for you"


On 09.09.15 13:08, Farkas Zsolt wrote:
it has been done at the start, but there is no visible reason for the 
abnormal behavior: limit is 15 but it is filtered with 13.1



Content analysis details:   (13.1 points, 15.0 required)

pts rule name  description
 -- --
3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
   [score: 1.]
2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
0.0 TVD_RCVD_IPTVD_RCVD_IP
1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for HELO
1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
   [113.167.11.149 listed in bb.barracudacentral.org]
0.5 MISSING_MIDMissing Message-Id: header
0.8 RDNS_NONE  Delivered to internal network by a host with no rDNS
1.4 MISSING_DATE   Missing Date: header
2.0 FROM_12LTRDOM  From a 12-letter domain
0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS



Return-Path: <>
X-Greylist: delayed 281 seconds by postgrey-1.34 at server5; Wed, 09 Sep 2015 
11:43:16 CEST
Received: from 113.167.11.149 (unknown [113.167.11.149])
by server5.xxx (Postfix) with SMTP id 849BA264029
for; Wed,  9 Sep 2015 11:43:16 +0200 (CEST)
Received: from unknown (HELO localhost) (mh...@lnainsurance.com@96.100.228.198)
by 113.167.11.149 with ESMTPA; Wed, 9 Sep 2015 16:41:05 +0700
From:mh...@lnainsurance.com
To:sgauvin@xx
Subject: Are you ready to impress your beloved one at night?


This looks like mail sent through your mailseerver to outside: numeric helo,
no rDNS, missing Date and Message-Id headers...

or, may tghe header be somehow broken? Could you post whole message, or at
least all headers and part of the textual body to pastebin?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]

Re: SA gone mad



badword rule is now disabled, SA running with limited functionality, 
higher limit (15 insted of 4.9), bayes disabled


there are two problems now:
-messages marked as SPAM but they are below limit (limit at 15 but at 
13.1 marked as spam) this has no connection with disabled or enabled badword
-if badword rule is enabled badword points added but there is no badword 
match in regex




2015.09.09. 13:14 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:


Content analysis details:   (13.1 points, 15.0 required)

   pts rule name  description
 --
-- 3.5 BAYES_99
BODY: Bayes spam probability is 99 to 100% [score: 1.]
   2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
   0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
   0.0 TVD_RCVD_IPTVD_RCVD_IP
   1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for HELO
   1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
  [113.167.11.149 listed in
bb.barracudacentral.org] 0.5 MISSING_MIDMissing Message-Id:
header
   0.8 RDNS_NONE  Delivered to internal network by a host with
no rDNS 1.4 MISSING_DATE   Missing Date: header
   2.0 FROM_12LTRDOM  From a 12-letter domain
   0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS
header

You said earlier "it gave 10 point for badword rule but there was no match".

I see no badword rule match getting 10 points in what you posted above.

Are you telling us about the same message each time you post bits of
information?


Antoy.

Re: SA doesn't respect my user_prefs

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens 
regularly by cron (all 3 hours), along with other jobs like sa-learn, 
sa-update and sa-compile.

> On 09.09.2015 11:12 Matus wrote:
>
> have you tried running spamassassin -D ? maybe there's somethign
> invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh - 
runs as, I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is 
clear, because it cannot know what user the file "belongs" to, in test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs 
to, it works, so no use in that.

How can I make use of the "-D" cmdline option in the normal mail-flow in 
a way it gets logged by journald? Can I simply add "-D" to the filter.sh 
script and it get's caught in journald's database?

How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA gone mad

On Wednesday 09 September 2015 at 13:42:31, Farkas Zsolt wrote:

> badword rule is now disabled, SA running with limited functionality,
> higher limit (15 insted of 4.9), bayes disabled

How have you disabled Bayes?  The message below has 3.5 points for Bayes_99


Antony.

> there are two problems now:
> -messages marked as SPAM but they are below limit (limit at 15 but at
> 13.1 marked as spam) this has no connection with disabled or enabled
> badword -if badword rule is enabled badword points added but there is no
> badword match in regex
> 
> 2015.09.09. 13:14 keltezéssel, Antony Stone írta:
> > On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:
> >> Content analysis details:   (13.1 points, 15.0 required)
> >> 
> >>pts rule name  description
> >> 
> >>  --
> >> -- 3.5 BAYES_99
> >> 
> >> BODY: Bayes spam probability is 99 to 100% [score: 1.]
> >>
> >>2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
> >>0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
> >>0.0 TVD_RCVD_IPTVD_RCVD_IP
> >>1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for
> >>HELO 1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
> >>
> >>   [113.167.11.149 listed in
> >> 
> >> bb.barracudacentral.org] 0.5 MISSING_MIDMissing Message-Id:
> >> header
> >> 
> >>0.8 RDNS_NONE  Delivered to internal network by a host
> >>with
> >> 
> >> no rDNS 1.4 MISSING_DATE   Missing Date: header
> >> 
> >>2.0 FROM_12LTRDOM  From a 12-letter domain
> >>0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS
> >> 
> >> header
> > 
> > You said earlier "it gave 10 point for badword rule but there was no
> > match".
> > 
> > I see no badword rule match getting 10 points in what you posted above.
> > 
> > Are you telling us about the same message each time you post bits of
> > information?
> > 
> > 
> > Antoy.

-- 
There are infinitely many other ways to set up an alternative universe: they 
range from having 101 dimensions and totally different laws to being identical 
with our universe except for 6 atoms of dysprosium in the core of the star 
Procyon that change into iodine on Thursdays.

   Please reply to the list;
 please *don't* CC me.

Re: SA gone mad


This was the email what i receive from SA for spamadmin email account
I am sending the links in few minutes


2015.09.09. 13:42 keltezéssel, Matus UHLAR - fantomas írta:

Hello,

what about starting with the log-entries showing the matched rules 
from some of that messages from the very first start? "it don't work 
as i expect" is not enough and the only correct answer would be "i 
am sorry for you"


On 09.09.15 13:08, Farkas Zsolt wrote:
it has been done at the start, but there is no visible reason for the 
abnormal behavior: limit is 15 but it is filtered with 13.1



Content analysis details:   (13.1 points, 15.0 required)

pts rule name  description
 -- 
--

3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
   [score: 1.]
2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
0.0 TVD_RCVD_IPTVD_RCVD_IP
1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for 
HELO

1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
   [113.167.11.149 listed in 
bb.barracudacentral.org]

0.5 MISSING_MIDMissing Message-Id: header
0.8 RDNS_NONE  Delivered to internal network by a host 
with no rDNS

1.4 MISSING_DATE   Missing Date: header
2.0 FROM_12LTRDOM  From a 12-letter domain
0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS



Return-Path: <>
X-Greylist: delayed 281 seconds by postgrey-1.34 at server5; Wed, 09 
Sep 2015 11:43:16 CEST

Received: from 113.167.11.149 (unknown [113.167.11.149])
by server5.xxx (Postfix) with SMTP id 849BA264029
for; Wed,  9 Sep 2015 11:43:16 +0200 (CEST)
Received: from unknown (HELO localhost) 
(mh...@lnainsurance.com@96.100.228.198)

by 113.167.11.149 with ESMTPA; Wed, 9 Sep 2015 16:41:05 +0700
From:mh...@lnainsurance.com
To:sgauvin@xx
Subject: Are you ready to impress your beloved one at night?


This looks like mail sent through your mailseerver to outside: numeric 
helo,

no rDNS, missing Date and Message-Id headers...

or, may tghe header be somehow broken? Could you post whole message, 
or at

least all headers and part of the textual body to pastebin?

Re: SA gone mad

On Wed, 9 Sep 2015 13:14:04 +0200
Antony Stone wrote:

> On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:
> 
> > Content analysis details:   (13.1 points, 15.0 required)
> > 

> You said earlier "it gave 10 point for badword rule but there was no
> match".
> 
> I see no badword rule match getting 10 points in what you posted
> above.
> 
> Are you telling us about the same message each time you post bits of 
> information?

The details of the rules are a distraction. The actual problem is that
mail is being classified as spam despite having a score under the
configured threshold.

The first thing to try is to reproduce it on the command line using
spamassassin -D , using the original emails from the reports. If that
doesn't work it's probably a problem with the glue, which hasn't been
mentioned yet.

Re: SA gone mad


you made a point, but /etc/spamassassin/local.cf looks like:

...
#   Use Bayesian classifier (default: 1)
#
#use_bayes 1


#   Bayesian classifier auto-learning (default: 1)
#
#bayes_auto_learn 1
...

KR, Zsolt

2015.09.09. 13:49 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:42:31, Farkas Zsolt wrote:


badword rule is now disabled, SA running with limited functionality,
higher limit (15 insted of 4.9), bayes disabled

How have you disabled Bayes?  The message below has 3.5 points for Bayes_99


Antony.


there are two problems now:
-messages marked as SPAM but they are below limit (limit at 15 but at
13.1 marked as spam) this has no connection with disabled or enabled
badword -if badword rule is enabled badword points added but there is no
badword match in regex

2015.09.09. 13:14 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:

Content analysis details:   (13.1 points, 15.0 required)

pts rule name  description

 --
-- 3.5 BAYES_99

 BODY: Bayes spam probability is 99 to 100% [score: 1.]

2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1

0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
0.0 TVD_RCVD_IPTVD_RCVD_IP
1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for
HELO 1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT

   [113.167.11.149 listed in


bb.barracudacentral.org] 0.5 MISSING_MIDMissing Message-Id:
header

0.8 RDNS_NONE  Delivered to internal network by a host
with

no rDNS 1.4 MISSING_DATE   Missing Date: header

2.0 FROM_12LTRDOM  From a 12-letter domain
0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS

header

You said earlier "it gave 10 point for badword rule but there was no
match".

I see no badword rule match getting 10 points in what you posted above.

Are you telling us about the same message each time you post bits of
information?


Antoy.

Re: SA gone mad


Hello,

please find the original incoming email saved by SA:
http://pastebin.com/mszg5nJe

the email notification received back from SA on spamadmin account
http://pastebin.com/0UTvT9tQ

KR, Zsolt

2015.09.09. 13:42 keltezéssel, Matus UHLAR - fantomas írta:

Hello,

what about starting with the log-entries showing the matched rules 
from some of that messages from the very first start? "it don't work 
as i expect" is not enough and the only correct answer would be "i 
am sorry for you"


On 09.09.15 13:08, Farkas Zsolt wrote:
it has been done at the start, but there is no visible reason for the 
abnormal behavior: limit is 15 but it is filtered with 13.1



Content analysis details:   (13.1 points, 15.0 required)

pts rule name  description
 -- 
--

3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
   [score: 1.]
2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
0.0 TVD_RCVD_IPTVD_RCVD_IP
1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for 
HELO

1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
   [113.167.11.149 listed in 
bb.barracudacentral.org]

0.5 MISSING_MIDMissing Message-Id: header
0.8 RDNS_NONE  Delivered to internal network by a host 
with no rDNS

1.4 MISSING_DATE   Missing Date: header
2.0 FROM_12LTRDOM  From a 12-letter domain
0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS



Return-Path: <>
X-Greylist: delayed 281 seconds by postgrey-1.34 at server5; Wed, 09 
Sep 2015 11:43:16 CEST

Received: from 113.167.11.149 (unknown [113.167.11.149])
by server5.xxx (Postfix) with SMTP id 849BA264029
for; Wed,  9 Sep 2015 11:43:16 +0200 (CEST)
Received: from unknown (HELO localhost) 
(mh...@lnainsurance.com@96.100.228.198)

by 113.167.11.149 with ESMTPA; Wed, 9 Sep 2015 16:41:05 +0700
From:mh...@lnainsurance.com
To:sgauvin@xx
Subject: Are you ready to impress your beloved one at night?


This looks like mail sent through your mailseerver to outside: numeric 
helo,

no rDNS, missing Date and Message-Id headers...

or, may tghe header be somehow broken? Could you post whole message, 
or at

least all headers and part of the textual body to pastebin?

Re: SA doesn't respect my user_prefs

PS:

I just did the following test:

As the user, filter.sh is executed as, I did test the following:

1. /usr/bin/vendor_perl/spamc -x -E -u ww < /tmp/spam

As the user, who owns the user_prefs, I did test the following:

2. /usr/bin/vendor_perl/spamc -x -E < /tmp/spam
3. spamassassin --test-mode -D < /tmp/spam

Unfortunately, spamc seems to not have a verbosity trigger ... so I can 
only judge on the results:

1. This is, how the filter.sh issues the command. This brings the same 
result like 2.

2. This brings the same result, I see in my Inbox: The mail gets 
processed, but ~ww/.spamassassin/user_prefs seems to be ignored; the 
mail only gets a score of 3.8.

3. When I execute the "spamassassin" - program, everything looks as if 
it gets processed correctly: the user_prefs is read, the mail gets a 
score of 100.1 and is considered spam.

So, does this mean I should switch to use "spamassassin" instead of 
"spamc" in my filter.sh script instead? Manpage of spamc reads:

"""
Spamc is the client half of the spamc/spamd pair.  It should be used in 
place of "spamassassin" in scripts to process mail.

[...]
Spamc has extremely low overhead in loading, so it should be much faster 
to load than the whole spamassassin program.

"""

- What is the "whole spamassassin program"; are there features missing 
in spamc (like respecting user_prefs file)?

- Is it wise to use spamassassin when the developers intend spamc to be 
used for this purpose?

- How do I get spamc to respect user_prefs file?

Best regards,
Marc

Am 09.09.2015 um 13:47 schrieb Marc Richter:

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens
regularly by cron (all 3 hours), along with other jobs like sa-learn,
sa-update and sa-compile.

 > On 09.09.2015 11:12 Matus wrote:
 >
 > have you tried running spamassassin -D ? maybe there's somethign
 > invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh -
runs as, I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is
clear, because it cannot know what user the file "belongs" to, in
test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs
to, it works, so no use in that.

How can I make use of the "-D" cmdline option in the normal mail-flow in
a way it gets logged by journald? Can I simply add "-D" to the filter.sh
script and it get's caught in journald's database?

How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA gone mad

On Wednesday 09 September 2015 at 13:54:44, Farkas Zsolt wrote:

> you made a point, but /etc/spamassassin/local.cf looks like:
> 
> ...
> #   Use Bayesian classifier (default: 1)
> #
> #use_bayes 1

So, you have not set it to 0, to disable Bayes; you have left it at the 
default, which is 1, therefore Bayes remains enabled.

Antony.

> #   Bayesian classifier auto-learning (default: 1)
> #
> #bayes_auto_learn 1
> ...
> 
> KR, Zsolt
> 
> 2015.09.09. 13:49 keltezéssel, Antony Stone írta:
> > On Wednesday 09 September 2015 at 13:42:31, Farkas Zsolt wrote:
> >> badword rule is now disabled, SA running with limited functionality,
> >> higher limit (15 insted of 4.9), bayes disabled
> > 
> > How have you disabled Bayes?  The message below has 3.5 points for
> > Bayes_99
> > 
> > 
> > Antony.
> > 
> >> there are two problems now:
> >> -messages marked as SPAM but they are below limit (limit at 15 but at
> >> 13.1 marked as spam) this has no connection with disabled or enabled
> >> badword -if badword rule is enabled badword points added but there is no
> >> badword match in regex
> >> 
> >> 2015.09.09. 13:14 keltezéssel, Antony Stone írta:
> >>> On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:
>  Content analysis details:   (13.1 points, 15.0 required)
>  
>  pts rule name  description
>  
>   --
>  -- 3.5 BAYES_99
>  
>   BODY: Bayes spam probability is 99 to 100% [score: 1.]
>  
>  2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
>  0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
>  0.0 TVD_RCVD_IPTVD_RCVD_IP
>  1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used
>  for HELO 1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
>  
> [113.167.11.149 listed in
>  
>  bb.barracudacentral.org] 0.5 MISSING_MIDMissing
>  Message-Id: header
>  
>  0.8 RDNS_NONE  Delivered to internal network by a host
>  with
>  
>  no rDNS 1.4 MISSING_DATE   Missing Date: header
>  
>  2.0 FROM_12LTRDOM  From a 12-letter domain
>  0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS
>  
>  header
> >>> 
> >>> You said earlier "it gave 10 point for badword rule but there was no
> >>> match".
> >>> 
> >>> I see no badword rule match getting 10 points in what you posted above.
> >>> 
> >>> Are you telling us about the same message each time you post bits of
> >>> information?
> >>> 
> >>> 
> >>> Antoy.

-- 
Atheism is a non-prophet-making organisation.

   Please reply to the list;
 please *don't* CC me.

Re: SA gone mad

On Wednesday 09 September 2015 at 13:52:31, RW wrote:

> On Wed, 9 Sep 2015 13:14:04 +0200
> 
> Antony Stone wrote:
> > On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:
> > > Content analysis details:   (13.1 points, 15.0 required)
> > 
> > You said earlier "it gave 10 point for badword rule but there was no
> > match".
> > 
> > I see no badword rule match getting 10 points in what you posted
> > above.
> > 
> > Are you telling us about the same message each time you post bits of
> > information?
> 
> The details of the rules are a distraction. The actual problem is that
> mail is being classified as spam despite having a score under the
> configured threshold.

I agree; however the inconsistent bits of information we're seeing from Farkas 
Zsolt make it very difficult to get a clear picture of quite what's going on 
under what circumstances.

> The first thing to try is to reproduce it on the command line using
> spamassassin -D , using the original emails from the reports. If that
> doesn't work it's probably a problem with the glue, which hasn't been
> mentioned yet.

I suspect you're spot on with that last part.


Antony.

-- 
BASIC is to computer languages what Roman numerals are to arithmetic.

   Please reply to the list;
 please *don't* CC me.

Re: SA gone mad



Am 09.09.2015 um 14:08 schrieb Farkas Zsolt:

please find the original incoming email saved by SA:
http://pastebin.com/mszg5nJe

the email notification received back from SA on spamadmin account
http://pastebin.com/0UTvT9tQ


i would say this is the wrong mailing-list since you are running amavis 
and not a pure spamassassin


X-Spam-Status: Yes, score=12.46 tag=-50 tag2=6.31 kill=6.5

what about the "kill=6.5" which don't exist on SA and is pretty sure 
written by the glue (amavis) which doe snothing else than make decisions 
based on the spamassassin headers


Received: from server5.xx.hu ([127.0.0.1]) by localhost 
(server5.x.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 
8HwwIRPqANxt for ; Wed,  9 Sep 2015 11:53:24 +0200 (CEST)



2015.09.09. 13:42 keltezéssel, Matus UHLAR - fantomas írta:

Hello,


what about starting with the log-entries showing the matched rules
from some of that messages from the very first start? "it don't work
as i expect" is not enough and the only correct answer would be "i
am sorry for you"


On 09.09.15 13:08, Farkas Zsolt wrote:

it has been done at the start, but there is no visible reason for the
abnormal behavior: limit is 15 but it is filtered with 13.1



Content analysis details:   (13.1 points, 15.0 required)

pts rule name  description
 --
--
3.5 BAYES_99   BODY: Bayes spam probability is 99 to 100%
   [score: 1.]
2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1
0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
0.0 TVD_RCVD_IPTVD_RCVD_IP
1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used for
HELO
1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
   [113.167.11.149 listed in
bb.barracudacentral.org]
0.5 MISSING_MIDMissing Message-Id: header
0.8 RDNS_NONE  Delivered to internal network by a host
with no rDNS
1.4 MISSING_DATE   Missing Date: header
2.0 FROM_12LTRDOM  From a 12-letter domain
0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS



Return-Path: <>
X-Greylist: delayed 281 seconds by postgrey-1.34 at server5; Wed, 09
Sep 2015 11:43:16 CEST
Received: from 113.167.11.149 (unknown [113.167.11.149])
by server5.xxx (Postfix) with SMTP id 849BA264029
for; Wed,  9 Sep 2015 11:43:16 +0200 (CEST)
Received: from unknown (HELO localhost)
(mh...@lnainsurance.com@96.100.228.198)
by 113.167.11.149 with ESMTPA; Wed, 9 Sep 2015 16:41:05 +0700
From:mh...@lnainsurance.com
To:sgauvin@xx
Subject: Are you ready to impress your beloved one at night?


This looks like mail sent through your mailseerver to outside: numeric
helo,
no rDNS, missing Date and Message-Id headers...

or, may tghe header be somehow broken? Could you post whole message,
or at
least all headers and part of the textual body to pastebin?




signature.asc
Description: OpenPGP digital signature

Re: SA doesn't respect my user_prefs

On Wed, 9 Sep 2015 13:47:01 +0200
Marc Richter wrote:


>  > On 09.09.2015 11:12 Matus wrote:
>  >
>  > have you tried running spamassassin -D ? maybe there's somethign
>  > invalid in SA's configuration or your user_prefs
> 
> When I issue "spamassassin --test-mode -D" as the user the filter.sh
> - runs as, I get this in the long output:
> 
> dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs
> 
> So, it tries to read the user_prefs from the daemon's home, what is 
> clear, because it cannot know what user the file "belongs" to, in
> test-mode.

you can use -p or alternately set HOME 

>  When I run that as the user (ww) the mail and desired
> user_prefs belongs to, it works, so no use in that.

Do you mean that ww is a unix user? The normal way to do this is to run
spamd as root and run spamc as the unix user. Passing -u to spamc is
really intended for virtual users, I'm not sure whether it works for
unix users.  Are you sure it worked before?

Re: SA gone mad

2015-09-09 Thread Kevin A. McGrail


On 9/9/2015 8:18 AM, Reindl Harald wrote:


Am 09.09.2015 um 14:08 schrieb Farkas Zsolt:

please find the original incoming email saved by SA:
http://pastebin.com/mszg5nJe

the email notification received back from SA on spamadmin account
http://pastebin.com/0UTvT9tQ


i would say this is the wrong mailing-list since you are running 
amavis and not a pure spamassassin
SA is both an API and a program so I think it's fairly normal to discuss 
glue and integration concerns.  You might get a better answer on a 
different mailing list but I wouldn't discourage too much!


Regards,
KAM

Re: Repository of rules

2015-09-09 Thread Joe Quinn


On 9/9/2015 5:43 AM, Sujit Acharyya-choudhury wrote:


Hi Joe,

I looked at the rule set and it was very interesting and I intend to 
use it. However, I did not see any *.pm file attached to it.  Is there 
any need for this?  Do you suggest, I increase the default score from 
5.0 to 6.0 if I include this rule?  I am interested in your view, 
especially the phishing rules will hit lot of mail which are coming 
through at present and causing mayhem.


Regards

Sujit

*From:*Joe Quinn [mailto:jqu...@pccc.com]
*Sent:* 08 September 2015 16:27
*To:* users@spamassassin.apache.org
*Subject:* Re: Repository of rules

On 9/8/2015 11:13 AM, Anthony Hoppe wrote:

Hey All,

This is likely a n00b question, so I apologize.

I've been a member of this list for a while.  Periodically, I see
rules develop based on submissions of samples from other members.
 Is there, by chance, a repository of rules like that somewhere I
can reference?  I'm not often able to keep up and would love to go
back and add rules that I think will benefit my environment.

Thanks!

~ Anthony

You can find some of them in 
http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf which 
updates regularly, but it only includes rules for myself and Kevin. 
Off the top of my head, I can't think of a similar file for anyone 
else on the list. A lot of rules eventually end up committed and then 
it's up to RuleQA to decide if they merit going to sa-update.


Keep replies on-list. Just put it into /etc/mail/spamassassin and 
restart anything you need to, and it will be loaded. You can bump the 
threshold if you like, as the file notes:


#This cf file is designed for systems with a threshold of 5.0 or higher.

Re: SA doesn't respect my user_prefs


On 09.09.15 13:47, Marc Richter wrote:
Regardless if it's necessary or not, I have done so. It also happens 
regularly by cron (all 3 hours), along with other jobs like sa-learn, 
sa-update and sa-compile.


reload should be enough, restart is rarely necessary.
Also, why do you check oftern than once a day?


On 09.09.2015 11:12 Matus wrote:

have you tried running spamassassin -D ? maybe there's somethign
invalid in SA's configuration or your user_prefs


When I issue "spamassassin --test-mode -D" as the user the filter.sh 
- runs as, I get this in the long output:


dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is 
clear, because it cannot know what user the file "belongs" to, in 
test-mode.
When I run that as the user (ww) the mail and desired user_prefs 
belongs to, it works, so no use in that.


How can I make use of the "-D" cmdline option in the normal mail-flow 
in a way it gets logged by journald? Can I simply add "-D" to the 
filter.sh script and it get's caught in journald's database?


how do you plug spamassassin into your mail flow? How do you call
spamassassin? mta, mail client ... ?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.

Re: SA gone mad


sorry, I did not know that, I thought it is disabled by default


2015.09.09. 14:12 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:54:44, Farkas Zsolt wrote:


you made a point, but /etc/spamassassin/local.cf looks like:

...
#   Use Bayesian classifier (default: 1)
#
#use_bayes 1

So, you have not set it to 0, to disable Bayes; you have left it at the
default, which is 1, therefore Bayes remains enabled.

Antony.


#   Bayesian classifier auto-learning (default: 1)
#
#bayes_auto_learn 1
...

KR, Zsolt

2015.09.09. 13:49 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:42:31, Farkas Zsolt wrote:

badword rule is now disabled, SA running with limited functionality,
higher limit (15 insted of 4.9), bayes disabled

How have you disabled Bayes?  The message below has 3.5 points for
Bayes_99


Antony.


there are two problems now:
-messages marked as SPAM but they are below limit (limit at 15 but at
13.1 marked as spam) this has no connection with disabled or enabled
badword -if badword rule is enabled badword points added but there is no
badword match in regex

2015.09.09. 13:14 keltezéssel, Antony Stone írta:

On Wednesday 09 September 2015 at 13:08:45, Farkas Zsolt wrote:

Content analysis details:   (13.1 points, 15.0 required)

 pts rule name  description

 --
-- 3.5 BAYES_99

  BODY: Bayes spam probability is 99 to 100% [score: 1.]
 
 2.3 FSL_HELO_BARE_IP_1 FSL_HELO_BARE_IP_1

 0.0 TVD_RCVD_IP4   TVD_RCVD_IP4
 0.0 TVD_RCVD_IPTVD_RCVD_IP
 1.2 RCVD_NUMERIC_HELO  Received: contains an IP address used
 for HELO 1.4 RCVD_IN_BRBL_LASTEXT   RBL: RCVD_IN_BRBL_LASTEXT
 
[113.167.11.149 listed in


bb.barracudacentral.org] 0.5 MISSING_MIDMissing
Message-Id: header

 0.8 RDNS_NONE  Delivered to internal network by a host
 with

no rDNS 1.4 MISSING_DATE   Missing Date: header

 2.0 FROM_12LTRDOM  From a 12-letter domain
 0.0 TO_NO_BRKTS_NORDNS To: misformatted and no rDNS

header

You said earlier "it gave 10 point for badword rule but there was no
match".

I see no badword rule match getting 10 points in what you posted above.

Are you telling us about the same message each time you post bits of
information?


Antoy.

Re: Resume / Doc Spam

2015-09-09 Thread Bill Cole


On 8 Sep 2015, at 17:38, Kevin A. McGrail wrote:

Overall, the default SA is designed for a 5.0 threshold.  You have 
raised it to 10.0.  That's largely the source of the issue.


+1

5.0 is a very safe threshold, and I've found that once the Bayes and AWL 
DBs are reasonably trained, 4.5 or even 4.0 will catch more spam without 
measurably increasing false positives for business email streams. For 
more ISP-like streams maybe 5.5 or 6.0 would make sense, but I haven't 
seen a >9 false positive from SA in many millions (billions? maybe...) 
of messages over many years of use. At 10.0 SA will miss most spam, 
unless you expose it to sorts of spam that can be blocked by much 
lower-cost and safer means (e.g. SMTP greeting pause, low-FP DNSBLs, 
etc.)

Problem with Bayes

2015-09-09 Thread Ben Whyall


Hi

I have a mail setup serving multiple domains that is running on ubuntu 
lts 14.04.03 lts.


Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82

We have bayes stored in a mysql database

When spamassassin is run from exim, it is always returning a hit on rule 
Bayes_00 with a score of 0., however if you run spamassassin < 
message, it will return a different bayes score.


in the /etc/defaults/spamassassin file we have the following

OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config --helper-home-dir 
-H -u spamd"


the output of sa-learn -u spamd --dump magic is

0.000 0  3  0  non-token data: bayes db version
0.000  0  13794  0  non-token data: nspam
0.000  0   5619  0  non-token data: nham
0.000  01157620  0  non-token data: ntokens
0.000  0 1398925984  0  non-token data: oldest atime
0.000  0 1441801878  0  non-token data: newest atime
0.000  0  0  0  non-token data: last journal 
sync atime

0.000  0 1441797057  0  non-token data: last expiry atime
0.000  05529600  0  non-token data: last expire 
atime delta
0.000  0 783449  0  non-token data: last expire 
reduction count


Could anyone suggest how I might find out what is going awry ?

Ben

--
Sent from Postbox 


--
Sent from Postbox

Re: Problem with Bayes

2015-09-09 Thread Axb


On 09/09/2015 04:03 PM, Ben Whyall wrote:

Hi

I have a mail setup serving multiple domains that is running on ubuntu
lts 14.04.03 lts.

Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82

We have bayes stored in a mysql database

When spamassassin is run from exim, it is always returning a hit on rule
Bayes_00 with a score of 0., however if you run spamassassin <
message, it will return a different bayes score.

in the /etc/defaults/spamassassin file we have the following

OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config --helper-home-dir
-H -u spamd"

the output of sa-learn -u spamd --dump magic is

0.000 0  3  0  non-token data: bayes db version
0.000  0  13794  0  non-token data: nspam
0.000  0   5619  0  non-token data: nham
0.000  01157620  0  non-token data: ntokens
0.000  0 1398925984  0  non-token data: oldest atime
0.000  0 1441801878  0  non-token data: newest atime
0.000  0  0  0  non-token data: last journal
sync atime
0.000  0 1441797057  0  non-token data: last expiry atime
0.000  05529600  0  non-token data: last expire
atime delta
0.000  0 783449  0  non-token data: last expire
reduction count

Could anyone suggest how I might find out what is going awry ?

Ben



No idea how Exim interfaces to SA but...

Do you have
bayes_sql_override_username
in local.cf ?
if no, try adding that and restart spamd

h2h

Re: SA doesn't respect my user_prefs


Hi Matus,

Am 09.09.2015 um 15:01 schrieb Matus UHLAR - fantomas:

On 09.09.15 13:47, Marc Richter wrote:

Regardless if it's necessary or not, I have done so. It also happens
regularly by cron (all 3 hours), along with other jobs like sa-learn,
sa-update and sa-compile.


reload should be enough, restart is rarely necessary.
Also, why do you check oftern than once a day?


You are right, this can be made more clever. I made it that way when I 
started with my server to make sure the frequently changed configs are 
reloaded often automatically. I left it that way since then.
It can be optimized for sure, but I doubt it has something to do with my 
actual troubles.



how do you plug spamassassin into your mail flow? How do you call
spamassassin? mta, mail client ... ?


I'm running postfix as my MTA. In it's master.cf there is configured to 
pipe my mail through a script:


smtp   inet  n  -  n  -  -  smtpd -o content_filter=spamassassin
spamassassin
   unix  -  n  n  -  -  pipe
   flags=Rq user=spamd argv=/var/lib/spamassassin/filter.sh -oi -f 
${sender} ${recipient}


In the script filter.sh spamc is invoked:

#!/bin/sh

SENDMAIL="/usr/sbin/sendmail -i"
SPAMASSASSIN=/usr/bin/vendor_perl/spamc
COMMAND="$SENDMAIL $@"
USER=`echo $COMMAND | awk '{ print $NF }' | sed 's/@.*$//'`
NEW_COMMAND=`echo $COMMAND | awk '{ $6 = "spamfilter"; NF = 6; print }'`

# Exit codes from 
EX_TEMPFAIL=75
EX_UNAVAILABLE=69

umask 077

OUTPUT="`mktemp /tmp/mailfilter.XX`"

if [ "$?" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Unable to create 
temporary file."

 exit $EX_TEMPFAIL
fi

# Clean up when done or when aborting.
trap "rm -f $OUTPUT" EXIT SIGTERM

$SPAMASSASSIN -x -E -u $USER > $OUTPUT
return="$?"
if [ "$return" == 1 ]; then
 $NEW_COMMAND < $OUTPUT
 exit $?
elif [ "$return" != 0 ]; then
 /usr/bin/logger -s -p mail.warning -t filter "Temporary SpamAssassin 
failure (spamc return $return)"

 exit $EX_TEMPFAIL
fi

$SENDMAIL "$@" < $OUTPUT
exit $?


Thus, in effect, mail is filtered as this command would have been executed:

spamc -x -E -u ww < MAILINPUT

Best regards,
Marc

Re: Problem with Bayes




Am 09.09.2015 um 16:03 schrieb Ben Whyall:

I have a mail setup serving multiple domains that is running on ubuntu
lts 14.04.03 lts.

Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82

We have bayes stored in a mysql database

When spamassassin is run from exim, it is always returning a hit on rule
Bayes_00 with a score of 0., however if you run spamassassin <
message, it will return a different bayes score.


sound like exim is running spamassassin as a different user and like 
using a different bayes


however, i observed that BAYES_xx for whatever reason *heavily* depends 
on Received-headers and so results are often differetn between spamc and 
running as milter for example


another example: in case there are no reveived headers at all bayes will 
never ever hit 999, put a purely random, valid received header on top 
and the same sample hits 999


observed that by stripa ton of headers out of our training-corpus with 
"formail" and rebuild bayes, hence now there is on single static 
Received-heaer on top of all samples and so they pass our masstests as 
expected




signature.asc
Description: OpenPGP digital signature

Re: Problem with Bayes

2015-09-09 Thread Ben Whyall


Hi

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.

I have restarted and will see what happens.

I am using the ubuntu exim-daemon heavy and it calls spamd directly.

Ben


Axb 
09 September 2015 15:10via Postbox 




No idea how Exim interfaces to SA but...

Do you have
bayes_sql_override_username
in local.cf ?
if no, try adding that and restart spamd

h2h
Ben Whyall 
09 September 2015 15:03via Postbox 


Hi

I have a mail setup serving multiple domains that is running on ubuntu 
lts 14.04.03 lts.


Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82

We have bayes stored in a mysql database

When spamassassin is run from exim, it is always returning a hit on 
rule Bayes_00 with a score of 0., however if you run spamassassin 
< message, it will return a different bayes score.


in the /etc/defaults/spamassassin file we have the following

OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config 
--helper-home-dir -H -u spamd"


the output of sa-learn -u spamd --dump magic is

0.000 0  3  0  non-token data: bayes db version
0.000  0  13794  0  non-token data: nspam
0.000  0   5619  0  non-token data: nham
0.000  01157620  0  non-token data: ntokens
0.000  0 1398925984  0  non-token data: oldest atime
0.000  0 1441801878  0  non-token data: newest atime
0.000  0  0  0  non-token data: last journal 
sync atime

0.000  0 1441797057  0  non-token data: last expiry atime
0.000  05529600  0  non-token data: last expire 
atime delta
0.000  0 783449  0  non-token data: last expire 
reduction count


Could anyone suggest how I might find out what is going awry ?

Ben



--
Sent from Postbox

Re: Resume / Doc Spam

2015-09-09 Thread Dianne Skoll

On Wed, 09 Sep 2015 09:23:44 +0200
Benny Pedersen  wrote:

> i would run "strings vbaProject.bin" and make clamav signature based
> on it

ClamAV is totally useless.

Here's a trick: Macro viruses must define a subroutine called "Document_Open"
So finding the string "Document_Open" case-insensitively in an MS
Office file is a red flag.  If you don't find it directly, use
unzip -p (the so called "pipe mode") to look for that same string
case-insensitively in the more modern MS Office files, which are really
just zip files in disguise.

There will be some false-positives because some legitimate MS Office files
(b) auto-execute macros on document open, but IMO the danger posed
by macro viruses makes the tradeoff worth it.

Regards,

Dianne.

Re: Resume / Doc Spam


On Wed, 09 Sep 2015 09:23:44 +0200 Benny Pedersen  wrote:

i would run "strings vbaProject.bin" and make clamav signature based
on it


On 09.09.15 10:44, Dianne Skoll wrote:

ClamAV is totally useless.


Do you mean generally, or in this case?


Here's a trick: Macro viruses must define a subroutine called "Document_Open"
So finding the string "Document_Open" case-insensitively in an MS
Office file is a red flag.  If you don't find it directly, use
unzip -p (the so called "pipe mode") to look for that same string
case-insensitively in the more modern MS Office files, which are really
just zip files in disguise.

There will be some false-positives because some legitimate MS Office files
(b) auto-execute macros on document open, but IMO the danger posed
by macro viruses makes the tradeoff worth it.


i believe some people will argument against this ;-)

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.

Re: Problem with Bayes


Ben Whyall 
09 September 2015 15:03via Postbox 



When spamassassin is run from exim, it is always returning a hit on 
rule Bayes_00 with a score of 0., however if you run 
spamassassin < message, it will return a different bayes score.


that loooks like spamassassin called from exim uses different username(s)...


Axb 
09 September 2015 15:10via Postbox 


No idea how Exim interfaces to SA but...


... so this is the most important question.

On 09.09.15 15:21, Ben Whyall wrote:

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.


I don't think you should set such options without understsanding what it may
cause.
it may work, but also may fsck up spam filtering.


I have restarted and will see what happens.

I am using the ubuntu exim-daemon heavy and it calls spamd directly.


that means how?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.

Re: Problem with Bayes

2015-09-09 Thread Axb


On 09/09/2015 04:55 PM, Matus UHLAR - fantomas wrote:

On 09.09.15 15:21, Ben Whyall wrote:

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.


I don't think you should set such options without understsanding what it
may
cause.
it may work, but also may fsck up spam filtering.


Did I take it for granted that the op would check the docs 
(http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.txt) 
before adding the option?

Re: Resume / Doc Spam

2015-09-09 Thread Dianne Skoll

On Wed, 9 Sep 2015 16:51:11 +0200
Matus UHLAR - fantomas  wrote:

> On 09.09.15 10:44, Dianne Skoll wrote:
> >ClamAV is totally useless.

> Do you mean generally, or in this case?

Generally, at least if you use the official signatures.  And the unofficial
ones have unacceptably high FP rates.

> >There will be some false-positives because some legitimate MS Office
> >files (b) auto-execute macros on document open, but IMO the
> >danger posed by macro viruses makes the tradeoff worth it.

> i believe some people will argument against this ;-)

I'm sure some will.  It's a tradeoff and everyone has a different opinion.

We've implemented this in our hosted scanning service and so far
haven't had any complaints (though to be sure, we quarantine rather
than outright reject messages that hit this rule.)

These are the subjects we've seen that have hit the rule so far
today; counts are on the left:

  1 
  1 Fv: fattura sospesa 8587917 del 12-07-2015
  1 Invio fattura convalida 2492412 del 25-03-2015
  1 RE: fattura sospesa 0585247 del 18-03-2015
  1 RE: fattura sospesa 2684935 del 04-03-2015
  1 RE: fattura sospesa 6857874 del 22-06-2015
  1 Re: fattura emessa 8939951 del 25-01-2015
  1 Re: fattura sospesa 3445841 del 09-02-2015
  1 
  1 Solicitud de Oferta SM No 123/2015 Proyecto 5070
229 Resume
255 RE:resume

Looks to me like one probable and one possible false positive out of
498; IMO that's a good tradeoff for quarantining.

Regards,

Dianne.

Re: Problem with Bayes


On 09/09/2015 04:55 PM, Matus UHLAR - fantomas wrote:

On 09.09.15 15:21, Ben Whyall wrote:

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.


I don't think you should set such options without understsanding what it
may
cause.
it may work, but also may fsck up spam filtering.


On 09.09.15 17:05, Axb wrote:

Did I take it for granted that the op would check the docs
(http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.txt)
before adding the option?


I would first search for answer to the question, how does calling SA from
exim work...

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!

Re: Problem with Bayes

2015-09-09 Thread Ben Whyall

Here is an explanation of how exim is talking to Spamassassin in my
setup
http://www.exim.org/exim-html-current/doc/html/spec_html/ch-content_scanning_at_acl_time.html

I suspect that as we have spam = nobody, SA was running the queries as
nobody and as such not correctly using the right spam corpus.

Which the override statement is sorting.

Ben

Matus UHLAR - fantomas
09 September 2015 15:55via Postbox

Ben Whyall
09 September 2015 15:03via Postbox

When spamassassin is run from exim, it is always returning a hit on
rule Bayes_00 with a score of 0., however if you run
spamassassin < message, it will return a different bayes score.

that loooks like spamassassin called from exim uses different
username(s)...

Axb
09 September 2015 15:10via Postbox

No idea how Exim interfaces to SA but...

... so this is the most important question.

On 09.09.15 15:21, Ben Whyall wrote:

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.

I don't think you should set such options without understsanding what
it may

cause.
it may work, but also may fsck up spam filtering.

I have restarted and will see what happens.

I am using the ubuntu exim-daemon heavy and it calls spamd directly.

that means how?

Ben Whyall
09 September 2015 15:21via Postbox

I didnt have that option but I did have bayes_sql_username set

I have added the sql_override as well.

I have restarted and will see what happens.

I am using the ubuntu exim-daemon heavy and it calls spamd directly.

Ben

Ben Whyall
09 September 2015 15:03via Postbox

I have a mail setup serving multiple domains that is running on ubuntu
lts 14.04.03 lts.

Spamassassin 3.4.0
Perl 5.18.2
Exim 4.82

We have bayes stored in a mysql database

When spamassassin is run from exim, it is always returning a hit on
rule Bayes_00 with a score of 0., however if you run spamassassin
< message, it will return a different bayes score.

in the /etc/defaults/spamassassin file we have the following

OPTIONS="-p 987 -s /var/log/spamd.log --nouser-config
--helper-home-dir -H -u spamd"

the output of sa-learn -u spamd --dump magic is

0.000 0 3 0 non-token data: bayes db version
0.000 0 13794 0 non-token data: nspam
0.000 0 5619 0 non-token data: nham
0.000 01157620 0 non-token data: ntokens
0.000 0 1398925984 0 non-token data: oldest atime
0.000 0 1441801878 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal
sync atime

0.000 0 1441797057 0 non-token data: last expiry atime
0.000 05529600 0 non-token data: last expire
atime delta
0.000 0 783449 0 non-token data: last expire
reduction count

Could anyone suggest how I might find out what is going awry ?

Ben

--
Sent from Postbox

Re: SA doesn't respect my user_prefs


Hi RW,


Do you mean that ww is a unix user? The normal way to do this is to run
spamd as root and run spamc as the unix user. Passing -u to spamc is
really intended for virtual users, I'm not sure whether it works for
unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.

When I issue spamc without the -u trigger as user ww, I get the same 
(wrong) results.


Best regards,
Marc

Re: Resume / Doc Spam

2015-09-09 Thread Benny Pedersen


Dianne Skoll skrev den 2015-09-09 16:44:


ClamAV is totally useless.


why ?

Here's a trick: Macro viruses must define a subroutine called 
"Document_Open"


thanks for that note i will keep in mind


So finding the string "Document_Open" case-insensitively in an MS
Office file is a red flag.


with can be used to reject in clamav milter no ?


If you don't find it directly, use
unzip -p (the so called "pipe mode") to look for that same string
case-insensitively in the more modern MS Office files, which are really
just zip files in disguise.


and i belived i was the only one that creates clamav signatures :=)

There will be some false-positives because some legitimate MS Office 
files
(b) auto-execute macros on document open, but IMO the danger 
posed

by macro viruses makes the tradeoff worth it.


pdf files with javascript are much better :=)

Re: SA doesn't respect my user_prefs

2015-09-09 Thread jdow

Is this line in your "local.cf" file? (And is it in the correct place?)

allow_user_rules 1

{^_^}

On 2015-09-09 04:47, Marc Richter wrote:

Hi jdow,
hi Matus,

thanks for your replies.

Regardless if it's necessary or not, I have done so. It also happens regularly
by cron (all 3 hours), along with other jobs like sa-learn, sa-update and
sa-compile.

 > On 09.09.2015 11:12 Matus wrote:
 >
 > have you tried running spamassassin -D ? maybe there's somethign
 > invalid in SA's configuration or your user_prefs

When I issue "spamassassin --test-mode -D" as the user the filter.sh - runs as,
I get this in the long output:

dbg: config: read file /var/lib/spamassassin/.spamassassin/user_prefs

So, it tries to read the user_prefs from the daemon's home, what is clear,
because it cannot know what user the file "belongs" to, in test-mode.
When I run that as the user (ww) the mail and desired user_prefs belongs to, it
works, so no use in that.

How can I make use of the "-D" cmdline option in the normal mail-flow in a way
it gets logged by journald? Can I simply add "-D" to the filter.sh script and it
get's caught in journald's database?

How else can I test this?

Sorry if I'm slow in understanding atm ...

Best regards,
Marc

Re: SA doesn't respect my user_prefs

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:

> Hi RW,
> 
> > Do you mean that ww is a unix user? The normal way to do this is to
> > run spamd as root and run spamc as the unix user. Passing -u to
> > spamc is really intended for virtual users, I'm not sure whether it
> > works for unix users.  Are you sure it worked before?
> 
> ww is a unix user, yes. And it worked before, yes.

Supporting that sounds like a really bad idea. It would mean that any
user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.

Re: SA doesn't respect my user_prefs

2015-09-09 Thread jdow


On 2015-09-09 13:51, RW wrote:

On Wed, 9 Sep 2015 17:27:54 +0200
Marc Richter wrote:


Hi RW,


Do you mean that ww is a unix user? The normal way to do this is to
run spamd as root and run spamc as the unix user. Passing -u to
spamc is really intended for virtual users, I'm not sure whether it
works for unix users.  Are you sure it worked before?


ww is a unix user, yes. And it worked before, yes.


Supporting that sounds like a really bad idea. It would mean that any
user could make a spamd child run as any unix user they choose -
possibly even root. It's an unnecessary risk of privilege escalation.

It also gives users too much access to each other's databases. A
malicious user would be able to miss-train another user's Bayes or
manipulate reputations in TxRep or AWL. It would also be possible to
infer some of the contents of another users TxRep database from
suitable test emails.


Why don't you try to run spamc -u root as a common user and see what happens 
then talk about the results if it is warranted?


{o.o}

Re: SA doesn't respect my user_prefs