subject:"Bayes Auto Learn"

Re: Bayes auto-learn - Solved

2017-08-11 Thread John Hardin


On Fri, 11 Aug 2017, John Hardin wrote:


On Fri, 11 Aug 2017, Scott wrote:


 I'm chicken.  :D

 I don't have much (almost no) experience overriding those yum packages.


It's pretty simple, just "yum install {local_filename}"


 And those warnings I got when I rebuilt from source made me nervous.


I suppose I could publish the Centos 7 x86/64 RPMs I build and use on my 
website. They wouldn't be signed...


  http://www.impsec.org/~jhardin/antispam/centos7/

You do need the epel-release package installed to use this.

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Je ne suis pas Charlie. Je suis armé.
---
 4 days until the 72nd anniversary of the end of World War II

Re: Bayes auto-learn - Solved

2017-08-11 Thread John Hardin


On Fri, 11 Aug 2017, Scott wrote:


I'm chicken.  :D

I don't have much (almost no) experience overriding those yum packages.


It's pretty simple, just "yum install {local_filename}"


And those warnings I got when I rebuilt from source made me nervous.


I suppose I could publish the Centos 7 x86/64 RPMs I build and use on my 
website. They wouldn't be signed...



--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  USMC Rules of Gunfighting #6: If you can choose what to bring
  to a gunfight, bring a long gun and a friend with a long gun.
---
 4 days until the 72nd anniversary of the end of World War II

Re: Bayes auto-learn - Solved

2017-08-11 Thread Scott

I'm chicken.  :D

I don't have much (almost no) experience overriding those yum packages.  And
those warnings I got when I rebuilt from source made me nervous.

Maybe when the dust settles...





--
View this message in context: 
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138316.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Bayes auto-learn - Solved

2017-08-11 Thread John Hardin


On Fri, 11 Aug 2017, Scott wrote:


Centos7 (selinux disabled at the time of testing)
Spamassassin 3.4.0


Next on your plate: upgrading to 3.4.1...

https://dl.fedoraproject.org/pub/fedora/linux/releases/25/Everything/source/tree/Packages/s/spamassassin-3.4.1-9.fc25.src.rpm

It works jes' fine here.

...ooo, time to update:
https://dl.fedoraproject.org/pub/fedora/linux/releases/26/Everything/source/tree/Packages/s/spamassassin-3.4.1-12.fc26.src.rpm


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  ...for a nation to tax itself into prosperity is like
  a man standing in a bucket and trying to
  lift himself up by the handle. -- Winston Churchill
---
 4 days until the 72nd anniversary of the end of World War II

Restored my last copy of my manually learned bayes database from the "bad"
directory to the new location. Let it cook for a day. Of the messages that
made it through postscreen, RBLs, etc since my logs rotated early this
morning, 93% were autolearn=no, 2% autolearn=spam, 5% autolearn=ham. ZERO
autolearn=unavailable.

This is solved.

Summary for posterity for anyone who may run into the same problem. Some
tags for searching:

Centos7 (selinux disabled at the time of testing)
Postfix 3.2.2
Amavisd-new amavisd amavis 2.11.0
Spamassassin 3.4.0
bayes
autolearn=unavailable

I strongly suspected bayes auto-learn was not functioning. Read the thread
for evidence. In local.cf had the bayes path set to:
/etc/mail/bayes/bayes

Don't remember if it came packaged that way or if I followed someone else's
"guide" to ed up with that bad location. I do see one well written guide
that specified that folder, honestly I'm not sure. No matter.

In any case, the end result was that any message that would have been
autolearned got "autolearn=unavailable" and did not learn.

The fix for this setup as listed above was to NOT have the directory under
/etc. Even with wide open (777) write permissions, amavisd/SA was
apparently unable to write there.

I moved the bayes database under /var/spool/amavisd/bayes and all now
functions properly. Note the default if no path is specified is
/var/spool/amavisd/.spamassassin/bayes IIRC (assuming /var/spool/amavisd is
the home directory for amavis) I tested both, it appears happy with either.
local.cf:
bayes_path /var/spool/amavisd/bayes/bayes

I now have a journal file FWIW:
[root@mail2 root]# ls -la /var/spool/amavisd/bayes
total 4280
drwx-- 2 amavis amavis4096 Aug 11 16:07 .
drwxr-xr-x 7 amavis amavis4096 Aug 10 22:18 ..
-rw-rw-rw- 1 amavis amavis 81888 Aug 11 16:07 bayes_journal
-rw-rw-rw- 1 amavis amavis 86016 Aug 11 16:07 bayes_seen
-rw-rw-rw- 1 amavis amavis 5267456 Aug 11 16:07 bayes_toks

And my database is happy:
[root@mail2 root]# su amavis -c 'sa-learn --dump magic'
0.000 0 3 0 non-token data: bayes db version
0.000 0359 0 non-token data: nspam
0.000 0494 0 non-token data: nham
0.000 0 149970 0 non-token data: ntokens

I now know way more about amavis-new and spamassassin than I did when I
started. Guess that's the silver lining to a few days of hair pulling.

Thanks,
Scott

--
View this message in context:
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138314.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Scott

Tom:
re selinux:
Yes, once I discovered the fix, I considered that could have been the casue. 
FWIW I'm not using it and it's disabled, so it *shouldn't* hose anything. 
But I would not be surprised if it were the culprit.






--
View this message in context: 
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138313.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Tom Hendrikx

xOn 11-08-17 17:05, Scott wrote:
> I'm going to go back and look at my build notes but I think that directory
> got created for me. It's just as possible i followed some "guide".  I am
> positive i did not think it up on my own LOL.   I remember more than set of
> instructions one with that path setting, and it very well could be the
> related Centos7 package.  Glad i found the casue though. Regardless of the
> source. 
> 
> In the FWIW department, as shown above, I still don't have it in the default
> location (I know, risks...), but why it is happy there and not under /etc I
> don't know.  And really don't care at this point.   
> 

I had to go way back in thread to look it up, but I noticed you're
running Centos, which has selinux.

Maybe your custom path is disallowed under the amavis/spamd/whatever
role? And manual testing when su'ing from the root role will not have
the same impact as running amavis using an init system.

Kind regards,
Tom



signature.asc
Description: OpenPGP digital signature

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Scott

I'm going to go back and look at my build notes but I think that directory
got created for me. It's just as possible i followed some "guide".  I am
positive i did not think it up on my own LOL.   I remember more than set of
instructions one with that path setting, and it very well could be the
related Centos7 package.  Glad i found the casue though. Regardless of the
source. 

In the FWIW department, as shown above, I still don't have it in the default
location (I know, risks...), but why it is happy there and not under /etc I
don't know.  And really don't care at this point.   





--
View this message in context: 
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138299.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread RW

On Fri, 11 Aug 2017 16:22:50 +0200
Matus UHLAR - fantomas wrote:


> don't set the path, that way it should work OOTB.

Maybe amavis is different and has it's own internl default location, but
the equivalent for spamd relies on the packager giving the spamd user a
unix home directory.

I once saw a Bayes howto that recommended:

 mkdir  /nonexistent

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Matus UHLAR - fantomas


On 10.08.17 20:15, Scott wrote:

About the only difference in my old, functioning box and this new "clean"
install was the location of the bayes files.

Old box:
/var/spool/amavisd/.spamassassin/
New box:
/etc/mail/bayes


On 11.08.17 16:22, Matus UHLAR - fantomas wrote:

Do did you change bayes path in first place?


I mean why, of course

[deleted]


don't set the path, that way it should work OOTB.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam is for losers who can't get business any other way.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Matus UHLAR - fantomas


On 10.08.17 20:15, Scott wrote:

About the only difference in my old, functioning box and this new "clean"
install was the location of the bayes files.

Old box:
/var/spool/amavisd/.spamassassin/
New box:
/etc/mail/bayes


Do did you change bayes path in first place?

amavis is the only one who processes the database, there's no need to change
it and play with permissions (which might be the reason why it does not
work).

you can still train as root:

sa-learn --dbpath /var/spool/amavis/.spamassassin/  ...


Finally to the path setting:

I tried setting the default path, and changing the path filename suffix of
bayes to mybayes for curiosity...
bayes_path /var/spool/amavisd/.spamassassin/mybayes

Upon sending a test message, SA promptly auto-learned as ham and created the
2 new files starting with "mybayes" instead of bayes.

So changing the filename didn't hurt autolearn


don't set the path, that way it should work OOTB.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
The only substitute for good manners is fast reflexes.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread Scott

Yeah, i don't know who the culprit is. sa-learn always worked. autolearn did
not. So far this am it's looking good.  An expected spread of autolearn no,
spam, and ham. Not a single unavailable. Will check this afternoon and
expect to call this done. Summary for other googlers to follow. 



--
View this message in context: 
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138295.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Bayes auto-learn - not happening, tentative success....

2017-08-11 Thread RW

On Thu, 10 Aug 2017 20:15:48 -0700 (MST)
Scott wrote:

>   For reasons beyond my skill set,
> SA will not auto-learn to a bayes db in a folder in /etc/mail/bayes.
> Regardless of wide open permissions on everything except /etc.  And
> the user's confirmed ability to write to the folder.

But sa-learn was working. It seems more likely that the difference is
between the ordinary SA scripts and amavis rather than between auto and
manual training. Amavis uses SA libraries from it's own perl code and
it's free to behave very differently.

Re: Bayes auto-learn - not happening - tentative success

2017-08-10 Thread Scott

Aug 10, 2017; 10:15pm Scottonline Scott Re: Bayes auto-learn - not
happening, tentative success

Well, here's a development...

About the only difference in my old, functioning box and this new "clean"
install was the location of the bayes files.

Old box:
/var/spool/amavisd/.spamassassin/
New box:
/etc/mail/bayes

The other details that caught my attention were that on the old box, the
ONLY bayes thing that was explicitly set was the path, which was:
bayes_path /home/amavis/.spamassassin/bayes

On this (new) box, I commented out most of the bayes settings similarly.
Restarted amavis/SA and got some errors about no bayes db. Ignored.

Sent an email anyway. Guess what? autolearn=ham right out of the gate, it
and created the database files.

And the very next message received was also shown as autolearn=ham.

** that eliminates any doubt that a minimum corpus is necessary to autolearn
(whether that's a good practice is a different topic).

I cleared the db's with sa-learn --clear. I received a new message,
autolearn=ham, again. Result:

[root@mail2 .spamassassin]# su amavis -c 'sa-learn --dump magic'
0.000 0 3 0 non-token data: bayes db version
0.000 0 0 0 non-token data: nspam
0.000 0 1 0 non-token data: nham

[root@mail2 root]# ll /var/spool/amavisd/.spamassassin
total 28
-rw-rw-rw- 1 amavis amavis 12288 Aug 10 21:20 bayes_seen
-rw-rw-rw- 1 amavis amavis 12288 Aug 10 21:20 bayes_toks
-rw-r--r-- 1 amavis amavis 1869 Aug 8 15:32 user_prefs

I gradually restored all the bayes settings in local.cf, restarting amavis
each time, clearing the db and sending that same test message.
Each time the result was autolearn=ham

Finally to the path setting:

I tried setting the default path, and changing the path filename suffix of
bayes to mybayes for curiosity...
bayes_path /var/spool/amavisd/.spamassassin/mybayes

Upon sending a test message, SA promptly auto-learned as ham and created the
2 new files starting with "mybayes" instead of bayes.

So changing the filename didn't hurt autolearn

Next I kept the new name and changed the folder back to where it was:
/etc/mail/bayes/mybayes
Send a test message
This time, NO new file was created. It apparently cannot write to it.
Permissions on mail, and on bayes are both amavis:amavis & 777.

autolearn=unavailable (As Matus expected)

Next, I log on as amavis. cd to /etc/mail/bayes. Create a file, edit it,
and then delete it. User amavis CAN write there. I verify I'm amavis, Try
to cd to some other user's folder, get "permission denied", check.

Finally, because I don't like the hidden directory anyway, I try to move the
bayes folder from the default. I configure the bayes path to:
bayes_path /var/spool/amavisd/bayes/bayes

Send my test message, voila, db files created, and autolearn=ham Success!
(tentative, cautiously optimistic)

I hope I have solved the mystery. For reasons beyond my skill set, SA will
not auto-learn to a bayes db in a folder in /etc/mail/bayes. Regardless of
wide open permissions on everything except /etc. And the user's confirmed
ability to write to the folder.

Bug I guess?

SHIT! What a PITA to figure out.

I'm gonna let this cook overnight and see how it does. Will report back.

--
View this message in context:
http://spamassassin.1065346.n5.nabble.com/Bayes-auto-learn-not-happening-tp138065p138268.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.