SA-Learn - OT (slightly) Bash Script help needed
Hello all,
For many years now I have been using SA on my home server which acts as
a mail server for my family only. It has, for all this time, been based
on an mbox structure.
I am now planning to change to the Maildir format.
(Do folks here think that's a good idea?)
Before I make the change I want to make sure SA and the tools I use with
will still work. This first thing that comes to mind is my sa-learn
script.
I have a folder structure that looks something like:
/home/mark/mail/
Home Inbox -mbox
Work Inbox -mbox
Hobby -mbox
Malware -directory
L Spam -mbox
L Virus -mbox
Misc -directory
L Clubs -mbox
L Car -mbox
L Insurance -mbox
L ...etc...
Personal -directory
L Mother -mbox
L AuntBessie -mbox
L UncleJohn -mbox
L ...etc...
Note the Malware directory.
The way I currently do sa-learning is with a nightly cron job that looks
like this (edited a bit):
Essentially it concatenates all the known ham mboxs and all the known
spam mboxs into two temporary files, runs sa-learn over them and then
deletes the temporary files.
8
# Current sa-learn mbox script
#!/bin/bash
MAILDIR=/home/mark/mail
cd ${MAILDIR}
echo Gathering Ham messages together...
cat ${MAILDIR}/Misc/* ${MAILDIR}/Work/* ${MAILDIR}/Personal/*
${MAILDIR}/School/* TempHam
echo Ham concatenation completed
echo Gathering Spam messages together...
cat ${MAILDIR}/Malware/* TempSpam
echo Spam concatenation completed
echo Starting SA-Learn for Spam messages...
sa-learn --spam --mbox ${MAILDIR}/TempSpam
echo Spam learning completed
echo Starting SA-Learn for Ham messages...
sa-learn --ham --mbox ${MAILDIR}/TempHam
echo Ham learning completed
echo Removing temporary Spam and Ham files...
rm ${MAILDIR}/TempHam
rm ${MAILDIR}/TempSpam
echo SA-Learning completed
8
When I switch to Maildir I will have a structure that looks like:
~/Maildir/.Hobby/{cur,new}
~/Maildir/.Misc.Clubs/{cur,new}
~/Maildir/.Misc.Car/{cur,new}
~/Maildir/.Misc.Insurance/{cur,new}
...etc...
So...
Will this work for sa-learn?
8
# Proposed sa-learn maildir script
#!/bin/bash
sa-learn --ham ~/Maildir/.Hobby/{cur,new}
sa-learn --ham ~/Maildir/.{Misc,Personal,etc}.*/{cur,new}
sa-learn --spam ~/Maildir/.Malware.*/{cur,new}
8
Is there a neater way of doing it?
Should I use --no-sync?
Thanks in advance for any help or suggestions
Mark
Re: SA-Learn - OT (slightly) Bash Script help needed
On 05/29/2014 12:22 PM, Arthur Dent wrote:
So...
Will this work for sa-learn?
8
# Proposed sa-learn maildir script
#!/bin/bash
sa-learn --ham ~/Maildir/.Hobby/{cur,new}
sa-learn --ham ~/Maildir/.{Misc,Personal,etc}.*/{cur,new}
sa-learn --spam ~/Maildir/.Malware.*/{cur,new}
8
new means unread - you really want to run sa-learn on stuff you
haven't looked at? (as in learning false negatives as ham?)
Re: SA-Learn - OT (slightly) Bash Script help needed
On 29/05/2014 11:43, Axb wrote:
On 05/29/2014 12:22 PM, Arthur Dent wrote:
So...
Will this work for sa-learn?
8
# Proposed sa-learn maildir script
#!/bin/bash
sa-learn --ham ~/Maildir/.Hobby/{cur,new}
sa-learn --ham ~/Maildir/.{Misc,Personal,etc}.*/{cur,new}
sa-learn --spam ~/Maildir/.Malware.*/{cur,new}
8
new means unread - you really want to run sa-learn on stuff you
haven't looked at? (as in learning false negatives as ham?)
If it was his Inbox then perhaps it would be best to avoid new, but a
lot of the time stuff that is unread in other folders generally means
that it has been looked at - perhaps not totally read, or perhaps put
aside for later inspection.
For me, I use unread / read as a marker to whether I have actioned a
particular email and keep messages unread until such time that they
are dealt with.
--
Regards,
Giles Coochey, CCNP, CCNA, CCNAS
NetSecSpec Ltd
+44 (0) 8444 780677
+44 (0) 7983 877438
http://www.coochey.net
http://www.netsecspec.co.uk
gi...@coochey.net
smime.p7s
Description: S/MIME Cryptographic Signature
Re: SA-Learn - OT (slightly) Bash Script help needed
On Thu, 2014-05-29 at 12:04 +0100, Giles Coochey wrote:
On 29/05/2014 11:43, Axb wrote:
On 05/29/2014 12:22 PM, Arthur Dent wrote:
So...
Will this work for sa-learn?
8
# Proposed sa-learn maildir script
#!/bin/bash
sa-learn --ham ~/Maildir/.Hobby/{cur,new}
sa-learn --ham ~/Maildir/.{Misc,Personal,etc}.*/{cur,new}
sa-learn --spam ~/Maildir/.Malware.*/{cur,new}
8
new means unread - you really want to run sa-learn on stuff you
haven't looked at? (as in learning false negatives as ham?)
If it was his Inbox then perhaps it would be best to avoid new, but a
lot of the time stuff that is unread in other folders generally means
that it has been looked at - perhaps not totally read, or perhaps put
aside for later inspection.
For me, I use unread / read as a marker to whether I have actioned a
particular email and keep messages unread until such time that they
are dealt with.
Yes, quite right. All the stuff in each of those mbox files has been
either put there manually, or by a well-tested procmail recipe from
known contacts etc. I do sometimes file an unread message for later
reading, so I think I will need new.
The (very few) FNs that slip through get hand-filed be me into the
Malware/Spam mbox and get re-learned on the next run of the script
(every night).
So - is the syntax correct for a maildir format?
In particular will it work with the current structure (i.e. will the
line sa-learn --ham ~/Maildir/.{Misc,Personal,etc}.*/{cur,new}
correctly catch:
.Misc.Clubs.cur
.Misc.Clubs.new
.Misc.Car.cur
.Misc.Car.new
etc...?
Should I use --no-sync?
Many thanks for the help so far...
Mark
I am getting lots of SPAM
Hello, recently I am getting loads of spam, more than usual. I have the following RBLs. reject_rbl_client b.barracudacentral.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, reject_rbl_client all.spamrats.com any recommendation? Bayes Headers: X-Spam-Flag: NO X-Spam-Score: 3.147 X-Spam-Level: *** X-Spam-Status: No, score=3.147 tagged_above=-999 required=5.3 tests=[BAYES_99=4.5, BAYES_999=0.2, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RDNS_NONE=2.013, T_REMOTE_IMAGE=0.01, URIBL_BLACK=1.7, USER_IN_WHITELIST_TO=-6] autolearn=no local.cf ## Optional Score Increase last 4.0 increase to 4.5 score BAYES_50 1.800 score BAYES_60 2.200 score BAYES_80 3.200 score BAYES_95 3.500 score BAYES_99 4.500 score BODY_ENHANCEMENT 2.513 score BODY_ENHANCEMENT2 1.513 score DRUGS_ERECTILE 3.513 score DRUG_ED_SILD 2.013 score HELO_DYNAMIC_DHCP 2.513 score HS_INDEX_PARAM 1.513 score ONLINE_PHARMACY 3.013 score RDNS_DYNAMIC 1.013 score RDNS_NONE 2.013 score STOX_REPLY_TYPE 2.013 score SUBJ_BUY 2.013 score TVD_VISIT_PHARMA 2.913 score TVD_SPACE_RATIO 1.913 help please!
Re: I am getting lots of SPAM
On Thu, 29 May 2014, motty cruz wrote: Hello, recently I am getting loads of spam, more than usual. X-Spam-Status: No, score=3.147 tagged_above=-999 required=5.3 tests=[BAYES_99=4.5, BAYES_999=0.2, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RDNS_NONE=2.013, T_REMOTE_IMAGE=0.01, URIBL_BLACK=1.7, USER_IN_WHITELIST_TO=-6] autolearn=no help please! If you whitelist the recipient you should expect them to get spam. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- You know things are bad when Pravda says we [the USA] have gone too far to the left. -- Joe Huffman --- 8 days until the 70th anniversary of D-Day
RE: Mystery SpamWare
Date: Thu, 22 May 2014 17:13:24 -0700 From: jdeb...@garlic.com To: users@spamassassin.apache.org Subject: Re: Mystery SpamWare On Thu, 22 May 2014 18:23:48 +0100 hospice admin hospice...@outlook.com wrote: Hi Team, All of a sudden I've started noticing a lot of spam coming in with some fairly unique headers like this: x-track-version: 4 x-track-source: notifire_XXX x-track-spooler-id: x-track-spooler-split-id: x-track-spooler-segment-id: x-render: render- Precedence: bulk x-track-contact-id: is some number which varies with user to some degree, XXX varies by spammer. Does anyone recognise where these headers come from? Those headers seem to be tracking headers for commercial email marketing campaigns. Possibly from Notifire.co.uk, an email massmarketing firm, calling itself a white label. Quite uncertain w/o more data. But those headers are enough to make a filter from or to use in header checks to reject such trash. jd Ah ... thank you so much ... our old 'friends' at Neteffekt. Very Helpful. Thanks again Judy.
Re: I am getting lots of SPAM
reject_rbl_client all.spamrats.com http://all.spamrats.com/ What's that? That doesn't really have a reputation here, and it's not going to be more effective than zen or barracuda. Set up your RBLs so they're weighted. Implement postscreen with postfix. X-Spam-Status: No, score=3.147 tagged_above=-999 required=5.3 tests=[BAYES_99=4.5, BAYES_999=0.2, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RDNS_NONE=2.013, T_REMOTE_IMAGE=0.01, URIBL_BLACK=1.7, USER_IN_WHITELIST_TO=-6] autolearn=no Why is this user whitelisted if you consider it to be spam? ## Optional Score Increase last 4.0 increase to 4.5 score BAYES_50 1.800 ... Don't modify the default scores. Something else is wrong if you have to do that. If you're still having difficulties, post a sample with full headers to pastebin.com with a link to it here so we can analyze it further. Regards, Alex On Thu, May 29, 2014 at 10:11 AM, motty cruz motty.c...@gmail.com wrote: Hello, recently I am getting loads of spam, more than usual. I have the following RBLs. reject_rbl_client b.barracudacentral.org, reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net, reject_rbl_client all.spamrats.com any recommendation? Bayes Headers: X-Spam-Flag: NO X-Spam-Score: 3.147 X-Spam-Level: *** X-Spam-Status: No, score=3.147 tagged_above=-999 required=5.3 tests=[BAYES_99=4.5, BAYES_999=0.2, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RDNS_NONE=2.013, T_REMOTE_IMAGE=0.01, URIBL_BLACK=1.7, USER_IN_WHITELIST_TO=-6] autolearn=no local.cf ## Optional Score Increase last 4.0 increase to 4.5 score BAYES_50 1.800 score BAYES_60 2.200 score BAYES_80 3.200 score BAYES_95 3.500 score BAYES_99 4.500 score BODY_ENHANCEMENT 2.513 score BODY_ENHANCEMENT2 1.513 score DRUGS_ERECTILE 3.513 score DRUG_ED_SILD 2.013 score HELO_DYNAMIC_DHCP 2.513 score HS_INDEX_PARAM 1.513 score ONLINE_PHARMACY 3.013 score RDNS_DYNAMIC 1.013 score RDNS_NONE 2.013 score STOX_REPLY_TYPE 2.013 score SUBJ_BUY 2.013 score TVD_VISIT_PHARMA 2.913 score TVD_SPACE_RATIO 1.913 help please!
Re: I am getting lots of SPAM
Ironical that whitelist to score can be changed so users does not make faults with shooting them self in foots ;) -- Sendt fra min Android telefon med K-9 Mail. Undskyld hvis jeg er lidt kortfattet.
