sa-learn script
Does anybody know a good way to script sa-learn to daily check on junk
e-mail folders? i'm currently trying the following line in a cron.daily
script, but its throwing up an error:
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam cur {} \;
Error:
Learned tokens from 0 message(s) (0 message(s) examined)
archive-iterator: unable to open cur: No such file or directory
I'm using vpopmail as you can see, and each user has a .Junk E-mail
folder under their Maildir.
Many thanks,
Nick Payne-Roberts
Re: sa-learn script
* Nicholas Payne-Roberts wrote (11/07/06 11:58):
Does anybody know a good way to script sa-learn to daily check on junk
e-mail folders? i'm currently trying the following line in a cron.daily
script, but its throwing up an error:
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam cur {} \;
Your --exec subcommand is the problem. The {} expands to the full path
of the found file. It doesn't change directory. A version that might work is
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam {}/cur \;
There's not much point using --showdots in cron, I would have thought,
but it's probably useful for testing.
To make sure your find command is right, you can do something like this:
find /home/vpopmail/domains -name .Junk E-mail -exec echo sa-learn
--showdots --spam {}/cur \;
which will simply echo a list of commands that would get executed.
Chris
Re: sa-learn script
Thats fantastic, thanks very much Chris!
Chris Lear wrote:
* Nicholas Payne-Roberts wrote (11/07/06 11:58):
Does anybody know a good way to script sa-learn to daily check on
junk e-mail folders? i'm currently trying the following line in a
cron.daily script, but its throwing up an error:
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam cur {} \;
Your --exec subcommand is the problem. The {} expands to the full path
of the found file. It doesn't change directory. A version that might
work is
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam {}/cur \;
There's not much point using --showdots in cron, I would have thought,
but it's probably useful for testing.
To make sure your find command is right, you can do something like this:
find /home/vpopmail/domains -name .Junk E-mail -exec echo sa-learn
--showdots --spam {}/cur \;
which will simply echo a list of commands that would get executed.
Chris
Re: sa-learn script
On Tue, Jul 11, 2006 at 12:13:22PM +0100, Chris Lear wrote:
Does anybody know a good way to script sa-learn to daily check on junk
e-mail folders? i'm currently trying the following line in a cron.daily
script, but its throwing up an error:
find /home/vpopmail/domains -name .Junk E-mail -exec sa-learn
--showdots --spam cur {} \;
Your --exec subcommand is the problem. The {} expands to the full path
of the found file. It doesn't change directory. A version that might work is
A bigger problem imo is that the file/path you're looking for contains
a space, and {} doesn't quote that for you. It may be as easy as just
putting single or double quotes around (see the find man page), but may
involve something more complicated ala:
find /home/vpopmail/domains -name .Junk E-mail -print0 | \
xargs -0 sa-learn --showdots --spam
or however you want to deal with it.
--
Randomly Generated Tagline:
My daddy shoots people!
--Ralph Wiggum
Last Tap Dance in Springfield (Episode BABF15)
pgph1dGMNaAA9.pgp
Description: PGP signature
Re: sa-learn script
On 7/11/06, Nicholas Payne-Roberts [EMAIL PROTECTED] wrote:
Does anybody know a good way to script sa-learn to daily check on junk
e-mail folders?
I use logrotate because it handles automatically removing or renaming
the files after learning, but I don't use maildir-format folders so I
can't provide a tested configuration.
Something like this:
notifempty
missingok
/home/vpopmail/domains/*/*/.Junk E-mail/cur/* {
rotate 0
daily
nomail
prerotate
spamc -t 20 -l -L spam $1
endscript
}
Be careful of that rotate 0 which means to delete the file. If
there's any chance that a false-positive might need to be recovered
later, you probably want to increase that and add an olddir
directive to tell logrotate where to archive the spam.
If you have logrotate running regularly as a system process, that
config would go in (for example, may vary by OS distribution)
/etc/logrotate.d/sa-learn. If not or if you have to run logrotate as
a user other than root, put that in a file somewhere in the correct
user's home directory (I like to use a subdirectory named .logrotate
and name the file conf) and install a crontab entry for that user,
similar to
1 3 * * * logrotate -f --state $HOME/.logrotate/state $HOME/.logrotate/conf
Re: sa-learn script
Paul Matthews wrote: ... what I want it to do is once it is marked as spam to move the e-mail from /var/spool/mail/user to $HOME/mail/Junk and same as marking it ham, move it from $HOME/mail/Junk to /var/spool/mail/user Can anyone tell me how-to do that? Well, the desire to do this is quite correct. But I don't know that I'd go to the point of assuming that everything in the user's inbox is ham. After all, how do you know they haven't simply not see a message they will move to Junk later? In my case, I keep a Junk folder and a sub-folder of that used to drop in messages that were mislabelled as Junk. I wonder, though, whether it would make sense to always register mail from known good folders as ham on an ongoing basis. In general, autolearn for ham over a certain threshold is about all I've done, except to specifically reprocess messages as ham that were incorrectly marked as spam to begin with. Bill
sa-learn script
Hi there, i'm running RHEL4 with spamassassin-3.0.5-3.el4 and i'm looking for a script that will make sa-learn go though everyone's Junk mail folder and 'learn' what is Junk. i've come up with this #!/bin/bash for i in $( ls /home/MYDOMAIN); do sa-learn --spam /home/MYDOMAIN/i$/mail/Junk done If i set it to run as a cron job once a week, Will that do what I want it to do?
Re: sa-learn script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Matthews wrote: Hi there, i'm running RHEL4 with spamassassin-3.0.5-3.el4 and i'm looking for a script that will make sa-learn go though everyone's Junk mail folder and 'learn' what is Junk. i've come up with this #!/bin/bash for i in $( ls /home/MYDOMAIN); do sa-learn --spam /home/MYDOMAIN/i$/mail/Junk done If i set it to run as a cron job once a week, Will that do what I want it to do? Almost certainly not, unless you change that i$ to $i ;-) C. - -- Craig McLeanhttp://fukka.co.uk [EMAIL PROTECTED] Where the fun never starts Powered by FreeBSD, and GIN! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFEdWGSMDDagS2VwJ4RAiRfAJ9/0LsDofegmY1FMQMLgQRL9MnwcACeJOYd LH64xcCF3cfXHfAo/KTO4zc= =KOF+ -END PGP SIGNATURE-
Re: sa-learn script
Almost certainly not, unless you change that i$ to $i ;-) okay, say I do change it, will that script work? if i just add it in a cron job? Also, i'm using the squirrelmail plugin spam_button. http://www.squirrelmail.org/plugin_view.php?id=242 But what I want it to do is once it is marked as spam to move the e-mail from /var/spool/mail/user to $HOME/mail/Junk and same as marking it ham, move it from $HOME/mail/Junk to /var/spool/mail/user Can anyone tell me how-to do that?
Re: sa-learn script
Also, i'm using the squirrelmail plugin spam_button. http://www.squirrelmail.org/plugin_view.php?id=242 But what I want it to do is once it is marked as spam to move the e-mail from /var/spool/mail/user to $HOME/mail/Junk and same as marking it ham, move it from $HOME/mail/Junk to /var/spool/mail/user Can anyone tell me how-to do that? I've slightly changed the script I was using, can anyone tell me: if it will still work? if it will work better, worse or no change? Am I missing anything in the script that should be there? I notice i the --help option for sa-learn there is a --sync is that something I have to do after this script has run? #!/bin/bash for i in $( ls /home/MYDOMAIN); do sa-learn -p /home/MYDOMAIN/$i/.spamassassin/user_prefs -u [EMAIL PROTECTED] --siteconfigpath=/etc/mail/spamassassin --mbox --spam /home/MYDOMAIN/$i/mail/Junk sa-learn -p /home/MYDOMAIN/$i/.spamassassin/user_prefs -u [EMAIL PROTECTED] --siteconfigpath=/etc/mail/spamassassin --mbox --ham /var/spool/mail/$i done
Re: SA-Learn script
It is not fully tested yet but here it is. NB that I changed the USER
env variable to USERNAME. I do not know if this is common on all
flavors of linux but USER does not transliterate under su conditions to
the child id but stays the parent. The var USERNAME does change to
reflect the child username. Also, this script is still localized
somewhat since it assumes all Junk folders are prefixed with Junk and I
did not adjust the courier IMAP code with my changes since I had no
system to test against. It should provide for some interesting ideas
nonetheless.
New features include cross version compatibility, higher speed (using
bayes journals), debugging and error controls, wider bayes training and
most importantly support for UWash based IMAP and mbox format
mailboxes.
Tom
Rubin Bennett wrote:
Hello all...
I figure I've asked enough questions of this list that it's about time I
gave something back... You may not want it,but here it is anyway :)
I've written a bash script that takes will run sa-learn against the
administrator specified False-Postive and False-Negative folders.
Run this script from cron, and have your users drag n' drop emails that
get misclassified by SA to the appropriate folders. The script will act
in 2 ways:
1.) Run it as root, and it will parse the administrator specified
USERLIST and run the internally defined autoLearn() function as each
user.
2.) Run it as an ordinary user and it will only learn from that user's
email.
I wrote it this way so that I could have a wrapper around sa-learn that
would make sure that the directories exist, create them if they don't
using maildirmake++, and not try to learn from directories with no
messages in them.
This is written to work with Courier IMAP and Maildir; I have not tried
it with anything else.
Someday I may get around to rewriting it in php and using php-imap to do
the moving around etc, but as a dirty hack this works ok. It also
doesn't need passwords etc. in config files...
I hope this benefits someone out there... if there's enough interest,
I'll put it on my website and do a proper CVS for it.
If anyone has ideas for making it better (or suck less), let me know.
Patches are always welcome...
#!/bin/bash
# Copyright (c) 2004 by Rubin Bennett [EMAIL PROTECTED]
# All Rights reserved.
#This program is free software; you can redistribute it and/or
#modify it under the terms of the GNU General Public License
#as published by the Free Software Foundation; either version 2
#of the License, or (at your option) any later version.
#
#This program is distributed in the hope that it will be useful,
#but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#
#You should have received a copy of the GNU General Public License
#along with this program; if not, write to the Free Software
#Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
# Usage: IMAP users can move misclassified emails into the "False Negative"
# or "Flase Positive" folders, and this script will learn from them and put
# them where they belong.
# Spam will be moved to the designated Spam folder, and Ham will be moved to
# the user's Inbox.
# This script should be called by CRON or a similar scheduler.
# Requires:
# Maildir style email storage (i.e. Courier IMAP) and IMAP server
# Settings - tweak as necessary.
MAILDIR="/home/$USER/Maildir"
FALSE_NEG_FOLDER="Undetected Spam"
FALSE_POS_FOLDER="Not Spam"
SPAMFOLDER="Spam"
# List of users to run the autoLearn funtcion as (space separated)...
USERLIST=""
autoLearn() {
# Checks to see if the specified FALSE_NEG_FOLDER and FALSE_POS_FOLDER exist,
# and creates them if necessary.
[ -d "${MAILDIR}/.${FALSE_NEG_FOLDER}" ] || /usr/bin/maildirmake++ -f "${FALSE_NEG_FOLDER}" "${MAILDIR}"
[ -d "${MAILDIR}/.${FALSE_POS_FOLDER}" ] || /usr/bin/maildirmake++ -f "${FALSE_POS_FOLDER}" "${MAILDIR}"
# Parses the designated Ham folder and then moves it's contents to the Inbox
hamCount=`find "${MAILDIR}/.${FALSE_POS_FOLDER}/cur" | wc -l`
if [ $hamCount -gt 2 ]
then
echo "Learning from $hamCount HAM's"
sa-learn --ham "${MAILDIR}/.${FALSE_POS_FOLDER}/cur/*"
mv "${MAILDIR}/.${FALSE_POS_FOLDER}/cur/"* ${MAILDIR}/cur/
fi
# Parses the "Undetected Spam" folder and then moved it's contents to Spam
spamCount=`find "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur" | wc -l`
if [ $spamCount -gt 2 ]
then
echo "Learning from $spamCount SPAM's"
sa-learn --spam "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/*"
mv "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/"* ${MAILDIR}/.${SPAMFOLDER}/cur/
fi
}
### End of function declaration ###
if [ "${USER}" == "root" ]
then
for USER in $USERLIST;
do
echo "learning for $USER"
su - $USER -c sa-autolearn
done
else
autoLearn
fi
#!/bin/bash
# Copyright (c) 2004 by Rubin Bennett [EMAIL PROTECTED]
# All
Re: SA-Learn script
This is exactly the kind of starting point I needed to get me to get in
gear and write something similar for my system. For me however, I am
using the std UWash based IMAP and a few other differences but the
important difference/addition is that I want to automatically train my
users emails accross all of their boxes including inbox and train on
junk that gets picked up but not auto learned. This way things that
pass the spam test but do not get auto trained will get picked up and
trained and vice versa. Even if some things are falsely trained on
because the script ran before they manually classified their FP/FN
mail, when they use the FP/FN boxes sa-learn is smart enough to relearn
things so this should work. When I get the script done I will post it
back for you to merge in with yours.
Thanks,
Tom
Rubin Bennett wrote:
Hello all...
I figure I've asked enough questions of this list that it's about time I
gave something back... You may not want it,but here it is anyway :)
I've written a bash script that takes will run sa-learn against the
administrator specified False-Postive and False-Negative folders.
Run this script from cron, and have your users drag n' drop emails that
get misclassified by SA to the appropriate folders. The script will act
in 2 ways:
1.) Run it as root, and it will parse the administrator specified
USERLIST and run the internally defined autoLearn() function as each
user.
2.) Run it as an ordinary user and it will only learn from that user's
email.
I wrote it this way so that I could have a wrapper around sa-learn that
would make sure that the directories exist, create them if they don't
using maildirmake++, and not try to learn from directories with no
messages in them.
This is written to work with Courier IMAP and Maildir; I have not tried
it with anything else.
Someday I may get around to rewriting it in php and using php-imap to do
the moving around etc, but as a dirty hack this works ok. It also
doesn't need passwords etc. in config files...
I hope this benefits someone out there... if there's enough interest,
I'll put it on my website and do a proper CVS for it.
If anyone has ideas for making it better (or suck less), let me know.
Patches are always welcome...
#!/bin/bash
# Copyright (c) 2004 by Rubin Bennett [EMAIL PROTECTED]
# All Rights reserved.
#This program is free software; you can redistribute it and/or
#modify it under the terms of the GNU General Public License
#as published by the Free Software Foundation; either version 2
#of the License, or (at your option) any later version.
#
#This program is distributed in the hope that it will be useful,
#but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#
#You should have received a copy of the GNU General Public License
#along with this program; if not, write to the Free Software
#Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
# Usage: IMAP users can move misclassified emails into the "False Negative"
# or "Flase Positive" folders, and this script will learn from them and put
# them where they belong.
# Spam will be moved to the designated Spam folder, and Ham will be moved to
# the user's Inbox.
# This script should be called by CRON or a similar scheduler.
# Requires:
# Maildir style email storage (i.e. Courier IMAP) and IMAP server
# Settings - tweak as necessary.
MAILDIR="/home/$USER/Maildir"
FALSE_NEG_FOLDER="Undetected Spam"
FALSE_POS_FOLDER="Not Spam"
SPAMFOLDER="Spam"
# List of users to run the autoLearn funtcion as (space separated)...
USERLIST=""
autoLearn() {
# Checks to see if the specified FALSE_NEG_FOLDER and FALSE_POS_FOLDER exist,
# and creates them if necessary.
[ -d "${MAILDIR}/.${FALSE_NEG_FOLDER}" ] || /usr/bin/maildirmake++ -f "${FALSE_NEG_FOLDER}" "${MAILDIR}"
[ -d "${MAILDIR}/.${FALSE_POS_FOLDER}" ] || /usr/bin/maildirmake++ -f "${FALSE_POS_FOLDER}" "${MAILDIR}"
# Parses the designated Ham folder and then moves it's contents to the Inbox
hamCount=`find "${MAILDIR}/.${FALSE_POS_FOLDER}/cur" | wc -l`
if [ $hamCount -gt 2 ]
then
echo "Learning from $hamCount HAM's"
sa-learn --ham "${MAILDIR}/.${FALSE_POS_FOLDER}/cur/*"
mv "${MAILDIR}/.${FALSE_POS_FOLDER}/cur/"* ${MAILDIR}/cur/
fi
# Parses the "Undetected Spam" folder and then moved it's contents to Spam
spamCount=`find "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur" | wc -l`
if [ $spamCount -gt 2 ]
then
echo "Learning from $spamCount SPAM's"
sa-learn --spam "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/*"
mv "${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/"* ${MAILDIR}/.${SPAMFOLDER}/cur/
fi
}
### End of function declaration ###
if [ "${USER}" == "root" ]
then
for USER in $USERLIST;
do
echo "learning for $USER"
su - $USER -c sa-autolearn
done
else
autoLearn
fi
Re: SA-Learn script
On Thu, 23 Sep 2004, Rubin Bennett said: I've written a bash script that takes will run sa-learn against the administrator specified False-Postive and False-Negative folders. Functionality-irrelevant pedant point: #!/bin/bash # Copyright (c) 2004 by Rubin Bennett [EMAIL PROTECTED] # All Rights reserved. #This program is free software; you can redistribute it and/or #modify it under the terms of the GNU General Public License #as published by the Free Software Foundation; either version 2 #of the License, or (at your option) any later version. These two phrases are contradictory: `All Rights Reserved' is phraseology required decades ago by an old Berne Convention to state that nobody was allowed to copy (c) your work. Then you go and say the opposite. :) There's never a need to say `All Rights Reserved' these days. -- `I agree that school is a learning environment, and learning to intimidate others -- aka social skills -- is part of that.' --- jabberwocky
SA-Learn script
Hello all...
I figure I've asked enough questions of this list that it's about time I
gave something back... You may not want it,but here it is anyway :)
I've written a bash script that takes will run sa-learn against the
administrator specified False-Postive and False-Negative folders.
Run this script from cron, and have your users drag n' drop emails that
get misclassified by SA to the appropriate folders. The script will act
in 2 ways:
1.) Run it as root, and it will parse the administrator specified
USERLIST and run the internally defined autoLearn() function as each
user.
2.) Run it as an ordinary user and it will only learn from that user's
email.
I wrote it this way so that I could have a wrapper around sa-learn that
would make sure that the directories exist, create them if they don't
using maildirmake++, and not try to learn from directories with no
messages in them.
This is written to work with Courier IMAP and Maildir; I have not tried
it with anything else.
Someday I may get around to rewriting it in php and using php-imap to do
the moving around etc, but as a dirty hack this works ok. It also
doesn't need passwords etc. in config files...
I hope this benefits someone out there... if there's enough interest,
I'll put it on my website and do a proper CVS for it.
If anyone has ideas for making it better (or suck less), let me know.
Patches are always welcome...
--
Rubin Bennett [EMAIL PROTECTED]
RB Technologies
#!/bin/bash
# Copyright (c) 2004 by Rubin Bennett [EMAIL PROTECTED]
# All Rights reserved.
#This program is free software; you can redistribute it and/or
#modify it under the terms of the GNU General Public License
#as published by the Free Software Foundation; either version 2
#of the License, or (at your option) any later version.
#
#This program is distributed in the hope that it will be useful,
#but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#
#You should have received a copy of the GNU General Public License
#along with this program; if not, write to the Free Software
#Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
# Usage: IMAP users can move misclassified emails into the False Negative
# or Flase Positive folders, and this script will learn from them and put
# them where they belong.
# Spam will be moved to the designated Spam folder, and Ham will be moved to
# the user's Inbox.
# This script should be called by CRON or a similar scheduler.
# Requires:
# Maildir style email storage (i.e. Courier IMAP) and IMAP server
# Settings - tweak as necessary.
MAILDIR=/home/$USER/Maildir
FALSE_NEG_FOLDER=Undetected Spam
FALSE_POS_FOLDER=Not Spam
SPAMFOLDER=Spam
# List of users to run the autoLearn funtcion as (space separated)...
USERLIST=
autoLearn() {
# Checks to see if the specified FALSE_NEG_FOLDER and FALSE_POS_FOLDER exist,
# and creates them if necessary.
[ -d ${MAILDIR}/.${FALSE_NEG_FOLDER} ] || /usr/bin/maildirmake++ -f ${FALSE_NEG_FOLDER} ${MAILDIR}
[ -d ${MAILDIR}/.${FALSE_POS_FOLDER} ] || /usr/bin/maildirmake++ -f ${FALSE_POS_FOLDER} ${MAILDIR}
# Parses the designated Ham folder and then moves it's contents to the Inbox
hamCount=`find ${MAILDIR}/.${FALSE_POS_FOLDER}/cur | wc -l`
if [ $hamCount -gt 2 ]
then
echo Learning from $hamCount HAM's
sa-learn --ham ${MAILDIR}/.${FALSE_POS_FOLDER}/cur/*
mv ${MAILDIR}/.${FALSE_POS_FOLDER}/cur/* ${MAILDIR}/cur/
fi
# Parses the Undetected Spam folder and then moved it's contents to Spam
spamCount=`find ${MAILDIR}/.${FALSE_NEG_FOLDER}/cur | wc -l`
if [ $spamCount -gt 2 ]
then
echo Learning from $spamCount SPAM's
sa-learn --spam ${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/*
mv ${MAILDIR}/.${FALSE_NEG_FOLDER}/cur/* ${MAILDIR}/.${SPAMFOLDER}/cur/
fi
}
### End of function declaration ###
if [ ${USER} == root ]
then
for USER in $USERLIST;
do
echo learning for $USER
su - $USER -c sa-autolearn
done
else
autoLearn
fi
signature.asc
Description: This is a digitally signed message part
