Re: [qmailtoaster] sa-learn-attach

[Martin Waschbüsch IT-Dienstleistungen]

Hi Eric,


Am 21.12.2011 um 19:48 schrieb Eric Shubert:

> Here's the script I use with a shared folder to learn ham and spam, fwiw:
> #!/bin/sh
> #
> # learn and remove spam and ham in shared folders
> #
> # shubes 3/26/08 - created
> #
> 
> learndir="/home/vpopmail/domains/shubes.net/sa-learn"
> hambox=.Ham
> spambox=.Spam
> 
> do_the_learning(){
> 
> learnas=$1
> maildir=$2
> 
> shopt -s extglob
> for spamfile in `find $maildir/+(cur|new)/* 2>/dev/null`; do
>  sudo -u vpopmail -H sa-learn --$learnas $spamfile
>  rc=$?
>  if [ $? != "0" ]; then
>echo "sa-learn failed, rc=$rc, spamfile=$spamfile"
>exit $rc
>  fi
>  rm $spamfile
> done
> }
> 
> do_the_learning ham  "$learndir/$hambox"
> do_the_learning spam "$learndir/$spambox"
> 
> exit 0

I am using a version of something I found on the wiki (or as part of qtp?).

It adds the following things:
- loop through all domains and users
- do not touch/learn special files (e.g. dovecot cache, etc.).
- move ham back to inbox (which is safe with dovecot - I asked the author.)
- learn items without syncing *before* making spamassassin sync the database 
(if you host several domains with users actively using the ham/spam feature, 
you'll be glad to do that, as perl / spam assassin is no lightweight)


# Let's define our folder conventions:
SPAMDIR=".Spam.Lernen"
HAMDIR=".Spam.Korrektur"

# find and process each SPAMDIR

for directory in $( find /home/vpopmail/domains -type d -name $SPAMDIR );
do
  # then find and process each file in SPAMDIR that is not a dovecot special 
file

  for file in $( find $directory -type f -not \( -name dovecot.index -o -name 
dovecot.index.log -o -name dovecot.index.cache -o -name dovecot-keywords -o 
-name dovecot-uidlist -o -name maildirfolder \) );
  do

# learn the file with sa-learn as Spam (use the vpopmail user so it ends up 
in the correct database)
   sudo -u vpopmail -H sa-learn --no-sync --spam ${file}  >/dev/null 2>&1

# Spam belongs to nirvana!
rm -f ${file}  >/dev/null 2>&1

  done
done

# find and process each HAMDIR
for directory in $( find /home/vpopmail/domains -type d -name $HAMDIR );
do

  # then find and process each file in HAMDIR that is not a dovecot special file
  for file in $( find $directory -type f -not \( -name dovecot.index -o -name 
dovecot.index.log -o -name dovecot.index.cache -o -name dovecot-keywords -o 
-name dovecot-uidlist -o -name maildirfolder \) );
  do

# learn the file with sa-learn as HAM (use the vpopmail user so it ends up 
in the correct database)
sudo -u vpopmail -H sa-learn --no-sync --ham ${file} >/dev/null 2>&1

# move the file back to the the INBOX.
mv ${file} ${directory}/../cur >/dev/null 2>&1

  done
done

# to speed up learning, we only sync the journal with the database at the end.
sudo -u vpopmail -H sa-learn --sync >/dev/null 2>&1

signature.asc
Description: Message signed with OpenPGP using GPGMail

RE: [qmailtoaster] Re: sa-learn-attach

[Helmut Fritz]

So I have decided to try something a little different.  I have my ham and
spam accounts on the server.  On the client using outlook I have these two
accounts added as IMAP accounts, and can then just drag and drop the SPAM
and HAM to the inbox of the appropriate account within outlook.  easy for
users to do.  Then I run my script to have sa-learn go through the cur and
new folders and then delete the emails.  I include only somewhat technically
savvy users and have instructed them to only use the accounts for putting up
SPAM and HAM - no folder creation, putting emails anywhere else, etc.  of
course it will bear some watching but this should be relatively easy and
effective.

Anyone see anything wrong with this method?

-Original Message-
From: Helmut Fritz [mailto:hel...@fritz.us.com]
Sent: Wednesday, December 21, 2011 11:29 AM
To: 'qmailtoaster-list@qmailtoaster.com
Subject: RE: [qmailtoaster] Re: sa-learn-attach

Thx Eric.  Just more questions now!

Do you think there is a downside to forwarding the emails as an attachment?

Or maybe sa-learn can be called within that attached script (i.e. modify it)
instead of duplicating its code?

Can sa-learn read an outlook .msg file or an exchange folder directly?

The other issue is that the two servers are not on the same internal
network, although I could make them that way.  Or perhaps do other things to
move the files to the qmt server.  But the answer to the question above
might demonstrate this is not useful?


I have also seen reference to a mime defang useage with sa-learn and a
script that uses some other attachment 'detaching' method as well.

-Original Message-
From: Eric Shubert [mailto:e...@shubes.net]
Sent: Wednesday, December 21, 2011 10:49 AM
To: qmailtoaster-list@qmailtoaster.com
Subject: [qmailtoaster] Re: sa-learn-attach

On 12/20/2011 10:57 PM, Helmut Fritz wrote:
> And a little background on this - my server is both a mail server and 
> acts as a smart host front end for an exchange server.  So I cannot 
> use the typical method of scanning users junk email folders, I can 
> only have known spams forwarded back to a spam mailbox and false positives
to a ham mailbox.
>
> -Original Message-
> From: Helmut Fritz [mailto:hel...@fritz.us.com]
> Sent: Tuesday, December 20, 2011 8:13 PM
> To: qmailtoaster-list@qmailtoaster.com
> Subject: [qmailtoaster] sa-learn-attach
>
> Any of you ever heard of this script?  Would it work to learn on 
> attachments vs actual emails?  This could be very handy for training 
> SpamAssassin if it would work, or at least easier on users.  I am 
> definitely not a coder/scripter, so wondering if anyone can take a 
> look.  Or is there a built in method for doing this in the newer versions
of SpamAssassin?  Thx!
>
>
>
>
> --
> ---

I haven't heard of it, but it appears to be what you're looking for. 
Personally, I would prefer to have it simply strip the attachment from the
input stream and pipe the result to the real sa-learn, instead of
duplicating sa-learn's code.
Also, this method has the drawback of getting users trained to use "Forward
as attachment".
Other than that, I'd say it's worth a try.

I wonder if you could create a shared folder on the exchange server for
spam/ham, and have people move/copy their messages there. Then you could
make the QMT a samba client to the exchange server, mount the shares as
cifs, and run a sa-learn script that way.

Here's the script I use with a shared folder to learn ham and spam, fwiw:
#!/bin/sh
#
# learn and remove spam and ham in shared folders
#
# shubes 3/26/08 - created
#

learndir="/home/vpopmail/domains/shubes.net/sa-learn"
hambox=.Ham
spambox=.Spam

do_the_learning(){

learnas=$1
maildir=$2

shopt -s extglob
for spamfile in `find $maildir/+(cur|new)/* 2>/dev/null`; do
   sudo -u vpopmail -H sa-learn --$learnas $spamfile
   rc=$?
   if [ $? != "0" ]; then
 echo "sa-learn failed, rc=$rc, spamfile=$spamfile"
 exit $rc
   fi
   rm $spamfile
done
}

do_the_learning ham  "$learndir/$hambox"
do_the_learning spam "$learndir/$spambox"

exit 0

--
-Eric 'shubes'



-
Qmailtoaster is sponsored by Vickers Consulting Group
(www.vickersconsulting.com)
Vickers Consulting Group offers Qmailtoaster support and installations.
  If you need professional help with your setup, contact them today!

-
 Please visit qmailtoaster.com for the latest news, updates, and
packages.
 
  To unsubscribe, e-mail: qmailtoaster-list-unsubscr...@qmailtoaster.com
 For additional commands, e-mail:
qmailtoaster-list-h...@qm