Re: antispam plugin, pipe backend, how to make it work?

[deano-dovecot]

Johannes -

I'm running 2.2.9 under Ubuntu 14.04.  I gave up on using the pipe 
backend, just could not get the damn thing to work.  I would up using 
spool2dir and incron, which works perfectly.  The issue was that 
sa-learn would cause a pthread_cancel error with libgcc_s.so.1


Below is an excerpt from my install script :


  # Enable antispam - Damn, not working right with pipe backend
  # now using spool2dir and incron
  if [ -e /etc/spamassassin ]; then
sed -i "
  s/^  #mail_plugins.*/  mail_plugins = \$mail_plugins antispam 
${DOVENOTIFY}/

  s/^  #mail_max_userip.*/mail_max_userip_connections = 20/
" /etc/dovecot/conf.d/20-imap.conf
cat > /etc/dovecot/conf.d/99-Installerbox-antispam.conf << EOF
##
## antispam configuration
##
plugin {
  antispam_debug_target = syslog
  # antispam_verbose_debug = 1
  antispam_trash_pattern = Trash;Deleted *
  antispam_spam = Junk;Spam

  antispam_backend = spool2dir
  antispam_spool2dir_spam = 
/var/cache/dovecot-antispam/spam/%%020lu-%u-%%05luS
  antispam_spool2dir_notspam =  
/var/cache/dovecot-antispam/ham/%%020lu-%u-%%05luH


  # pipe backend not working with sa-learn - causes pthread_cancel error 
with libgcc_s.so.1

  # antispam_backend = pipe
  # antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
  # antispam_pipe_program_args = --for;%u
  # antispam_pipe_program_spam_arg = --spam
  # antispam_pipe_program_notspam_arg = --ham
  # antispam_pipe_tmpdir = /tmp
}
EOF

# incron watches the spam/ham spool dirs, calls sa-learn-pipe.sh to 
handle

echo "root" >> /etc/incron.allow
mkdir -p /var/cache/dovecot-antispam/spam 
/var/cache/dovecot-antispam/ham

chown -R ${VMAIL_ID}.dovecot /var/cache/dovecot-antispam/
cat > /var/spool/incron/root << "EOF"
/var/cache/dovecot-antispam/spam IN_CLOSE_WRITE 
/usr/local/bin/sa-learn-pipe.sh --spam 
/var/cache/dovecot-antispam/spam/$#
/var/cache/dovecot-antispam/ham IN_CLOSE_WRITE 
/usr/local/bin/sa-learn-pipe.sh --ham /var/cache/dovecot-antispam/ham/$#

EOF
chgrp incron /var/spool/incron/root
chmod 600 /var/spool/incron/root

# inotify needs a little more room to breath - default of 128 too 
low

cat > /etc/sysctl.d/60-inotify.conf << EOF
# inotify changes for Dovecot
# http://dovecot.org/list/dovecot/2011-March/058300.html

# Defaults are
# fs.inotify.max_queued_events = 16384
# fs.inotify.max_user_instances = 128
# fs.inotify.max_user_watches = 8192

fs.inotify.max_user_instances = 2048
EOF

# spamassassin learning script
cat > /usr/local/bin/sa-learn-pipe.sh << "EOFSPAM"
#!/bin/bash
# Pipe script to learn/unlearn single email file
# Set to read from file or from stdin
# From stdin to accomodate dovecot-antispam pipe backend (nor currently 
working)


# echo /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt
FILE=`echo $* | sed "s/^.* //"`
echo "$$-start ($*)" >> /var/log/sa-learn-pipe.log
echo -n "$$ " >> /var/log/sa-learn-pipe.log
egrep --no-filename "^Subject: " /tmp/sendmail-msg-$$.txt ${FILE} | head 
-1 >> /var/log/sa-learn-pipe.log

cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn --progress $* /tmp/sendmail-msg-$$.txt >> 
/tmp/sa-learn-pipe.$$.log 2>&1

echo $$ sa-learn rc=$? id=$(id) HOME=$HOME >> /var/log/sa-learn-pipe.log

while read line; do
  echo $$-sa-learn "$line" >> /var/log/sa-learn-pipe.log
done < /tmp/sa-learn-pipe.$$.log

rm -f /tmp/sendmail-msg-$$.txt /tmp/sa-learn-pipe.$$.log
rm -f ${FILE}
echo "$$-end" >> /var/log/sa-learn-pipe.log

exit 0
EOFSPAM
chmod 755 /usr/local/bin/sa-learn-pipe.sh
touch /var/log/sa-learn-pipe.log
chown ${VMAIL_ID}.dovecot /var/log/sa-learn-pipe.log
chmod 660 /var/log/sa-learn-pipe.log
cat > /etc/logrotate.d/sa-learn-pipe.log << EOFLOG
/var/log/sa-learn-pipe.log {
daily
missingok
rotate 10
compress
delaycompress
notifempty
create 660 ${VMAIL_ID} dovecot
}
EOFLOG
  fi # spamassassin




On 2016-04-12 14:14, Johannes Rohr wrote:

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
the antispam plugin in version  2.0+20120225-2 and spamassassin at
version 3.2.2

I have been trying and failed to get the pipe backend of the antispam
plugin to work. Spamassin by itself works, a manual call of sa-learn
works fine. Bayes data is stored in a mysql DB.

I have the following configuration in 
/etc/dovecot/conf.d/90-plugin.conf


plugin {
  #setting_name = value
  sieve=~/.dovecot.sieve
  sieve_dir=~/sieve
antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done by 
dovecot

antispam_trash = trash;Trash;Deleted Items;Deleted Messages
antispam_spam = SPAM;Junk
antispam_backend = pipe
antispam_verbose_debug = 1
antispam_debug_target = syslog
antispam_pipe_tmpdir = 

Re: antispam plugin, pipe backend, how to make it work?

[Florian Mutter]

> Am 13.04.2016 um 12:14 schrieb Johannes Rohr :
> 
> Yikes! Our server is running a fairly dated Ubuntu Precise, what's your 
> version? Did an upgrade fail to fix it? I am not seeing this error on Debian 
> Sid but I am not using virtual users over here, so that might be the cause. 
> Did you report the bug to the Spamassassin bugzilla?

I’m running Debian 8 (Jessie). Dovecot is version 2.2.13, Dovecot Antispam is 
2.0+20130912-2 and SpamAssassin is 3.4.0. I did not fill a bug but I see you 
already did it.

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Am 13.04.2016 um 11:54 schrieb Florian Mutter:

Am 13.04.2016 um 10:35 schrieb Johannes Rohr :

tweaked the wrapper script to make it work. It seems like the sa-learn call 
silently fails:

"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johan...@rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the script & 
sa-learn manually works fine. do virtual users change something how dovecot calls 
programs, such as chrooting them?

Here [1] is an old thread about that problem.In short: It seems to be a problem 
in sa-learn that is triggered by the privilege drop that dovecot does when 
starting.

I have the same problem and I have not found a solution yet.


I have just filed a bug with spamassassin, 
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7309


I am not sure that they will act upon it, as it is an old version, and 
SA isn't the ultimate cause anyway.


Cheers,

Johannes




[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html


Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Am 13.04.2016 um 11:54 schrieb Florian Mutter:

Am 13.04.2016 um 10:35 schrieb Johannes Rohr :

tweaked the wrapper script to make it work. It seems like the sa-learn call 
silently fails:

"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johan...@rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the script & 
sa-learn manually works fine. do virtual users change something how dovecot calls 
programs, such as chrooting them?

Here [1] is an old thread about that problem.In short: It seems to be a problem 
in sa-learn that is triggered by the privilege drop that dovecot does when 
starting.

I have the same problem and I have not found a solution yet.


Yikes! Our server is running a fairly dated Ubuntu Precise, what's your 
version? Did an upgrade fail to fix it? I am not seeing this error on 
Debian Sid but I am not using virtual users over here, so that might be 
the cause. Did you report the bug to the Spamassassin bugzilla?


Cheers,

Johannes


[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html


Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Am 13.04.2016 um 11:31 schrieb Steffen Kaiser:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 13 Apr 2016, Johannes Rohr wrote:

tweaked the wrapper script to make it work. It seems like the 
sa-learn call


See the page again.


Can you kindly be more precise? The wrapper script at the page is 
broken, it cannot work, as it fails to write the message passed from the 
plugin to  /tmp/sendmail-msg-$$.txt. So the input passed to sa-learn is 
missing, which I fixed by inserting "cat > /tmp/sendmail-msg-$$.txt" at 
the beginning of the script. However, this is not the actual issue, as 
sa-learn even fails even when the script is fixed.



silently fails:

"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johan...@rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the 
script & sa-learn manually works fine. do virtual users change 
something how dovecot calls programs, such as chrooting them?


Maybe, the script uses different search paths? Uncomment "# env" to 
verify.


I'll try later, although I doubt that environment variables are to 
blame. (For now we have a cronjob which is scanning the Junk directories 
periodically as a workaround.) I have to change back a whole bunch of 
config files to enable the plugin again. It seems to be a bug which hits 
daemons that run as a limited user, see 
https://bugs.launchpad.net/ubuntu/+source/gcc-3.3/+bug/40285/comments/30


I am a bit worried because the same error is reported from newer Ubuntu 
versions, so unless the spamassassin guys have fixed it their way, it 
might not go away by upgrading


Cheers,

Johannes




Cheers,

Johannes


Am 13.04.2016 um 09:45 schrieb Johannes Rohr:

Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:

On 13-04-16 08:27, Johannes Rohr wrote:

Hi List,

does the antispam plugin any way of actually seeing how it is
calling the pipe program and how it is failing? I can't reproduce the
failure when I call sa-learn manually. Can someone tell me, with
which UID the call happens, is it the UID the current dovecot process
is running under (which would be vmail, as it is a virtual user
setup) Cheers,


Hi Johannes,
http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
This is exactly what the wrapper script is supposed to log for you, or
which is trivially to add to its output. The wrapper script is not
actually useful for production, but it is for debugging. That is why I
asked for its logging, and any differences between manual run and a
dovecot run.


When I used the wrapper script recommended at 
http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the 
PID and parameters (--username=s...@na.me --ham). The wrapper script 
noted in the dovecot wiki seems broken. Note the commented out line:


#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt

It seems like the second half of the line "cat<&0 >> 
/tmp/sendmail-msg-$$.txt" was originally a separate line, catting 
the message, which has been passed by the plugin via STDOUT to a 
file. Without that command, the call of sa-learn in the next line 
has no file to read from:


/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
/tmp/sendmail-msg-$$.txt &


So I moved the cat command to a separate uncommented line and to be 
sure I also commented out the


rm -f /tmp/sendmail-msg-$$.txt &

to verify that the file actually contains something, (why is there 
an "&" terminating the line, which would have the rm command 
executed in the background?)


But it seems the

/tmp/sendmail-msg-$$.txt &

was empty and the whole operation continued to fail. Could it be 
that the plugin fails to pass the actual message?


Cheers,

Johannes



Regards,
Tom



Johannes

Am 12.04.2016 um 23:09 schrieb Johannes Rohr:

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
with the antispam plugin in version  2.0+20120225-2 and
spamassassin at version 3.2.2

I have been trying and failed to get the pipe backend of the
antispam plugin to work. Spamassin by itself works, a manual
call of sa-learn works fine. Bayes data is stored in a mysql
DB.

I have the following configuration in
/etc/dovecot/conf.d/90-plugin.conf

plugin { #setting_name = value sieve=~/.dovecot.sieve
sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done
by dovecot

You need to specify the argument list as a list seperated by
semicolons, per example on the wiki:

antispam_pipe_program_args = --username;%u

Not sure about that, because sa-learn expects the parameter
--username=u...@na.me, this is one parameter, not two, and it
works for me locally, 

Re: antispam plugin, pipe backend, how to make it work?

[Florian Mutter]

Am 13.04.2016 um 10:35 schrieb Johannes Rohr :
> 
> tweaked the wrapper script to make it work. It seems like the sa-learn call 
> silently fails:
> 
> "/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
> root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
> starting log
> 25200-start (--username=johan...@rohr.org --spam)
> libgcc_s.so.1 must be installed for pthread_cancel to work
> 25200-end
> 
> Does the above error message have anything to do with it? Running the script 
> & sa-learn manually works fine. do virtual users change something how dovecot 
> calls programs, such as chrooting them?

Here [1] is an old thread about that problem.In short: It seems to be a problem 
in sa-learn that is triggered by the privilege drop that dovecot does when 
starting.

I have the same problem and I have not found a solution yet.

[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html

> 
> Cheers,
> 
> Johannes

Re: antispam plugin, pipe backend, how to make it work?

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 13 Apr 2016, Johannes Rohr wrote:


tweaked the wrapper script to make it work. It seems like the sa-learn call


See the page again.


silently fails:

"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johan...@rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the script 
& sa-learn manually works fine. do virtual users change something how dovecot 
calls programs, such as chrooting them?


Maybe, the script uses different search paths? Uncomment "# env" to 
verify.




Cheers,

Johannes


Am 13.04.2016 um 09:45 schrieb Johannes Rohr:

Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:

On 13-04-16 08:27, Johannes Rohr wrote:

Hi List,

does the antispam plugin any way of actually seeing how it is
calling the pipe program and how it is failing? I can't reproduce the
failure when I call sa-learn manually. Can someone tell me, with
which UID the call happens, is it the UID the current dovecot process
is running under (which would be vmail, as it is a virtual user
setup) Cheers,


Hi Johannes,
http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
This is exactly what the wrapper script is supposed to log for you, or
which is trivially to add to its output. The wrapper script is not
actually useful for production, but it is for debugging. That is why I
asked for its logging, and any differences between manual run and a
dovecot run.


When I used the wrapper script recommended at 
http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and 
parameters (--username=s...@na.me --ham). The wrapper script noted in the 
dovecot wiki seems broken. Note the commented out line:


#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt

It seems like the second half of the line "cat<&0 >> 
/tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
message, which has been passed by the plugin via STDOUT to a file. Without 
that command, the call of sa-learn in the next line has no file to read 
from:


/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
/tmp/sendmail-msg-$$.txt &


So I moved the cat command to a separate uncommented line and to be sure I 
also commented out the


rm -f /tmp/sendmail-msg-$$.txt &

to verify that the file actually contains something, (why is there an "&" 
terminating the line, which would have the rm command executed in the 
background?)


But it seems the

/tmp/sendmail-msg-$$.txt &

was empty and the whole operation continued to fail. Could it be that the 
plugin fails to pass the actual message?


Cheers,

Johannes



Regards,
Tom



Johannes

Am 12.04.2016 um 23:09 schrieb Johannes Rohr:

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
with the antispam plugin in version  2.0+20120225-2 and
spamassassin at version 3.2.2

I have been trying and failed to get the pipe backend of the
antispam plugin to work. Spamassin by itself works, a manual
call of sa-learn works fine. Bayes data is stored in a mysql
DB.

I have the following configuration in
/etc/dovecot/conf.d/90-plugin.conf

plugin { #setting_name = value sieve=~/.dovecot.sieve
sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done
by dovecot

You need to specify the argument list as a list seperated by
semicolons, per example on the wiki:

antispam_pipe_program_args = --username;%u

Not sure about that, because sa-learn expects the parameter
--username=u...@na.me, this is one parameter, not two, and it
works for me locally, where I also set up dovecot to test. The
main differences are that the versions I have here are newer, I'm
running Debian sid, the server is on Ubuntu Precise, and second,
the server is handling virtual users, locally I have only real Unix
users.

Also, dovecot doesn't complain about configuration errors regard
--username=%u. What other way would be there to get the parameter
passed to sa-learn the way I want?

btw, I tried this, just to be sure, but the result was the same.


The verbose debug should give you some logging to work with.

Yes, and I pasted the output. I can of course repeat:

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
12 22:53:02 vm10 imap: antispam: 
mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 
0 Apr 12

22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
12 

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

tweaked the wrapper script to make it work. It seems like the sa-learn 
call silently fails:


"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johan...@rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the 
script & sa-learn manually works fine. do virtual users change something 
how dovecot calls programs, such as chrooting them?


Cheers,

Johannes


Am 13.04.2016 um 09:45 schrieb Johannes Rohr:

Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:

On 13-04-16 08:27, Johannes Rohr wrote:

Hi List,

does the antispam plugin any way of actually seeing how it is
calling the pipe program and how it is failing? I can't reproduce the
failure when I call sa-learn manually. Can someone tell me, with
which UID the call happens, is it the UID the current dovecot process
is running under (which would be vmail, as it is a virtual user
setup) Cheers,


Hi Johannes,
http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
This is exactly what the wrapper script is supposed to log for you, or
which is trivially to add to its output. The wrapper script is not
actually useful for production, but it is for debugging. That is why I
asked for its logging, and any differences between manual run and a
dovecot run.


When I used the wrapper script recommended at 
http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID 
and parameters (--username=s...@na.me --ham). The wrapper script noted 
in the dovecot wiki seems broken. Note the commented out line:


#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt

It seems like the second half of the line "cat<&0 >> 
/tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
message, which has been passed by the plugin via STDOUT to a file. 
Without that command, the call of sa-learn in the next line has no 
file to read from:


/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
/tmp/sendmail-msg-$$.txt &


So I moved the cat command to a separate uncommented line and to be 
sure I also commented out the


rm -f /tmp/sendmail-msg-$$.txt &

to verify that the file actually contains something, (why is there an 
"&" terminating the line, which would have the rm command executed in 
the background?)


But it seems the

/tmp/sendmail-msg-$$.txt &

was empty and the whole operation continued to fail. Could it be that 
the plugin fails to pass the actual message?


Cheers,

Johannes



Regards,
Tom



Johannes

Am 12.04.2016 um 23:09 schrieb Johannes Rohr:

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
with the antispam plugin in version  2.0+20120225-2 and
spamassassin at version 3.2.2

I have been trying and failed to get the pipe backend of the
antispam plugin to work. Spamassin by itself works, a manual
call of sa-learn works fine. Bayes data is stored in a mysql
DB.

I have the following configuration in
/etc/dovecot/conf.d/90-plugin.conf

plugin { #setting_name = value sieve=~/.dovecot.sieve
sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done
by dovecot

You need to specify the argument list as a list seperated by
semicolons, per example on the wiki:

antispam_pipe_program_args = --username;%u

Not sure about that, because sa-learn expects the parameter
--username=u...@na.me, this is one parameter, not two, and it
works for me locally, where I also set up dovecot to test. The
main differences are that the versions I have here are newer, I'm
running Debian sid, the server is on Ubuntu Precise, and second,
the server is handling virtual users, locally I have only real Unix
users.

Also, dovecot doesn't complain about configuration errors regard
--username=%u. What other way would be there to get the parameter
passed to sa-learn the way I want?

btw, I tried this, just to be sure, but the result was the same.


The verbose debug should give you some logging to work with.

Yes, and I pasted the output. I can of course repeat:

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
12 22:53:02 vm10 imap: antispam: 
mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 
0 Apr 12

22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:

On 13-04-16 08:27, Johannes Rohr wrote:

Hi List,

does the antispam plugin any way of actually seeing how it is
calling the pipe program and how it is failing? I can't reproduce the
failure when I call sa-learn manually. Can someone tell me, with
which UID the call happens, is it the UID the current dovecot process
is running under (which would be vmail, as it is a virtual user
setup) Cheers,


Hi Johannes,
http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
This is exactly what the wrapper script is supposed to log for you, or
which is trivially to add to its output. The wrapper script is not
actually useful for production, but it is for debugging. That is why I
asked for its logging, and any differences between manual run and a
dovecot run.


When I used the wrapper script recommended at 
http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID 
and parameters (--username=s...@na.me --ham). The wrapper script noted 
in the dovecot wiki seems broken. Note the commented out line:


#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt

It seems like the second half of the line "cat<&0 >> 
/tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
message, which has been passed by the plugin via STDOUT to a file. 
Without that command, the call of sa-learn in the next line has no file 
to read from:


/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
/tmp/sendmail-msg-$$.txt &


So I moved the cat command to a separate uncommented line and to be sure 
I also commented out the


rm -f /tmp/sendmail-msg-$$.txt &

to verify that the file actually contains something, (why is there an 
"&" terminating the line, which would have the rm command executed in 
the background?)


But it seems the

/tmp/sendmail-msg-$$.txt &

was empty and the whole operation continued to fail. Could it be that 
the plugin fails to pass the actual message?


Cheers,

Johannes



Regards,
Tom



Johannes

Am 12.04.2016 um 23:09 schrieb Johannes Rohr:

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
with the antispam plugin in version  2.0+20120225-2 and
spamassassin at version 3.2.2

I have been trying and failed to get the pipe backend of the
antispam plugin to work. Spamassin by itself works, a manual
call of sa-learn works fine. Bayes data is stored in a mysql
DB.

I have the following configuration in
/etc/dovecot/conf.d/90-plugin.conf

plugin { #setting_name = value sieve=~/.dovecot.sieve
sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done
by dovecot

You need to specify the argument list as a list seperated by
semicolons, per example on the wiki:

antispam_pipe_program_args = --username;%u

Not sure about that, because sa-learn expects the parameter
--username=u...@na.me, this is one parameter, not two, and it
works for me locally, where I also set up dovecot to test. The
main differences are that the versions I have here are newer, I'm
running Debian sid, the server is on Ubuntu Precise, and second,
the server is handling virtual users, locally I have only real Unix
users.

Also, dovecot doesn't complain about configuration errors regard
--username=%u. What other way would be there to get the parameter
passed to sa-learn the way I want?

btw, I tried this, just to be sure, but the result was the same.


The verbose debug should give you some logging to work with.

Yes, and I pasted the output. I can of course repeat:

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
12 22:53:02 vm10 imap: antispam: 
mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX):
 0 Apr 12
22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running
mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10
imap: antispam: running mailtrain backend program
/usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running
mailtrain backend program parameter 1 --username=johan...@rohr.org
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend
program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run
program failed with exit code -1





Also, what does the wrapper script log when dovecot runs it?

I have actually ditched  the wrapper script because it works
without locally, I have configured the plugin to call sa-learn
directly.


How 

Re: antispam plugin, pipe backend, how to make it work?

[Tom Hendrikx]

On 13-04-16 08:27, Johannes Rohr wrote:
> Hi List,
> 
> does the antispam plugin any way of actually seeing how it is
> calling the pipe program and how it is failing? I can't reproduce the
> failure when I call sa-learn manually. Can someone tell me, with
> which UID the call happens, is it the UID the current dovecot process
> is running under (which would be vmail, as it is a virtual user
> setup) Cheers,
> 

Hi Johannes,

This is exactly what the wrapper script is supposed to log for you, or
which is trivially to add to its output. The wrapper script is not
actually useful for production, but it is for debugging. That is why I
asked for its logging, and any differences between manual run and a
dovecot run.

Regards,
Tom


> Johannes
> 
> Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
>> Hi Tom,
>> 
>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
>>> On 12-04-16 20:14, Johannes Rohr wrote:
 Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise 
 with the antispam plugin in version  2.0+20120225-2 and
 spamassassin at version 3.2.2
 
 I have been trying and failed to get the pipe backend of the
 antispam plugin to work. Spamassin by itself works, a manual
 call of sa-learn works fine. Bayes data is stored in a mysql
 DB.
 
 I have the following configuration in 
 /etc/dovecot/conf.d/90-plugin.conf
 
 plugin { #setting_name = value sieve=~/.dovecot.sieve 
 sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam 
 antispam_pipe_program_notspam_arg  = --ham 
 antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh 
 antispam_pipe_program_args = --username=%u # % expansion done
 by dovecot
>>> You need to specify the argument list as a list seperated by
>>> semicolons, per example on the wiki:
>>> 
>>> antispam_pipe_program_args = --username;%u
>> 
>> Not sure about that, because sa-learn expects the parameter 
>> --username=u...@na.me, this is one parameter, not two, and it
>> works for me locally, where I also set up dovecot to test. The
>> main differences are that the versions I have here are newer, I'm
>> running Debian sid, the server is on Ubuntu Precise, and second,
>> the server is handling virtual users, locally I have only real Unix
>> users.
>> 
>> Also, dovecot doesn't complain about configuration errors regard 
>> --username=%u. What other way would be there to get the parameter 
>> passed to sa-learn the way I want?
>> 
>> btw, I tried this, just to be sure, but the result was the same.
>> 
>>> 
>>> The verbose debug should give you some logging to work with.
>> Yes, and I pasted the output. I can of course repeat:
>> 
>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
>> 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0 Apr 12
>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 
>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running
>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10
>> imap: antispam: running mailtrain backend program 
>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running
>> mailtrain backend program parameter 1 --username=johan...@rohr.org 
>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend
>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run
>> program failed with exit code -1
>> 
>> 
>> 
>> 
>>> Also, what does the wrapper script log when dovecot runs it?
>> 
>> I have actually ditched  the wrapper script because it works
>> without locally, I have configured the plugin to call sa-learn
>> directly.
>> 
>>> How does that differ from a manual run?
>> sa-learn works just fine with a manual run. It seems as if the
>> plugin is passing the parameters wrongly, but the debugging output
>> doesn't reveal the actual command, so it's just guesswork.
>> 
>> Cheers,
>> 
>> Johannes
>> 
>>> 
>>> 
 antispam_trash = trash;Trash;Deleted Items;Deleted Messages 
 antispam_spam = SPAM;Junk antispam_backend = pipe 
 antispam_verbose_debug = 1 antispam_debug_target = syslog 
 antispam_pipe_tmpdir = /tmp }
 
 I have tried the following wrapper script: #!/bin/bash # set
 -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >>
 $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >>
 /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D
 /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f
 /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
 
 It somehow isn't working. I can call the wrapper script
 manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh
 --username=s...@mail.user --ham \ < someham

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Hi List,

does the antispam plugin any way of actually seeing how it is calling 
the pipe program and how it is failing? I can't reproduce the failure 
when I call sa-learn manually. Can someone tell me, with which UID the 
call happens, is it the UID the current dovecot process is running under 
(which would be vmail, as it is a virtual user setup)

Cheers,

Johannes

Am 12.04.2016 um 23:09 schrieb Johannes Rohr:

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:
 Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise 
with

the antispam plugin in version  2.0+20120225-2 and spamassassin at
version 3.2.2

I have been trying and failed to get the pipe backend of the antispam
plugin to work. Spamassin by itself works, a manual call of sa-learn
works fine. Bayes data is stored in a mysql DB.

I have the following configuration in 
/etc/dovecot/conf.d/90-plugin.conf


plugin {
   #setting_name = value
   sieve=~/.dovecot.sieve
   sieve_dir=~/sieve
 antispam_pipe_program_spam_arg = --spam
 antispam_pipe_program_notspam_arg  = --ham
 antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
 antispam_pipe_program_args = --username=%u # % expansion done by
dovecot

You need to specify the argument list as a list seperated by semicolons,
per example on the wiki:

antispam_pipe_program_args = --username;%u


Not sure about that, because sa-learn expects the parameter 
--username=u...@na.me, this is one parameter, not two, and it works 
for me locally, where I also set up dovecot to test. The main 
differences are that the versions I have here are newer, I'm running 
Debian sid, the server is on Ubuntu Precise, and second, the server is 
handling virtual users, locally I have only real Unix users.


Also, dovecot doesn't complain about configuration errors regard 
--username=%u. What other way would be there to get the parameter 
passed to sa-learn the way I want?


btw, I tried this, just to be sure, but the result was the same.



The verbose debug should give you some logging to work with.

Yes, and I pasted the output. I can of course repeat:

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0
Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to 
trash: 0

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 
1, src unsure: 0
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
/usr/bin/sa-learn
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
/usr/bin/sa-learn
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
parameter 1 --username=johan...@rohr.org
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
parameter 2 --spam

Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1





  Also, what
does the wrapper script log when dovecot runs it?


I have actually ditched  the wrapper script because it works without 
locally, I have configured the plugin to call sa-learn directly.



How does that differ
from a manual run?
sa-learn works just fine with a manual run. It seems as if the plugin 
is passing the parameters wrongly, but the debugging output doesn't 
reveal the actual command, so it's just guesswork.


Cheers,

Johannes





 antispam_trash = trash;Trash;Deleted Items;Deleted Messages
 antispam_spam = SPAM;Junk
 antispam_backend = pipe
 antispam_verbose_debug = 1
 antispam_debug_target = syslog
 antispam_pipe_tmpdir = /tmp
}

I have tried the following wrapper script:
#!/bin/bash
# set -x
LOGFILE=/tmp/sa-learn-pipe.log
echo "$$-start ($*)" >> $LOGFILE
#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
rm -f /tmp/sendmail-msg-$$.txt
echo "$$-end" >> $LOGFILE
exit 0

It somehow isn't working. I can call the wrapper script manually e.g.
sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
--ham \
< someham

But it doesn't work from dovecot. It seems the script is called. but no
new patterns show up in the bayes DB. Unfortunately, the plugin 
seems to

be documented poorly. How does it call the programme it pipes to? Is it
something like $programme --par1 --par2 < message ? Why can't /
shouldn't I call sa-learn directly? All the instructions I found on the
in internet contained a wrapper script, but none bothered to explain 
why

you would actually need it.

I would be very grateful, if someone could enlighten me on that...

Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Hi Tom,

Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:

On 12-04-16 20:14, Johannes Rohr wrote:

 Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
the antispam plugin in version  2.0+20120225-2 and spamassassin at
version 3.2.2

I have been trying and failed to get the pipe backend of the antispam
plugin to work. Spamassin by itself works, a manual call of sa-learn
works fine. Bayes data is stored in a mysql DB.

I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf

plugin {
   #setting_name = value
   sieve=~/.dovecot.sieve
   sieve_dir=~/sieve
 antispam_pipe_program_spam_arg = --spam
 antispam_pipe_program_notspam_arg  = --ham
 antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
 antispam_pipe_program_args = --username=%u # % expansion done by
dovecot

You need to specify the argument list as a list seperated by semicolons,
per example on the wiki:

antispam_pipe_program_args = --username;%u


Not sure about that, because sa-learn expects the parameter 
--username=u...@na.me, this is one parameter, not two, and it works for 
me locally, where I also set up dovecot to test. The main differences 
are that the versions I have here are newer, I'm running Debian sid, the 
server is on Ubuntu Precise, and second, the server is handling virtual 
users, locally I have only real Unix users.


Also, dovecot doesn't complain about configuration errors regard 
--username=%u. What other way would be there to get the parameter passed 
to sa-learn the way I want?


btw, I tried this, just to be sure, but the result was the same.



The verbose debug should give you some logging to work with.

Yes, and I pasted the output. I can of course repeat:

Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0
Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0
Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 
1, src unsure: 0
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
/usr/bin/sa-learn
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
/usr/bin/sa-learn
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
parameter 1 --username=johan...@rohr.org
Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program 
parameter 2 --spam

Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1





  Also, what
does the wrapper script log when dovecot runs it?


I have actually ditched  the wrapper script because it works without 
locally, I have configured the plugin to call sa-learn directly.



How does that differ
from a manual run?
sa-learn works just fine with a manual run. It seems as if the plugin is 
passing the parameters wrongly, but the debugging output doesn't reveal 
the actual command, so it's just guesswork.


Cheers,

Johannes





 antispam_trash = trash;Trash;Deleted Items;Deleted Messages
 antispam_spam = SPAM;Junk
 antispam_backend = pipe
 antispam_verbose_debug = 1
 antispam_debug_target = syslog
 antispam_pipe_tmpdir = /tmp
}

I have tried the following wrapper script:
#!/bin/bash
# set -x
LOGFILE=/tmp/sa-learn-pipe.log
echo "$$-start ($*)" >> $LOGFILE
#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
rm -f /tmp/sendmail-msg-$$.txt
echo "$$-end" >> $LOGFILE
exit 0

It somehow isn't working. I can call the wrapper script manually e.g.
sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
--ham \
< someham

But it doesn't work from dovecot. It seems the script is called. but no
new patterns show up in the bayes DB. Unfortunately, the plugin seems to
be documented poorly. How does it call the programme it pipes to? Is it
something like $programme --par1 --par2 < message ? Why can't /
shouldn't I call sa-learn directly? All the instructions I found on the
in internet contained a wrapper script, but none bothered to explain why
you would actually need it.

I would be very grateful, if someone could enlighten me on that...

Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[Tom Hendrikx]

On 12-04-16 20:14, Johannes Rohr wrote:
> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
> the antispam plugin in version  2.0+20120225-2 and spamassassin at
> version 3.2.2
> 
> I have been trying and failed to get the pipe backend of the antispam
> plugin to work. Spamassin by itself works, a manual call of sa-learn
> works fine. Bayes data is stored in a mysql DB.
> 
> I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
> 
> plugin {
>   #setting_name = value
>   sieve=~/.dovecot.sieve
>   sieve_dir=~/sieve
> antispam_pipe_program_spam_arg = --spam
> antispam_pipe_program_notspam_arg  = --ham
> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
> antispam_pipe_program_args = --username=%u # % expansion done by
> dovecot

You need to specify the argument list as a list seperated by semicolons,
per example on the wiki:

antispam_pipe_program_args = --username;%u

The verbose debug should give you some logging to work with. Also, what
does the wrapper script log when dovecot runs it? How does that differ
from a manual run?


> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
> antispam_spam = SPAM;Junk
> antispam_backend = pipe
> antispam_verbose_debug = 1
> antispam_debug_target = syslog
> antispam_pipe_tmpdir = /tmp
> }
> 
> I have tried the following wrapper script:
> #!/bin/bash
> # set -x
> LOGFILE=/tmp/sa-learn-pipe.log
> echo "$$-start ($*)" >> $LOGFILE
> #echo $* > /tmp/sendmail-parms.txt
> cat<&0 >> /tmp/sendmail-msg-$$.txt
> /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
> rm -f /tmp/sendmail-msg-$$.txt
> echo "$$-end" >> $LOGFILE
> exit 0
> 
> It somehow isn't working. I can call the wrapper script manually e.g.
> sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
> --ham \
> < someham
> 
> But it doesn't work from dovecot. It seems the script is called. but no
> new patterns show up in the bayes DB. Unfortunately, the plugin seems to
> be documented poorly. How does it call the programme it pipes to? Is it
> something like $programme --par1 --par2 < message ? Why can't /
> shouldn't I call sa-learn directly? All the instructions I found on the
> in internet contained a wrapper script, but none bothered to explain why
> you would actually need it.
> 
> I would be very grateful, if someone could enlighten me on that...
> 
> Cheers,
> 
> Johannes




signature.asc
Description: OpenPGP digital signature

Re: antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Hi Jürgen,

what do you mean by "It's already integrated"? Of course I added the 
configuration, see my initial mail below. I still am struggling to 
understand what the reason  for the wrapper script is and how the plugin 
is supposed to call he helper programme, is the message passed to it via 
stdout?


This is what I find in the syslog: Apr 12 21:54:32 vm10 imap: antispam: 
plugin initialising (2.0-notgit)

Apr 12 21:54:32 vm10 imap: antispam: verbose debug enabled
Apr 12 21:54:32 vm10 imap: antispam: "SPAM" is exact match spam folder
Apr 12 21:54:32 vm10 imap: antispam: "Junk" is exact match spam folder
Apr 12 21:54:32 vm10 imap: antispam: no unsure folders
Apr 12 21:54:32 vm10 imap: antispam: "trash" is exact match trash folder
Apr 12 21:54:32 vm10 imap: antispam: "Trash" is exact match trash folder
Apr 12 21:54:32 vm10 imap: antispam: "Deleted Items" is exact match 
trash folder
Apr 12 21:54:32 vm10 imap: antispam: "Deleted Messages" is exact match 
trash folder

Apr 12 21:54:32 vm10 imap: antispam: pipe backend spam argument = --spam
Apr 12 21:54:32 vm10 imap: antispam: pipe backend not-spam argument = --ham
Apr 12 21:54:32 vm10 imap: antispam: pipe backend program = 
/usr/bin/sa-learn
Apr 12 21:54:32 vm10 imap: antispam: pipe backend program arg[0] = 
--username=[blanked out]@rohr.org

Apr 12 21:54:32 vm10 imap: antispam: pipe backend tmpdir /tmp

It doesn't hint at any particular error, however, when I tried to move a 
message to junk, nothing happens, the message just stays at its original 
place and I see that the contents of the bayes db is unchanged, so 
somehow nothing happens. When I disable the plugin, moving messages to 
Junk works again.


I wonder whether this is a bug in a particular version, it is quite 
ancient software, the server is running on Ubuntu precise.


 Cheers,

Johannes




Am 12.04.2016 um 20:18 schrieb J. Echter:

Hi,

here (Centos 7) its already integrated...

Just add something like this:

plugin {


   antispam_mail_sendmail = /usr/bin/sa-learn
   antispam_mail_sendmail_args = -u amavis
   antispam_mail_spam  = --spam
   antispam_mail_notspam  = --ham
}

to dovecot.conf

Cheers

Jürgen

Am 12.04.2016 um 20:14 schrieb Johannes Rohr:

 Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
the antispam plugin in version  2.0+20120225-2 and spamassassin at
version 3.2.2

I have been trying and failed to get the pipe backend of the antispam
plugin to work. Spamassin by itself works, a manual call of sa-learn
works fine. Bayes data is stored in a mysql DB.

I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf

plugin {
   #setting_name = value
   sieve=~/.dovecot.sieve
   sieve_dir=~/sieve
 antispam_pipe_program_spam_arg = --spam
 antispam_pipe_program_notspam_arg  = --ham
 antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
 antispam_pipe_program_args = --username=%u # % expansion done by
dovecot
 antispam_trash = trash;Trash;Deleted Items;Deleted Messages
 antispam_spam = SPAM;Junk
 antispam_backend = pipe
 antispam_verbose_debug = 1
 antispam_debug_target = syslog
 antispam_pipe_tmpdir = /tmp
}

I have tried the following wrapper script:
#!/bin/bash
# set -x
LOGFILE=/tmp/sa-learn-pipe.log
echo "$$-start ($*)" >> $LOGFILE
#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
rm -f /tmp/sendmail-msg-$$.txt
echo "$$-end" >> $LOGFILE
exit 0

It somehow isn't working. I can call the wrapper script manually e.g.
sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
--ham \
< someham

But it doesn't work from dovecot. It seems the script is called. but no
new patterns show up in the bayes DB. Unfortunately, the plugin seems to
be documented poorly. How does it call the programme it pipes to? Is it
something like $programme --par1 --par2 < message ? Why can't /
shouldn't I call sa-learn directly? All the instructions I found on the
in internet contained a wrapper script, but none bothered to explain why
you would actually need it.

I would be very grateful, if someone could enlighten me on that...

Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[J. Echter]

dovecot 2.2.10

Am 12.04.2016 um 20:18 schrieb J. Echter:
> Hi,
> 
> here (Centos 7) its already integrated...
> 
> Just add something like this:
> 
> plugin {
> 
> 
>   antispam_mail_sendmail = /usr/bin/sa-learn
>   antispam_mail_sendmail_args = -u amavis
>   antispam_mail_spam  = --spam
>   antispam_mail_notspam  = --ham
> }
> 
> to dovecot.conf
> 
> Cheers
> 
> Jürgen
> 
> Am 12.04.2016 um 20:14 schrieb Johannes Rohr:
>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
>> the antispam plugin in version  2.0+20120225-2 and spamassassin at
>> version 3.2.2
>>
>> I have been trying and failed to get the pipe backend of the antispam
>> plugin to work. Spamassin by itself works, a manual call of sa-learn
>> works fine. Bayes data is stored in a mysql DB.
>>
>> I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
>>
>> plugin {
>>   #setting_name = value
>>   sieve=~/.dovecot.sieve
>>   sieve_dir=~/sieve
>> antispam_pipe_program_spam_arg = --spam
>> antispam_pipe_program_notspam_arg  = --ham
>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
>> antispam_pipe_program_args = --username=%u # % expansion done by
>> dovecot
>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
>> antispam_spam = SPAM;Junk
>> antispam_backend = pipe
>> antispam_verbose_debug = 1
>> antispam_debug_target = syslog
>> antispam_pipe_tmpdir = /tmp
>> }
>>
>> I have tried the following wrapper script:
>> #!/bin/bash
>> # set -x
>> LOGFILE=/tmp/sa-learn-pipe.log
>> echo "$$-start ($*)" >> $LOGFILE
>> #echo $* > /tmp/sendmail-parms.txt
>> cat<&0 >> /tmp/sendmail-msg-$$.txt
>> /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
>> rm -f /tmp/sendmail-msg-$$.txt
>> echo "$$-end" >> $LOGFILE
>> exit 0
>>
>> It somehow isn't working. I can call the wrapper script manually e.g.
>> sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
>> --ham \
>> < someham
>>
>> But it doesn't work from dovecot. It seems the script is called. but no
>> new patterns show up in the bayes DB. Unfortunately, the plugin seems to
>> be documented poorly. How does it call the programme it pipes to? Is it
>> something like $programme --par1 --par2 < message ? Why can't /
>> shouldn't I call sa-learn directly? All the instructions I found on the
>> in internet contained a wrapper script, but none bothered to explain why
>> you would actually need it.
>>
>> I would be very grateful, if someone could enlighten me on that...
>>
>> Cheers,
>>
>> Johannes

antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with 
the antispam plugin in version  2.0+20120225-2 and spamassassin at 
version 3.2.2


I have been trying and failed to get the pipe backend of the antispam 
plugin to work. Spamassin by itself works, a manual call of sa-learn 
works fine. Bayes data is stored in a mysql DB.


I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf

plugin {
  #setting_name = value
  sieve=~/.dovecot.sieve
  sieve_dir=~/sieve
antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg  = --ham
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --username=%u # % expansion done by 
dovecot

antispam_trash = trash;Trash;Deleted Items;Deleted Messages
antispam_spam = SPAM;Junk
antispam_backend = pipe
antispam_verbose_debug = 1
antispam_debug_target = syslog
antispam_pipe_tmpdir = /tmp
}

I have tried the following wrapper script:
#!/bin/bash
# set -x
LOGFILE=/tmp/sa-learn-pipe.log
echo "$$-start ($*)" >> $LOGFILE
#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
rm -f /tmp/sendmail-msg-$$.txt
echo "$$-end" >> $LOGFILE
exit 0

It somehow isn't working. I can call the wrapper script manually e.g.
sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user 
--ham \

< someham

But it doesn't work from dovecot. It seems the script is called. but no 
new patterns show up in the bayes DB. Unfortunately, the plugin seems to 
be documented poorly. How does it call the programme it pipes to? Is it 
something like $programme --par1 --par2 < message ? Why can't / 
shouldn't I call sa-learn directly? All the instructions I found on the 
in internet contained a wrapper script, but none bothered to explain why 
you would actually need it.


I would be very grateful, if someone could enlighten me on that...

Cheers,

Johannes

Re: antispam plugin, pipe backend, how to make it work?

[J. Echter]

Hi,

here (Centos 7) its already integrated...

Just add something like this:

plugin {


  antispam_mail_sendmail = /usr/bin/sa-learn
  antispam_mail_sendmail_args = -u amavis
  antispam_mail_spam  = --spam
  antispam_mail_notspam  = --ham
}

to dovecot.conf

Cheers

Jürgen

Am 12.04.2016 um 20:14 schrieb Johannes Rohr:
> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with
> the antispam plugin in version  2.0+20120225-2 and spamassassin at
> version 3.2.2
> 
> I have been trying and failed to get the pipe backend of the antispam
> plugin to work. Spamassin by itself works, a manual call of sa-learn
> works fine. Bayes data is stored in a mysql DB.
> 
> I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
> 
> plugin {
>   #setting_name = value
>   sieve=~/.dovecot.sieve
>   sieve_dir=~/sieve
> antispam_pipe_program_spam_arg = --spam
> antispam_pipe_program_notspam_arg  = --ham
> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
> antispam_pipe_program_args = --username=%u # % expansion done by
> dovecot
> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
> antispam_spam = SPAM;Junk
> antispam_backend = pipe
> antispam_verbose_debug = 1
> antispam_debug_target = syslog
> antispam_pipe_tmpdir = /tmp
> }
> 
> I have tried the following wrapper script:
> #!/bin/bash
> # set -x
> LOGFILE=/tmp/sa-learn-pipe.log
> echo "$$-start ($*)" >> $LOGFILE
> #echo $* > /tmp/sendmail-parms.txt
> cat<&0 >> /tmp/sendmail-msg-$$.txt
> /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \
> rm -f /tmp/sendmail-msg-$$.txt
> echo "$$-end" >> $LOGFILE
> exit 0
> 
> It somehow isn't working. I can call the wrapper script manually e.g.
> sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=s...@mail.user
> --ham \
> < someham
> 
> But it doesn't work from dovecot. It seems the script is called. but no
> new patterns show up in the bayes DB. Unfortunately, the plugin seems to
> be documented poorly. How does it call the programme it pipes to? Is it
> something like $programme --par1 --par2 < message ? Why can't /
> shouldn't I call sa-learn directly? All the instructions I found on the
> in internet contained a wrapper script, but none bothered to explain why
> you would actually need it.
> 
> I would be very grateful, if someone could enlighten me on that...
> 
> Cheers,
> 
> Johannes