Ldap, User Preferences and Milters
Hello all, I am about to discuss a antispam/antivirus policy with a client of mine. There are however a couple of "gray areas", for me, in the documentations. I will be Using an ldap server where users reside and posibly a web interface for user preferences. The questions that have risen are: 1. From the preferences stated here: http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.html#user_preferences (and also the rest of them) how many can I delegate to the ldap backend? are there any spamassassin specific limitations? Does the Ldap subsystem use any kind of caching mechanism or works in a one message, one recipient, one query way, has anyone noted what happens in multiple recipients per message? 2. The final one is a bit of performance oriented question the next part of the equation is the milter that will be used. I would like to hear some opinions regarding the way they operate and specificly with multiple rcpt to headers used? Thanks again for the time ... -- Vaggelis Georgatos IT&T Specialist
Re: Warnings in procmail log
Bart Schaefer wrote: On 7/8/06, Geoff Soper <[EMAIL PROTECTED]> wrote: Bart Schaefer wrote: I think I need to specify the .procmailrc as the .procmailrc file is per e-mail address, not per user or even system-wide I think we've just uncovered a crucial bit of missing information. You're apparently running procmail in some kind of virtual-user environment, where there is no user login name corresponding to the email address being processed. You need to explain these sorts of things up front. All the answers so far have assumed you have a traditional unix/linux type environment where mail is delivered to individual user accounts that have /etc/passwd file entries, separate home directories, etc. So, forget everything that's been said, and let's start over. Apologies, I've little idea of what is traditional and didn't realise my situation was unusual! > Where is this "./.procmailrc" file that you are trying to read with > the -m option? That is, what do you expect the current directory > (./) to be at the time procmail runs? the .procmailrc file is in /var/qmail/mailnames/domain.tld/test alongside the .qmail file and the Maildir directory In that case you need to tell spamassassin to look for its configuration files in that location. I don't want to have a per address/user configuration file, one configuration file per system is more than enough. There may be a way to finagle the options to "spamassassin" itself to make this work, but the easiest approach is to run spamd: spamd --create-prefs --virtual-config-dir=/var/qmail/mailnames/%d/%u (see "man spamd" for other options you might want to pass, such as "-m" to limit the number of simultaneous processes, etc.). This is a daemon that needs to run as a system service; you may already have an /etc/rc.d/init.d/spamassassin or similar script for managing this service. It depends on your OS and whether you built SA yourself or installed it with some kind of package management tool (other than CPAN). Then in each appropriate .procmailrc file, :0fw * < 256000 | /usr/bin/spamc -u [EMAIL PROTECTED] where you'll need to get the equivalent of "[EMAIL PROTECTED]" for each virtual address from somewhere; I don't know enough about qmail to tell you how, but if it's not in an environment variable, perhaps you can add it to the procmail command line after the ./.procmailrc and then refer to it here as $1. Just to confirm, the .procmailrc file isn't common to all users but the SA setup is. I'm no longer sure I understand what you consider to be "the SA setup." I mean all users should have the same rules and spam threshold, subject rewriting setting etc.
Re: Warnings in procmail log
jdow wrote:
From: "Geoff Soper" <[EMAIL PROTECTED]>
Bart Schaefer wrote:
On 7/8/06, Geoff Soper <[EMAIL PROTECTED]> wrote:
.qmail contains the lines: | true ./Maildir/
Caveat: I don't use qmail, and don't even particularly like qmail,
so what I'm about to say are really educated guesses rather than
definitive answers.
which I've altered to: | true | /usr/bin/procmail -m ./.procmailrc
No, don't use the -m option. Just use
| /usr/bin/procmail
and let procmail figure out where the $HOME/.procmailrc file is on
its own. If you want any options to procmail there at all, you want
the "-d recipient" option (where you'll have to get the value for
"recipient" from qmail somehow, I don't know how).
I think I need to specify the .procmailrc as the .procmailrc file is per
e-mail address, not per user or even system-wide
This only works if the number of email addresses is equal to or fewer
than the number of users. The .procmailrc file must reside in the
user's home directory as "~/.procmailrc". Now you could bend, fold,
and mutilate a generic .procmailrc file into a global "/etc/procmailrc"
file. But you still need a place for Bayes and user_prefs. If you go
to the extent of having SQL based Bayes and user_prefs database then
you can, I believe, break free of this restriction. You still need
the mail spool files for delivery to the individual email addresses.
The setup I have with the .procmailrc in each addresses'
/var/qmail/mailnames/domain.tld/address directory and referenced by
/var/qmail/mailnames/domain.tld/address/.qmail seems to work perfectltly
well, though I realise it might not be conventional.
and in that .procmailrc : DIR="./Maildir/"
What exactly do you think that's accomplishing? If you never refer
to $DIR again anywhere, this is meaningless. If you want to change
directories, assign to MAILDIR. If you are trying to force procmail
to deliver in maildir format, I think what you want is
I'm using $DIR in the recipes which I haven't quoted for clarity when
delivering to sub folders or to the main folder. An example recipe is:
:0
* ^X-Spam-Status: Yes
${DIR}.spam/
I don't know if my use of $DIR in this way is correct but it works.
DEFAULT="$HOME/Maildir/"
I'm not sure about the $HOME part, but DEFAULT should never be a
relative path (never one starting with "./" or with no directory
reference at all).
The recipes (that I didn't quote) refer to $DIR when delivering to that
location or any child folders so I'm happy that this is OK, bear in mind
my only known problem is these errors relating to SpamAssassin, the
.procmailrc filtering side of things works just fine. It may not be
the most elegant set up as I'm not totally sure of what I'm doing but
it appears to work.
Out of addled curiosity what does procmail think is the $HOME when
it is running? Have you ever figured that out?
I'll get back to you on that one! I doubt it's the
/var/qmail/mailnames/domain.tld/address directory though.
I've no desire to run different configurations for different users
or addresses, the single configuration is fine, I just want to
solve these errors I'm seeing in the procmail_log file.
Where is this "./.procmailrc" file that you are trying to read with
the -m option? That is, what do you expect the current directory
(./) to be at the time procmail runs?
the .procmailrc file is in /var/qmail/mailnames/domain.tld/test
alongside the .qmail file and the Maildir directory
And you REALLY think "./.procmailrc" points to that file? Off
hand I don't THINK so.
Seriously, it works almost perfectly, the various recipes in the
.procmail work very well in moving messages around.
If you really want exactly this same config for all users, then you
should move that ./.procmailrc file (wherever it is) to
/etc/procmailrc (with no dot) and insert DROPPRIVS=yes somewhere
before the recipe that runs spamassassin, probably at the very top of
the file (unless you want all users to write to the same log file as
well). If you later add things to /etc/procmailrc, you'll need to
research whether they belong above or below the DROPPRIVS (below will
usually be safe, but not always correct).
Just to confirm, the .procmailrc file isn't common to all users but
the SA setup is.
All I can say is good luck and quit running procmail as root that way.
I don't think I'm running procmail as root, my suspicion from looking at
the ownership of the files in the
/var/qmail/mailnames/domain.tld/address directory is that's running as
popuser. I'll try and confirm my suspicions.
Thanks,
Geoff
Spamassassin check failure
Hi everybody! I have a FreeBSD 5.4 server which is an mail proxy/server. In the last time there is a lot of incomming spam. Thas is why I decided to install an antivirus and spammassasin. The problem is that after I added the spamassassin configuration to my sendmail configuration, there is a very big delay sending the messages (up to 10 minutes). Jul 10 15:01:05 mail sm-mta[793]: k6ABsdbq000715: to=ludmila, delay=00:06:22, xdelay=00:00:01, mailer=local, pri=208760, relay=local, dsn=2.0.0, stat=Sent Jul 10 15:01:14 mail cucipop[797]: ana 192.168.1.4 0, 0 (0), 0 (0) Jul 10 15:01:21 mail sm-mta[742]: k6ABvIRw000742: Milter (spamassassin): timeout before data read Jul 10 15:01:22 mail sm-mta[742]: k6ABvIRw000742: Milter (spamassassin): to error state Jul 10 15:01:27 mail sm-mta[798]: k6ABvIRw000742: to=<[EMAIL PROTECTED]>, ctladdr=<[EMAIL PROTECTED]> (1003/1003), delay=00:04:07, xdelay=00:00:01, mailer=local, pri=80786, relay=local, dsn=2.0.0, stat=Sent --- Another problem (2) is the user which should run spamd daemon. I configured to run with root, but it tells (maillog) that it should not run with unexisting user or with root. I saw that spamassassin tries to read/write to files in $HOME/.spamassassin/ folder: user_prefs (), that is the reason why I'm using root. the last question: how to reject from the start messages that are sended to unexiting urser: Jul 10 15:02:38 mail sm-mta[805]: k6AC2WxH000805: <[EMAIL PROTECTED]>... User unknown Cheers, Sergiu -- View this message in context: http://www.nabble.com/Spamassassin-check-failure-tf1917969.html#a5250539 Sent from the SpamAssassin - Users forum at Nabble.com.
Re: set permission? how
Hey hello: some update whate i did to try spamassassin running The first thing I want to say is that it looks like spamassassin is running ?! Okay first of all what I did today: 1: I said # spamass-milter -p /var/run/spamass.sock -f -b [EMAIL PROTECTED] So he puts the spam to spambucket mailbox 2: I put some lines in the /etc/mail/local.cf like Matthias said: Bayes_path /home/spambucket/spamassassin/bayes Auto_whitelist_path /home/spambucket/spamassassin/auto-whitelist (I hope that this is the normal way to config it here, looks a little strange, but it seems to work). 3: I said # spamd ?u spambucket Okey hey created the file auto-whitelist in /home/spambucket/spamassassin/ now (looks good) 4: I sent myself some spam and look at the mail log : Jul 10 14:19:56 localhost spamd[3561]: auto-whitelist: open of auto-whitelist file failed: auto-whitelist: cannot open auto_whitelist_path /home/spambucket/spamassassin/auto-whitelist: Ongeschikte I/O besturingsoperatie voor apparaat So he says thate there is a I/O error. Now I looked at the file(/home/spambucket/spamassassin/auto-whitelist) it has user and group (root permission). 5: I changed this file to spambucket permission (I hope thate I did good by this???) Now it seems to work when I see my mail-log!! Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: from=<[EMAIL PROTECTED]>, size=2992, class=0, nrcpts=1, msgid=<[EMAIL PROTECTED]>, proto=ESMTP, daemon=MTA, relay=smtp-vbr9.xs4all.nl [194.109.24.29] Jul 10 14:44:18 localhost spamd[4022]: spamd: connection from localhost.localdomain [127.0.0.1] at port 33922 Jul 10 14:44:18 localhost spamd[4022]: spamd: processing message <[EMAIL PROTECTED]> for root:508 Jul 10 14:44:18 localhost spamd[4022]: spamd: identified spam (1001.3/5.0) for root:508 in 0.2 seconds, 3387 bytes. Jul 10 14:44:18 localhost spamd[4022]: spamd: result: Y 1001 - GTUBE,HTML_MESSAGE,MISSING_SUBJECT scantime=0.2,size=3387,user=root,uid=508,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=33922,mid=<[EMAIL PROTECTED]>,autolearn=no Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: X-Spam-Flag: YES Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: X-Spam-Status: Yes, score=1001.3 required=5.0 tests=GTUBE,HTML_MESSAGE,\n\tMISSING_SUBJECT autolearn=no version=3.1.3 Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: rcpt: [EMAIL PROTECTED] Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: X-Spam-Orig-To: <[EMAIL PROTECTED]> Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter delete: rcpt <[EMAIL PROTECTED]> Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: X-Spam-Level: ** Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on ProdecoBoy Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: header: Subject: *SPAM* Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter change: header Content-Type: from multipart/alternative;\n\tboundary="=_NextPart_000_003B_01C6A42F.4DC852B0" to multipart/mixed; boundary="--=_44B24BA2.69B131D0" Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter message: body replaced Jul 10 14:44:19 localhost spamd[4020]: prefork: child states: II Jul 10 14:44:19 localhost sendmail[4027]: k6ACiIWi004024: [EMAIL PROTECTED], delay=00:00:01, xdelay=00:00:00, mailer=local, pri=37064, dsn=2.0.0, stat=Sent Jul 10 14:44:20 localhost ipop3d[4029]: pop3 service init from 192.168.0.2 Okey seems to work now?? Okey some questions about what I did today: 1: When I restart my linuxbox spamassassin falls back to user root again! How can I prevent this to keep it -u spambucket? 2: In the beginning I told spamass-milter to sent the spam to spambucket mailbox, but it does not happen ;( Little mail-log: Jul 10 14:44:18 localhost sendmail[4024]: k6ACiIWi004024: Milter add: rcpt: [EMAIL PROTECTED] But I get the spam mail in my normal mailbox. How can I set this right? 3: did i do good to set permission on auto-whitelist to spambucket? Many credits and thanks for reply Matthias and Jdow -- View this message in context: http://www.nabble.com/set-permission--how-tf1903359.html#a5251716 Sent from the SpamAssassin - Users forum at Nabble.com.
Re: Warnings in procmail log
On 7/9/06, Geoff Soper <[EMAIL PROTECTED]> wrote: Apologies, I've little idea of what is traditional and didn't realise my situation was unusual! I didn't say it was unusual ... it's just not the assumed default state of affairs. I mean all users should have the same rules and spam threshold, subject rewriting setting etc. Including sharing a bayes database (if you're using bayes)? In that case all you have to do is make sure that the user who is running spamassassin is not root and has a writable home directory. If your conjecture that procmail is running as the user "popuser" is correct, make sure that "popuser" does not have its home directory set to "/".
Re: Strange problem
Sanford Whiteman wrote: Both servers have exactly the same config except for the auto-learn and bayes/user prefs are stored in mysql on the FreeBSD server. Thanks to all who replied. I found the problem and it's related to ixhash, the timeout doesn't work correctly / work at all. I see Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at/etc/mail/spamassassin/ixhash.pm line 91, line 2226. In the logs and the child never exits from processing the message. I've cc'd Dallas to see if he has any insights into the problem. Regards, Rick
Userprefs via X-Envelope-To header.
Hey all, Has anyone managed to successfully create an easy way to have a custom query look up prefs based on the X-Envelope-To header -- preferably with domain and username via custom query? I just need a few pointers here. -Dan -- "Oh, and we just recently got an invoice..." "Congratulations!" -JC and DM, regarding Unpredictable Billing, 8/18/2001 Dan Mahoney Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---
RE: Strange problem
> -Original Message-
> From: Rick Macdougall [mailto:[EMAIL PROTECTED]
> Sent: Monday, July 10, 2006 11:59
> To: [EMAIL PROTECTED]
> Cc: users@spamassassin.apache.org
> Subject: Re: Strange problem
>
> Sanford Whiteman wrote:
> >> Both servers have exactly the same config except for the
> auto-learn
> >> and bayes/user prefs are stored in mysql on the FreeBSD server.
> >
>
> Thanks to all who replied.
>
> I found the problem and it's related to ixhash, the timeout
> doesn't work correctly / work at all.
>
> I see
>
> Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
> at /etc/mail/spamassassin/ixhash.pm line 91, line 2226.
>
> Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
> at/etc/mail/spamassassin/ixhash.pm line 91, line 2226.
>
> In the logs and the child never exits from processing the message.
>
> I've cc'd Dallas to see if he has any insights into the problem.
>
the warns are being generated because the timeout value has been exceeded...
my $timeout = $permsgstatus->{main}->{conf}->{'ixhash_timeout'} ||
5;
eval {
Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die "ixhash
timeout reached"; });
the code is right.. you need to figure out why it times out. have you
hardcoded ixhash_timeout to some other value? have you tried manual
lookups from that box?
# host -tA abc.ix.dnsbl.manitu.net
Host abc.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
d
Re: Strange problem
Hi all,
Dallas, I think the problem isn't the request timing out - Rick says
'the child never exits from processing the message'...how can this be??
Rick, as a workaround either raise the timeout limit (both my servers
are located near Cologne/Germany). And take a look at how fast DNS is
for you
Dirk
Dallas Engelken schrieb:
-Original Message-
From: Rick Macdougall [mailto:[EMAIL PROTECTED]
Sent: Monday, July 10, 2006 11:59
To: [EMAIL PROTECTED]
Cc: users@spamassassin.apache.org
Subject: Re: Strange problem
Sanford Whiteman wrote:
Both servers have exactly the same config except for the
auto-learn
and bayes/user prefs are stored in mysql on the FreeBSD server.
Thanks to all who replied.
I found the problem and it's related to ixhash, the timeout
doesn't work correctly / work at all.
I see
Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
at /etc/mail/spamassassin/ixhash.pm line 91, line 2226.
Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
at/etc/mail/spamassassin/ixhash.pm line 91, line 2226.
In the logs and the child never exits from processing the message.
I've cc'd Dallas to see if he has any insights into the problem.
the warns are being generated because the timeout value has been exceeded...
my $timeout = $permsgstatus->{main}->{conf}->{'ixhash_timeout'} ||
5;
eval {
Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die "ixhash
timeout reached"; });
the code is right.. you need to figure out why it times out. have you
hardcoded ixhash_timeout to some other value? have you tried manual
lookups from that box?
# host -tA abc.ix.dnsbl.manitu.net
Host abc.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
d
Re: Strange problem
Dirk Bonengel wrote:
Hi all,
Dallas, I think the problem isn't the request timing out - Rick says
'the child never exits from processing the message'...how can this be??
Rick, as a workaround either raise the timeout limit (both my servers
are located near Cologne/Germany). And take a look at how fast DNS is
for you
Dirk
Dallas Engelken schrieb:
-Original Message-
From: Rick Macdougall [mailto:[EMAIL PROTECTED] Sent: Monday, July
10, 2006 11:59
To: [EMAIL PROTECTED]
Cc: users@spamassassin.apache.org
Subject: Re: Strange problem
Sanford Whiteman wrote:
Both servers have exactly the same config except for the
auto-learn
and bayes/user prefs are stored in mysql on the FreeBSD server.
Thanks to all who replied.
I found the problem and it's related to ixhash, the timeout doesn't
work correctly / work at all.
I see
Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at
/etc/mail/spamassassin/ixhash.pm line 91, line 2226.
Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
at/etc/mail/spamassassin/ixhash.pm line 91, line 2226.
In the logs and the child never exits from processing the message.
I've cc'd Dallas to see if he has any insights into the problem.
the warns are being generated because the timeout value has been
exceeded...
my $timeout =
$permsgstatus->{main}->{conf}->{'ixhash_timeout'} ||
5;
eval {
Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die "ixhash
timeout reached"; });
the code is right.. you need to figure out why it times out. have you
hardcoded ixhash_timeout to some other value? have you tried manual
lookups from that box?
# host -tA abc.ix.dnsbl.manitu.net
Host abc.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
d
Hi,
Lookups are fast, very fast. I'm running djb's dnscache server locally.
time host -tA abcd.ix.dnsbl.manitu.net
Host abcd.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
real0m0.138s
user0m0.003s
sys 0m0.005s
Like I said in my first message, it happens very rarely, so it's most
likely a network glitch. I haven't changed the timeout and it is still
at 10.
I've turned it off on that box for now but I'm more than willing to turn
it back on if someone wants me to test something for them.
As Dirk said, it should exit after the timeout is reached but it never
does, it just keeps logging errors about ixhash timing out.
Regards,
Rick
RE: Strange problem
> -Original Message-
> From: Rick Macdougall [mailto:[EMAIL PROTECTED]
> Sent: Monday, July 10, 2006 14:18
> To: users@spamassassin.apache.org
> Subject: Re: Strange problem
>
> Dirk Bonengel wrote:
> > Hi all,
> >
> > Dallas, I think the problem isn't the request timing out -
> Rick says
> > 'the child never exits from processing the message'...how
> can this be??
> >
> > Rick, as a workaround either raise the timeout limit (both
> my servers
> > are located near Cologne/Germany). And take a look at how
> fast DNS is
> > for you
> >
> > Dirk
> >
> > Dallas Engelken schrieb:
> >>> -Original Message-
> >>> From: Rick Macdougall [mailto:[EMAIL PROTECTED] Sent: Monday,
> >>> July 10, 2006 11:59
> >>> To: [EMAIL PROTECTED]
> >>> Cc: users@spamassassin.apache.org
> >>> Subject: Re: Strange problem
> >>>
> >>> Sanford Whiteman wrote:
> >>>
> > Both servers have exactly the same config except for
> the
> >>> auto-learn
> > and bayes/user prefs are stored in mysql on the FreeBSD server.
> >
> >>> Thanks to all who replied.
> >>>
> >>> I found the problem and it's related to ixhash, the
> timeout doesn't
> >>> work correctly / work at all.
> >>>
> >>> I see
> >>>
> >>> Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at
> >>> /etc/mail/spamassassin/ixhash.pm line 91, line 2226.
> >>>
> >>> Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached
> >>> at/etc/mail/spamassassin/ixhash.pm line 91, line 2226.
> >>>
> >>> In the logs and the child never exits from processing the message.
> >>>
> >>> I've cc'd Dallas to see if he has any insights into the problem.
> >>>
> >>>
> >>
> >> the warns are being generated because the timeout value has been
> >> exceeded...
> >>
> >>
> >> my $timeout =
> >> $permsgstatus->{main}->{conf}->{'ixhash_timeout'} || 5;
> >> eval {
> >> Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die
> >> "ixhash timeout reached"; });
> >>
> >> the code is right.. you need to figure out why it times
> out. have you
> >> hardcoded ixhash_timeout to some other value? have you
> tried manual
> >> lookups from that box?
> >>
> >> # host -tA abc.ix.dnsbl.manitu.net
> >> Host abc.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
> >>
> >> d
> >>
> >>
> >
>
> Hi,
>
> Lookups are fast, very fast. I'm running djb's dnscache
> server locally.
>
> time host -tA abcd.ix.dnsbl.manitu.net
> Host abcd.ix.dnsbl.manitu.net not found: 3(NXDOMAIN)
>
> real0m0.138s
> user0m0.003s
> sys 0m0.005s
>
> Like I said in my first message, it happens very rarely, so
> it's most likely a network glitch. I haven't changed the
> timeout and it is still at 10.
>
> I've turned it off on that box for now but I'm more than
> willing to turn it back on if someone wants me to test
> something for them.
>
> As Dirk said, it should exit after the timeout is reached but
> it never does, it just keeps logging errors about ixhash timing out.
>
well, then maybe there is a bug in the plugin code of SA? because the
timeout code is implemented in the same fashion pyzor/razor/etc are.
are you seeing the same issue with pyzor and/or razor enabled?
d
Re: Strange problem
Dallas L. Engelken wrote: -Original Message- Both servers have exactly the same config except for the auto-learn and bayes/user prefs are stored in mysql on the FreeBSD server. Thanks to all who replied. I found the problem and it's related to ixhash, the timeout doesn't work correctly / work at all. I see Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at/etc/mail/spamassassin/ixhash.pm line 91, line 2226. As Dirk said, it should exit after the timeout is reached but it never does, it just keeps logging errors about ixhash timing out. well, then maybe there is a bug in the plugin code of SA? because the timeout code is implemented in the same fashion pyzor/razor/etc are. are you seeing the same issue with pyzor and/or razor enabled? Hi, No, no problems with Razor and I'm not currently using Pyzor. It's entirely possible that it could be a problem with the plugin code. Regards, Rick
Strange timeouts
Hello to all ! I was looking to my sendmail log and I saw that there a big timeouts while sending messages. Also there are big timeout while sending from client (Mozilla thinderbird) up to 10 minutes. Next is proposed a part of my sendmail lod. think it could be useful :) -- Jul 10 23:03:24 mail sm-mta[2763]: k6AJwcbr002763: Milter (spamassassin): to error state Jul 10 23:03:26 mail sm-mta[2795]: k6AJwcbr002763: to=adriano, delay=00:04:45, xdelay=00:00:00, mailer=local, pri=33559, relay=local, dsn=2.0.0, stat=Sent Jul 10 23:03:56 mail sm-mta[2794]: k6AK36e2002794: Milter add: header: X-Virus-Scanned: ClamAV version 0.88.2, clamav-milter version 0.88.2 on mail.server.md Jul 10 23:03:56 mail sm-mta[2794]: k6AK36e2002794: Milter add: header: X-Virus-Status: Clean Jul 10 23:05:43 mail spamd[2781]: spamd: clean message (0.1/5.0) for ludmila:1008 in 249.0 seconds, 2385 bytes. Jul 10 23:05:43 mail spamd[2780]: spamd: identified spam (5.8/5.0) for ludmila:1008 in 249.0 seconds, 1165 bytes. Jul 10 23:05:43 mail spamd[2781]: spamd: result: . 0 - HTML_50_60,HTML_MESSAGE scantime=249.0,size=2385,user=valentino,uid=1008,required_score=5.0,rhost=localhost.ns1.globnet.md,raddr=127.0.0.1,rport=50989,mid=<[EMAIL PROTECTED]>,autolearn=no Jul 10 23:05:43 mail spamd[2780]: spamd: result: Y 5 - INFO_TLD,MSGID_FROM_MTA_ID,SUBJ_ILLEGAL_CHARS scantime=249.0,size=1165,user=ludmila,uid=1008,required_score=5.0,rhost=localhost.ns1.globnet.md,raddr=127.0.0.1,rport=56132,mid=<[EMAIL PROTECTED]>,autolearn=no Jul 10 23:06:12 mail spamd[2780]: spamd: copy_config timeout, respawning child process after 1 messages at /usr/local/bin/spamd line 966. Jul 10 23:06:12 mail spamd[2781]: spamd: copy_config timeout, respawning child process after 1 messages at /usr/local/bin/spamd line 966. can someone explain me the cause of this timeout and how can I reduce them ? thanks a lot, Humer -- View this message in context: http://www.nabble.com/Strange-timeouts-tf1920574.html#a5258347 Sent from the SpamAssassin - Users forum at Nabble.com.
Spamassassin and SPF
I have a question regarding Spamassassin's behaviour with SPF. My configuration is running Postfix v2.1.5 and Spamassassin v3.0.1 is called by Amavis-new v2.2.0 as is installed as part of Mac OS X Server 10.4.x All domains handled by the email server have strict SPF records that require all account holders to send their email through the server. As such all account holders authenticate with the server through the submission port 587 when they want to send email from outside the network. I noticed that Spamassassin is scoring an SPF_FAIL with these users based on the fact that they are connecting with IPs that fall outside what is in their domain's SPF record. It seems to me that this is the incorrect behaviour. Spamassassin should not be evaluating the IP address from which the clients are connecting but should be ignoring that altogether. The Postfix policy daemon is handling this correctly , as it should, ignoring where the client is connecting from since it is the outbound server. It's not evaluating itself against the SPF record. Is there something that is configured incorrectly that is causing this to happen? Has this behaviour changed in a more recent version of Spamassassin? Any suggestions would be appritiated. -- Gino Cerullo Pixel Point Studios 21 Chesham Drive Toronto, ON M3M 1W6 T: 416-247-7740 F: 416-247-7503 smime.p7s Description: S/MIME cryptographic signature
RE: Strange problem
> -Original Message- > From: Rick Macdougall [mailto:[EMAIL PROTECTED] > Sent: Monday, July 10, 2006 14:42 > To: Dallas L. Engelken > Cc: users@spamassassin.apache.org > Subject: Re: Strange problem > > Dallas L. Engelken wrote: > >> -Original Message- > > > >>> Both servers have exactly the same config except for > >> the > > auto-learn > >>> and bayes/user prefs are stored in mysql on the > FreeBSD server. > >>> > > Thanks to all who replied. > > > > I found the problem and it's related to ixhash, the > >> timeout doesn't > > work correctly / work at all. > > > > I see > > > > Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at > > /etc/mail/spamassassin/ixhash.pm line 91, line 2226. > > > > Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached > > at/etc/mail/spamassassin/ixhash.pm line 91, > line 2226. > >> > >> As Dirk said, it should exit after the timeout is reached but it > >> never does, it just keeps logging errors about ixhash timing out. > >> > > > > well, then maybe there is a bug in the plugin code of SA? > because the > > timeout code is implemented in the same fashion pyzor/razor/etc are. > > are you seeing the same issue with pyzor and/or razor enabled? > > Hi, > > No, no problems with Razor and I'm not currently using Pyzor. > > It's entirely possible that it could be a problem with the > plugin code. > well, the pyzor/razor plugins now (svn) use the Mail::SpamAssassin::Timeout code, instead of M::SA::Util->trap_sigalrm_fully i'm not sure if that would help or not, but you'd have to be running something newer than 3.1.3. anyways, check by spamd.log's, and is see these once in a while. twice yesterday. but the 17605 child that timed out didnt have any problem processing its next message. 2006-07-10 03:38:40.216974500 [17605] error: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:38:40.217110500 [17605] error: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:38:40.217307500 [17605] warn: spamd: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:39:44.843887500 [17605] info: spamd: connection from localhost [127.0.0.1] at port 35831 2006-07-10 03:39:44.967460500 [17605] info: spamd: identified spam (100.0/5.0) for $global:200 in 0.1 seconds, 1978 bytes. d
Re: Spamassassin and SPF
If the MSA in question is *ONLY* an MSA, you're easiest quickest fix is to just not trust it (or mark it as trusted but not internal). If SA is running on that host, or it's also doing another mail function, MX or intermediate relay, etc., then describe your mail topology and we'll help you out. Daryl
Re: Strange problem
Dallas L. Engelken wrote: -Original Message- well, the pyzor/razor plugins now (svn) use the Mail::SpamAssassin::Timeout code, instead of M::SA::Util->trap_sigalrm_fully i'm not sure if that would help or not, but you'd have to be running something newer than 3.1.3. anyways, check by spamd.log's, and is see these once in a while. twice yesterday. but the 17605 child that timed out didnt have any problem processing its next message. 2006-07-10 03:38:40.216974500 [17605] error: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:38:40.217110500 [17605] error: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:38:40.217307500 [17605] warn: spamd: ixhash timeout reached at /usr/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Plugin/iXhash.pm line 105. 2006-07-10 03:39:44.843887500 [17605] info: spamd: connection from localhost [127.0.0.1] at port 35831 2006-07-10 03:39:44.967460500 [17605] info: spamd: identified spam (100.0/5.0) for $global:200 in 0.1 seconds, 1978 bytes. d No, Still running 3.1.3, if it wasn't a diskless server I'd update to a more recent version but that would affect all the diskless servers, not just one to test on and I don't want to do that without testing it first. My logs show Jul 10 11:12:41 spa010 spamd[29830]: spamd: connection from mta011.aei.ca [206.123.6.241] at port 53484 Jul 10 11:12:41 spa010 spamd[29830]: spamd: processing message <[EMAIL PROTECTED]> for [EMAIL PROTECTED]:89 Jul 10 11:12:51 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:12:51 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:13:01 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Jul 10 11:13:11 spa010 spamd[29830]: ixhash timeout reached at /etc/mail/spamassassin/ixhash.pm line 91, line 2226. Which is strange since I only have the standard 3 lookups in the .cf file. The child hung around until I killed with signal 9, signal 15 did nothing to it. Stuff in /var and /etc is r/w, just about everything else is read-only so if there is a patched rule or something I can try I can do it if it can go in either of those two places. Rick
Re: Strange problem
Rick Macdougall wrote: Dallas L. Engelken wrote: -Original Message- well, the pyzor/razor plugins now (svn) use the Mail::SpamAssassin::Timeout code, instead of M::SA::Util->trap_sigalrm_fully i'm not sure if that would help or not, but you'd have to be running something newer than 3.1.3. No, Still running 3.1.3, if it wasn't a diskless server I'd update to a more recent version but that would affect all the diskless servers, not just one to test on and I don't want to do that without testing it first. 3.1.3 has M::SA::Timeout, which will definitely work better for timing out eval blocks. Daryl
Re: Strange timeouts
humer wrote: Hello to all ! I was looking to my sendmail log and I saw that there a big timeouts while sending messages. Also there are big timeout while sending from client (Mozilla thinderbird) up to 10 minutes. Next is proposed a part of my sendmail lod. think it could be useful :) -- Jul 10 23:03:24 mail sm-mta[2763]: k6AJwcbr002763: Milter (spamassassin): to error state Jul 10 23:03:26 mail sm-mta[2795]: k6AJwcbr002763: to=adriano, delay=00:04:45, xdelay=00:00:00, mailer=local, pri=33559, relay=local, dsn=2.0.0, stat=Sent Jul 10 23:03:56 mail sm-mta[2794]: k6AK36e2002794: Milter add: header: X-Virus-Scanned: ClamAV version 0.88.2, clamav-milter version 0.88.2 on mail.server.md Jul 10 23:03:56 mail sm-mta[2794]: k6AK36e2002794: Milter add: header: X-Virus-Status: Clean Jul 10 23:05:43 mail spamd[2781]: spamd: clean message (0.1/5.0) for ludmila:1008 in 249.0 seconds, 2385 bytes. Jul 10 23:05:43 mail spamd[2780]: spamd: identified spam (5.8/5.0) for ludmila:1008 in 249.0 seconds, 1165 bytes. Jul 10 23:05:43 mail spamd[2781]: spamd: result: . 0 - HTML_50_60,HTML_MESSAGE scantime=249.0,size=2385,user=valentino,uid=1008,required_score=5.0,rhost=localhost.ns1.globnet.md,raddr=127.0.0.1,rport=50989,mid=<[EMAIL PROTECTED]>,autolearn=no Jul 10 23:05:43 mail spamd[2780]: spamd: result: Y 5 - INFO_TLD,MSGID_FROM_MTA_ID,SUBJ_ILLEGAL_CHARS scantime=249.0,size=1165,user=ludmila,uid=1008,required_score=5.0,rhost=localhost.ns1.globnet.md,raddr=127.0.0.1,rport=56132,mid=<[EMAIL PROTECTED]>,autolearn=no Jul 10 23:06:12 mail spamd[2780]: spamd: copy_config timeout, respawning child process after 1 messages at /usr/local/bin/spamd line 966. Jul 10 23:06:12 mail spamd[2781]: spamd: copy_config timeout, respawning child process after 1 messages at /usr/local/bin/spamd line 966. can someone explain me the cause of this timeout and how can I reduce them ? thanks a lot, Humer It looks like you're using spamass-milter. I tried it for a couple of weeks and had constant problems with it hanging up it's children. This of course causes Sendmail to sit idle until it's timeouts for that milter are reached. I gave up on it and ended up writing my own milter. I suggest you look at using one of the many other milters available. Or ask for help from the spamass-milter folks. Daryl
Re: Strange problem
Daryl C. W. O'Shea wrote: Rick Macdougall wrote: Dallas L. Engelken wrote: -Original Message- well, the pyzor/razor plugins now (svn) use the Mail::SpamAssassin::Timeout code, instead of M::SA::Util->trap_sigalrm_fully i'm not sure if that would help or not, but you'd have to be running something newer than 3.1.3. No, Still running 3.1.3, if it wasn't a diskless server I'd update to a more recent version but that would affect all the diskless servers, not just one to test on and I don't want to do that without testing it first. 3.1.3 has M::SA::Timeout, which will definitely work better for timing out eval blocks. Have any snippets of code I could try ? Regards, Rick
Re: Strange problem
Daryl C. W. O'Shea wrote:
Rick Macdougall wrote:
Dallas L. Engelken wrote:
-Original Message-
well, the pyzor/razor plugins now (svn) use the
Mail::SpamAssassin::Timeout code, instead of
M::SA::Util->trap_sigalrm_fully
i'm not sure if that would help or not, but you'd have to be running
something newer than 3.1.3.
No, Still running 3.1.3, if it wasn't a diskless server I'd update to
a more recent version but that would affect all the diskless servers,
not just one to test on and I don't want to do that without testing it
first.
3.1.3 has M::SA::Timeout, which will definitely work better for timing
out eval blocks.
Daryl
Does this patch below look even vaguely correct ? (and of course the
same loop around hash 2 etc)
--- ixhash.pm 2006-07-04 18:46:55.707706000 -0400
+++ ixhash.pm.new 2006-07-10 19:00:03.964058000 -0400
@@ -85,12 +85,6 @@
# here we implement proper alarms, ala Pyzor, Razor2 plugins.
# keep the alarm as $oldalarm, so we dont loose the
timeout-child alarm
# see
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3828#c123
-my $oldalarm = 0;
-my $timeout = $permsgstatus->{main}->{conf}->{'ixhash_timeout'}
|| 5;
-eval {
-Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die
"ixhash
timeout reached"; });
-$oldalarm = alarm($timeout);
-
#--
-
# Creation of hash # 1 if following conditions are met:
# - mail contains at least 16 spaces or tabs
@@ -112,7 +106,12 @@
dbg ("IXHASH: Computed hash-value $digest via method 1");
dbg ("IXHASH: Now checking $digest.$dnsserver");
# Now check via DNS query
-$answer = $resolver->search($digest.'.'.$dnsserver,
"A", "IN");
+ my $timeout =
$permsgstatus->{main}->{conf}->{'ixhash_timeout'}
|| 5;
+ my $timer = Mail::SpamAssassin::Timeout->new({ secs =>
$timeout
});
+ my $err = $time->run_and_catch(sub {
+ $answer =
$resolver->search($digest.'.'.$dnsserver, "A","IN");
+ }
+
if ($answer) {
foreach $rr ($answer->answer) {
next unless $rr->type eq "A";
@@ -120,6 +119,11 @@
$hits = 1 if $rr->address;
}
}
+
+ if ($timer->timed_out()) {
+ dbg("ixhash: check timed out after $timeout
seconds");
+ return 0;
+ }
}
# End of computing hash #1
#-
Re: Spamassassin and SPF
On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote: If the MSA in question is *ONLY* an MSA, you're easiest quickest fix is to just not trust it (or mark it as trusted but not internal). If I'm understanding you correctly, I'm fairly new to this, then no, the MSA (Mail Submission Agent) is also the MTA (Mail Transfer Agent.) All mail for the domains in question are received and delivered by this one server. trusted_networks lists my public and private IP addresses (WAN & LAN) internal_networks lists only private IP addresses (LAN) I think that's how those are suppose to work. I haven't bothered to try and list the IPs where outside users may connect from since that can be anywhere. If SA is running on that host, or it's also doing another mail function, MX or intermediate relay, etc., then describe your mail topology and we'll help you out. Okay, this one server is running Postfix, Amavisd-new, Spamassassin, ClamAV, pretty standard stuff. This server does not relay to any other servers for mail delivery and is not a relay for any servers. Both incoming and outgoing mail is handled by Postfix which hands it off to Amavisd-new which calls Spamassassin and ClamAV to scan the message. Clean email is handed back to Postfix to complete delivery. Bad email is quarantined by Amavis-new. At least that's my understanding of how it works. Generally, because of SPF, all users submit directly to this server and all outgoing email for the domains handled by this server are delivered directly by this server, no relays involved. Fairly simple stuff and it all works as expected except for SA. Daryl This is what I'm seeing if it helps. 0.9 SPF_FAIL SPF: sender does not match SPF record (fail) [SPF failed: Please see http://www.openspf.org/why.html? sender=dmaguire% 40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstudios .lan] maguiremarketing.com is hosted by the server in question. 24.42.90.104 is the IP address that he is connecting from. In this case it's the IP assigned dynamically to his router by his cable company (ISP). Like I said the Postfix policy daemon that checks SPF correctly ignores this IP address as it represents the MUA (Mail User Agent.) I guess I'm expecting SA to know that as well but I guess it doesn't. -- Gino Cerullo Pixel Point Studios 21 Chesham Drive Toronto, ON M3M 1W6 T: 416-247-7740 F: 416-247-7503 smime.p7s Description: S/MIME cryptographic signature
Re: Strange problem
Rick Macdougall wrote:
Does this patch below look even vaguely correct ? (and of course the
same loop around hash 2 etc)
I've just had my first brief (20 second) look at this plugin code. Your
patch looks reasonable. You'll also need to lose the current alarm
stuff near the end though:
if (defined $oldalarm) {
alarm $oldalarm; $oldalarm = undef;
}
Also, the original plugin, unless I'm blind, seems to have an unbalanced
set of curly braces:
eval {
Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die
"ixhash timeout reached"; });
$oldalarm = alarm($timeout);
Further, I don't know why it even uses it's own DNS resolver. It would
make much more sense to use SA's and do all this in the background.
Daryl
RE: Strange problem
> || 5;
> + my $timer =
> Mail::SpamAssassin::Timeout->new({ secs =>
> $timeout
> });
> + my $err = $time->run_and_catch(sub {
> + $answer =
I assume you mean $timer-> (vs $time->) there?
D
Re: Spamassassin and SPF
Gino Cerullo wrote:
On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote:
If the MSA in question is *ONLY* an MSA, you're easiest quickest fix
is to just not trust it (or mark it as trusted but not internal).
If I'm understanding you correctly, I'm fairly new to this, then no, the
MSA (Mail Submission Agent) is also the MTA (Mail Transfer Agent.) All
mail for the domains in question are received and delivered by this one
server.
trusted_networks lists my public and private IP addresses (WAN & LAN)
internal_networks lists only private IP addresses (LAN)
These are incorrect for your setup. Trusted and internal networks
should both list YOUR public and private IP addresses. IP addresses of
other people could go in trusted networks only, but don't even bother
with that.
The only 'one' of your addresses that wouldn't go in internal networks
would be your MSA if it was only an MSA, but yours isn't as you've said
it's also working as an MX.
I think that's how those are suppose to work. I haven't bothered to try
and list the IPs where outside users may connect from since that can be
anywhere.
So... fixing the above will resolve your SPF (and other) issues for LAN
users. Add all the IP space you control to trusted networks (including
your internal RFC 1938 ranges). Don't even bother with setting internal
networks. In your case they'll be the same as your trusted networks, so
SA will just copy them for you if you leave internal networks unset.
That still leaves a problem with your roaming users not being trusted.
Postfix makes this oh so fun.
If SA is running on that host, or it's also doing another mail
function, MX or intermediate relay, etc., then describe your mail
topology and we'll help you out.
Okay, this one server is running Postfix, Amavisd-new, Spamassassin,
ClamAV, pretty standard stuff. This server does not relay to any other
servers for mail delivery and is not a relay for any servers. Both
incoming and outgoing mail is handled by Postfix which hands it off to
Amavisd-new which calls Spamassassin and ClamAV to scan the message.
Clean email is handed back to Postfix to complete delivery. Bad email is
quarantined by Amavis-new. At least that's my understanding of how it
works.
Generally, because of SPF, all users submit directly to this server and
all outgoing email for the domains handled by this server are delivered
directly by this server, no relays involved. Fairly simple stuff and it
all works as expected except for SA.
This is what I'm seeing if it helps.
0.9 SPF_FAIL SPF: sender does not match SPF record (fail)
[SPF failed: Please see
http://www.openspf.org/why.html?sender=dmaguire%40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstudios.lan]
maguiremarketing.com is hosted by the server in question.
24.42.90.104 is the IP address that he is connecting from. In this case
it's the IP assigned dynamically to his router by his cable company (ISP).
Like I said the Postfix policy daemon that checks SPF correctly ignores
this IP address as it represents the MUA (Mail User Agent.) I guess I'm
expecting SA to know that as well but I guess it doesn't.
You can't expect SA to do something based on information that Postfix
has and SA doesn't (because Postfix doesn't give it up, at least not
easily), that is, SA doesn't know the user is an authenticated user
while Postfix does.
You have two options:
1) Quoting mouss...
mouss wrote:
> martin f krafft a écrit :
>> Well, sure, this makes sense, but how can I support this standard
>> use-case? Postfix adding a SASL-header that causes Spamassassin then
>> to ignore the message isn't the solution as spammers would simply do
>> that sooner or later. Short of whitelisting people, what should
>> I do?
>>
>
> how do you integrate SA with postfix?
>
> If using content_filter, then you could skip SA for authenticated users.
> for instance:
>
> content_filter=
> #or this to still check for viruses:
> #content_filter=scan:[clamsmtp.domain.example]:10020
> smtpd_recipient_restrictions =
>...
>permit_sasl_authenticated
>permit_mynetworks
>reject_unauth_destination
>check_client_access pcre:/etc/postfix/default_filter.pcre
>...
>
> == default_filter.pcre:
> /./FILTER scan:[amavisd.domain.example]:10024
>
> ('scan' is the name of the transport as defined in master.cf).
or
2) Upgrade to Postfix 2.3, if necessary, set
"smtpd_sasl_authenticated_header yes" in your Postfix config and then
offer to buy me lunch next time I'm in the city to persuade me to make a
patch to support this. :)
Related info:
http://www.postfix.org/postconf.5.html#smtpd_sasl_authenticated_header
http://wiki.apache.org/spamassassin/DynablockIssues
Daryl
Re: Strange problem
Dallas L. Engelken wrote:
|| 5;
+ my $timer =
Mail::SpamAssassin::Timeout->new({ secs =>
$timeout
});
+ my $err = $time->run_and_catch(sub {
+ $answer =
I assume you mean $timer-> (vs $time->) there?
D
This diff seems to work. I'm going to let it run while I can watch it
and see what happens.
Anyone let me know if you'd rather an attachment.
If anyone sees anything really stupid, please let me know, I'm thick
skinned :)
Regards,
Rick
--- ixhash.pm.old 2006-07-10 19:43:50.22640 -0400
+++ ixhash.pm 2006-07-10 19:53:10.49773 -0400
@@ -32,6 +32,7 @@
use Mail::SpamAssassin;
use Mail::SpamAssassin::Plugin;
use Mail::SpamAssassin::Util;
+use Mail::SpamAssassin::Timeout;
use Digest::MD5 qw(md5 md5_hex md5_base64);
use Net::DNS;
use Net::DNS::Resolver;
@@ -74,6 +75,7 @@
my ($digest,$answer,$ixdigest,$body) = "";
my @body = $permsgstatus->{msg}->get_body();
my $resolver = Net::DNS::Resolver->new;
+ my $timeout = $permsgstatus->{main}->{conf}->{'ixhash_timeout'}
|| 5;
my $body_copy = "";
foreach (@body) {
$body .= join "", @$_;
@@ -85,12 +87,6 @@
# here we implement proper alarms, ala Pyzor, Razor2 plugins.
# keep the alarm as $oldalarm, so we dont loose the
timeout-child alarm
# see
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3828#c123
-my $oldalarm = 0;
-my $timeout = $permsgstatus->{main}->{conf}->{'ixhash_timeout'}
|| 5;
-eval {
-Mail::SpamAssassin::Util::trap_sigalrm_fully(sub { die
"ixhash
timeout reached"; });
-$oldalarm = alarm($timeout);
-
#--
-
# Creation of hash # 1 if following conditions are met:
# - mail contains at least 16 spaces or tabs
@@ -112,7 +108,11 @@
dbg ("IXHASH: Computed hash-value $digest via method 1");
dbg ("IXHASH: Now checking $digest.$dnsserver");
# Now check via DNS query
-$answer = $resolver->search($digest.'.'.$dnsserver,
"A", "IN");
+ my $timer = Mail::SpamAssassin::Timeout->new({ secs =>
$timeout
});
+ my $err = $timer->run_and_catch(sub {
+ $answer =
$resolver->search($digest.'.'.$dnsserver, "A",
"IN");
+ });
+
if ($answer) {
foreach $rr ($answer->answer) {
next unless $rr->type eq "A";
@@ -120,6 +120,11 @@
$hits = 1 if $rr->address;
}
}
+
+ if ($timer->timed_out()) {
+ dbg("ixhash: check timed out after $timeout
seconds");
+ return 0;
+ }
}
# End of computing hash #1
#-
@@ -146,7 +151,10 @@
dbg ("IXHASH: Computed hash-value $digest via method 2");
dbg ("IXHASH: Now checking $digest.$dnsserver");
# Now check via DNS query
-$answer = $resolver->search($digest.'.'.$dnsserver,
"A", "IN");
+ my $timer = Mail::SpamAssassin::Timeout->new({ secs =>
$timeout
});
+ my $err = $timer->run_and_catch(sub {
+ $answer =
$resolver->search($digest.'.'.$dnsserver, "A",
"IN");
+ });
if ($answer) {
foreach $rr ($answer->answer) {
next unless $rr->type eq "A";
@@ -154,6 +162,10 @@
$hits = 1 if $rr->address;
}
}
+ if ($timer->timed_out()) {
+ dbg("ixhash: check timed out after $timeout
seconds");
+ return 0;
+ }
}
# End of computing hash #2
@@ -178,7 +190,10 @@
dbg ("IXHASH: Computed hash-value $digest via method 3");
dbg ("IXHASH: Now checking $digest.$dnsserver");
# Check via DNS
-$answer = $resolver->search($digest.'.'.$dnsserver,
"A", "IN");
+ my $timer = Mail::SpamAssassin::Timeout->new({ secs =>
$timeout
});
+ my $err = $timer->run_and_catch(sub {
+ $answer =
$resolver->search($digest.'.'.$dnsserver, "A",
"IN");
+ });
if ($answer) {
foreach $rr ($answer->answer) {
next unless $rr->type eq "A";
@@ -186,18 +201,11 @@
$hits = 1 if $rr->address;
}
}
+ if ($timer->timed_out()) {
+ dbg("ixhash: check timed out after $timeout
seconds");
Re: Strange problem
Daryl C. W. O'Shea wrote: Rick Macdougall wrote: Further, I don't know why it even uses it's own DNS resolver. It would make much more sense to use SA's and do all this in the background. I'm just assuming that the original author wasn't aware of it's existence or was too lazy to be bothered to use it. If my current patch does work I'll look at adding SA's resolver tomorrow instead of it's own. Regards, Rick
Re: Spamassassin and SPF
On 10-Jul-06, at 7:39 PM, Daryl C. W. O'Shea wrote:
Gino Cerullo wrote:
On 10-Jul-06, at 5:49 PM, Daryl C. W. O'Shea wrote:
If the MSA in question is *ONLY* an MSA, you're easiest quickest
fix is to just not trust it (or mark it as trusted but not
internal).
If I'm understanding you correctly, I'm fairly new to this, then
no, the MSA (Mail Submission Agent) is also the MTA (Mail Transfer
Agent.) All mail for the domains in question are received and
delivered by this one server.
trusted_networks lists my public and private IP addresses (WAN & LAN)
internal_networks lists only private IP addresses (LAN)
These are incorrect for your setup. Trusted and internal networks
should both list YOUR public and private IP addresses. IP
addresses of other people could go in trusted networks only, but
don't even bother with that.
The only 'one' of your addresses that wouldn't go in internal
networks would be your MSA if it was only an MSA, but yours isn't
as you've said it's also working as an MX.
I think that's how those are suppose to work. I haven't bothered
to try and list the IPs where outside users may connect from since
that can be anywhere.
So... fixing the above will resolve your SPF (and other) issues for
LAN users. Add all the IP space you control to trusted networks
(including your internal RFC 1938 ranges). Don't even bother with
setting internal networks. In your case they'll be the same as
your trusted networks, so SA will just copy them for you if you
leave internal networks unset.
Thanks for the clarification. I'll make the changes.
That still leaves a problem with your roaming users not being
trusted. Postfix makes this oh so fun.
If SA is running on that host, or it's also doing another mail
function, MX or intermediate relay, etc., then describe your mail
topology and we'll help you out.
Okay, this one server is running Postfix, Amavisd-new,
Spamassassin, ClamAV, pretty standard stuff. This server does not
relay to any other servers for mail delivery and is not a relay
for any servers. Both incoming and outgoing mail is handled by
Postfix which hands it off to Amavisd-new which calls Spamassassin
and ClamAV to scan the message. Clean email is handed back to
Postfix to complete delivery. Bad email is quarantined by Amavis-
new. At least that's my understanding of how it works.
Generally, because of SPF, all users submit directly to this
server and all outgoing email for the domains handled by this
server are delivered directly by this server, no relays involved.
Fairly simple stuff and it all works as expected except for SA.
This is what I'm seeing if it helps.
0.9 SPF_FAIL SPF: sender does not match SPF record
(fail)
[SPF failed: Please see http://www.openspf.org/why.html?
sender=dmaguire%
40maguiremarketing.com&ip=24.42.90.104&receiver=server.pixelpointstud
ios.lan] maguiremarketing.com is hosted by the server in question.
24.42.90.104 is the IP address that he is connecting from. In this
case it's the IP assigned dynamically to his router by his cable
company (ISP).
Like I said the Postfix policy daemon that checks SPF correctly
ignores this IP address as it represents the MUA (Mail User
Agent.) I guess I'm expecting SA to know that as well but I guess
it doesn't.
You can't expect SA to do something based on information that
Postfix has and SA doesn't (because Postfix doesn't give it up, at
least not easily), that is, SA doesn't know the user is an
authenticated user while Postfix does.
This is what I thought. Thanks for confirming this as well.
You have two options:
1) Quoting mouss...
mouss wrote:
> martin f krafft a écrit :
>> Well, sure, this makes sense, but how can I support this standard
>> use-case? Postfix adding a SASL-header that causes Spamassassin
then
>> to ignore the message isn't the solution as spammers would
simply do
>> that sooner or later. Short of whitelisting people, what should
>> I do?
>>
>
> how do you integrate SA with postfix?
>
> If using content_filter, then you could skip SA for authenticated
users.
> for instance:
>
> content_filter=
> #or this to still check for viruses:
> #content_filter=scan:[clamsmtp.domain.example]:10020
> smtpd_recipient_restrictions =
>...
>permit_sasl_authenticated
>permit_mynetworks
>reject_unauth_destination
>check_client_access pcre:/etc/postfix/default_filter.pcre
>...
>
> == default_filter.pcre:
> /./FILTER scan:[amavisd.domain.example]:10024
>
> ('scan' is the name of the transport as defined in master.cf).
I see but is the trade-off here that if SA skips authenticated users
and they are compromised then they can become spam sources that
wouldn't be caught locally or does it only skip SPF and still do all
other scans?
or
2) Upgrade to Postfix 2.3, if necessary, set
"smtpd_sasl_authenticated_header yes" in your Postfix config and
then offer to b
Re: Spamassassin and SPF
On 7/10/2006 8:47 PM, Gino Cerullo wrote: On 10-Jul-06, at 7:39 PM, Daryl C. W. O'Shea wrote: You have two options: 1) Quoting mouss... mouss wrote: > martin f krafft a écrit : >> Well, sure, this makes sense, but how can I support this standard >> use-case? Postfix adding a SASL-header that causes Spamassassin then >> to ignore the message isn't the solution as spammers would simply do >> that sooner or later. Short of whitelisting people, what should >> I do? >> > > how do you integrate SA with postfix? > > If using content_filter, then you could skip SA for authenticated users. I see but is the trade-off here that if SA skips authenticated users and they are compromised then they can become spam sources that wouldn't be caught locally or does it only skip SPF and still do all other scans? Skipping SA for auth'd users in your Postfix config would skip all of SA, not just SPF checks in SA. I wouldn't necessarily call not spam checking local spam zombies a trade-off. Personally, I'd want to receive the spam from these local machines/users so that I can take action against/for those users. or 2) Upgrade to Postfix 2.3, if necessary, set "smtpd_sasl_authenticated_header yes" in your Postfix config and then offer to buy me lunch next time I'm in the city to persuade me to make a patch to support this. :) Are you a contributor to SA's development? I've been known to commit enough good, and apparently not enough bad, code to the SA code base. I guess the same compromise will exist in this scenario? Would it be difficult to get SA to see that the user is authenticated and just skip SPF but still do everything else? In this case (SA knows the user is authenticated), SA does The Right Thing (tm). All regex based tests and network tests against body content are done and the appropriate network based checks are done against the appropriate hosts. I see from the header in the message you sent that you have deployed DKIM. I'm hoping to do that as well but not for a while yet. Do similar problems arise with DKIM and SA as we've discussed here with SPF? DKIM doesn't rely on any defined set of relays. It uses the envelope sender (usually just the domain) and the signature found in the headers. Also note that SPF isn't the only thing suffering from your trust path issues, it's just a symptom you've noticed. You'll also currently be doing dynablock checks against users you'd rather not be, along with a whole slew of other tests that will FP when SA thinks it's testing mail from some random system/zombie and not an authenticated user. Let me know if you're running Postfix 2.3 and can enable the auth headers in your config. I'll probably get to making a patch tonight as long as the rain doesn't stop and I don't get distracted by the big stash of fireworks I've accumulated. :) Daryl
Re: Spamassassin and SPF
On 10-Jul-06, at 9:16 PM, Daryl C. W. O'Shea wrote: Snip, snip and more snip. Thanks for all that good info. I see from the header in the message you sent that you have deployed DKIM. I'm hoping to do that as well but not for a while yet. Do similar problems arise with DKIM and SA as we've discussed here with SPF? DKIM doesn't rely on any defined set of relays. It uses the envelope sender (usually just the domain) and the signature found in the headers. Someday I'll have some time to play with this and get a better understanding of DKIM. Also note that SPF isn't the only thing suffering from your trust path issues, it's just a symptom you've noticed. You'll also currently be doing dynablock checks against users you'd rather not be, along with a whole slew of other tests that will FP when SA thinks it's testing mail from some random system/zombie and not an authenticated user. So what you're saying is that I'm better off not scanning authenticated users. I'll take your word on that. Let me know if you're running Postfix 2.3 and can enable the auth headers in your config. I'll probably get to making a patch tonight as long as the rain doesn't stop and I don't get distracted by the big stash of fireworks I've accumulated. :) Well I'm running Postfix v2.1.3 (standard in OS X Server 10.4.x.) I'm waiting until Apple previews OS X 10.5 in August to see whether 10.5 includes Postfix v2.3. If not than I may do the upgrade myself in 10.4. Since SA is being called by Amavisd-new shouldn't the changes to ignore authenticated user happen there? I think I read that somewhere, maybe the Amavis mailing list. That's the problem with being subscribed to all these lists. They all start to run into each other in your head. Off to the archives I go. BIG STASH OF FIREWORKS! Boy I'm glad you don't live in my neighbourhood. :-O Daryl -- Gino Cerullo Pixel Point Studios 21 Chesham Drive Toronto, ON M3M 1W6 T: 416-247-7740 F: 416-247-7503 smime.p7s Description: S/MIME cryptographic signature
Re: Spamassassin and SPF
On 7/10/2006 9:57 PM, Gino Cerullo wrote: On 10-Jul-06, at 9:16 PM, Daryl C. W. O'Shea wrote: Snip, snip and more snip. Also note that SPF isn't the only thing suffering from your trust path issues, it's just a symptom you've noticed. You'll also currently be doing dynablock checks against users you'd rather not be, along with a whole slew of other tests that will FP when SA thinks it's testing mail from some random system/zombie and not an authenticated user. So what you're saying is that I'm better off not scanning authenticated users. I'll take your word on that. There's probably just as many advantages as there are disadvantages for doing it either way. If you can't inform SA of a user's auth status then you've got to skip the SA check. If you can provide the auth info, then SA will work fine (if it supports parsing of your particular auth info), and it's a matter of personal preference. Let me know if you're running Postfix 2.3 and can enable the auth headers in your config. I'll probably get to making a patch tonight as long as the rain doesn't stop and I don't get distracted by the big stash of fireworks I've accumulated. :) Well I'm running Postfix v2.1.3 (standard in OS X Server 10.4.x.) I'm waiting until Apple previews OS X 10.5 in August to see whether 10.5 includes Postfix v2.3. If not than I may do the upgrade myself in 10.4. AFAIK, v2.3 is the first to support adding auth headers, so, yeah, unless you upgrade you'll have to go with the first option. Since SA is being called by Amavisd-new shouldn't the changes to ignore authenticated user happen there? I think I read that somewhere, maybe the Amavis mailing list. That's the problem with being subscribed to all these lists. They all start to run into each other in your head. Off to the archives I go. I know of, but am not at all familiar with Amavisd-new configuration, so I have no idea. Do what works. BIG STASH OF FIREWORKS! Boy I'm glad you don't live in my neighbourhood. :-O My closest neighbor is about 4 km away so I don't have to worry about blowing up the neighborhood. I do, however, need lots of fireworks if I want to annoy them. :) Daryl
Postfix auth headers (Re: Problem with false-positives for SASL users)
For those using Postfix, bug 4980 has a patch to support the auth headers, available in Postfix 2.3 and later, that can be used to extend your trust path to authenticated users. http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4980 Daryl
Bayes only works sometimes
I have an odd problem where given the /same/ input spamd, bayes will be triggered sometimes, and not others. I cannot replicate the problem sending the input to spamassassin (bayes always shows up when piping to spamassassin). for example: given the soruce at: http://jeremy.kister.net/tmp/1152571494.I9198eV400ff04M962772P11599.penny i get the following results: http://jeremy.kister.net/tmp/result1.txt http://jeremy.kister.net/tmp/result2.txt http://jeremy.kister.net/tmp/result3.txt my logs show: [25639] info: spamd: result: Y 32 - FORGED_RCVD_HELO,HTML_50_60,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_XBL,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC_SURBL scantime=1.2,size=2498,user=root,uid=1010,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=37429,mid=<[EMAIL PROTECTED]>,autolearn=unavailable [25639] info: spamd: result: Y 36 - BAYES_99,FORGED_RCVD_HELO,HTML_50_60,HTML_MESSAGE,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_XBL,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,URIBL_SBL,URIBL_SC_SURBL scantime=1.6,size=2498,user=root,uid=1010,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=37434,mid=<[EMAIL PROTECTED]>,bayes=0.9997122134,autolearn=unavailable with no errors before or after. I have confirmed this behavior with 3.1.2 and 3.1.3. any clues ?? -- Jeremy Kister http://jeremy.kister.net./
Util.pm error
Hello, after a fresh SpamAssassin-3.1.3 install from sources the following errors appear raddomly in log file: @400044b2ce89145fc454 [354] error: Minute '60' out of range 0..59 at /usr/local/spamassasin/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 @400044b2d6ca1bcbda34 [8264] error: Hour '24' out of range 0..23 at /usr/local/spamassasin/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 GL
Re: Util.pm error
On 7/11/2006 1:12 AM, George Lekatsas wrote: after a fresh SpamAssassin-3.1.3 install from sources the following errors appear raddomly in log file: @400044b2ce89145fc454 [354] error: Minute '60' out of range 0..59 at /usr/local/spamassasin/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 @400044b2d6ca1bcbda34 [8264] error: Hour '24' out of range 0..23 at /usr/local/spamassasin/lib/perl5/site_perl/5.8.5/Mail/SpamAssassin/Util.pm line 429 Fairly harmless. If they bother you there's a patch here: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4978 Daryl
