whitelist_from_rcvd hits only sometimes
Hi, I'm trying to find out why a message sometimes hits whitelist_from_rcvd and sometimes does not. I checked the headers again and again but cannot see the difference. whitelist_from_rcvd quarant...@eu.quarantine.symantec.com messagelabs.com whitelist_from_rcvd quarant...@eu.quarantine.symantec.com messagelabs.net Hit: X-Spam-Score: -17.777 X-Spam-Level: X-Spam-Status: No, score=-17.777 tagged_above=- required=6.3 tests=[BAYES_50=1.5, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, USER_IN_WHITELIST=-20] autolearn=no autolearn_force=no Received: from deaugmail02.mydomain.com ([127.0.0.1]) by localhost (deaugmail02.mydomain.com [127.0.0.1]) (amavisd-new,port 10024) with ESMTP id QJysMQERq-OY for ; Tue, 26 Feb 2019 01:10:19 +0100 (CET) Received: from deaugmail01-in.mydomain.com (deaugmail01-in.mydomain.com[172.20.16.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by deaugmail02.mydomain.com (Postfix) with ESMTPS for ; Tue, 26 Feb 2019 01:10:19 +0100 (CET) Received: from mail6.bemta26.messagelabs.com (mail6.bemta26.messagelabs.com [85.158.142.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256bits)) (No client certificate requested) by deaugmail01-in.mydomain.com (Postfix) with ESMTPS id 05CD8D3ABE1 for ; Tue, 26 Feb 2019 01:10:18 +0100 (CET) Received: from [85.158.142.194] (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits)) by server-4.bemta.az-b.eu-central-1.aws.symcld.net id 06/5B-03001-AE3847C5; Tue, 26 Feb 2019 00:10:18 + X-Env-Sender: bounce-notifications-verp-1abcbf9c040cf77c0...@eu.quarantine.symantec.com X-Msg-Ref: server-21.tower-239.messagelabs.com!1551139817!1629604!1 X-Originating-IP: [95.131.104.177] X-StarScan-Received: X-StarScan-Version: 9.31.5; banners=-,-,- X-VirusChecked: Checked Received: (qmail 17691 invoked from network); 26 Feb 2019 00:10:18 - Received: from mail-css2-1.ld1.messagelabs.net (HELO inbound.prqfe006002.mgmt.messagelabs.net) (95.131.104.177) by server-21.tower-239.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 26 Feb 2019 00:10:18 - Received: from [127.0.0.1] ([127.0.0.1:53790] helo=prqfe006002.mgmt.messagelabs.net) by prqfe006002.mgmt.messagelabs.net (envelope-from ) (ecelerity 4.2.28.58446 r(Core:4.2.28.1)) with ESMTPS (cipher=AES256-SHA256) id 38/2F-02400-9E3847C5; Tue, 26 Feb 2019 00:10:17 + To: hel...@mydomain.com Date: Tue, 26 Feb 2019 00:10:17 + Message-Id: <20190226001017.439d763f554cfe22dfd4...@quarantine.messagelabs.com> From: Email Quarantine Miss: X-Spam-Score: 19.767 X-Spam-Level: *** X-Spam-Status: Yes, score=19.767 tagged_above=- required=6.3 tests=[BAYES_99=6.5, BAYES_999=6.5, HELO_MISC_IP=0.25, HTML_MESSAGE=0.001, INTERNETX_UCE_NOT_REG=5, MIME_HTML_ONLY=0.723, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793] autolearn=no autolearn_force=no Received: from deaugmail02.mydomain.com ([127.0.0.1]) by localhost (deaugmail02.mydomain.com [127.0.0.1]) (amavisd-new,port 10024) with ESMTP id TbYATLBnkUKk for ; Tue, 26 Feb 2019 01:19:03 +0100 (CET) MIME-Version: 1.0 Subject: [mydomain Content Filter] [EXT] Email Quarantine: You have 2 new emails Received: from deaugmail01-in.mydomain.com (mailin.desog.mydomain.com [172.20.16.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by deaugmail02.mydomain.com (Postfix) with ESMTPS for ; Tue, 26 Feb 2019 01:19:03 +0100 (CET) Received: from mail6.bemta25.messagelabs.com (mail6.bemta25.messagelabs.com [195.245.230.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256bits)) (No client certificate requested) by deaugmail01-in.mydomain.com (Postfix) with ESMTPS id CC521D3AD2F for ; Tue, 26 Feb 2019 01:19:03 +0100 (CET) Received: from [46.226.52.194] (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits)) by server-2.bemta.az-b.eu-west-1.aws.symcld.net id 45/A1-14990-7F5847C5; Tue, 26 Feb 2019 00:19:03 + Received: (qmail 17246 invoked from network); 26 Feb 2019 00:19:02 - Received: from mail-css2-1.ld1.messagelabs.net (HELO inbound.prqfe006003.mgmt.messagelabs.net) (95.131.104.177) by server-22.tower-282.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 26 Feb 2019 00:19:02 - Received: from [127.0.0.1] ([127.0.0.1:38688] helo=prqfe006003.mgmt.messagelabs.net) by prqfe006003.mgmt.messagelabs.net (envelope-from ) (ecelerity 4.2.28.58446 r(Core:4.2.28.1)) with ESMTPS(cipher=AES256-SHA256) id DB/F9-02397-6F5847C5; Tue, 26 Feb 2019 00:19:02 + To: intern...@mydomain.com Date: Tue, 26 Feb 2019 00:19:02 + Message-Id: <20190226001902.43540a5f10d008b5d2c8...@quarantine.messagelabs.com> From: Email Quarantine Thank you!
Re: remaining relays will be considered trusted, but no longer internal
Helmut Schneider wrote: > when further investigating my issue that ALL_TRUSTED is always true I > came along the following lines when debugging SA: > > Apr 15 11:44:43.211 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: parsed as [ ip=172.20.12.10 rdns=relay-in > helo=mail2 by=mail01 ident= envfrom= intl=0 id= auth= msa=0 ] > Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: netset: trusted_networks lookup on 172.20.12.10, 5 networks, > result: 1, 0.617 ms > Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: netset: internal_networks lookup on 172.20.12.10, 5 networks, > result: 1, 0.204 ms > Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: relay 172.20.12.10 trusted? yes internal? yes > msa? no > Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: parsed as [ ip=195.245.231.135 > rdns=mail6.bemta5.messagelabs.com helo=mail6.bemta5.messagelabs.com > by=mail2 ident= envfrom= intl=0 id=0CC1B30E auth= msa=0 ] > Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: netset: trusted_networks lookup on 195.245.231.135, 5 networks, > result: 0, 0.204 ms > Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: originating, 195.245.231.135 and remaining > relays will be considered trusted, but no longer internal > Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: relay 195.245.231.135 trusted? yes internal? no > msa? no > Apr 15 11:44:43.216 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: parsed as [ ip=85.158.139.35 rdns= helo= > by=server-4.bemta-5.messagelabs.com ident= envfrom= intl=0 > id=B6/BA-18387-A08B0175 auth= msa=0 ] > Apr 15 11:44:43.216 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA > dbg: received-header: relay 85.158.139.35 trusted? yes internal? no > msa? no > > So SA correctly identifies an relay as external but still trusts the > whole path. Why? For the archives: There might be other solutions but exclude your postfix instances from @mynetworks in amavisd.conf and your fine.
Re: remaining relays will be considered trusted, but no longer internal
RW wrote:
> On Fri, 15 Apr 2016 14:08:15 + (UTC)
> Helmut Schneider wrote:
>
> > RW wrote:
> >
> > > On Fri, 15 Apr 2016 12:35:24 +0100
> > > RW wrote:
> > >
> > > > On Fri, 15 Apr 2016 10:10:13 + (UTC)
> > > > Helmut Schneider wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > when further investigating my issue that ALL_TRUSTED is always
> > > > > true I came along the following lines when debugging SA:
> > > > >
> > > > > ...
> > > > > Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]:
> > > > > (09991-02) SA dbg: received-header: originating,
> > > > > 195.245.231.135 and remaining relays will be considered
> > > > > trusted, but no longer internal ...
> > > > >
> > > > > So SA correctly identifies an relay as external but still
> > > > > trusts the whole path. Why?
> > > >
> > > > It looks like it's being seen as mail submission. Do you have
> > > > msa_networks set?
> > >
> > > I had a look at the code, and it looks like that particular
> > > message with "but no longer internal" can only be be reached when
> > > a flag is set that asserts that the message was submitted. This
> > > causes the point at which trust would otherwise be broken to be
> > > treated as a submission server.
> >
> > msa_networks is not set.
>
> It's when a mail client submits outgoing mail to an mta. This should
> involve some form of authentication
>
> For some reason amavisd thinks that all of your mail is being
> submitted locally. SA is finding that it's ALL_TRUSTED because amavisd
> is telling SA that it is via the SA perl library interface.
Thank you, this helped a lot:
I have 2 servers with 3 postfix instances each, postfix-in, postfix-out
and postfix-amavis with different IPs each.
All mail is received by the postfix-in instances. For some domains I
forward mails directly to their final destinations, for some I do SPAM
filtering on the postfix-amavis instances.
It seems that ALL mail is treated as relayed internally as soon as I
forward those mails to the postfix-amavis instance:
Passed CLEAN {RelayedInbound}, [52.71.20.6]:55081
52.71.20.6 is an external IP adress.
Now I have to figure out how to prevent amavis from behaving like that.
Re: remaining relays will be considered trusted, but no longer internal
Reindl Harald wrote: > > Am 15.04.2016 um 16:08 schrieb Helmut Schneider: > > What does "submission" mean in this context? > > ESMT(S)A > > https://en.wikipedia.org/wiki/SMTP_Authentication I'm neither using authentication nor smtp submission (TCP587).
Re: remaining relays will be considered trusted, but no longer internal
RW wrote: > On Fri, 15 Apr 2016 12:35:24 +0100 > RW wrote: > > > On Fri, 15 Apr 2016 10:10:13 + (UTC) > > Helmut Schneider wrote: > > > > > Hi, > > > > > > when further investigating my issue that ALL_TRUSTED is always > > > true I came along the following lines when debugging SA: > > > > > > ... > > > Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) > > > SA dbg: received-header: originating, 195.245.231.135 and > > > remaining relays will be considered trusted, but no longer > > > internal ... > > > > > > So SA correctly identifies an relay as external but still trusts > > > the whole path. Why? > > > > It looks like it's being seen as mail submission. Do you have > > msa_networks set? > > I had a look at the code, and it looks like that particular message > with "but no longer internal" can only be be reached when a flag is > set that asserts that the message was submitted. This causes the point > at which trust would otherwise be broken to be treated as a submission > server. msa_networks is not set. What does "submission" mean in this context?
remaining relays will be considered trusted, but no longer internal
Hi, when further investigating my issue that ALL_TRUSTED is always true I came along the following lines when debugging SA: Apr 15 11:44:43.211 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: parsed as [ ip=172.20.12.10 rdns=relay-in helo=mail2 by=mail01 ident= envfrom= intl=0 id= auth= msa=0 ] Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: netset: trusted_networks lookup on 172.20.12.10, 5 networks, result: 1, 0.617 ms Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: netset: internal_networks lookup on 172.20.12.10, 5 networks, result: 1, 0.204 ms Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: relay 172.20.12.10 trusted? yes internal? yes msa? no Apr 15 11:44:43.212 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: parsed as [ ip=195.245.231.135 rdns=mail6.bemta5.messagelabs.com helo=mail6.bemta5.messagelabs.com by=mail2 ident= envfrom= intl=0 id=0CC1B30E auth= msa=0 ] Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: netset: trusted_networks lookup on 195.245.231.135, 5 networks, result: 0, 0.204 ms Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: originating, 195.245.231.135 and remaining relays will be considered trusted, but no longer internal Apr 15 11:44:43.213 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: relay 195.245.231.135 trusted? yes internal? no msa? no Apr 15 11:44:43.216 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: parsed as [ ip=85.158.139.35 rdns= helo= by=server-4.bemta-5.messagelabs.com ident= envfrom= intl=0 id=B6/BA-18387-A08B0175 auth= msa=0 ] Apr 15 11:44:43.216 mail /usr/sbin/amavisd-new[9991]: (09991-02) SA dbg: received-header: relay 85.158.139.35 trusted? yes internal? no msa? no So SA correctly identifies an relay as external but still trusts the whole path. Why? Thanky you
Re: Fixing ALL_TRUSTED=-1
Helmut Schneider wrote: > Bill Cole wrote: > > > On 12 Apr 2016, at 9:03, Helmut Schneider wrote: > > > > > Bill Cole wrote: > > > > > > > Pipe that message into "spamassassin -t -D > > > > dns,received-header,metadata" *running as the same user that > > > > runs your Amavisd* and examine the first ~20 line of the debug > > > > output, which will show you how SA is parsing those Received > > > > headers as well as what version of Net::DNS you're using. > > > > > > Good point! Running spamassassin from command line works fine and > > > does not trigger ALL_TRUSTED: > > > > > > Apr 12 09:49:27.475 [13767] dbg: metadata: > > > X-Spam-Relays-Untrusted: [ ip=193.109.254.103 > > > rdns=mail6.bemta14.messagelabs.com > > > helo=mail6.bemta14.messagelabs.com by=XX ident= envfrom= intl=0 > > > id=0423F30E auth= msa=0 ] [ ip=85.158.140.195 rdns= helo= > > > by=server-10.bemta-14.messagelabs.com ident= envfrom= intl=0 > > > id=04/85-02972-8943C075 auth= msa=0 ] [ ip=104.47.100.68 > > > rdns=mail-ma1ind01on0068.outbound.protection.outlook.com > > > helo=IND01-MA1-obe.outbound.protection.outlook.com > > > by=server-9.tower-193.messagelabs.com ident= envfrom= intl=0 id= > > > auth= msa=0 ] [ ip=115.114.122.40 rdns=115.114.122.40 > > > helo=115.114.122.40 by=BM1PR01MB0596.INDPRD01.PROD.OUTLOOK.COM > > > ident= envfrom= intl=0 id=15.1.453.26 auth= msa=0 ] [ > > > ip=115.114.122.40 rdns= helo= by= ident= envfrom= intl=0 id= > > > auth= msa=0 ] > > > > > > Amavisd runs chrooted, how can I debug SA while running from > > > amavisd? > > > > I cannot say, as I do not run Amavisd. There seem to be instructions > > at https://www.ijs.si/software/amavisd/README.chroot.txt > > Unfortunately I contributed many of those instructions myself. I'll > try strace. Thank you. Too bad, the issue also occurs without chroot. So I'll head over to the amavisd-new list.
Re: Fixing ALL_TRUSTED=-1
Bill Cole wrote: > On 12 Apr 2016, at 9:03, Helmut Schneider wrote: > > > Bill Cole wrote: > > > > > Pipe that message into "spamassassin -t -D > > > dns,received-header,metadata" *running as the same user that runs > > > your Amavisd* and examine the first ~20 line of the debug output, > > > which will show you how SA is parsing those Received headers as > > > well as what version of Net::DNS you're using. > > > > Good point! Running spamassassin from command line works fine and > > does not trigger ALL_TRUSTED: > > > > Apr 12 09:49:27.475 [13767] dbg: metadata: X-Spam-Relays-Untrusted: > > [ ip=193.109.254.103 rdns=mail6.bemta14.messagelabs.com > > helo=mail6.bemta14.messagelabs.com by=XX ident= envfrom= intl=0 > > id=0423F30E auth= msa=0 ] [ ip=85.158.140.195 rdns= helo= > > by=server-10.bemta-14.messagelabs.com ident= envfrom= intl=0 > > id=04/85-02972-8943C075 auth= msa=0 ] [ ip=104.47.100.68 > > rdns=mail-ma1ind01on0068.outbound.protection.outlook.com > > helo=IND01-MA1-obe.outbound.protection.outlook.com > > by=server-9.tower-193.messagelabs.com ident= envfrom= intl=0 id= > > auth= msa=0 ] [ ip=115.114.122.40 rdns=115.114.122.40 > > helo=115.114.122.40 by=BM1PR01MB0596.INDPRD01.PROD.OUTLOOK.COM > > ident= envfrom= intl=0 id=15.1.453.26 auth= msa=0 ] [ > > ip=115.114.122.40 rdns= helo= by= ident= envfrom= intl=0 id= auth= > > msa=0 ] > > > > Amavisd runs chrooted, how can I debug SA while running from > > amavisd? > > I cannot say, as I do not run Amavisd. There seem to be instructions > at https://www.ijs.si/software/amavisd/README.chroot.txt Unfortunately I contributed many of those instructions myself. I'll try strace. Thank you.
Re: Fixing ALL_TRUSTED=-1
Bill Cole wrote: > On 11 Apr 2016, at 10:55, Helmut Schneider wrote: > > > Hi, > > > > for more than 6 months I'm trying to fix ALL_TRUSTED=-1 without > > success. > > Did it just start showing up 6 months ago on a previously-working > SpamAssassin installation, of was SA just set up 6 months ago and has > been broken the whole time? I don't recall that it ever worked. > > Received: from XXX (XXX [172.20.12.10]) > > (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) > > (No client certificate requested) > > by XXX (Postfix) with ESMTPS > > for ; Mon, 11 Apr 2016 15:01:32 +0200 (CEST) > > Pipe that message into "spamassassin -t -D > dns,received-header,metadata" *running as the same user that runs > your Amavisd* and examine the first ~20 line of the debug output, > which will show you how SA is parsing those Received headers as well > as what version of Net::DNS you're using. Good point! Running spamassassin from command line works fine and does not trigger ALL_TRUSTED: Apr 12 09:49:27.475 [13767] dbg: metadata: X-Spam-Relays-Untrusted: [ ip=193.109.254.103 rdns=mail6.bemta14.messagelabs.com helo=mail6.bemta14.messagelabs.com by=XX ident= envfrom= intl=0 id=0423F30E auth= msa=0 ] [ ip=85.158.140.195 rdns= helo= by=server-10.bemta-14.messagelabs.com ident= envfrom= intl=0 id=04/85-02972-8943C075 auth= msa=0 ] [ ip=104.47.100.68 rdns=mail-ma1ind01on0068.outbound.protection.outlook.com helo=IND01-MA1-obe.outbound.protection.outlook.com by=server-9.tower-193.messagelabs.com ident= envfrom= intl=0 id= auth= msa=0 ] [ ip=115.114.122.40 rdns=115.114.122.40 helo=115.114.122.40 by=BM1PR01MB0596.INDPRD01.PROD.OUTLOOK.COM ident= envfrom= intl=0 id=15.1.453.26 auth= msa=0 ] [ ip=115.114.122.40 rdns= helo= by= ident= envfrom= intl=0 id= auth= msa=0 ] Amavisd runs chrooted, how can I debug SA while running from amavisd?
Re: Fixing ALL_TRUSTED=-1
Martin Gregorie wrote: > On Mon, 2016-04-11 at 14:55 +0000, Helmut Schneider wrote: > > > > Hi, > > > > for more than 6 months I'm trying to fix ALL_TRUSTED=-1 without > > success. > > > > I have read https://wiki.apache.org/spamassassin/TrustPath and > > https://wiki.apache.org/spamassassin/FixingAllTrusted carefully, put > > > > trusted_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 > > internal_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 > > > You do realise that 172.12.0.0 is equal to 172.20.8.13 if you compare > them using only the top 12 bits? > > 172.16 is 10101100,0001 and 20 is 10101100,00010100 but mask the > bit patterns to retain only the top 12 bits, which is what specifying > /12 effectively does when you're comparing IPV4 addresses, and both > become 10101100,0001 Sure, but 85.158.139.19 and 103.208.153.18 aren't: Received: from [85.158.139.19] by server-11.bemta-5.messagelabs.com id BD/80-27787-C20AB075; Mon, 11 Apr 2016 13:01:32 + [...] Received: from unknown (HELO ns2.Host1.yourdomainname.com) (103.208.153.18) Or did I miss something?
Re: Fixing ALL_TRUSTED=-1
Bowie Bailey wrote: > On 4/11/2016 10:55 AM, Helmut Schneider wrote: > > Hi, > > > > for more than 6 months I'm trying to fix ALL_TRUSTED=-1 without > > success. > > > > I have read https://wiki.apache.org/spamassassin/TrustPath and > > https://wiki.apache.org/spamassassin/FixingAllTrusted carefully, put > > > > trusted_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 > > internal_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 > > add_header all RelaysUntrusted RELAYSUNTRUSTED (this does not seem > > to work at all, no header seems added) > > > > into local.cf and still ALL_TRUSTED gets fired. Any help would be > > appreciated. > > Step one is to make sure you're putting the settings into the right > file. Run this to check if you are using the right file: > > $ spamassassin -D config --lint 2>&1 | grep local.cf > Apr 11 11:40:56.509 [6692] dbg: config: read file > /etc/mail/spamassassin/local.cf > > Once you have your settings in the right file, then make sure you > have restarted amavisd-new to load the new settings. mail:~$ spamassassin -D config --lint 2>&1 | grep local.cf Apr 11 17:54:12.525 [31265] dbg: config: read file /usr/share/spamassassin/local.cf Apr 11 17:54:12.526 [31265] dbg: config: read file /etc/spamassassin/local.cf mail:~$ grep -iE '(^trusted|internal)' /etc/spamassassin/local.cf trusted_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 internal_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 mail:~$ grep -iE '(^trusted|internal)' /usr/share/spamassassin/local.cf mail:~$ Restarted amavisd-new?! I guess I restarted the server more than 20 times within the last 6 months ;)
Fixing ALL_TRUSTED=-1
Hi, for more than 6 months I'm trying to fix ALL_TRUSTED=-1 without success. I have read https://wiki.apache.org/spamassassin/TrustPath and https://wiki.apache.org/spamassassin/FixingAllTrusted carefully, put trusted_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 internal_networks 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 add_header all RelaysUntrusted _RELAYSUNTRUSTED_ (this does not seem to work at all, no header seems added) into local.cf and still ALL_TRUSTED gets fired. Any help would be appreciated. mail:~$ sudo spamassassin -V SpamAssassin version 3.4.0 running on Perl version 5.18.2 mail:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04.4 LTS Release:14.04 Codename: trusty mail:~$ Received: from XXX ([172.20.8.31]) by XXX (IBM Domino Release 9.0.1FP4) with ESMTP id 2016041115014726-193867 ; Mon, 11 Apr 2016 15:01:47 +0200 Received: from localhost (localhost [127.0.0.1]) by XXX (Postfix) with ESMTP id 3BD0618E for ; Mon, 11 Apr 2016 15:01:43 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at XXX X-Spam-Flag: NO X-Spam-Score: 5.607 X-Spam-Level: * X-Spam-Status: No, score=5.607 tagged_above=- required=6.3 tests=[ALL_TRUSTED=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, HTML_MIME_NO_HTML_TAG=0.377, INTERNETX_UCE=5, MIME_HTML_ONLY=0.723, MISSING_MID=0.497, SPF_HELO_PASS=-0.001, T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=no autolearn_force=no Authentication-Results: XXX (amavisd-new); domainkeys=neutral (2048-bit key) reason="invalid (bad identity)" header.sender=x...@ncrprop.biz header.d=ncrprop.biz; dkim=pass (2048-bit key) header.d=ncrprop.biz Received: from XXX ([127.0.0.1]) by localhost (XXX [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rzCBYBjiHHbC for ; Mon, 11 Apr 2016 15:01:32 +0200 (CEST) Received: from XXX (XXX [172.20.12.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by XXX (Postfix) with ESMTPS for ; Mon, 11 Apr 2016 15:01:32 +0200 (CEST) Received: from mail6.bemta5.messagelabs.com (mail6.bemta5.messagelabs.com [195.245.231.135]) by XXX (Postfix) with ESMTP id 63B4C335 for ; Mon, 11 Apr 2016 15:01:32 +0200 (CEST) Received: from [85.158.139.19] by server-11.bemta-5.messagelabs.com id BD/80-27787-C20AB075; Mon, 11 Apr 2016 13:01:32 + X-Brightmail-Tracker: H4sIA+NgFtrDIsWRWlGSWpSXmKPExsWSfmGmkK72Au5 wgyVt8hYzt5o7MHosm7CYPYAxijUzLym/IoE148KtPSwFTWIVN479ZmpgbBTpYuTkkBAwkJj x dj4zhK0s8XzTZ7YuRi4OFoGTzBInj2xnhXCaWSR6Jr1mA6niFRCUODnzCQuILSygLXFo+x1W E JtNQEdi3paNYLaIgLTEpgmvwKYKCRhL/Jz4ByzOIqAg8fLdKUaIXjWJnS97wGYyC+hKnD6+j Q XClpfY/nYOWK+AgIDEgaaJYL2cQHbb5WOsExj5ZyE5YxaS9llI2hcwMq9i1ChOLSpLLdI1NN J LKspMzyjJTczM0TU0MNXLTS0uTkxPzUlMKtZLzs/dxAgMQwYg2MHYN8v5EKMkB5OSKG/CXO5 w Ib6k/JTKjMTijPii0pzU4kOMMhwcShK8B+cB5QSLUtNTK9Iyc4ARAZOW4OBREuFdDJLmLS5I z C3OTIdInWI05vh09MFaJo5n3TP3MAmx5OXnpUqJ834FKRUAKc0ozYMbBIvUS4yyUsK8jECnC f EUpBblZpagyr9iFOdgVBLmXQIyhSczrwRu3yugU5iATnn2jxPklJJEhJRUA6Ns3Ee3Dbfmpc 7 mecCeuKHF4qbb70INmU/ce0pesj4NWbTx3Z++c+pLH/HcZRN86Tc7o+j/Fc0vqiwq0qmPmS3 / mKm94Pb9++O5xRGVLY2G6X3pux88fvlN4px87cfAsNWJiQf2P9JKWn34jeeljP1ezJXfBc4Y W LgrTJzAvqHH7LZt0dl7nE+VWIozEg21mIuKEwF0Fbe9zwIAAA== X-Env-Sender: x...@ncrprop.biz X-Msg-Ref: server-12.tower-178.messagelabs.com!1460379690!32840337!1 X-Originating-IP: [103.208.153.18] X-SpamReason: No, hits=2.7 required=7.0 tests=msgid: No Message-ID, HTML_60_70,HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 8.28; banners=-,-,- X-VirusChecked: Checked Received: (qmail 4785 invoked from network); 11 Apr 2016 13:01:31 - Received: from unknown (HELO ns2.Host1.yourdomainname.com) (103.208.153.18) by server-12.tower-178.messagelabs.com with SMTP; 11 Apr 2016 13:01:31 - X-Sender: "Sonam Singh" X-Receiver: XXX DomainKey-Signature: a=rsa-sha1; c=simple; d=ncrprop.biz; h=From:To:Subject; q=dns; s=jsmtp ; b=j8TzR3hoYHUafVg9yI0iyVfuGnrFlWf3/D8TdvVWoHxShJW6kPhZkgAAPzynTB79KtzOJb adDxZ437AC+/dePYCtQx5DLVSuPNGGP8l/B0HgkVZ7gs8Rlbv1SlbTEEDFPkIDhhBzBCgy2f ORIToDXhJVd4fW+NeIeReZ2ZCHcjD6AxMcac/2uIniGz34CHWqkellaF+ckP3p/LrTt+R8Ua bKqG/mqOq+Rbxea1Poam6ORIAYhAekOrhQchzsVXC7jvc0eSWJB6F2CLGoxQEEwzqbAcc1Fc nzFPi2Ps6JW3hJ9vyMEtSK6j0wPkj/hsdR71NnBfDGfs4E9roRuYw0lQ==; DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=ncrprop.biz; h=From:To:Subject; q=dns/txt; s=jsmtp ; t=1460379276; bh=A9wE/QPGFnmy8ntNIHK6uqqeU/Q=; b=Rp+RKP73ntQxhIU0tNJyX4RW1N2yLciYAC9+rK+Be0UO1qHPBBl/W+6on/Xtz/cXlBYdyY evEsLtIVz4vNkbsBlwGLDmk8YTuwMesYxbqSuJyWy0AyAZZJrRVt7W5RfCSk7Q4
Re: meta test HEXHASH_WORD has undefined dependency '__KAM_BODY_LENGTH_LT_512'
John Hardin wrote:
> On Sun, 6 Apr 2014, Helmut Schneider wrote:
>
> > John Hardin wrote:
> >
> > > On Sun, 6 Apr 2014, Helmut Schneider wrote:
> > >
> > > > over the last weeks I constantly run into issues when I cannot
> > > > get SA up again because of "broken" rule sets. Today it's
> > > >
> > > > Apr 6 17:06:01.960 [31092] dbg: rules: meta test HEXHASH_WORD
> > > > has undefined dependency '__KAM_BODY_LENGTH_LT_512'
> > > >
> > > > Is something wrong in my process or do we have a problem with QA
> > > > these days.
> > >
> > > Both in part. Do you have the BodyEval plugin disabled?
> >
> > No, it's enabled.
>
> Interesting. That subrule is in an ifplugin for that plugin, so if
> you have that plugin enabled that subrule should be defined.
>
> What version of SA are you running?
u1dd_hr:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 10.04.4 LTS
Release:10.04
Codename: lucid
u1dd_hr:~$ sudo spamassassin -V
SpamAssassin version 3.3.1
running on Perl version 5.10.1
u1dd_hr:~$ spamassassin -D --lint 2>&1 | grep -Ei '(failed|undefined
dependency)'
Apr 7 21:14:40.718 [18171] dbg: diag: [...] module not installed:
IP::Country::Fast ('require' failed)
Apr 7 21:14:40.718 [18171] dbg: diag: [...] module not installed:
Net::Ident ('require' failed)
Apr 7 21:14:40.719 [18171] dbg: diag: [...] module not installed:
Encode::Detect ('require' failed)
Apr 7 21:14:40.839 [18171] dbg: plugin: loading
Mail::SpamAssassin::Plugin::BodyEval from @INC
Apr 7 21:14:42.484 [18171] dbg: rules: meta test HEXHASH_WORD has
undefined dependency '__KAM_BODY_LENGTH_LT_512'
u1dd_hr:~$
SA 3.3.2 on Ubuntu 12.04 does not complain btw:
helmut:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 12.04.4 LTS
Release:12.04
Codename: precise
helmut:~$ spamassassin -V
SpamAssassin version 3.3.2
running on Perl version 5.14.2
helmut:~$ spamassassin -D --lint 2>&1 | grep -Ei '(failed|undefined
dependency)'
Apr 7 21:12:07.412 [16845] dbg: diag: [...] module not installed:
Digest::SHA1 ('require' failed)
Apr 7 21:12:07.413 [16845] dbg: diag: [...] module not installed:
IP::Country::Fast ('require' failed)
Apr 7 21:12:07.413 [16845] dbg: diag: [...] module not installed:
Net::Ident ('require' failed)
Apr 7 21:12:07.413 [16845] dbg: diag: [...] module not installed: DBI
('require' failed)
Apr 7 21:12:07.414 [16845] dbg: diag: [...] module not installed:
Encode::Detect ('require' failed)
Apr 7 21:12:07.497 [16845] dbg: plugin: loading
Mail::SpamAssassin::Plugin::BodyEval from @INC
helmut:~$
---
Diese E-Mail ist frei von Viren und Malware, denn der avast! Antivirus Schutz
ist aktiv.
http://www.avast.com
Re: meta test HEXHASH_WORD has undefined dependency '__KAM_BODY_LENGTH_LT_512'
Dave Funk wrote: > On Sun, 6 Apr 2014, Helmut Schneider wrote: > > > over the last weeks I constantly run into issues when I cannot get > > SA up again because of "broken" rule sets. Today it's > > > > Apr 6 17:06:01.960 [31092] dbg: rules: meta test HEXHASH_WORD has > > undefined dependency '__KAM_BODY_LENGTH_LT_512' > > > > Is something wrong in my process or do we have a problem with QA > > these days. > > > > Don't get me wrong, I appreciate your work very much. > > That is effectively a warning, not a fatal error message. That one > particular kind of warning should not stop SA from running. Then it's my fault and I apologize.
Re: meta test HEXHASH_WORD has undefined dependency '__KAM_BODY_LENGTH_LT_512'
John Hardin wrote: > On Sun, 6 Apr 2014, Helmut Schneider wrote: > > > over the last weeks I constantly run into issues when I cannot get > > SA up again because of "broken" rule sets. Today it's > > > > Apr 6 17:06:01.960 [31092] dbg: rules: meta test HEXHASH_WORD has > > undefined dependency '__KAM_BODY_LENGTH_LT_512' > > > > Is something wrong in my process or do we have a problem with QA > > these days. > > Both in part. Do you have the BodyEval plugin disabled? No, it's enabled. > Fixing... Thank you.
meta test HEXHASH_WORD has undefined dependency '__KAM_BODY_LENGTH_LT_512'
Hi, over the last weeks I constantly run into issues when I cannot get SA up again because of "broken" rule sets. Today it's Apr 6 17:06:01.960 [31092] dbg: rules: meta test HEXHASH_WORD has undefined dependency '__KAM_BODY_LENGTH_LT_512' Is something wrong in my process or do we have a problem with QA these days. Don't get me wrong, I appreciate your work very much. Thanks, Helmut
Re: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIb'
Kevin A. McGrail wrote: > > all over sudden all my installations (3.3.1 and 3.3.2) show > > > > rules: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency > > '__AC_RHASH_URIb' > > > > when checking rules. Whats's wrong? > > That should not have auto promoted and has already been fixed. Will > hopefully get the rules.update engine working tonight. Regards, > KAM Thanks for the update. Will sa-update fix the issue (later).
Re: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIb'
Helmut Schneider wrote: > all over sudden all my installations (3.3.1 and 3.3.2) show To be more precise: Feb 18 20:48:03.261 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIb' Feb 18 20:48:03.261 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIc' Feb 18 20:48:03.268 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS7 has undefined dependency '__AC_RHASH2_URIb' Feb 18 20:48:03.268 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS7 has undefined dependency '__AC_RHASH2_URIc' Feb 18 20:48:03.289 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS5 has undefined dependency '__AC_SEQHASH_URIb' Feb 18 20:48:03.289 [68576] dbg: rules: meta test AC_SPAMMY_URI_PATTERNS5 has undefined dependency '__AC_SEQHASH_URIc'
meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIb'
Hi, all over sudden all my installations (3.3.1 and 3.3.2) show rules: meta test AC_SPAMMY_URI_PATTERNS6 has undefined dependency '__AC_RHASH_URIb' when checking rules. Whats's wrong? Thanks, Helmut
Re: whitelisting despite of trusted_networks
Benny Pedersen wrote: > Helmut Schneider skrev den 2013-03-13 15:19: > > > How can I whiltelist(_auth) senders now? > > if sender ip is whitelisted, does it then make sense to whitelist > based on dkim/spf ? > > note here dkim is not using ip at all ? I want to whitelist email adresses, not IPs. And it worked time ago (I guess it worked until I changed trusted_networks).
whitelisting despite of trusted_networks
Hi, after a discussion here on September 12th I added MessageLabs to trusted_networks. If I understood posts in the net correctly this might be the reason why whitelist_ does not work anymore. Mar 13 14:44:04.119 [17641] dbg: spf: relayed through one or more trusted relays, cannot use header-based Envelope-From, skipping How can I whiltelist(_auth) senders now? Thanks, Helmut
Re: Exclude from RCVD_IN_DNSWL_MED
Helmut Schneider wrote: > Kris Deugau wrote: > > > Helmut Schneider wrote: > > but if their support refuses to tell you, I'd be looking at > > switching providers > > I guess they would if they knew themselves. But project "switch" is > ongoing... :) http://images.messagelabs.com/EmailResources/ImplementationGuides/Subnet_IP.pdf
Re: Exclude from RCVD_IN_DNSWL_MED
Matthias Leisi wrote: > On Mon, Sep 10, 2012 at 8:34 PM, Helmut Schneider > wrote: > > >> It looks like RCVD_IN_DNSWL_MED examines "firstuntrusted" and if he > >> trusts his MX/relays correctly then this shouldn't be happening. > > In general, setting up the trustpath correctly is sufficient. > > > If I understood you correctly I'd need to add all relays of > > MessageLabs to trusted_networks and also track any IP address > > changes... > > In theory, you need to do this for all DNSxL lookups. In practise they all resolve fine to *.messagelabs.com. > As for dnswl.org, one of the data download files is in "SpamAssassin > format", ie .cf files with trusted_networks entries separated into > four files, one for each trust level, so users can choose which [not] > to include. I appreciate the work of dnswl.org very much and only want to exclude a (few) record(s) and not the whole (or a larger part of the) list. I'll check the trusted_networks-way.
Re: Exclude from RCVD_IN_DNSWL_MED
Kris Deugau wrote: > Helmut Schneider wrote: > > If I understood you correctly I'd need to add all relays of > > MessageLabs to trusted_networks and also track any IP address > > changes... > > If you don't have that info, and their support refuses to tell you, > tailing your inbound logs for a while should give you a pretty good > idea what segments of their system your mail flows through... I'll check that. > but if their support refuses to tell you, I'd be looking at switching > providers I guess they would if they knew themselves. But project "switch" is ongoing... :) > knowing where your mail will legitimately go through your > filter provider's systems is important. They even won't tell you what rules applied. That's another reason why I'm about to switch.
Re: Exclude from RCVD_IN_DNSWL_MED
Dave Funk wrote: > On Mon, 10 Sep 2012, John Hardin wrote: > > > On Mon, 10 Sep 2012, Helmut Schneider wrote: > > > > > Short story: > > > Can I exclude hosts from RCVD_IN_DNSWL_LOW/MED/HI? > > > > > > Long story: > > > We are using an external provider to filter SPAM. We also use SA > > > internally. Sometimes mails are not recognized as SPAM externally > > > and forwarded to SA. The mailrelays of the external provider are > > > listed in RCVD_IN_DNSWL_MED and therefore SA subtracts -2.3 > > > points. While SA would recognize and filter the SPAM correctly it > > > does not because of RCVD_IN_DNSWL_MED. So I would like to exclude > > > those mailrelays from (e.g.) RCVD_IN_DNSWL_MED. > > > > > > I know I can write a rule that adds a score to those mailrelays > > > but that seems to be "not perfect" as membership of that host > > > might change from RCVD_IN_DNSWL_MED to RCVD_IN_DNSWL_HI/LOW and > > > v.v. and then would receive different scores. > > > > Make a subrule that looks for your mail service host's name in > > Received headers, and add a meta that fires on that rule + > > RCVD_IN_DNSWL_MED and adds compensating points. > > If he's got his "trusted_networks" configured correctly (has his > MX/relays listed) shouldn't that take care of the problem? > > It looks like RCVD_IN_DNSWL_MED examines "firstuntrusted" and if he > trusts his MX/relays correctly then this shouldn't be happening. If I understood you correctly I'd need to add all relays of MessageLabs to trusted_networks and also track any IP address changes...
Re: Exclude from RCVD_IN_DNSWL_MED
John Hardin wrote: > On Mon, 10 Sep 2012, Helmut Schneider wrote: > > > Short story: > > Can I exclude hosts from RCVD_IN_DNSWL_LOW/MED/HI? > > > > Long story: > > We are using an external provider to filter SPAM. We also use SA > > internally. Sometimes mails are not recognized as SPAM externally > > and forwarded to SA. The mailrelays of the external provider are > > listed in RCVD_IN_DNSWL_MED and therefore SA subtracts -2.3 points. > > While SA would recognize and filter the SPAM correctly it does not > > because of RCVD_IN_DNSWL_MED. So I would like to exclude those > > mailrelays from (e.g.) RCVD_IN_DNSWL_MED. > > > > I know I can write a rule that adds a score to those mailrelays but > > that seems to be "not perfect" as membership of that host might > > change from RCVD_IN_DNSWL_MED to RCVD_IN_DNSWL_HI/LOW and v.v. and > > then would receive different scores. > > Make a subrule that looks for your mail service host's name in > Received headers, and add a meta that fires on that rule + > RCVD_IN_DNSWL_MED and adds compensating points. Isn't that what I'm doing with > > I know I can write a rule that adds a score to those mailrelays but > > that seems to be "not perfect" as membership of that host might > > change from RCVD_IN_DNSWL_MED to RCVD_IN_DNSWL_HI/LOW and v.v. and > > then would receive different scores. ? If not, do you have additional ressources to read on?
Exclude from RCVD_IN_DNSWL_MED
Hi, Short story: Can I exclude hosts from RCVD_IN_DNSWL_LOW/MED/HI? Long story: We are using an external provider to filter SPAM. We also use SA internally. Sometimes mails are not recognized as SPAM externally and forwarded to SA. The mailrelays of the external provider are listed in RCVD_IN_DNSWL_MED and therefore SA subtracts -2.3 points. While SA would recognize and filter the SPAM correctly it does not because of RCVD_IN_DNSWL_MED. So I would like to exclude those mailrelays from (e.g.) RCVD_IN_DNSWL_MED. I know I can write a rule that adds a score to those mailrelays but that seems to be "not perfect" as membership of that host might change from RCVD_IN_DNSWL_MED to RCVD_IN_DNSWL_HI/LOW and v.v. and then would receive different scores. Thanks, Helmut
Re: Bayes expiration
RW wrote: > On Wed, 19 Jan 2011 13:16:09 + (UTC) > "Helmut Schneider" wrote: > > > Michael Scheidell wrote: > > > > > On 1/19/11 7:56 AM, Helmut Schneider wrote: > > > > Michael Scheidell wrote: > > > > > > > > > On 1/19/11 6:04 AM, Helmut Schneider wrote: > > > > > > bayes_auto_expire 1 > > > > > disable auto expire and run a cronjob. > > > > OK...but..why? :) > > > > > > > to fix your problem. > > autoexpiry runs while scanning an email OK, thanks.
Re: Bayes expiration
Michael Scheidell wrote: > On 1/19/11 8:16 AM, Helmut Schneider wrote: > > 1295442708, > > Last: 1295442672, atime: 0, count: 0, newdelta: 0, ratio: 0, period: > > 43200 > or, it has been trying (automatically) for a while. remove auto > expire (at least) done. > change to this also > bayes_store_module Mail::SpamAssassin::BayesStore::MySQL done. > let it run for a hour. at 300K tokens, I hope you have less then > 2000 users, 300K tokens isn't enough for 2000 users. ~200. > after SA runs for an hour after those fixes, it its still broke, > expire bayes, clear bayes and reimport. I'll try. Thanks, Helmut
Re: Bayes expiration
Michael Scheidell wrote: > On 1/19/11 7:56 AM, Helmut Schneider wrote: > > Michael Scheidell wrote: > > > > > On 1/19/11 6:04 AM, Helmut Schneider wrote: > > > > bayes_auto_expire 1 > > > disable auto expire and run a cronjob. > > OK...but..why? :) > > > to fix your problem. [helmut@BSDHelmut ~]$ sudo sa-learn --force-expire -D | grep bayes [...] Jan 19 14:11:48.275 [4221] dbg: bayes: bayes journal sync starting Jan 19 14:11:48.275 [4221] dbg: bayes: bayes journal sync completed Jan 19 14:11:48.276 [4221] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x804a24a08) implements 'learner_expire_old_training', priority 0 Jan 19 14:11:48.276 [4221] dbg: bayes: expiry starting Jan 19 14:11:48.277 [4221] dbg: bayes: expiry check keep size, 0.75 * max: 225000 Jan 19 14:11:48.277 [4221] dbg: bayes: token count: 644941, final goal reduction size: 419941 Jan 19 14:11:48.278 [4221] dbg: bayes: first pass? current: 1295442708, Last: 1295442672, atime: 0, count: 0, newdelta: 0, ratio: 0, period: 43200 Jan 19 14:11:48.278 [4221] dbg: bayes: can't use estimation method for expiry, unexpected result, calculating optimal atime delta (first pass) Jan 19 14:11:48.278 [4221] dbg: bayes: expiry max exponent: 9 Jan 19 14:11:48.280 [4221] dbg: bayes: atime token reduction Jan 19 14:11:48.281 [4221] dbg: bayes: === Jan 19 14:11:48.281 [4221] dbg: bayes: 43200 640988 Jan 19 14:11:48.281 [4221] dbg: bayes: 86400 635805 Jan 19 14:11:48.281 [4221] dbg: bayes: 172800 629424 Jan 19 14:11:48.282 [4221] dbg: bayes: 345600 62 Jan 19 14:11:48.282 [4221] dbg: bayes: 691200 620466 Jan 19 14:11:48.282 [4221] dbg: bayes: 1382400 620466 Jan 19 14:11:48.282 [4221] dbg: bayes: 2764800 620466 Jan 19 14:11:48.283 [4221] dbg: bayes: 5529600 620466 Jan 19 14:11:48.283 [4221] dbg: bayes: 11059200 620466 Jan 19 14:11:48.283 [4221] dbg: bayes: 22118400 620466 Jan 19 14:11:48.283 [4221] dbg: bayes: couldn't find a good delta atime, need more token difference, skipping expire Jan 19 14:11:48.284 [4221] dbg: bayes: expiry completed Jan 19 14:11:48.284 [4221] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x804a24a08) implements 'learner_close', priority 0 [helmut@BSDHelmut ~]$ Corrupted database?! > plus auto expire can seriously degrade the performance of your system > during peak times. run at during maint (slow, quiet) period. OK.
Re: Bayes expiration
Michael Scheidell wrote: > On 1/19/11 6:04 AM, Helmut Schneider wrote: > > bayes_auto_expire 1 > disable auto expire and run a cronjob. OK...but..why? :) > make sure you run the cronjob for each user in bayes. The database ist global...
Bayes expiration
Hi, I set use_bayes 1 bayes_auto_learn1 bayes_expiry_max_db_size30 bayes_auto_expire 1 bayes_store_module Mail::SpamAssassin::BayesStore::SQL bayes_sql_dsn DBI:mysql:$mysqlserver at local.cf. According to http://spamassassin.apache.org/full/3.3.x/doc/sa-learn.html#expiration I expected SA to automatically expire/cleanup/shrink the database. But after 2 weeks the database grew to ~650k tokens. [helmut@BSDHelmut ~]$ sudo sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 354123 0 non-token data: nspam 0.000 0 157550 0 non-token data: nham 0.000 0 644732 0 non-token data: ntokens 0.000 0 1271332927 0 non-token data: oldest atime 0.000 0 1295434465 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 1295395414 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count [helmut@BSDHelmut ~]$ sudo spamassassin -V SpamAssassin version 3.3.1 running on Perl version 5.10.1 [helmut@BSDHelmut ~]$ Did I miss anything? Thanks, Helmut
Re: Performance problem body tests
Helmut Schneider wrote: > with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance > problem: I might have been able to "catch" a non-confident example mail[1] (bad example because of the size, but an example). While SA 3.2.5 needs ~45 seconds, with SA 3.3.1: Jun 4 03:36:41.029 [56496] dbg: timing: total 103627 ms - init: 1615 (1.6%), parse: 52 (0.0%), extract_message_metadata: 1971 (1.9%), poll_dns_idle: 772 (0.7%), get_uri_detail_list: 348 (0.3%), tests_pri_-1000: 467 (0.5%), compile_gen: 292 (0.3%), compile_eval: 35 (0.0%), tests_pri_-950: 23 (0.0%), tests_pri_-900: 28 (0.0%), tests_pri_-400: 35 (0.0%), tests_pri_0: 99167 (95.7%), dkim_load_modules: 59 (0.1%), check_dkim_signature: 26 (0.0%), check_razor2: 2061 (2.0%), check_pyzor: 1.52 (0.0%), tests_pri_500: 188 (0.2%) Jun 4 03:35:02.226 [56496] dbg: rules: run_generic_tests - compiling eval code: body, priority 0 Jun 4 03:35:02.227 [56496] dbg: rules: compiled body tests Jun 4 03:35:18.067 [56496] dbg: rules: ran body rule __I_INHERIT ==> got hit: "I inherited" Jun 4 03:35:23.007 [56496] dbg: rules: ran body rule __YOU_HAVE_WON ==> got hit: "you won" Jun 4 03:35:24.733 [56496] dbg: rules: ran body rule __FB_MA ==> got hit: "MA" Jun 4 03:35:24.964 [56496] dbg: rules: ran body rule __MASTERS ==> got hit: "Masters" Jun 4 03:35:32.730 [56496] dbg: rules: ran body rule __DOS_BODY_WED ==> got hit: "Wednesday" Jun 4 03:35:33.415 [56496] dbg: rules: ran body rule __FRAUD_IRJ ==> got hit: "holding company" Jun 4 03:35:34.848 [56496] dbg: rules: ran body rule __KAM_LOTTO3 ==> got hit: "claim" Jun 4 03:35:35.281 [56496] dbg: rules: ran body rule __MILLIONS ==> got hit: "millions of dollar" Jun 4 03:35:36.387 [56496] dbg: rules: ran body rule __DEAL ==> got hit: "the deal" Jun 4 03:35:41.524 [56496] dbg: rules: ran body rule __FB_NATIONAL ==> got hit: "National" Jun 4 03:35:46.119 [56496] dbg: rules: ran body rule __F_LARGE_MONEY_2 ==> got hit: "10 million" Jun 4 03:36:04.770 [56496] dbg: rules: ran body rule __MBA ==> got hit: "mba" Jun 4 03:36:06.071 [56496] dbg: rules: ran body rule __DOS_BODY_FRI ==> got hit: "Friday" Jun 4 03:36:07.273 [56496] dbg: rules: ran body rule __DOS_LINK ==> got hit: "link" Jun 4 03:36:09.234 [56496] dbg: rules: ran body rule __DOS_BODY_SAT ==> got hit: "sat" Jun 4 03:36:09.838 [56496] dbg: rules: ran body rule __FILL_THIS_FORM_FRAUD_PHISH ==> got hit: "password. Jun 4 03:36:09.839 [56496] dbg: rules: [...] " Jun 4 03:36:15.269 [56496] dbg: rules: ran body rule __SUBSCRIPTION_INFO ==> got hit: "opt out" Jun 4 03:36:15.521 [56496] dbg: rules: ran body rule __HAS_ANY_EMAIL ==> got hit: "m...@wsj.c" Jun 4 03:36:16.798 [56496] dbg: rules: ran body rule __FB_NUM_PERCNT ==> got hit: "2%" Jun 4 03:36:16.998 [56496] dbg: rules: ran body rule __YOU_WON_01 ==> got hit: "you won" Jun 4 03:36:17.226 [56496] dbg: rules: ran body rule __NONEMPTY_BODY ==> got hit: "A" Jun 4 03:36:21.053 [56496] dbg: rules: ran body rule __FB_PICK ==> got hit: "pick" Jun 4 03:36:23.851 [56496] dbg: rules: ran body rule __FB_GAME ==> got hit: "Game" Jun 4 03:36:30.641 [56496] dbg: rules: ran body rule __FRAUD_DBI ==> got hit: "dollars" Jun 4 03:36:34.278 [56496] dbg: rules: ran body rule __F_LARGE_MONEY ==> got hit: "200,000" Jun 4 03:36:36.247 [56496] dbg: rules: ran body rule __HUSH_HUSH ==> got hit: "private" Jun 4 03:36:36.785 [56496] dbg: rules: ran body rule __LOTSA_MONEY_03 ==> got hit: "$300 million" Jun 4 03:36:38.060 [56496] dbg: rules: ran body rule __FB_S_PRICE ==> got hit: "price" Jun 4 03:36:38.066 [56496] dbg: async: select found 1 responses ready (t.o.=0.0) Jun 4 03:36:38.066 [56496] dbg: async: completed in 96.650 s: URI-DNSBL, DNSBL:zen.spamhaus.org.:2.46.246.72 Jun 4 03:36:38.067 [56496] dbg: dns: harvested completed queries Jun 4 03:36:38.068 [56496] dbg: rules: running uri tests; score so far=1.206 sa-compile didn't make a difference: Jun 4 04:15:10.870 [84689] dbg: timing: total 103477 ms - init: 1573 (1.5%), parse: 40 (0.0%), extract_message_metadata: 1196 (1.2%), poll_dns_idle: 6 (0.0%), get_uri_detail_list: 333 (0.3%), tests_pri_-1000: 468 (0.5%), compile_gen: 262 (0.3%), compile_eval: 28 (0.0%), tests_pri_-950: 29 (0.0%), tests_pri_-900: 40 (0.0%), tests_pri_-400: 25 (0.0%), tests_pri_0: 99833 (96.5%), dkim_load_modules: 58 (0.1%), check_dkim_signature: 20 (0.0%), check_razor2: 266 (0.3%), check_pyzor: 3 (0.0%), tests_pri_500: 196 (0.2%) I also can reproduce this with Ubuntu 10.4. [1] http://www.charlieroot.de/downloads/email.txt -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: Performance problem body tests
Helmut Schneider wrote: > with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance > problem: [...] > Any idea where to start? Appendix: I set up a fresh and clean FreeBSD 8.0 with only SA 3.3.1 and Perl 5.10.1_1 and the problem still persists. I then removed all packages, compiled perl 5.8.9_3 and compiled SA 3.3.1 and the problem still persists. I then started from scratch and tried with SA 3.2.5. The particular body_tests take only 5 seconds (instead of 30). -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: [sa] Performance problem body tests
Charles Gregory wrote: > On Wed, 2 Jun 2010, Helmut Schneider wrote: > > with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance > > problem: > > What distinguishes 'certain mails'? Length? Content? Mime > attachements? It's around 1 of 1000, I caught one that was a HTML mail, 100kB, no MIME attachments. But that one reliable, I can reproduce that problem on all (4) of my installations. Unfortunately it's a company internal legal mail so I can't share. > > So the body tests take ~ 30 of 37 seconds. It's not a load problem, > > I noticed a significant increase in processing time when I upgraded > from 3.2 to 3.3. but it was pretty much for all messages. > > You might want to raise the level of debugging How? -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: Performance problem body tests
David Michaels wrote: > Quoting "Helmut Schneider" : > > > Hi, > > > > with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance > > problem: [...] > > timing: total 36840 ms - init: 3827 (10.4%), parse: 43 (0.1%), > > extract_message_metadata: 822 (2.2%), get_uri_detail_list: 178 > > (0.5%), tests_pri_-1000: 212 (0.6%), compile_gen: 538 (1.5%), > > compile_eval: 79 (0.2%), tests_pri_-950: 52 (0.1%), tests_pri_-900: > > 35 (0.1%), tests_pri_-400: 392 (1.1%), check_bayes: 359 (1.0%), > > tests_pri_0: 30780 (83.6%), dkim_load_modules: 99 (0.3%), > > check_dkim_signature: 20 (0.1%), check_dkim_adsp: 13 (0.0%), > > check_spf: 81 (0.2%), poll_dns_idle: 0.57 (0.0%), check_dcc: 240 > > (0.7%), check_pyzor: 2 (0.0%), tests_pri_500: 258 (0.7%), > > tests_pri_1000: 61 (0.2%), total_awl: 28 (0.1%), check_awl: 6 > > (0.0%), update_awl: 4 (0.0%), learn: 118 (0.3%) [/var/amavis/tmp]# > > > > So the body tests take ~ 30 of 37 seconds. It's not a load problem, > > under load it takes >2 minutes and it is reproducable with certain > > mails only. > > > > Any idea where to start? > > are you escaping the white spaces and @? Yes, and also dots. But I also completely removed my custom-rules.cf if you refer to that. -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Performance problem body tests
Hi, with certain mails on FreeBSD 8.0 and SA 3.3.1 I have a performance problem: [/var/amavis/tmp]# spamassassin -D -lint < /var/amavis/tmp/amavis-20100602T192227-44802/email.txt Jun 2 21:37:08.809 [50826] warn: The -l option has been deprecated and is no longer supported, ignoring. Jun 2 21:37:08.810 [50826] dbg: logger: adding facilities: all Jun 2 21:37:08.811 [50826] dbg: logger: logging level is DBG Jun 2 21:37:08.811 [50826] dbg: generic: SpamAssassin version 3.3.1 Jun 2 21:37:08.812 [50826] dbg: generic: Perl 5.010001, PREFIX=/usr/local, DEF_RULES_DIR=/usr/local/share/spamassassin, LOCAL_RULES_DIR=/usr/local/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/db/spamassassin [...] Jun 2 21:37:15.626 [50826] dbg: rules: run_generic_tests - compiling eval code: body, priority 0 Jun 2 21:37:15.627 [50826] dbg: rules: compiled body tests Jun 2 21:37:20.524 [50826] dbg: rules: ran body rule __DOS_BODY_THU ==> got hit: "Thursday" Jun 2 21:37:22.444 [50826] dbg: rules: ran body rule __YOU_ASSIST ==> got hit: "your assistance" Jun 2 21:37:26.018 [50826] dbg: rules: ran body rule __FILL_THIS_FORM_PARTIAL ==> got hit: "Tel : " Jun 2 21:37:26.042 [50826] dbg: rules: ran body rule __FILL_THIS_FORM_PARTIAL ==> got hit: "Tel : " Jun 2 21:37:26.424 [50826] dbg: rules: ran body rule __DOS_BODY_TUE ==> got hit: "Tuesday" Jun 2 21:37:27.410 [50826] dbg: rules: ran body rule __PLS_REVIEW ==> got hit: "Please see attached" Jun 2 21:37:33.503 [50826] dbg: rules: ran body rule __DOS_BODY_FRI ==> got hit: "Friday" Jun 2 21:37:37.334 [50826] dbg: rules: ran body rule __SUBSCRIPTION_INFO ==> got hit: "Register" Jun 2 21:37:37.409 [50826] dbg: rules: ran body rule __HAS_ANY_EMAIL ==> got hit: "s...@vodafone.c" Jun 2 21:37:37.885 [50826] dbg: rules: ran body rule __NONEMPTY_BODY ==> got hit: "R" Jun 2 21:37:43.231 [50826] dbg: rules: ran body rule __HUSH_HUSH ==> got hit: "confidential" Jun 2 21:37:44.008 [50826] dbg: rules: running uri tests; score so far=0 [...] Jun 2 21:37:45.723 [50826] dbg: learn: initializing learner Jun 2 21:37:45.789 [50826] dbg: check: is spam? score=0.012 required=5 Jun 2 21:37:45.790 [50826] dbg: check: tests=HTML_FONT_SIZE_LARGE,HTML_MESSAGE,T_FILL_THIS_FORM Jun 2 21:37:45.791 [50826] dbg: check: subtests=__ANY_TEXT_ATTACH,__ANY_TEXT_ATTACH_DOC,__COMMENT_EXISTS,__CT,_ _CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__CTYPE_MULTIPART_ANY,__DKIM_D EPENDABLE,__DOS_BODY_FRI,__DOS_BODY_THU,__DOS_BODY_TUE,__DOS_HAS_ANY_URI ,__DOS_RCVD_WED,__DOS_REF_2_WK_DAYS,__DOS_REF_NEXT_WK_DAY,__DOS_RELAYED_ EXT,__FILL_THIS_FORM_PARTIAL,__FILL_THIS_FORM_PARTIAL,__FILL_THIS_FORM_P ARTIAL_RAW,__FILL_THIS_FORM_PARTIAL_RAW,__FILL_THIS_FORM_PARTIAL_RAW,__F ILL_THIS_FORM_PARTIAL_RAW,__FILL_THIS_FORM_PARTIAL_RAW,__FILL_THIS_FORM_ PARTIAL_RAW,__FILL_THIS_FORM_PARTIAL_RAW,__FILL_THIS_FORM_PARTIAL_RAW,__ HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_DATE,__HAS_MESSAGE_ID,__HAS_MIMEOLE,__ HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HS_SUBJ_RE_FW,__HUSH_HUSH,__IMS_MSG ID,__IS_EXCH,__LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUT H,__MIME_HTML,__MIME_QP,__MIME_VERSION,__NONEMPTY_BODY,__PLS_REVIEW,__RC VD_IN_2WEEKS,__SANE_MSGID,__SUBJ_RE,__SUBSCRIPTION_INFO,__TAG_EXISTS_BOD Y,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__ TVD_MIME_ATT_TP,__YOU_ASSIST Jun 2 21:37:45.793 [50826] dbg: timing: total 36840 ms - init: 3827 (10.4%), parse: 43 (0.1%), extract_message_metadata: 822 (2.2%), get_uri_detail_list: 178 (0.5%), tests_pri_-1000: 212 (0.6%), compile_gen: 538 (1.5%), compile_eval: 79 (0.2%), tests_pri_-950: 52 (0.1%), tests_pri_-900: 35 (0.1%), tests_pri_-400: 392 (1.1%), check_bayes: 359 (1.0%), tests_pri_0: 30780 (83.6%), dkim_load_modules: 99 (0.3%), check_dkim_signature: 20 (0.1%), check_dkim_adsp: 13 (0.0%), check_spf: 81 (0.2%), poll_dns_idle: 0.57 (0.0%), check_dcc: 240 (0.7%), check_pyzor: 2 (0.0%), tests_pri_500: 258 (0.7%), tests_pri_1000: 61 (0.2%), total_awl: 28 (0.1%), check_awl: 6 (0.0%), update_awl: 4 (0.0%), learn: 118 (0.3%) [/var/amavis/tmp]# So the body tests take ~ 30 of 37 seconds. It's not a load problem, under load it takes >2 minutes and it is reproducable with certain mails only. Any idea where to start? Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: URIDNSBL
I am using the 3.0 line of SpamAssassin and it's being invoked through amavisd-maia (Maia Mailguard.) I have a certain domain name that's blocked in several of the URIDNSBL lists as "fm.interia.pl" however my DNSBL checks are only doing interia.pl Just as I'm curious, what does SA score that mail? X-Spam-Status: Yes, score=35.341 tag=- tag2=6.3 kill=6.3 tests=[BAYES_99=6.5, DOS_OE_TO_MX=2.75, FH_HELO_EQ_D_D_D_D=0.001, FM_SEX_HELO=1.851, HELO_DYNAMIC_HCC=4.295, HELO_DYNAMIC_IPADDR2=4.395, LOGINHASH=4.5, LOGINHASH2=2.5, RCVD_IN_BL_SPAMCOP_NET=1.96, RCVD_IN_PBL=0.905, RCVD_IN_SORBS_WEB=0.619, RCVD_IN_XBL=3.033, RDNS_DYNAMIC=0.1, STOX_REPLY_TYPE=0.001, TVD_RCVD_IP=1.931] autolearn=spam Using amavisd-new 2.6.2 and SA 3.2.5.
Re: bayes options
Matt Kettler wrote: Helmut Schneider wrote: where can I find a complete set of (bayes) options for local.cf? Either it's well hidden or even http://spamassassin.apache.org/ does not provide such a list. Thanks, Helmut http://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Conf.html# learning_options Or, on your machine, man Mail::SpamAssassin::Conf, and page to the "Learning Options" section. Excellent. Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
bayes options
Hi, where can I find a complete set of (bayes) options for local.cf? Either it's well hidden or even http://spamassassin.apache.org/ does not provide such a list. Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: move bayes-db to mysql, unable to initialize database for amavis user, aborting!
Micke Andersson wrote: Helmut Schneider wrote: Hi, I tried to move a global (not per user) bayes-db to mysql5.0 to use it with 3 different machines. From local.cf: use_bayes 1 bayes_store_module Mail::SpamAssassin::BayesStore::SQL bayes_sql_dsn DBI:mysql:SpamAssassin bayes_sql_username amavis bayes_sql_password amavis bayes_sql_override_username amavis auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList user_awl_dsnDBI:mysql:SpamAssassin user_awl_sql_username amavis user_awl_sql_password amavis spamassassin --lint -D [12722] dbg: bayes: using username: amavis [12722] dbg: bayes: database connection established [12722] dbg: bayes: found bayes db version 3 [12722] dbg: bayes: unable to initialize database for amavis user, aborting! [12722] dbg: bayes: database connection established [12722] dbg: bayes: found bayes db version 3 [12722] dbg: bayes: unable to initialize database for amavis user, aborting! What did I miss? I read README.bayes carefully, but... Have you tried to connect to your MySQL database with user amavis and password amavis? Forgot to initialize the database: http://wiki.apache.org/spamassassin/BetterDocumentation/SqlReadmeBayes, bottom. Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
move bayes-db to mysql, unable to initialize database for amavis user, aborting!
Hi, I tried to move a global (not per user) bayes-db to mysql5.0 to use it with 3 different machines. From local.cf: use_bayes 1 bayes_store_module Mail::SpamAssassin::BayesStore::SQL bayes_sql_dsn DBI:mysql:SpamAssassin bayes_sql_username amavis bayes_sql_password amavis bayes_sql_override_username amavis auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList user_awl_dsnDBI:mysql:SpamAssassin user_awl_sql_username amavis user_awl_sql_password amavis spamassassin --lint -D [12722] dbg: bayes: using username: amavis [12722] dbg: bayes: database connection established [12722] dbg: bayes: found bayes db version 3 [12722] dbg: bayes: unable to initialize database for amavis user, aborting! [12722] dbg: bayes: database connection established [12722] dbg: bayes: found bayes db version 3 [12722] dbg: bayes: unable to initialize database for amavis user, aborting! What did I miss? I read README.bayes carefully, but... Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: Filter from *and* to
Helmut Schneider wrote: can I (and if how) create a filter that catches mails _from_and_to_ specific email addresses? It should only apply if a specific sender sendw an email to a specific recipient. I can use a meta rule, thanks. -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Filter from *and* to
Hi, can I (and if how) create a filter that catches mails _from_and_to_ specific email addresses? It should only apply if a specific sender sendw an email to a specific recipient. Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: whitelist_from/whitelist_auth and custom score
On Wed, 2008-07-02 at 11:12 +0200, Helmut Schneider wrote: I would like to do some whitelisting for an external mailing list. I found "whitelist_from" and "whitelist_auth" but they automatically score -100. Is there a way to use whitelist_* or something similiar with a custom score? amavisd-new provides "soft-whitelisting" where you can put in a custom score per recipient. I changed the default score for one of my whitelists: score USER_IN_SPF_WHITELIST -10.000 Seems I have to use whitelist_to, does it check To:, or envelope-to:?
whitelist_from/whitelist_auth and custom score
Hi, I would like to do some whitelisting for an external mailing list. I found "whitelist_from" and "whitelist_auth" but they automatically score -100. Is there a way to use whitelist_* or something similiar with a custom score? Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: sa-update and location of rules
"Theo Van Dinter" <[EMAIL PROTECTED]> wrote: On Tue, Jun 17, 2008 at 09:56:49PM +0200, Helmut Schneider wrote: >FWIW, the directories and their order are well documented in the >spamassassin >POD. Could you please point me to the exact location? Thanks. (you could also use "man") $ perldoc spamassassin [...] CONFIGURATION FILES The SpamAssassin rule base, text templates, and rule description text are loaded from configuration files. Default configuration data is loaded from the first existing directory in: /var/lib/spamassassin/3.002005 /usr/share/spamassassin [...] Ah, OK, I searched here: http://spamassassin.apache.org/full/3.2.x/doc/ Thanks.
Re: sa-update and location of rules
"Theo Van Dinter" <[EMAIL PROTECTED]> wrote: On Tue, Jun 17, 2008 at 10:42:41AM +0200, Helmut Schneider wrote: So /var/db/spamassassin//updates_spamassassin_org has precedence over /usr/local/etc/mail/spamassassin? Some kind of version checking or rather the existence of the rules file? What happens if /usr/local/etc/mail/spamassassin contains obsolete rules? /usr/local/etc/mail/spamassassin sounds like your like site rules dir, so if you have obsolete rules in there you will continue to have them. Typo, I meant /usr/local/share/spamassassin/ FWIW, the directories and their order are well documented in the spamassassin POD. Could you please point me to the exact location? Thanks.
Re: SpamAssassin 3.2.5 committed to FreeBSD ports
Helmut Schneider <[EMAIL PROTECTED]> wrote: Len Conrad <[EMAIL PROTECTED]> wrote: Does libkrb5.so.8 exist (usually in /usr/lib/)?! no. installed heimdal then krb5 from ports, no problem. re-booted. same msgs as before in sshd logs. sshd won't allow any logins. and complains same as before. Did you install security/krb5 or security/heimdal from ports? yes, after your first msg. Check your make.conf. If there are no entries about kerberos, remove security/heimdal and then: cd /usr/src/kerberos5/lib/libkrb5 && make && make install && make clean I still don't see why a port upgrade should remove base conponents but you should consider rebuilding the system[1]. Alternatively use sysinstall and "fixit". [1] http://www.freebsd.org/doc/en/books/handbook/makeworld.html -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: SpamAssassin 3.2.5 committed to FreeBSD ports
Len Conrad <[EMAIL PROTECTED]> wrote: Does libkrb5.so.8 exist (usually in /usr/lib/)?! no. installed heimdal then krb5 from ports, no problem. re-booted. same msgs as before in sshd logs. sshd won't allow any logins. and complains same as before. Did you install security/krb5 or security/heimdal from ports? yes, after your first msg. Check your make.conf. If there are no entries about kerberos, remove security/heimdal and then: cd /usr/src/kerberos5/lib/libkrb5 && make && make install && make clean I still don't see why a port upgrade should remove base conponents but you should consider rebuilding the system[1]. Alternatively use sysinstall and "fixit". -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: SpamAssassin 3.2.5 committed to FreeBSD ports
Please don't post HTML, thanks... for sshd: /libexec/ld-elf.so.1: shared object "libkrb5.so.8" not found required by "sshd" Both sshd and libkrb5.so.8 are part of the base system so I guess you messed up something else. Does libkrb5.so.8 exist (usually in /usr/lib/)?! Did you install security/krb5 or security/heimdal from ports?
Re: sa-update and location of rules
Michael Scheidell <[EMAIL PROTECTED]> wrote: running FreeBSD I have two directories with rules in it: /usr/local/share/spamassassin /var/db/spamassassin/3.002005/updates_spamassassin_org Which is the correct directory, which rules are used? SpamAssassin will use the default, distributed rules in /usr/local/share/spamassassin plus /usr/local/etc/mail/spamassassin UNTIL YOU RUN SA-UPDATE. Then it uses the rules in /var/db/spamassassin//updates_spamassassin_org plus /usr/local/etc/mail/spamassassin. So /var/db/spamassassin//updates_spamassassin_org has precedence over /usr/local/etc/mail/spamassassin? Some kind of version checking or rather the existence of the rules file? What happens if /usr/local/etc/mail/spamassassin contains obsolete rules? I'm running amavisd chroot'ed, 'cp -rp /var/db/spamassassin /var/amavisd/var/db' is all I need to do? -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
sa-update and location of rules
Hi, running FreeBSD I have two directories with rules in it: /usr/local/share/spamassassin /var/db/spamassassin/3.002005/updates_spamassassin_org Which is the correct directory, which rules are used? Thanks, Helmut -- No Swen today, my love has gone away My mailbox stands for lorn, a symbol of the dawn
Re: OT: digest version of mailing list
From: "SM" <[EMAIL PROTECTED]> At 07:41 18-07-2007, Helmut Schneider wrote: sorry if I missed something but is there also a digest version of the mailing list? I searched http://wiki.apache.org/spamassassin/MailingLists but only found subscribe and unsubscribe. Send an email to [EMAIL PROTECTED] Thanks a lot.
OT: digest version of mailing list
Hi, sorry if I missed something but is there also a digest version of the mailing list? I searched http://wiki.apache.org/spamassassin/MailingLists but only found subscribe and unsubscribe. Thanks, Helmut
Re: "report_safe" does not work
From: "Wolfgang Zeikat" <[EMAIL PROTECTED]> On 07/12/07 15:47, Helmut Schneider wrote: Hi, I use amavisd-new 2.52 and SA3.21 chroot'ed. Is there a setting that only mail with a hit greater than X is modified? Or did I miss anything else? AFAIK, amavisd-new has it's own ways of using SA, and that includes ignoring some local.cf options. You can try and put them into the amavisd config file with something like $sa_report_safe = 1; I don't have amavisd-new installed, but learned lately that the $sa_* way works with some options, see the sample / default cf files in the documentation. $defang_spam HTH, Yes. Thanks, Helmut
"report_safe" does not work
Hi, I use amavisd-new 2.52 and SA3.21 chroot'ed. [EMAIL PROTECTED] ~]# grep -ir report_safe /var/amavis/usr/local/etc/mail/spamassassin/local.cf # report_safe 1 report_safe 2 [EMAIL PROTECTED] ~]# spamassassin --lint -d [...] [15632] dbg: config: read file /usr/local/etc/mail/spamassassin/local.cf [...] [EMAIL PROTECTED] ~]# I do not use any user_prefs. Nevertheless the email is delivered unchanged to my inbox. It is tagged so SA seems to work. Is there a setting that only mail with a hit greater than X is modified? Or did I miss anything else? Thanks, Helmut
Re: AMaViS/SA chrroted: Error creating a DNS resolver socket: No such file or directory
From: "Leon Kolchinsky" <[EMAIL PROTECTED]> I've never run amavisd-new in chroot, but may be you'll find some tips here - http://www.ijs.si/software/amavisd/README.chroot Seems to me like a resolver issue (probably need to configure FreeBSD a little different than OpenBSD). Best Regards, Well, you should. Everyone should run amavis and the virus engine chroot'ed... :) BTW, I know this document very well, see notes at bottom and: http://flakshack.com/anti-spam/wiki/index.php?page=Introduction ;))
Re: AMaViS/SA chrroted: Error creating a DNS resolver socket: No such file or directory
From: "Helmut Schneider" <[EMAIL PROTECTED]>
[problems resolving a host]
Damn!!
[EMAIL PROTECTED] ~]# cat /var/amavis/test2.pl
#!/usr/bin/perl -w
use Net::DNS;
my $res = Net::DNS::Resolver->new;
my $query = $res->search("www.google.de");
if ($query) {
foreach my $rr ($query->answer) {
next unless $rr->type eq "A";
print $rr->address, "\n";
}
} else {
warn "query failed: ", $res->errorstring, "\n";
}
[EMAIL PROTECTED] ~]# ktrace chroot -u vscan -g vscan /var/amavis/ /test2.pl
query failed: could not get socket
[EMAIL PROTECTED] ~]# kdump -f ./ktrace.out
[...]
40635 perl5.8.8 CALL open(0x28275464,0,0x1b6)
40635 perl5.8.8 NAMI "/etc/protocols"
40635 perl5.8.8 RET open -1 errno 2 No such file or directory
40635 perl5.8.8 CALL open(0x28275464,0,0x1b6)
40635 perl5.8.8 NAMI "/etc/protocols"
40635 perl5.8.8 RET open -1 errno 2 No such file or directory
40635 perl5.8.8 CALL open(0x28275464,0,0x1b6)
40635 perl5.8.8 NAMI "/etc/protocols"
40635 perl5.8.8 RET open -1 errno 2 No such file or directory
40635 perl5.8.8 CALL open(0x28275464,0,0x1b6)
40635 perl5.8.8 NAMI "/etc/protocols"
40635 perl5.8.8 RET open -1 errno 2 No such file or directory
40635 perl5.8.8 CALL break(0x82c0800)
40635 perl5.8.8 RET break 0
40635 perl5.8.8 CALL break(0x82c1000)
40635 perl5.8.8 RET break 0
40635 perl5.8.8 CALL write(0x2,0x82beee0,0x23)
40635 perl5.8.8 GIO fd 2 wrote 35 bytes
"query failed: could not get socket
"
40635 perl5.8.8 RET write 35/0x23
40635 perl5.8.8 CALL break(0x82c1800)
40635 perl5.8.8 RET break 0
40635 perl5.8.8 CALL break(0x82c2000)
40635 perl5.8.8 RET break 0
40635 perl5.8.8 CALL exit(0)
[EMAIL PROTECTED] ~]# cp /etc/protocols /var/amavis/etc/
[EMAIL PROTECTED] ~]# chroot -u vscan -g vscan /var/amavis/ /test2.pl
209.85.135.104
209.85.135.147
209.85.135.99
209.85.135.103
[EMAIL PROTECTED] ~]#
[40648] dbg: dns: is_dns_available() last checked 1184068231 seconds ago;
re-checking
[40648] dbg: dns: name server: 192.168.0.90, LocalAddr: 0.0.0.0
[40648] dbg: dns: testing resolver nameservers: 192.168.0.90, 192.168.0.80
[40648] dbg: dns: trying (3) msn.com...
[40648] dbg: dns: looking up NS for 'msn.com'
[40648] dbg: dns: NS lookup of msn.com using 192.168.0.90 succeeded => DNS
available (set dns_available to override)
[40648] dbg: dns: is DNS available? 1
Thanks a lot, Helmut :)
Re: AMaViS/SA chrroted: Error creating a DNS resolver socket: No such file or directory
From: "Leon Kolchinsky" <[EMAIL PROTECTED]> Stupid question, but $MYHOME = /var/amavis ? Yes Also you can try to debug it with strace. Look for all the files you're missing in the jail and copy them into it. (here is a little example on how to use strace to find requirements outside the jail - http://olivier.sessink.nl/jailkit/howtos_debug_jails.html ) Best Regards, Leon Kolchinsky I run amavisd/SA/clamav successfully on OpenBSD for years now but the same config does not work with FreeBSD. I put 'host' and 'nslookup' into the jail: [EMAIL PROTECTED] ~]# chroot -u vscan -g vscan /var/amavis/ /usr/bin/host www.google.de www.google.de is an alias for www.google.com. www.google.com is an alias for www.l.google.com. www.l.google.com has address 209.85.135.104 www.l.google.com has address 209.85.135.147 www.l.google.com has address 209.85.135.99 www.l.google.com has address 209.85.135.103 [EMAIL PROTECTED] ~]# chroot -u vscan -g vscan /var/amavis/ /usr/bin/nslookup www.google.de Server: 192.168.0.90 Address: 192.168.0.90#53 Non-authoritative answer: www.google.de canonical name = www.google.com. www.google.com canonical name = www.l.google.com. Name: www.l.google.com Address: 209.85.135.104 Name: www.l.google.com Address: 209.85.135.147 Name: www.l.google.com Address: 209.85.135.99 Name: www.l.google.com Address: 209.85.135.103 [EMAIL PROTECTED] ~]# But amavisd debug-sa says: [...] [7930] dbg: util: final PATH set to: /usr/local/sbin:/usr/local/bin:/usr/bin [7930] dbg: dns: no ipv6 [7930] dbg: dns: is Net::DNS::Resolver available? yes [7930] dbg: dns: Net::DNS version: 0.60 [...] [7930] dbg: dns: is_dns_available() last checked 1184065522 seconds ago; re-checking [7930] dbg: dns: name server: 192.168.0.90, LocalAddr: 0.0.0.0 Error creating a DNS resolver socket: No such file or directory at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm line 227. [7930] dbg: dns: testing resolver nameservers: 192.168.0.90, 192.168.0.80 [7930] dbg: dns: trying (3) ebay.com... [7930] dbg: dns: looking up NS for 'ebay.com' [7930] dbg: dns: NS lookup of ebay.com using 192.168.0.90 failed, no results found [7930] dbg: dns: trying (2) motorola.com... [7930] dbg: dns: looking up NS for 'motorola.com' [7930] dbg: dns: NS lookup of motorola.com using 192.168.0.90 failed, no results found [7930] dbg: dns: trying (1) yahoo.com... [7930] dbg: dns: looking up NS for 'yahoo.com' [7930] dbg: dns: NS lookup of yahoo.com using 192.168.0.90 failed, no results found [7930] dbg: dns: NS lookups failed, removing nameserver 192.168.0.90 from list [7930] dbg: dns: trying (3) google.com... [7930] dbg: dns: looking up NS for 'google.com' [7930] dbg: dns: NS lookup of google.com using 192.168.0.80 failed, no results found [7930] dbg: dns: trying (2) kernel.org... [7930] dbg: dns: looking up NS for 'kernel.org' [7930] dbg: dns: NS lookup of kernel.org using 192.168.0.80 failed, no results found [7930] dbg: dns: trying (1) linux.org... [7930] dbg: dns: looking up NS for 'linux.org' [7930] dbg: dns: NS lookup of linux.org using 192.168.0.80 failed, no results found [7930] dbg: dns: NS lookups failed, removing nameserver 192.168.0.80 from list [7930] dbg: dns: all NS queries failed => DNS unavailable (set dns_available to override) [7930] dbg: dns: is DNS available? 0 What would a perl command look like to resolve a host? Because I think it is a perl issue. Helmut
Re: AMaViS/SA chrroted: Error creating a DNS resolver socket: No such file or directory
From: "Leon Kolchinsky" <[EMAIL PROTECTED]> I tried to set up SA with AMaViS in a chrooted environment ($daemon_chroot_dir = $MYHOME). I (thought I) copied all necessary files to the jail but when SA is starting I get an error: Jul 10 10:44:02 TEG /usr/local/sbin/amavisd[6817]: SpamControl: initializing Mail::SpamAssassin Error creating a DNS resolver socket: No such file or directory at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm line 227. Jul 10 10:44:05 TEG /usr/local/sbin/amavisd[6817]: SpamControl: init_pre_fork done Any idea what is missing? Do you have this /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm File in your chrooted environment? Yes: [EMAIL PROTECTED] ~]# ls -la /var/amavis/usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm -rwxr-x--- 1 root vscan 14970 Jun 8 14:55 /var/amavis/usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm [EMAIL PROTECTED] ~]# Can I run a perl script using DNSResolver to test things? BTW, $CHROOT/etc/resolv.conf is of course present, too.
AMaViS/SA chrroted: Error creating a DNS resolver socket: No such file or directory
Hi, I tried to set up SA with AMaViS in a chrooted environment ($daemon_chroot_dir = $MYHOME). I (thought I) copied all necessary files to the jail but when SA is starting I get an error: Jul 10 10:44:02 TEG /usr/local/sbin/amavisd[6817]: SpamControl: initializing Mail::SpamAssassin Error creating a DNS resolver socket: No such file or directory at /usr/local/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/DnsResolver.pm line 227. Jul 10 10:44:05 TEG /usr/local/sbin/amavisd[6817]: SpamControl: init_pre_fork done Any idea what is missing? Thanks, Helmut
Re: sa-update, can't resolve 'localhost' to address
From: "Duncan Hill" <[EMAIL PROTECTED]> On Fri, June 15, 2007 08:20, Helmut Schneider wrote: From: "Justin Mason" <[EMAIL PROTECTED]> [EMAIL PROTECTED] ~]# sa-update --nogpg can't resolve "localhost" to address at /usr local/libdata/perl5/site_perl/i386-openbsd/Net/DNS/Resolver/Base.pm line 751. [EMAIL PROTECTED] ~]# A guess-- you have server localhost in /etc/resolv.conf. In deed, localhost (which is a public nameserver itself) is the last of 5 nameservers in the list. Do I have to understand that? :) You can't use a name for a nameserver. Put 127.0.0.1. Although it is listed in /etc/hosts?! Well, OK, good to know. Thanks, Helmut
Re: sa-update, can't resolve "localhost" to address
Hi, From: "Justin Mason" <[EMAIL PROTECTED]> [EMAIL PROTECTED] ~]# sa-update --nogpg can't resolve "localhost" to address at /usr/local/libdata/perl5/site_perl/i386-openbsd/Net/DNS/Resolver/Base.pm line 751. [EMAIL PROTECTED] ~]# A guess-- you have server localhost in /etc/resolv.conf. In deed, localhost (which is a public nameserver itself) is the last of 5 nameservers in the list. Do I have to understand that? :) Thanks, Helmut
sa-update, can't resolve "localhost" to address
Hi,
[EMAIL PROTECTED] ~]# uname -rs
OpenBSD 4.0
[EMAIL PROTECTED] ~]#
When I run sa-update I get the error below:
[EMAIL PROTECTED] ~]# sa-update --nogpg
can't resolve "localhost" to address at
/usr/local/libdata/perl5/site_perl/i386-openbsd/Net/DNS/Resolver/Base.pm
line 751.
[EMAIL PROTECTED] ~]#
Could anyone please tell me what's wrong?
[EMAIL PROTECTED] ~]# sa-update -V
sa-update version svn454083
running on Perl version 5.8.8
[EMAIL PROTECTED] ~]# spamassassin -V
SpamAssassin version 3.1.7
running on Perl version 5.8.8
[EMAIL PROTECTED] ~]#
[EMAIL PROTECTED] ~]# sa-update --nogpg -D
[28203] dbg: logger: adding facilities: all
[28203] dbg: logger: logging level is DBG
[28203] dbg: generic: SpamAssassin version 3.1.7
[28203] dbg: config: score set 0 chosen.
[28203] dbg: message: MIME PARSER START
[28203] dbg: message: main message type: text/plain
[28203] dbg: message: parsing normal part
[28203] dbg: message: added part, type: text/plain
[28203] dbg: message: MIME PARSER END
[28203] dbg: dns: is Net::DNS::Resolver available? yes
[28203] dbg: dns: Net::DNS version: 0.59
[28203] dbg: generic: sa-update version svn454083
[28203] dbg: generic: using update directory: /var/lib/spamassassin/3.001007
[28203] dbg: diag: perl platform: 5.008008 openbsd
[28203] dbg: diag: module installed: Digest::SHA1, version 2.11
[28203] dbg: diag: module installed: MIME::Base64, version 3.07
[28203] dbg: diag: module installed: HTML::Parser, version 3.55
[28203] dbg: diag: module installed: DB_File, version 1.814
[28203] dbg: diag: module installed: Net::DNS, version 0.59
[28203] dbg: diag: module installed: Net::SMTP, version 2.31
[28203] dbg: diag: module installed: Mail::SPF::Query, version 1.999001
[28203] dbg: diag: module installed: IP::Country::Fast, version 604.001
[28203] dbg: diag: module installed: Razor2::Client::Agent, version 2.67
[28203] dbg: diag: module not installed: Net::Ident ('require' failed)
[28203] dbg: diag: module installed: IO::Socket::INET6, version 2.51
[28203] dbg: diag: module not installed: IO::Socket::SSL ('require' failed)
[28203] dbg: diag: module installed: Time::HiRes, version 1.86
[28203] dbg: diag: module not installed: DBI ('require' failed)
[28203] dbg: diag: module installed: Getopt::Long, version 2.35
[28203] dbg: diag: module installed: LWP::UserAgent, version 2.033
[28203] dbg: diag: module installed: HTTP::Date, version 1.47
[28203] dbg: diag: module installed: Archive::Tar, version 1.32
[28203] dbg: diag: module installed: IO::Zlib, version 1.04
[28203] dbg: channel: attempting channel updates.spamassassin.org
[28203] dbg: channel: update directory
/var/lib/spamassassin/3.001007/updates_spamassassin_org
[28203] dbg: channel: channel cf file
/var/lib/spamassassin/3.001007/updates_spamassassin_org.cf
[28203] dbg: channel: channel pre file
/var/lib/spamassassin/3.001007/updates_spamassassin_org.pre
can't resolve "localhost" to address at
/usr/local/libdata/perl5/site_perl/i386-openbsd/Net/DNS/Resolver/Base.pm
line 751.
[EMAIL PROTECTED] ~]#
Thanks, Helmut
