Re: SPF and Upgrade to SA 3.1

Ben Lentz Wed, 28 Sep 2005 18:43:19 -0700

Okay, I've added
always_trust_envelope_sender 1
trusted_networks 10.1.0.0/16
trusted_networks 205.246.7.107

and restarted. Still not acknoledgement that SPF is working forgmail.com. SPF-based whitelisting might be great, but at this point I'mstill not confident that SPF is working for me.

Here's the debug info. The line "domain of sender wproxy.gmail.com doesnot designate mailers" leads me to believe that SA thinks gmail's SPFrecord is bad because they don't have any IN TXT wproxy.gmail.com SPFrecord, they're only publishing at the domain level.


Thanks for all the help!

[28984] dbg: prefork: ordered 28988 to accept

[28988] info: spamd: connection from localhost.localdomain [127.0.0.1]at port 50129

[28984] dbg: prefork: child 28988: entering state 2
[28984] dbg: prefork: new lowest idle kid: 28989

[28988] warn: spamd: still running as root: user not specified with -u,not found, or set to root, falling back to nobody at /usr/bin/spamd line1150, <GEN8> line 3.[28988] info: spamd: checking message <[EMAIL PROTECTED]> for(unknown):99

[28988] dbg: dns: name server: 10.1.200.0, family: 2, ipv6: 0

[28988] dbg: received-header: parsed as [ ip=10.1.200.36rdns=smtp.channing-bete.com helo=smtp.channing-bete.comby=spam.channing-bete.com ident= envfrom= intl=0 id=j8T1cnxH028990 auth= ]

[28988] dbg: received-header: relay 10.1.200.36 trusted? yes internal? yes

[28988] dbg: received-header: parsed as [ ip=64.233.184.199rdns=wproxy.gmail.com helo=wproxy.gmail.com by=smtp.channing-bete.comident= envfrom= intl=0 id=j8T1cgHY012157 auth= ]

[28988] dbg: received-header: relay 64.233.184.199 trusted? no internal? no
[28988] dbg: dns: looking up PTR record for '67.20.144.224'
[28988] dbg: dns: PTR for '67.20.144.224': ''

[28988] dbg: received-header: parsed as [ ip=67.20.144.224 rdns= helo=by=mx.gmail.com ident= envfrom= intl=0id=35sm106614wra.2005.09.28.18.38.50 auth= ]

[28988] dbg: received-header: relay 67.20.144.224 trusted? no internal? no

[28988] dbg: metadata: X-Spam-Relays-Trusted: [ ip=10.1.200.36rdns=smtp.channing-bete.com helo=smtp.channing-bete.comby=spam.channing-bete.com ident= envfrom= intl=1 id=j8T1cnxH028990 auth= ][28988] dbg: metadata: X-Spam-Relays-Untrusted: [ ip=64.233.184.199rdns=wproxy.gmail.com helo=wproxy.gmail.com by=smtp.channing-bete.comident= envfrom= intl=0 id=j8T1cgHY012157 auth= ] [ ip=67.20.144.224rdns= helo= by=mx.gmail.com ident= envfrom= intl=0id=35sm106614wra.2005.09.28.18.38.50 auth= ]

[28988] dbg: metadata: X-Relay-Countries: US US
[28988] dbg: message: ---- MIME PARSER START ----
[28988] dbg: message: main message type: multipart/alternative

[28988] dbg: message: parsing multipart, got boundary:------------050703040406040403090403[28988] dbg: message: found part of type text/plain, boundary:------------050703040406040403090403

[28988] dbg: message: parsing normal part
[28988] dbg: message: added part, type: text/plain

[28988] dbg: message: found part of type text/html, boundary:------------050703040406040403090403

[28988] dbg: message: parsing normal part
[28988] dbg: message: added part, type: text/html
[28988] dbg: message: ---- MIME PARSER END ----
[28988] dbg: message: decoding other encoding type (7bit), ignoring
[28988] dbg: message: decoding other encoding type (7bit), ignoring
[28988] dbg: textcat: message too short for language analysis
[28988] dbg: textcat: X-Languages: "", X-Languages-Length: 18
[28988] dbg: uridnsbl: domains to query:
[28988] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl-notfirsthop

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS A query for199.184.233.64.sbl-xbl.spamhaus.org. in background[28988] dbg: dns: checking RBL sa-accredit.habeas.com., sethabeas-firsttrusted[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS A query for199.184.233.64.sa-accredit.habeas.com. in background

[28988] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl

[28988] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224

[28988] dbg: dns: launching DNS TXT query for224.144.20.67.sa-other.bondedsender.org. in background

[28988] dbg: dns: checking RBL combined.njabl.org., set njabl-notfirsthop

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS A query for199.184.233.64.combined.njabl.org. in background

[28988] dbg: dns: checking RBL combined.njabl.org., set njabl

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224,64.233.184.199[28988] dbg: dns: launching DNS A query for224.144.20.67.combined.njabl.org. in background[28988] dbg: dns: checking RBLcombined-HIB.dnsiplists.completewhois.com., set whois[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224,64.233.184.199[28988] dbg: dns: launching DNS A query for224.144.20.67.combined-HIB.dnsiplists.completewhois.com. in background[28988] dbg: dns: launching DNS A query for199.184.233.64.combined-HIB.dnsiplists.completewhois.com. in background

[28988] dbg: dns: checking RBL list.dsbl.org., set dsbl-notfirsthop

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS TXT query for199.184.233.64.list.dsbl.org. in background

[28988] dbg: dns: checking RBL bl.spamcop.net., set spamcop

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:[28988] dbg: dns: only inspecting the following IPs: 67.20.144.224,64.233.184.199[28988] dbg: dns: launching DNS TXT query for224.144.20.67.bl.spamcop.net. in background[28988] dbg: dns: launching DNS TXT query for199.184.233.64.bl.spamcop.net. in background[28988] dbg: dns: checking RBL sa-trusted.bondedsender.org., setbsp-firsttrusted[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS TXT query for199.184.233.64.sa-trusted.bondedsender.org. in background[28988] dbg: dns: checking RBLcombined-HIB.dnsiplists.completewhois.com., set whois-notfirsthop[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199
[28988] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-notfirsthop

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS A query for199.184.233.64.dnsbl.sorbs.net. in background

[28988] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs

[28988] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted

[28988] dbg: dns: IPs found: full-external: 64.233.184.199,67.20.144.224 untrusted: 64.233.184.199, 67.20.144.224 originating:

[28988] dbg: dns: only inspecting the following IPs: 64.233.184.199

[28988] dbg: dns: launching DNS A query for199.184.233.64.iadb.isipp.com. in background

[28988] dbg: check: running tests for priority: 0
[28988] dbg: rules: running header regexp tests; score so far=0
[28988] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
[28988] dbg: rules: ran header rule __CT ======> got hit: "m"
[28988] dbg: rules: ran header rule __HAS_RCVD ======> got hit: "f"

[28988] dbg: rules: ran header rule __SANE_MSGID ======> got hit:"<[EMAIL PROTECTED]>

[28988] dbg: rules: "

[28988] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit:"@gmail.com>"

[28988] dbg: rules: ran header rule __MIME_VERSION ======> got hit: "1"

[28988] dbg: rules: ran header rule __CTYPE_MULTIPART_ALT ======> gothit: "multipart/alternative"

[28988] dbg: rules: ran header rule __TOCC_EXISTS ======> got hit: "B"

[28988] dbg: rules: ran header rule __MOZILLA_MSGID ======> got hit:"<[EMAIL PROTECTED]>"

[28988] dbg: rules: ran header rule __USER_AGENT ======> got hit: "M"
[28988] dbg: rules: ran header rule __HAS_SUBJECT ======> got hit: "t"

[28988] dbg: rules: ran header rule __CTYPE_HAS_BOUNDARY ======> gothit: "boundary"[28988] dbg: rules: ran header rule __BAT_BOUNDARY ======> got hit:"boundary="----------"

[28988] dbg: spf: checking HELO (helo=wproxy.gmail.com, ip=64.233.184.199)

[28988] dbg: spf: query for /64.233.184.199/wproxy.gmail.com: result:none, comment: SPF: domain of sender wproxy.gmail.com does not designatemailers

[28988] dbg: eval: all '*From' addrs: [EMAIL PROTECTED]

[28988] dbg: eval: trying Received header date for real time: 28 Sep2005 21:38:49 -0400[28988] dbg: eval: time_t from date=1127957929, rcvd= 28 Sep 200521:38:49 -0400[28988] dbg: eval: trying Received header date for real time: 28 Sep2005 21:38:42 -0400[28988] dbg: eval: time_t from date=1127957922, rcvd= 28 Sep 200521:38:42 -0400[28988] dbg: eval: trying Received header date for real time: 28 Sep2005 18:38:51 -0700[28988] dbg: eval: time_t from date=1127957931, rcvd= 28 Sep 200518:38:51 -0700[28988] dbg: eval: trying Received header date for real time: 28 Sep2005 18:38:51 -0700[28988] dbg: eval: time_t from date=1127957931, rcvd= 28 Sep 200518:38:51 -0700[28988] dbg: eval: trying Received header date for real time: 28 Sep2005 18:38:50 -0700[28988] dbg: eval: time_t from date=1127957930, rcvd= 28 Sep 200518:38:50 -0700

[28988] dbg: eval: all '*To' addrs: [EMAIL PROTECTED]
[28988] dbg: spf: cannot get Envelope-From, cannot use SPF

[28988] dbg: eval: forged-HELO: from=gmail.com helo=gmail.comby=channing-bete.com

[28988] dbg: eval: forged-HELO: from= helo= by=gmail.com
[28988] dbg: rules: ran eval rule DK_SIGNED ======> got hit
[28988] dbg: rules: ran eval rule DK_VERIFIED ======> got hit

[28988] dbg: spf: def_spf_whitelist_from: could not find useableenvelope sender

[28988] dbg: eval: date chosen from message: Wed Sep 28 21:38:42 2005
[28988] dbg: spf: spf_whitelist_from: could not find useable envelope sender
[28988] dbg: rules: running body-text per-line regexp tests; score so far=0
[28988] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "t"
[28988] dbg: uri: running uri tests; score so far=0
[28988] dbg: rules: ran eval rule __HTML_LENGTH_512 ======> got hit
[28988] dbg: bayes: not scoring message, returning undef
[28988] dbg: bayes: opportunistic call attempt failed, DB not readable
[28988] dbg: rules: ran eval rule __TAG_EXISTS_BODY ======> got hit
[28988] dbg: rules: ran eval rule __HTML_LENGTH_0000_1024 ======> got hit
[28988] dbg: rules: ran eval rule __MIME_HTML ======> got hit
[28988] dbg: rules: ran eval rule HTML_MESSAGE ======> got hit
[28988] dbg: rules: ran eval rule __HTML_LENGTH_384 ======> got hit
[28988] dbg: rules: ran eval rule __TAG_EXISTS_HTML ======> got hit
[28988] dbg: eval: text words: 1, html words: 1
[28988] dbg: eval: madiff: left: 0, orig: 1, max-difference: 0.00%
[28988] dbg: rules: ran eval rule __TAG_EXISTS_HEAD ======> got hit
[28988] dbg: rules: ran eval rule __TAG_EXISTS_META ======> got hit

[28988] dbg: rules: running raw-body-text per-line regexp tests; scoreso far=0.001

[28988] dbg: rules: running full-text regexp tests; score so far=0.001
[28988] dbg: info: entering helper-app run mode
[28988] dbg: info: leaving helper-app run mode
[28988] dbg: razor2: part=0 engine=4 contested=0 confidence=0
[28988] dbg: razor2: part=1 engine=4 contested=0 confidence=0
[28988] dbg: razor2: results: spam? 0
[28988] dbg: razor2: results: engine 8, highest cf score: 0
[28988] dbg: razor2: results: engine 4, highest cf score: 0
[28988] dbg: pyzor: pyzor is available: /usr/bin/pyzor
[28988] dbg: info: entering helper-app run mode

[28988] dbg: pyzor: opening pipe: /usr/bin/pyzor --homedir /etc/pyzorcheck < /tmp/.spamassassin28988HHDFz8tmp

[28992] dbg: util: changing real uid from 0 to match effective uid 99
[28992] dbg: util: setuid: ruid=99 euid=99
[28988] dbg: pyzor: [28992] finished:  exit=0x0100
[28988] dbg: pyzor: got response: 66.250.40.33:24441_(200, 'OK')_0_0
[28988] dbg: info: leaving helper-app run mode
[28988] dbg: dcc: dccifd is available: /etc/dcc/dccifd
[28988] dbg: info: entering helper-app run mode

[28988] dbg: dcc: dccifd got response: X-DCC-dcc.uncw.edu-Metrics:spam.channing-bete.com 1201; Body=1 Fuz1=6

[28988] dbg: info: leaving helper-app run mode
[28988] dbg: check: running tests for priority: 500
[28988] dbg: dns: success for 15 of 15 queries
[28988] dbg: rules: running meta tests; score so far=0.001
[28988] dbg: rules: running header regexp tests; score so far=0.001

[28988] dbg: rules: running body-text per-line regexp tests; score sofar=0.001

[28988] dbg: uri: running uri tests; score so far=0.001

[28988] dbg: rules: running raw-body-text per-line regexp tests; scoreso far=0.001

[28988] dbg: rules: running full-text regexp tests; score so far=0.001
[28988] dbg: check: is spam? score=0.001 required=4
[28988] dbg: check: tests=DK_SIGNED,DK_VERIFIED,HTML_MESSAGE

[28988] dbg: check:subtests=__BAT_BOUNDARY,__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HTML_LENGTH_0000_1024,__HTML_LENGTH_384,__HTML_LENGTH_512,__MIME_HTML,__MIME_VERSION,__MOZILLA_MSGID,__MSGID_OK_HOST,__NONEMPTY_BODY,__RCVD_IN_NJABL,__RCVD_IN_SORBS,__SANE_MSGID,__TAG_EXISTS_BODY,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__USER_AGENT[28988] info: spamd: clean message (0.0/4.0) for (unknown):99 in 1.7seconds, 2230 bytes.[28988] info: spamd: result: . 0 - DK_SIGNED,DK_VERIFIED,HTML_MESSAGEscantime=1.7,size=2230,user=(unknown),uid=99,required_score=4.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=50129,mid=<[EMAIL PROTECTED]>,autolearn=disabled

[28988] dbg: config: copying current conf from backup
[28984] dbg: prefork: child 28988: entering state 1
[28984] dbg: prefork: new lowest idle kid: 28988
[28984] dbg: prefork: child reports idle
[28984] info: prefork: child states: II
[28988] dbg: prefork: sysread(7) not ready, wait max 300 secs


----- Original Message -----
*From:* "Daryl C. W. O'Shea" <[EMAIL PROTECTED]>
*Sent:* 09/28/2005 9:34:57 PM -0400
*To:* Ben Lentz <[EMAIL PROTECTED]>
*Cc:* users@spamassassin.apache.org
*Subject:* SPF and Upgrade to SA 3.1

Ben Lentz wrote:
Thanks for the info. I just added "always_trust_envelope_sender 1" tomy local.cf and restarted. I then resent an email from gmail andstill got no SPF. So, that didn't solve my problem.
Am I incorrectly implimenting the standard? Do I need my TXT recordto be located at IN TXT smtp.channing-bete.com -instead of- or -inaddition to- a IN TXT channing-bete.com record?
It would be both.  You'd want something like:

channing-bete.com.      IN      TXT     "v=spf1 mx -all"
smtp.channing-bete.com. IN      TXT     "v=spf1 a -all"
You only need the first one for "regular" return-path based SPFchecks. The second one only applies to HELO based SPF checks.
In any case, although it's good to know, it doesn't apply to whyyou're not seeing SPF results for mail from gmail.com (or likelyanyone else).
My internal setup might be the problem. I'd appreciate arecommendation if you have a moment... You may be able to see, basedon the header I pasted and my DNS information, that my external MX issmtp.channing-bete.com (205.246.7.107), which is NATd by our firewallto 10.1.200.36. Mail is then routed internally tospam.channing-bete.com at 10.1.200.40 based on LDAP information(whether a given user is interested in being filtered through SA ornot). This is where SpamAssassin is running, and where 3.0.4 used tobe able to check records. :)
I don't know why it would work with that setup in the past. Itshouldn't have and didn't for any of my networks.
Therefore, my trusted_networks is 10.1.0.0/16. Correct? Or does itnot make a difference? I've read that the gateway system *has* to bethe one that does the SPF checking... and that's not the case in mysetup. spam.channing-bete.com is one hop in from the external MX.
Setting always_trust_envelope_sender to 1 removes the requirement forSA to be on the first hop -- provided your trust path is set correctly.
Setting trusted_networks correctly is the single most important stepin setting up SpamAssassin. You need to include BOTH your external IPand your internal IPs to trusted_networks:
trusted_networks    10.1/16        # internal hosts
trusted_networks    205.246.7.107    # public mx ip
Lots of background, if you're interested, about trusted_networks andwhat other things it affects here:
http://wiki.apache.org/spamassassin/TrustPath
Honestly, I'd rather just run the former SA SPF checks on my system,crippled or inaccurate as they were, if they're not going to workwith my configuration.
The way regular SPF checks are done haven't changed from 3.0. Youcould probably use the old module but you'd almost definetely get thesame results you're getting now (along with a few Perl warnings).You'd also miss out on SPF based whitelisting.
The good news though is that 3.1's SPF plugin works just fine -- yourconfig is just a little incorrect. I'm assuming you're missing thesecond trusted_networks line (above)... adding that should fix yourproblem. If it doesn't, running the message in question throughspamassassin manually with some debug output will help.
The output from both of the below would tell you a lot:

spamassassin -Dspf < test.msg
spamassassin -Dreceived-header < test.msg


Daryl

Re: SPF and Upgrade to SA 3.1

Reply via email to