This is killing me here.... dozens of spams this morning getting through (with bayes, RDJ+SARE, razor, dcc). Without the SpamCopURI working, my detection rate plummets.
Any ideas why SpamCopURI would only be querying multi.surbl.org even though all of them are configured in my spamcop_uri.cf? I'm using SA 2.6.4, but with a somewhat old version of perl... other than that, everything is pretty up to date. Tried the latest Net::DNS, but no change. thanks!! johnS -----Original Message----- From: Stewart, John Sent: Tuesday, May 10, 2005 11:33 AM To: 'Jeff Chan'; SpamAssassin Users Subject: RE: SpamCopURI not working, was RE: More Messed Up www URLs Jeff Chan wrote: > Have you tried spamassassin -D < some_message and spamassassin > --lint? SA lints fine... running it in debug mode, it appears to not be checking anything but the multi records. See below. I've grepped through /usr/share/spamassassin and /etc/mail/spamassasin, and the only URI_RBL reference I find in any .cf file is in /etc/mail/spamassasin/spamcop_uri.cf, which is the config file included with SpamCopURI-0.25 (which has rules and scores for 7 different _URI_RBL's). The only one I'm seeing *ever* hit in my logfiles is SPAMCOP_URL_RBL. This is really killing my spam scanning performance...! [...] debug: using "/usr/share/spamassassin" for default rules dir debug: using "/etc/mail/spamassassin" for site rules dir debug: using "/var/amavis/.spamassassin" for user state dir debug: using "/var/amavis/.spamassassin/user_prefs" for user prefs file [...] debug: Razor2 results: spam? 0 highest cf score: 0 debug: running raw-body-text per-line regexp tests; score so far=0 debug: running uri tests; score so far=0 debug: uri tests: Done uriRE debug: checking url: http://www.achat-montre-rolex.net./ debug: querying for achat-montre-rolex.net.multi.surbl.org debug: Query failed for achat-montre-rolex.net.multi.surbl.org debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 2 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b20414) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 4 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b20414) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 32 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b20414) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 64 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b20414) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 16 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b20414) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 8 debug: no match debug: running full-text regexp tests; score so far=0 debug: Razor2 is available [...] I'll also attach the full debug run. It just seems like SA is not testing all the surbl.org servers. johnS
debug: Score set 0 chosen. debug: running in taint mode? yes debug: Running in taint mode, removing unsafe env vars, and resetting PATH debug: PATH included '/bin', keeping. debug: PATH included '/usr/bin', keeping. debug: PATH included '/usr/local/bin', keeping. debug: PATH included '/usr/bin/X11', keeping. debug: PATH included '/usr/X11R6/bin', keeping. debug: Final PATH set to: /bin:/usr/bin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin debug: using "/usr/share/spamassassin" for default rules dir debug: using "/etc/mail/spamassassin" for site rules dir debug: using "/var/amavis/.spamassassin" for user state dir debug: using "/var/amavis/.spamassassin/user_prefs" for user prefs file debug: bayes: 30299 tie-ing to DB file R/O /var/amavis/bayes_toks debug: bayes: 30299 tie-ing to DB file R/O /var/amavis/bayes_seen debug: bayes: found bayes db version 2 debug: Score set 3 chosen. debug: Initialising learner debug: received-header: parsed as [ ip=10.64.16.58 rdns=spaminator.heurikon.com helo= by=c3po.heurikon.com ident= ] debug: received-header: ignoring localhost handover debug: received-header: ignoring localhost handover debug: received-header: parsed as [ ip=10.64.49.2 rdns=frankfurterINT.heurikon.com helo=bratwurst.heurikon.com by=spaminator.heurikon.com ident= ] debug: received-header: parsed as [ ip=128.255.17.47 rdns=server07.icaen.uiowa.edu helo=server07.icaen.uiowa.edu by=bratwurst.heurikon.com ident= ] debug: received-header: parsed as [ ip=128.255.17.51 rdns=server11.icaen.uiowa.edu helo=server11.icaen.uiowa.edu by=server07.icaen.uiowa.edu ident= ] debug: received-header: parsed as [ ip=128.255.17.30 rdns=d-is00.icaen.uiowa.edu helo=d-is00.icaen.uiowa.edu by=server11.icaen.uiowa.edu ident= ] debug: received-header: relay 10.64.16.58 trusted? yes debug: received-header: relay 10.64.49.2 trusted? yes debug: received-header: relay 128.255.17.47 trusted? no debug: received-header: relay 128.255.17.51 trusted? no debug: received-header: relay 128.255.17.30 trusted? no debug: is Net::DNS::Resolver available? yes debug: trying (3) akamai.com... debug: looking up MX for 'akamai.com' debug: MX for 'akamai.com' exists? 1 debug: MX lookup of akamai.com succeeded => Dns available (set dns_available to hardcode) debug: is DNS available? 1 debug: all '*From' addrs: [EMAIL PROTECTED] debug: running header regexp tests; score so far=0 debug: running body-text per-line regexp tests; score so far=0 debug: bayes corpus size: nspam = 295777, nham = 17750 debug: uri tests: Done uriRE debug: tokenize: header tokens for *M = " FC102D41F595D311ACA7009027DE2C842CEE97 c3po heurikon com " debug: tokenize: header tokens for *F = "U*dbfunk D*engineering.uiowa.edu D*uiowa.edu D*edu" debug: tokenize: header tokens for To = "U*johns D*artesyncp.com D*com" debug: tokenize: header tokens for Cc = "U*users D*spamassassin.apache.org D*apache.org D*org" debug: tokenize: header tokens for MIME-Version = "" debug: tokenize: header tokens for *c = "/plain; charset="iso-8859-1"" debug: tokenize: header tokens for *r = " d-is00.icaen.uiowa.edu (d-is00.icaen.uiowa.edu [128.255.17]) by server11.icaen.uiowa.edu (8.13.2/smtp-serv-1.7) (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168); (envelope- <[EMAIL PROTECTED]>) " debug: tokenize: header tokens for *r = " d-is00.icaen.uiowa.edu (d-is00.icaen.uiowa.edu [128.255.17]) by server11.icaen.uiowa.edu (8.13.2/smtp-serv-1.7) (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168); (envelope- <[EMAIL PROTECTED]>) server11.icaen.uiowa.edu (server11.icaen.uiowa.edu [128.255.17]) by server07.icaen.uiowa.edu (8.13.3/8.12.9) ; (envelope- <[EMAIL PROTECTED]>) " debug: bayes token '0.9' => 0.999859617137648 debug: bayes token 'AWL' => 0.999805145508224 debug: bayes token 'N:H*r:sk:serverN' => 0.998229885057471 debug: bayes token 'H*r:sk:server1' => 0.997909502262443 debug: bayes token 'HCc:D*org' => 0.00292626227296213 debug: bayes token 'UD:0' => 0.996940397350993 debug: bayes token 'flagged' => 0.996940397350993 debug: bayes token 'UD:edu' => 0.00351441681541351 debug: bayes token 'Postmaster' => 0.994296296296296 debug: bayes token '0.22' => 0.992426229508197 debug: bayes token 'H*r:version' => 0.0109050506930717 debug: bayes token 'H*r:168' => 0.988731707317073 debug: bayes token 'tagged' => 0.988731707317073 debug: bayes token 'UD:cf' => 0.0131219512195122 debug: bayes token 'N:H*r:TLSvN' => 0.985096774193548 debug: bayes token 'H*r:cipher' => 0.985096774193548 debug: bayes token '2.6.4' => 0.985096774193548 debug: bayes token 'N:H*r:SSLvN' => 0.985096774193548 debug: bayes token 'Funk' => 0.985096774193548 debug: bayes token '319' => 0.0150372231514047 debug: bayes token 'URLs' => 0.017384177294992 debug: bayes token 'Dave' => 0.018259669022877 debug: bayes token 'sk:TO_ADDR' => 0.978 debug: bayes token 'catches' => 0.978 debug: bayes token '1256' => 0.0256190476190476 debug: bayes token 'N:N.N.N' => 0.0261436574169202 debug: bayes token 'Stewart' => 0.0268162585289933 debug: bayes token '999.0' => 0.971779296975069 debug: bayes token 'X-Spam-Status' => 0.971522314424863 debug: bayes token 'wrote' => 0.0285642530311501 debug: bayes token 'tagged_above' => 0.971424735207125 debug: bayes token 'N:H*M:sk:FCNNNDN' => 0.969501753518303 debug: bayes token 'H*M:sk:FC102D4' => 0.969501753518303 debug: bayes token 'H*F:D*edu' => 0.0358141290917653 debug: bayes token 'FAX' => 0.0362386535360371 debug: bayes token 'sk:www.ach' => 0.958 debug: bayes token 'UD:h' => 0.0480326168497531 debug: bayes token 'H*r:8.13.3' => 0.0489090909090909 debug: bayes token 'N:BAYES_NN' => 0.941105711767485 debug: bayes token 'N:NNN-NNNN' => 0.0695835477617935 debug: bayes token 'H*c:plain' => 0.079070037303936 debug: bayes token 'sk:enginee' => 0.0795370894615744 debug: bayes token 'H*r:8.13.2' => 0.0795370894615744 debug: bayes token 'newer' => 0.0843124390751361 debug: bayes token 'John' => 0.0909747431124613 debug: bayes token 'Engineering' => 0.0933308339700968 debug: bayes token 'Iowa' => 0.0958969995790821 debug: bayes token 'references' => 0.0960434490305194 debug: bayes token 'May' => 0.0979294193839739 debug: bayes token 'hitting' => 0.103469761861657 debug: bayes token 'Fri' => 0.120092665346632 debug: bayes token 'Better' => 0.874265322878964 debug: bayes token 'wasn't' => 0.150723595468282 debug: bayes: score = 0.500002968877464 debug: bayes: 30299 untie-ing debug: bayes: 30299 untie-ing db_toks debug: bayes: 30299 untie-ing db_seen debug: Razor2 is available : 17 items read from /var/amavis/.razor/server.thrill.cloudmark.com.conf May 10 11:29:55.643776 check[30299]: [ 5] read_file: 14 items read from /var/amavis/.razor/server.anxiety.cloudmark.com.conf May 10 11:29:55.645929 check[30299]: [ 5] read_file: 14 items read from /var/amavis/.razor/server.anxiety.cloudmark.com.conf May 10 11:29:55.648410 check[30299]: [ 5] read_file: 16 items read from /var/amavis/.razor/server.tension.cloudmark.com.conf May 10 11:29:55.650795 check[30299]: [ 5] read_file: 16 items read from /var/amavis/.razor/server.tension.cloudmark.com.conf May 10 11:29:55.652898 check[30299]: [ 5] read_file: 13 items read from /var/amavis/.razor/server.solace.cloudmark.com.conf May 10 11:29:55.654923 check[30299]: [ 5] read_file: 13 items read from /var/amavis/.razor/server.solace.cloudmark.com.conf May 10 11:29:55.657461 check[30299]: [ 5] read_file: 17 items read from /var/amavis/.razor/server.wonder.cloudmark.com.conf May 10 11:29:55.660041 check[30299]: [ 5] read_file: 17 items read from /var/amavis/.razor/server.wonder.cloudmark.com.conf May 10 11:29:55.662462 check[30299]: [ 5] read_file: 16 items read from /var/amavis/.razor/server.robust.cloudmark.com.conf May 10 11:29:55.664811 check[30299]: [ 5] read_file: 16 items read from /var/amavis/.razor/server.robust.cloudmark.com.conf May 10 11:29:55.665568 check[30299]: [ 5] 151062 seconds before closest server discovery May 10 11:29:55.666196 check[30299]: [ 6] shock.cloudmark.com is a Catalogue Server srl 5081; computed min_cf=6, Server se: C8 May 10 11:29:55.666764 check[30299]: [ 8] Computed supported_engines: 4 May 10 11:29:55.667236 check[30299]: [ 8] Using next closest server shock.cloudmark.com:2703, cached info srl 5081 May 10 11:29:55.667696 check[30299]: [ 8] mail 1 Subject: RE: More Messed Up www URLs May 10 11:29:55.672595 check[30299]: [ 6] preproc: mail 1.0 went from 1359 bytes to 1275 May 10 11:29:55.673076 check[30299]: [ 6] computing sigs for mail 1.0, len 1275 May 10 11:29:55.678487 check[30299]: [ 6] skipping whitelist file (empty?): /var/amavis/.razor/razor-whitelist May 10 11:29:55.678907 check[30299]: [ 5] Connecting to shock.cloudmark.com ... May 10 11:29:55.922498 check[30299]: [ 8] Connection established May 10 11:29:55.922945 check[30299]: [ 4] shock.cloudmark.com >> 36 server greeting: sn=C&srl=5081&a=l&a=cg&ep4=7542-10 May 10 11:29:55.924265 check[30299]: [ 4] shock.cloudmark.com << 25 May 10 11:29:55.924519 check[30299]: [ 6] cn=razor-agents&cv=2.36 May 10 11:29:55.925245 check[30299]: [ 6] shock.cloudmark.com is a Catalogue Server srl 5081; computed min_cf=6, Server se: C8 May 10 11:29:55.925810 check[30299]: [ 8] Computed supported_engines: 4 May 10 11:29:55.926285 check[30299]: [ 8] mail 1.0 e4 sig: UOAUTCd0_6hdon_P9_baNi_twGoA May 10 11:29:55.926834 check[30299]: [ 8] preparing 1 queries May 10 11:29:55.927438 check[30299]: [ 8] sending 1 batches May 10 11:29:55.927943 check[30299]: [ 4] shock.cloudmark.com << 52 May 10 11:29:55.928188 check[30299]: [ 6] a=c&e=4&ep4=7542-10&s=UOAUTCd0_6hdon_P9_baNi_twGoA May 10 11:29:56.306511 check[30299]: [ 4] shock.cloudmark.com >> 5 May 10 11:29:56.306804 check[30299]: [ 6] response to sent.2 p=0 May 10 11:29:56.308080 check[30299]: [ 6] mail 1.0 e=4 sig=UOAUTCd0_6hdon_P9_baNi_twGoA: sig not found. May 10 11:29:56.308494 check[30299]: [ 7] method 4: mail 1.0: no-contention part, spam=0 May 10 11:29:56.308760 check[30299]: [ 7] method 4: mail 1: all non-contention parts not spam, mail not spam May 10 11:29:56.309019 check[30299]: [ 3] mail 1 is not known spam. May 10 11:29:56.309354 check[30299]: [ 5] disconnecting from server shock.cloudmark.com May 10 11:29:56.309871 check[30299]: [ 4] shock.cloudmark.com << 5 May 10 11:29:56.310112 check[30299]: [ 6] a=q debug: Razor2 results: spam? 0 highest cf score: 0 debug: running raw-body-text per-line regexp tests; score so far=0 debug: running uri tests; score so far=0 debug: uri tests: Done uriRE debug: checking url: http://www.achat-montre-rolex.net./ debug: querying for achat-montre-rolex.net.multi.surbl.org debug: Query failed for achat-montre-rolex.net.multi.surbl.org debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 2 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b1f1e4) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 4 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b1f1e4) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 32 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b1f1e4) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 64 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b1f1e4) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 16 debug: no match debug: checking url: http://www.achat-montre-rolex.net./ debug: returning cached data : achat-montre-rolex.net.multi.surbl.org -> ARRAY(0x9b1f1e4) debug: Receieved match prefix: 127.0.0 debug: Receieved mask: 8 debug: no match debug: running full-text regexp tests; score so far=0 debug: Razor2 is available debug: DCCifd is not available: no r/w dccifd socket found. debug: Current PATH is: /bin:/usr/bin:/usr/local/bin:/usr/bin/X11:/usr/X11R6/bin debug: executable for dccproc was found at /usr/local/bin/dccproc debug: DCC is available: /usr/local/bin/dccproc debug: entering helper-app run mode debug: DCC: got response: X-DCC-NIET-Metrics: spaminator.heurikon.com 1080; Body=1 Fuz1=93 Fuz2=93 debug: leaving helper-app run mode debug: Pyzor is not available: pyzor not found debug: all '*To' addrs: [EMAIL PROTECTED] [EMAIL PROTECTED] users@spamassassin.apache.org debug: forged-HELO: from=uiowa.edu helo=uiowa.edu by=heurikon.com debug: forged-HELO: from=uiowa.edu helo=uiowa.edu by=uiowa.edu debug: forged-HELO: from=uiowa.edu helo=uiowa.edu by=uiowa.edu debug: DNS MX records found: 2 debug: RBL: success for 19 of 22 queries debug: RBL: timeout for rfci after 7 seconds debug: RBL: timeout for rfci after 7 seconds debug: RBL: timeout for rfci after 7 seconds debug: running meta tests; score so far=0 debug: is spam? score=0 required=5 tests= Received: from spaminator.heurikon.com ([10.64.16.58]) by c3po.heurikon.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id JW6KBGQY; Fri, 6 May 2005 17:09:14 -0500 Received: by spaminator.heurikon.com (Postfix) id 6AC8218331; Fri, 6 May 2005 17:09:14 -0500 (CDT) Received: from localhost (localhost.heurikon.com [127.0.0.1]) by spaminator.heurikon.com (Postfix) with ESMTP id F171B18327 for <[EMAIL PROTECTED]>; Fri, 6 May 2005 17:09:13 -0500 (CDT) Received: from spaminator.heurikon.com ([127.0.0.1]) by localhost (spaminator.heurikon.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 26798-10 for <[EMAIL PROTECTED]>; Fri, 6 May 2005 17:09:08 -0500 (CDT) Received: from bratwurst.heurikon.com (frankfurterINT.heurikon.com [10.64.49.2]) by spaminator.heurikon.com (Postfix) with ESMTP id E461E18336 for <[EMAIL PROTECTED]>; Fri, 6 May 2005 17:08:47 -0500 (CDT) Received: by bratwurst.heurikon.com (Postfix) id 843D815123; Fri, 6 May 2005 17:08:46 -0500 (CDT) Received: from server07.icaen.uiowa.edu (server07.icaen.uiowa.edu [128.255.17.47]) by bratwurst.heurikon.com (Postfix) with ESMTP id 769B415072 for <[EMAIL PROTECTED]>; Fri, 6 May 2005 17:08:45 -0500 (CDT) Received: from server11.icaen.uiowa.edu (server11.icaen.uiowa.edu [128.255.17.51]) by server07.icaen.uiowa.edu (8.13.3/8.12.9) with ESMTP id j46M8ihN023640; (envelope-from <[EMAIL PROTECTED]>) Fri, 6 May 2005 17:08:44 -0500 (CDT) Received: from d-is00.icaen.uiowa.edu (d-is00.icaen.uiowa.edu [128.255.17.30]) by server11.icaen.uiowa.edu (8.13.2/smtp-serv-1.7) with ESMTP id j46M8iwe012506 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168); (envelope-from <[EMAIL PROTECTED]>) Fri, 6 May 2005 17:08:44 -0500 (CDT) Message-ID: <[EMAIL PROTECTED]> From: David B Funk <[EMAIL PROTECTED]> To: "Stewart, John" <[EMAIL PROTECTED]> Cc: users@spamassassin.apache.org Subject: RE: More Messed Up www URLs Date: Fri, 6 May 2005 17:08:44 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Spam-Relays: [ ip=128.255.17.47 rdns=server07.icaen.uiowa.edu helo=server07.icaen.uiowa.edu by=bratwurst.heurikon.com ident= ] [ ip=128.255.17.51 rdns=server11.icaen.uiowa.edu helo=server11.icaen.uiowa.edu by=server07.icaen.uiowa.edu ident= ] [ ip=128.255.17.30 rdns=d-is00.icaen.uiowa.edu helo=d-is00.icaen.uiowa.edu by=server11.icaen.uiowa.edu ident= ] X-Spam-Checker-Version: SpamAssassin 2.64 (2004-01-11) on spaminator.heurikon.com X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.64 X-Spam-Level: On Fri, 6 May 2005, Stewart, John wrote: > > > > I'm starting to see references in messages that look like this: > > > > > > www.achat-montre-rolex.net./ > > > > > Upgrade to SA-2.6.4+SpamCopURI, catches those just fine. ;) > > > > I'm running 2.6.4 with SpamCopURI - is this being flagged on your install as > being in the URI-BL? This email wasn't tagged here as such... all I get is: > > X-Spam-Status: No, hits=-0.9 tagged_above=-999.0 required=5.0 tests=AWL, > BAYES_40, TO_ADDRESS_EQ_REAL Yes, all the discussions on this list that contain the above text are being flagged by my SA as hitting the OB SURBL list. That particular host/URL is only registered in the OB list, do you have a check against: check_spamcop_uri_rbl('multi.surbl.org','127.0.0.0+16') in your spamcop_uri.cf (or what ever ".cf" that you've got your check_spamcop_uri_rbl rules)? Also which version of SpamCopURI are you running? (need 0.22 or newer). -- Dave Funk University of Iowa <dbfunk (at) engineering.uiowa.edu> College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527 #include <std_disclaimer.h> Better is not better, 'standard' is better. B{
