Re: uribl result not triggering meta rule

[Wolfgang Breyha]

On 02/04/2021 13:46, Wolfgang Breyha wrote:

Hi!

It seems that 3.4.5 changed the behavior of URIBL lookups in a quite bad 
way compared to 3.4.4.


Just as a pointer:
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7897

Greetings,
Wolfgang

Re: URIBL randomly not triggered for the same message

[Benny Pedersen]

On 2016-07-26 11:39, Reindl Harald wrote:


sadly it don't work as expected
https://bugzilla.redhat.com/show_bug.cgi?id=1360222


add forward-first: yes to forward zone

without you are qquery stale data in unbound

no i do not use bind9 now :=)

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 17:40 schrieb Reindl Harald:

Am 06.07.2016 um 17:35 schrieb John Hardin:

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?


sounds good and at leat i don't get any error by using
unbound-1.5.8-2.fc23.x86_64 and the follwoing settings

cache-min-ttl: 600
cache-max-ttl: 43200
cache-max-negative-ttl: 100

when it works as expected it should lead in not so often expire heavily
used crap domains without take too long for realize new listings and at
least makes the problem nit so big as now


sadly it don't work as expected
https://bugzilla.redhat.com/show_bug.cgi?id=1360222



signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 17:35 schrieb John Hardin:

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?


sounds good and at leat i don't get any error by using 
unbound-1.5.8-2.fc23.x86_64 and the follwoing settings


cache-min-ttl: 600
cache-max-ttl: 43200
cache-max-negative-ttl: 100

when it works as expected it should lead in not so often expire heavily 
used crap domains without take too long for realize new listings and at 
least makes the problem nit so big as now


thanks god then normal DNSBL/DNSWL are not affected becaus ethey are 
used also in prostscreen for weighting and so at the moment SA is using 
them the cache is always hot





signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Reindl Harald wrote:




Am 06.07.2016 um 14:36 schrieb RW:

 On Tue, 5 Jul 2016 14:01:17 +0200
 Reindl Harald wrote:

>  since there is a local unbound-cache with
> 
>cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i observe is 
that after TTL is reached and unbound needs to query again the at least first 
question leads to a negativeresult in spamassassin while the next cache hit 
correctly has URIBL_BLACK again


Does that cache-min-ttl also affect NXDOMAIN? Is it possible to configure 
different TTL for NXDOMAIN (relatively low) and positive results 
(relatively high)?


If not, you might want to file a bug with unbound to ask them to make that 
possible.




--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[Paul Stead]

On 06/07/16 16:16, John Hardin wrote:

Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
configure different TTL for NXDOMAIN (relatively low) and positive
results (relatively high)?


For this cache-max-negative-ttl exists :)

Paul
--
Paul Stead
Systems Engineer
Zen Internet

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 14:36 schrieb RW:

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:


since there is a local unbound-cache with

  cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i 
observe is that after TTL is reached and unbound needs to query again 
the at least first question leads to a negativeresult in spamassassin 
while the next cache hit correctly has URIBL_BLACK again


so at the moment there is a tradeoff between get new domains fast enough 
and don't miss already known hits *and* that also affects SPF and so 
whitelist_auth in a bad way



You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.




signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[RW]

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:

> since there is a local unbound-cache with
> 
>   cache-min-ttl: 300

You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.

Re: URIBL randomly not triggered (and SPF too)

[Reindl Harald]

see also https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335

BTW: the bugtracker has also a major bug - click on "My Bugs" leads to 
the URL below listing a ton of bugreports back to the year 2011 and 
pretends they are reported by me


https://bz.apache.org/SpamAssassin/buglist.cgi?bug_status=UNCONFIRMED_status=NEW_status=ASSIGNED_status=REOPENED_to1=1=1=exact&%20%20%20%20%20%20%20%20%20email1=h.reindl%40thelounge.net=bug_status=notequals=UNCONFIRMED=reporter=equals=h.reindl%40thelounge.net

Am 05.07.2016 um 14:10 schrieb Reindl Harald:

Am 05.07.2016 um 14:01 schrieb Reindl Harald:

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in
the ABUSE SURBL blocklist

50% of all tries against spamd it does NOT hit while the scantime for
the whole message is arounnd 3 seconds - since there is a local
unbound-cache with

 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe
the same behavior randomly with URIBL_LOCAL where the unbound dns cache
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053

that smells why ever very unrelieable and frankly i observed similar
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same
message and one get USER_IN_SPF_WHITELIST while the other goes through
all tests


that below too MUST NOT happen because one triggers
USER_IN_SPF_WHITELIST and the other don't have any SPF test and given
that there is a python-policyd-spf waiting 20 seconds for the response
in 'smtpd_recipient_restrictions' long before the contentfilters the
dns-results are cached

Jul  4 11:34:51 mail-gw postfix/smtpd[13648]: 3rjhgb71LVzB47:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:34:52 mail-gw spamd[12535]: spamd: processing message
<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:34:56 mail-gw spamd[12535]: spamd: result: . -4 -
BAYES_00,CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,HTML_MESSAGE,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD
scantime=4.2,size=18438,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>,bayes=0.00,autolearn=disabled,shortcircuit=no


Jul  4 11:57:01 mail-gw postfix/smtpd[14837]: 3rjj993Bk8zB7P:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: processing message
<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: result: . -100 -
CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,CUST_SHORTCIRCUIT,RCVD_IN_MSPIKE_H2,SHORTCIRCUIT,USER_IN_SPF_WHITELIST
scantime=0.1,size=15685,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>,autolearn=disabled,shortcircuit=spam




signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered (and SPF too)

[Reindl Harald]

Am 05.07.2016 um 14:01 schrieb Reindl Harald:

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in
the ABUSE SURBL blocklist

50% of all tries against spamd it does NOT hit while the scantime for
the whole message is arounnd 3 seconds - since there is a local
unbound-cache with

 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe
the same behavior randomly with URIBL_LOCAL where the unbound dns cache
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053

that smells why ever very unrelieable and frankly i observed similar
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same
message and one get USER_IN_SPF_WHITELIST while the other goes through
all tests


that below too MUST NOT happen because one triggers 
USER_IN_SPF_WHITELIST and the other don't have any SPF test and given 
that there is a python-policyd-spf waiting 20 seconds for the response 
in 'smtpd_recipient_restrictions' long before the contentfilters the 
dns-results are cached


Jul  4 11:34:51 mail-gw postfix/smtpd[13648]: 3rjhgb71LVzB47: 
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:34:52 mail-gw spamd[12535]: spamd: processing message 
<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail> 
for sa-milt:189
Jul  4 11:34:56 mail-gw spamd[12535]: spamd: result: . -4 - 
BAYES_00,CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,HTML_MESSAGE,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD 
scantime=4.2,size=18438,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>,bayes=0.00,autolearn=disabled,shortcircuit=no


Jul  4 11:57:01 mail-gw postfix/smtpd[14837]: 3rjj993Bk8zB7P: 
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: processing message 
<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail> 
for sa-milt:189
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: result: . -100 - 
CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,CUST_SHORTCIRCUIT,RCVD_IN_MSPIKE_H2,SHORTCIRCUIT,USER_IN_SPF_WHITELIST 
scantime=0.1,size=15685,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>,autolearn=disabled,shortcircuit=spam





signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Alex]

Hi,

>> Is there any reason to not use the bl.score.sendrescore.com with
>> postscreen? I don't understand the distinction
>
> why?
>
> postscreen is supposed to be configured with sensible scoring to reject most
> spam without false positives long before it reachs smtpd or even expesnive
> contentfilters
>
> hence the scoring and any sensible setup would use postscreen combined with
> several whitelists
>
> that way your contentfilter has only to deal with the remaining 10% of junk
> and when you optimize postscreen to use a honeypot-MX (backup mx on a second
> IP with a postscreen whitelist_veto) and enforce pre-greet tests with a
> larger wait time there is not much for SpamAssasin to deal with

No, no, no. That's not at all what I mean. I know what the purpose and
benefit of postscreen is.

My issue relates to why is score.senderscore.com used with postscreen,
and not bl.score.senderscore.com as it is with SA?

Perhaps it should be as well?

The postscreen weights for score.senderscore.com are such that they
are relative to the threshold, so a reputation of say, 70 would
receive a higher score than a reputation of say, 90. In fact, 90
removes points.

And why is only bl.score.senderscore.com used with SA, and not the
reputation system?

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 03.03.2016 um 02:44 schrieb Alex:

Is there any reason to not use the bl.score.sendrescore.com with
postscreen? I don't understand the distinction


why?

postscreen is supposed to be configured with sensible scoring to reject 
most spam without false positives long before it reachs smtpd or even 
expesnive contentfilters


hence the scoring and any sensible setup would use postscreen combined 
with several whitelists


that way your contentfilter has only to deal with the remaining 10% of 
junk and when you optimize postscreen to use a honeypot-MX (backup mx on 
a second IP with a postscreen whitelist_veto) and enforce pre-greet 
tests with a larger wait time there is not much for SpamAssasin to deal with




signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Alex]

Hi,

Some time ago, David Jones wrote:
> In a related note, I have found that using the senderscore.org score combined
> with postscreen's weighting is very effective in quickly catching new 
> spammers.
>
> postscreen_dnsbl_sites =
>   score.senderscore.com=127.0.4.[60..69]*2
>   score.senderscore.com=127.0.4.[50..59]*4
>   score.senderscore.com=127.0.4.[30..49]*6
>   score.senderscore.com=127.0.4.[0..29]*8
>   score.senderscore.com=127.0.4.[90..100]*-6
>   score.senderscore.com=127.0.4.[80..89]*-4
>   score.senderscore.com=127.0.4.[70..79]*-2

This has been quite effective, but there have also been some
false-positives which I've had to whitelist. I've lowered the 0-29
result a bit so as to not make it a poison pill in my case.

I also probably should have asked at the time what your
postscreen_dnsbl_threshold is? Mine is 8.

Can someone explain how this differs from the bl.score.senderscore.com
that's used in the RCVD_IN_RP_RNBL rule?

Is there any reason to not use the bl.score.sendrescore.com with
postscreen? I don't understand the distinction.

Does anyone know where the return result codes are defined? I've
looked all over the senderscore website and can't find them.

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Noel Butler]

On 16/02/2016 01:08, Shawn Bakhtiar wrote:



There are A LOT more people out there, far greater than just the
Googles and Yahoos of the world, and to block IP addresses/subnets
without an automated system using definable metric (that usually is
enterprise specific), invariably IT will be inundated with complaints
about users not receiving legitimate vendor emails.




Thats the entire point though, as it has been for over 20 years.

admins shrug off badguy-complaints, badguy complaints go to rbl, rbl 
blocks, rbl gets notified badguy uses more resources,  rbl blocks wider 
range due to other IPs used


It's much much harder for admins to shrug off their own customers 
complaints, so admin gets off lazy useless arse and sorts out the badguy 
like should have in first place, rbl then removes blocks... life goes 
on..


--
If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[Shawn Bakhtiar]

I use to spend a lot of time blocking hosts and subnets, using IP tables, of 
malicious providers who would let any tom, dick, and Harry (no pun intended) to 
host spam hosts/relays on their servers. What I ended up doing is also blocking 
a lot SMB vendors from sending legitimate emails to users because most SMBs 
outsource their services without really comprehending the consequences of the 
provider they choose, this is especially true for low tech industries such as 
toll and process manufacturing companies, and frankly led to a management 
nightmare.

There are A LOT more people out there, far greater than just the Googles and 
Yahoos of the world, and to block IP addresses/subnets without an automated 
system using definable metric (that usually is enterprise specific), invariably 
IT will be inundated with complaints about users not receiving legitimate 
vendor emails.

It is much more effective to use existing RBLs, and supplementing it with your 
own honeypot RBL that uses metrics developed in house that can react to what 
your organization will consider the critical mass of spam it can take. That, 
along with the proper training of SA, is perhaps the best defense you can have. 
Using metric like last seen, total count, and frequency seem to provide the 
best metrics for me, my private RBL (based on honeypot addresses) can react 
faster than the big guys, on both ends of the equation (to block and to 
release), It's not that Google doesn't sometimes land on my RBL, it's that it 
also drops off fast as they remedy the issue, and the time outs are reached and 
they drop off my list.



> On Feb 14, 2016, at 10:19 PM, Noel Butler  wrote:
> 
> On 15/02/2016 09:02, Reindl Harald wrote:
>> Am 14.02.2016 um 23:34 schrieb Noel Butler:
>>> On 14/02/2016 01:46, Alex wrote:
 rejecting outright at the SMTP level for IPs reaching my honeypots
 could be dangerous if not checked.
>>> how so? if your honey pots use specific non human used (ever) addresses,
>>> then there should never ever be a genuine mail destined for it.
>>> I dont care who the connector is, be it foobar.com or gmail.com if they
>>> relay it, they are listed, its where spamhaus and I always disagreed,
>>> because what they are doing is sending a clear message to spammers to
>>> simply "use gmail" to avoid being listed in spamhaus.
>>> You are never too big to be stuffed into a dnsbl, there are a number of
>>> well known bl's that have been around for over ten years that also take
>>> that approach.
>> you missed to say that you are the type RBL operator which lists whole
>> subnets (in not only personal RBL's) because you don't like specific
>> people on mailing-lists
> 
> 
> Ohh, so you wanna bring this up again in public do you, fine by me... lets 
> have some history though shall we Harry...
> 
> Most DNSBL's blacklist spam *and* abusive hosts, there is no question about 
> you spamming, I know you don't and would never do that, but you are/were a 
> very very aggressively abusive person - this is supported by all those 
> mailing lists bannings/moderations you've copped over recent years which we 
> need both hands to count, the listing I placed on you was not just because of 
> the abuse and blackmailing you leveled at me, but number of complaints we 
> received also.
> 
> Further more, most people who've had interactions with you over the past 
> couple of years, espeically those that you've disagreed with also know how 
> you used to act, and occasionally still come close to, because you think you 
> are always right and anyone who disagrees with you is the anti christ or 
> something.
> 
> Ordinarily this does just warrant a /32 listing, however as a system 
> administrator with access to at least a /24, and evidence of your mailing 
> list ghost accounts, including at least one I recall from another IP in that 
> /24 a while back, yes, I took the step to block your /24.
> 
> 
>> also you don't realize that this don't stop any single mail from a
>> list sent by that person but just harms other domains using the SMTP
>> server
> 
> I realise a lot more than you think, as I've told you, and told you, and told 
> you, its up to lists what DNSBL's if any they use, but you are known to, on 
> the lists youve been moderated on, send abusive messages to recipients 
> directly since you can't via the lists
> so it does have a catching effect of those who use it.
> 
>> so *you* are hardly in the position for education about RBL's since
>> you don't care about any collateral damage but only your ego
> 
> You are entitled to your opinion, I care about valid collateral damage, if 
> you abuse an employers resources and your employers customers are caught up 
> on it, your employer, if they care, would take appropriate action, it is no 
> different than blocking a domain for spamming, forcing the host to clean up 
> its act and get rid of its spamming clients, of course at no time did I wish 
> to see your employment 

Re: URIBL/DNSBL from a database

[Noel Butler]

On 15/02/2016 09:02, Reindl Harald wrote:

Am 14.02.2016 um 23:34 schrieb Noel Butler:

On 14/02/2016 01:46, Alex wrote:


rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.


how so? if your honey pots use specific non human used (ever) 
addresses,

then there should never ever be a genuine mail destined for it.

I dont care who the connector is, be it foobar.com or gmail.com if 
they

relay it, they are listed, its where spamhaus and I always disagreed,
because what they are doing is sending a clear message to spammers to
simply "use gmail" to avoid being listed in spamhaus.

You are never too big to be stuffed into a dnsbl, there are a number 
of
well known bl's that have been around for over ten years that also 
take

that approach.


you missed to say that you are the type RBL operator which lists whole
subnets (in not only personal RBL's) because you don't like specific
people on mailing-lists




Ohh, so you wanna bring this up again in public do you, fine by me... 
lets have some history though shall we Harry...


Most DNSBL's blacklist spam *and* abusive hosts, there is no question 
about you spamming, I know you don't and would never do that, but you 
are/were a very very aggressively abusive person - this is supported by 
all those mailing lists bannings/moderations you've copped over recent 
years which we need both hands to count, the listing I placed on you was 
not just because of the abuse and blackmailing you leveled at me, but 
number of complaints we received also.


Further more, most people who've had interactions with you over the past 
couple of years, espeically those that you've disagreed with also know 
how you used to act, and occasionally still come close to, because you 
think you are always right and anyone who disagrees with you is the anti 
christ or something.


Ordinarily this does just warrant a /32 listing, however as a system 
administrator with access to at least a /24, and evidence of your 
mailing list ghost accounts, including at least one I recall from 
another IP in that /24 a while back, yes, I took the step to block your 
/24.




also you don't realize that this don't stop any single mail from a
list sent by that person but just harms other domains using the SMTP
server



I realise a lot more than you think, as I've told you, and told you, and 
told you, its up to lists what DNSBL's if any they use, but you are 
known to, on the lists youve been moderated on, send abusive messages to 
recipients directly since you can't via the lists

so it does have a catching effect of those who use it.


so *you* are hardly in the position for education about RBL's since
you don't care about any collateral damage but only your ego


You are entitled to your opinion, I care about valid collateral damage, 
if you abuse an employers resources and your employers customers are 
caught up on it, your employer, if they care, would take appropriate 
action, it is no different than blocking a domain for spamming, forcing 
the host to clean up its act and get rid of its spamming clients, of 
course at no time did I wish to see your employment terminated, just 
actions reigned in, resulting in cleaner transmissions, allowing for 
removal of blocking, just like networks that clean up spam.


I have seen you have remarkable behaved yourself in past 6 months 
compared to how you used to carry on, your still no saint, but no one 
including me is either.


This list is also off topic and I apologise to Gunther and co for 
replying to it on list, but some things needed to be said. No doubt 
Harry will rant and rave and carry on trollbaiting me, but I will try 
with-hold any further responses since, we are, well and truly OT.


Have a nice day.

--


If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 14.02.2016 um 23:34 schrieb Noel Butler:

On 14/02/2016 01:46, Alex wrote:


rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.


how so? if your honey pots use specific non human used (ever) addresses,
then there should never ever be a genuine mail destined for it.

I dont care who the connector is, be it foobar.com or gmail.com if they
relay it, they are listed, its where spamhaus and I always disagreed,
because what they are doing is sending a clear message to spammers to
simply "use gmail" to avoid being listed in spamhaus.

You are never too big to be stuffed into a dnsbl, there are a number of
well known bl's that have been around for over ten years that also take
that approach.


you missed to say that you are the type RBL operator which lists whole 
subnets (in not only personal RBL's) because you don't like specific 
people on mailing-lists


also you don't realize that this don't stop any single mail from a list 
sent by that person but just harms other domains using the SMTP server


so *you* are hardly in the position for education about RBL's since you 
don't care about any collateral damage but only your ego





signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Noel Butler]

On 14/02/2016 01:46, Alex wrote:




rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.




how so? if your honey pots use specific non human used (ever) addresses, 
then there should never ever be a genuine mail destined for it.


I dont care who the connector is, be it foobar.com or gmail.com if they 
relay it, they are listed, its where spamhaus and I always disagreed, 
because what they are doing is sending a clear message to spammers to 
simply "use gmail" to avoid being listed in spamhaus.


You are never too big to be stuffed into a dnsbl, there are a number of 
well known bl's that have been around for over ten years that also take 
that approach.



--
If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[John Hardin]

On Sun, 14 Feb 2016, Allen Chen wrote:


On 2/12/2016 8:48 AM, Axb wrote:

 On 02/12/2016 02:39 PM, Alex wrote:
>  For some time now I've been cycling URLs and IPs through  a mariadb
>  database gathered from incoming mail on a honeypot I've created.
>  Surprising how many are received ahead of spamhaus/barracuda.
> 
>  I'm looking for ideas on how to now make this information available to

>  spamassassin on my production system. I'd like to somehow export the
>  IPs, any URLs in the body, and email addresses to spamassassin.
> 
>  Is it possible for spamassassin to query a database directly?


Did you try iptables to block/allow IPs?


If you're getting that much abuse from specific IPs and you're sure that 
it's all spam, then set up a TCP tarpit.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Ignorance is no excuse for a law.
---
 8 days until George Washington's 284th Birthday

Re: URIBL/DNSBL from a database

[Allen Chen]

On 2/12/2016 8:48 AM, Axb wrote:

On 02/12/2016 02:39 PM, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?

Did you try iptables to block/allow IPs?



You'd need a custom plugin query the DB directly.



I'm familiar with how to create a uridnsbl, but is DNS the best
approach here?

DNS is cheap/reliable and simple to deploy / load balance.


The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


rbldnsd can check and load fresh data instantly within seconds.
If your dataset is not HUGE (loading 100MB zones is slow) rbldnspy 
will take inmemory updates so instant listings...

https://github.com/gryphius/rbldnspy






--
Allen Chen
Network Administrator
IT

Harbourfront Centre

235 Queens Quay West, Toronto, ON
M5J 2G8, Canada | harbourfrontcentre.com 
Office: +1 416 973 7973
Cell: +1 416 556 2493

Re: URIBL/DNSBL from a database

[David Jones]

>> DNS is very effective to block at the MTA level.  I setup my own private
>> RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
>> rbldnsd formatted zone file and setup your private RBL zone (doesn't
>> have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
>> will detect changes to it's zone files and reload them automatically to
>> keep current.

>Do you have some kind of whitelist that includes gmail, yahoo, etc?

Yes. My database query excludes FREEMAIL hits.   I also use/parse SPF
records of many of the large FREEMAIL domains to allow these in before
RBL checks.  You also have to whitelist many of these from greylisting too
and let SA score them.

>I'm not looking to compete with spamhaus, just compliment it, but
>rejecting outright at the SMTP level for IPs reaching my honeypots
>could be dangerous if not checked.

I don't have any honeypots so I can't speak from experience but I
would think you would need to filter these differently -- much more
relaxed than real user domains and mailboxes.   If your honeypot
addresses are on a different domain, send them through a different
MTA config that doesn't have all of these RBL checks.

>I've now got rbldnsd implemented. I've also known for a while it's
>faster/better than bind, but bind has always been in place.

>I have rbldnsd running on port 530, alongside bind on 53. How do I
>specify a urirhsbl in spamassassin to query the DNS server running on
>530 instead of 53?

You setup BIND to forward that zone of your own RBL to localhost:530.
http://www.surbl.org/setup-local-rbl-mirror  (toward the bottom)
rbldnsd only has to be listening on 127.0.0.1:530

>> In a related note, I have found that using the senderscore.org score combined
>> with postscreen's weighting is very effective in quickly catching new 
>> spammers.
>>
>> postscreen_dnsbl_sites =
>>   score.senderscore.com=127.0.4.[60..69]*2
>>   score.senderscore.com=127.0.4.[50..59]*4
>>   score.senderscore.com=127.0.4.[30..49]*6
>>   score.senderscore.com=127.0.4.[0..29]*8
>>   score.senderscore.com=127.0.4.[90..100]*-6
>>   score.senderscore.com=127.0.4.[80..89]*-4
>>   score.senderscore.com=127.0.4.[70..79]*-2
>>
>> You should monitor your own outbound IPs for their sender score.  If your
>> IP goes below 90, it's a good indication that you have been sending spam
>> and that your users are going to start experiencing delivery issues to the
>> Internet.

>Do you use this on inbound mail as well?

Yes.  Definitely use this primarily on inbound email.  I also use
some RBLs on outbound email to help detect compromised
accounts but make sure you have your internal_networks and
trusted_networks properly so SA will work with external IPs
properly.

>How does it fit with the other postscreen dnsbls? I already have at
>least six various dnsbls with varying weights...

I have more than a dozen in addition to the ones above.  You simply
list as many RBLs as you want with the proper weighting you think
based on their reliability/trustworthiness for your environment.
Negative numbers are used for reliable RBLs that show a good reputation
for the sending mail server IP.  Positive numbers go higher toward
the threshold number (I use 8 like many examples I have seen).  Set
your own private RBL at or slightly above your threshold along with
other trustworthy RBLs like zen.spamhaus.org.  Only use negative
number weighting for those RBLs that you have confirmed to be
good sources for good reputation.

Re: URIBL/DNSBL from a database

[Alex]

Hi,

> DNS is very effective to block at the MTA level.  I setup my own private
> RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
> rbldnsd formatted zone file and setup your private RBL zone (doesn't
> have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
> will detect changes to it's zone files and reload them automatically to
> keep current.

Do you have some kind of whitelist that includes gmail, yahoo, etc?

I'm not looking to compete with spamhaus, just compliment it, but
rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.

I've now got rbldnsd implemented. I've also known for a while it's
faster/better than bind, but bind has always been in place.

I have rbldnsd running on port 530, alongside bind on 53. How do I
specify a urirhsbl in spamassassin to query the DNS server running on
530 instead of 53?

> In a related note, I have found that using the senderscore.org score combined
> with postscreen's weighting is very effective in quickly catching new 
> spammers.
>
> postscreen_dnsbl_sites =
>   score.senderscore.com=127.0.4.[60..69]*2
>   score.senderscore.com=127.0.4.[50..59]*4
>   score.senderscore.com=127.0.4.[30..49]*6
>   score.senderscore.com=127.0.4.[0..29]*8
>   score.senderscore.com=127.0.4.[90..100]*-6
>   score.senderscore.com=127.0.4.[80..89]*-4
>   score.senderscore.com=127.0.4.[70..79]*-2
>
> You should monitor your own outbound IPs for their sender score.  If your
> IP goes below 90, it's a good indication that you have been sending spam
> and that your users are going to start experiencing delivery issues to the
> Internet.

Do you use this on inbound mail as well?

How does it fit with the other postscreen dnsbls? I already have at
least six various dnsbls with varying weights...

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 13.02.2016 um 16:46 schrieb Alex:

DNS is very effective to block at the MTA level.  I setup my own private
RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
rbldnsd formatted zone file and setup your private RBL zone (doesn't
have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
will detect changes to it's zone files and reload them automatically to
keep current.


Do you have some kind of whitelist that includes gmail, yahoo, etc?

I'm not looking to compete with spamhaus, just compliment it, but
rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked


something PTR based like below is a good start

snippet of our in PHP written honeypot daemon at the bottom, and yes you 
can write a proper network service in PHP listening not only on port 25

_

 /** chroot to runtime directory and change basedir for later operations */
 if(chroot(__DIR__))
 {
  $chroot_basedir = '/honeypot-chroot';
 }
 else
 {
  $chroot_basedir = __DIR__;
 }

 /** drop privileges to 'nobody' */
 if(!posix_initgroups('nobody', $nobody_group) || 
!posix_setgid($nobody_group) || !posix_setuid($nobody_user))

 {
  error_log('ERROR: Drop privileges failed (' . $port . ')');
  exit('ERROR: Drop privileges failed (' . $port . ')' . "\n");
 }
_

 /**
  * Grosse Provider und offensichtliche Mailserver von automatischem
  * Blacklisting ausnehmen Basis ist der Reverse-DNS
  *
  * Gibt 'true' zurueck wenn die IP zu ignorieren ist
  * Honeypot speichert somit nur die Spam-Samples
  *
  * @param  string $ptr
  * @return boolean
  * @access public
 */
 function ignore_blacklist_ptr($ptr)
 {
  /** Sonderbehandlung */
  if(strpos($ptr, 'smtp') !== false || strpos($ptr, 'mail') !== false 
|| strpos($ptr, 'mxout') !== false)

  {
   return true;
  }
  /** Zu ignorierende PTR-Ends */
  $ignored = array
  (
   '.ac.at',
   '.apple.com',
   '.ebay.com',
   '.eyepin.com',
   '.facebook.com',
   '.gmx.at',
   '.gmx.com',
   '.gmx.de',
   '.gmx.net',
   '.google.com',
   '.gv.at',
   '.itronic.at',
   '.itronic.at',
   '.kundenserver.de',
   '.microsoft.com',
   '.mx.aol.com',
   '.mx.aol.com',
   '.observer.at',
   '.office-vienna.at',
   '.orf.at',
   '.outlook.com',
   '.paylife.at',
   '.paypal.com',
   '.phx3.secureserver.net',
   '.pinterest.com',
   '.skype.com',
   '.smtp-out.amazonses.com',
   '.thelounge.net',
   '.twitter.com',
   '.web.de',
   '.wetransfer.com',
   '.xing.com',
   '.yahoo.co.jp',
   '.yahoo.com',
   'taro.utanet.at',
   'tatiana.utanet.at',
  );
  /** Durchlaufen und gegen PTR testen */
  foreach($ignored as $test)
  {
   if(strpos($ptr, $test) !== false)
   {
$xtest = substr($ptr, strlen($ptr)-strlen($test));
if($xtest == $test)
{
 return true;
 break;
}
   }
  }
  /** Wenn nicht gelistet 'false' zurueckgeben */
  false;
 }



signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Dave Funk]

On Sat, 13 Feb 2016, Alex wrote:


I've now got rbldnsd implemented. I've also known for a while it's
faster/better than bind, but bind has always been in place.

I have rbldnsd running on port 530, alongside bind on 53. How do I
specify a urirhsbl in spamassassin to query the DNS server running on
530 instead of 53?


One way to do this is to set up a "forward only" zone in your bind config.

For example, assume you're authoritative for "example.com" and you've got
your rbldnsd set up to serve up your data as zone "mybl.example.com" and
it's bound to 192.168.124.23/530

Then in your bind config file create a zone:

zone "mybl.example.com" {
type forward;
forward only;
forwarders {
192.168.124.23 port 530;
};
};

Then when your clients (spamd or regular dns tools) query
"blah.com.mybl.example.com" it will hit your bind and then
get passed on to your rbldnsd for an answer.

If you want to hide that resource from the world put that zone
in a private 'view' in your bind. You could control access via an
ACL but by putting it inside a private view they'll never even see it
to try pounding on it.

To provide fault tolerance, you can set up rbldnsd's on multiple
machines and put multiple addresses in that 'forwarders' stanza.
You will need to put that zone definition in your primary bind and
each secondary.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: URIBL/DNSBL from a database

[Martin Gregorie]

On Fri, 2016-02-12 at 08:39 -0500, Alex wrote:
> Is it possible for spamassassin to query a database directly?
> 
Yes, with a plugin. 

I've been doing the opposite for some years now: I archive all my
outgoing mail and most of my non-spam incoming mail in a Postgres
database and use this as a whitelist: incoming mail from anybody that
I've sent mail to gets whitelisted. I use a plugin to query the
database via a view: the view is there to present the list of addresses
to which I've sent mail to the plugin's SQL query: its needed for
performance reasons because the database uses a many-to-many structure
to associate addresses with the messages they send or receive. 

It should be simple enough to change my plugin's query to work with
your database, particularly if you already have a table containing the
addresses you'd like to blacklist. Likewise, its probably fairly simple
to extend it to deal with the URLs and IPs from message bodies. 

If you'd like a copy of the plugin plus the associated .cf file[*],
contact me offlist.


Martin

[*] this loads and configures the plugin with database login details
and defines the rule that whitelists hits.

Re: URIBL/DNSBL from a database

[David Jones]

>
>From: Alex 

>For some time now I've been cycling URLs and IPs through  a mariadb
>database gathered from incoming mail on a honeypot I've created.
>Surprising how many are received ahead of spamhaus/barracuda.

Major RBLs like that keep up with lots of data points for IP reputation
over time so that can give a little extra time for normally reputable IPs
that happen to have a compromised account -- which happens to us
all.  But if you don't detect compromised accounts on your system
through feedback loops and abuse reports, then a reputable IP can
eventually get listed on those major RBLs.

>Is anyone else doing this, and are you just rejecting the IPs at the
>SMTP level outright?

DNS is very effective to block at the MTA level.  I setup my own private
RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
rbldnsd formatted zone file and setup your private RBL zone (doesn't
have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
will detect changes to it's zone files and reload them automatically to
keep current.

Then I have a nightly script that goes through my list of IPs in my private
RBL to remove them if they show up in another major RBL that I use.  This
prevents my list from becoming stale in the event that the IP becomes
delisted from the public RBLs.

In a related note, I have found that using the senderscore.org score combined
with postscreen's weighting is very effective in quickly catching new spammers.

postscreen_dnsbl_sites =
  score.senderscore.com=127.0.4.[60..69]*2
  score.senderscore.com=127.0.4.[50..59]*4
  score.senderscore.com=127.0.4.[30..49]*6
  score.senderscore.com=127.0.4.[0..29]*8
  score.senderscore.com=127.0.4.[90..100]*-6
  score.senderscore.com=127.0.4.[80..89]*-4
  score.senderscore.com=127.0.4.[70..79]*-2

You should monitor your own outbound IPs for their sender score.  If your
IP goes below 90, it's a good indication that you have been sending spam
and that your users are going to start experiencing delivery issues to the
Internet.

Dave

Re: URIBL/DNSBL from a database

[Axb]

On 02/12/2016 02:39 PM, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?


You'd need a custom plugin query the DB directly.



I'm familiar with how to create a uridnsbl, but is DNS the best
approach here?

DNS is cheap/reliable and simple to deploy / load balance.


The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


rbldnsd can check and load fresh data instantly within seconds.
If your dataset is not HUGE (loading 100MB zones is slow) rbldnspy will 
take inmemory updates so instant listings...

https://github.com/gryphius/rbldnspy

Re: URIBL/DNSBL from a database

[Shawn Bakhtiar]

On Feb 12, 2016, at 5:39 AM, Alex 
> wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spam assassin.

DNSBLs are very effective at this task, and I would recommend using before you 
filter the email with SA, unless you specifically want to score, due to 
uncertainty.


Is it possible for spamassassin to query a database directly?

It is:
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html

But even than I find it more effective having the server running the DNSBL 
manage the the block list using some metrics such as number of times the IP 
address appears, and/or not recording ip addresses in a whitelist table etc... 
Once (either via blacklist or metric) the IP gets into the DNSBL there is no 
need for me to worry about SA, simply reject. I find URI tend to change A LOT, 
so IP based blocking can be much more effective. But I think that's more of a 
preference.


I'm familiar with how to create a uridnsbl, but is DNS the best
approach here? The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


That's the way I do it. using bind DLZ http://bind-dlz.sourceforge.net/
We have a delegated subdomain off our main domain that hosts a DNS exclusively 
used for block list, created from incoming mail sent to honeypot email address 
(ones that are no never were/or are no longer valid). Again I tend to focus on 
the IP address not the URI as a find that URI are dime a dozen and change quite 
frequently.

Is anyone else doing this, and are you just rejecting the IPs at the
SMTP level outright?

We use sendmail features to reject long before it gets to SA. It works better 
(IMHO) since there is much lower over head for sendmail doing a quick DNS 
lookup than engaging the milter that runs the email through it's passes with SA.

http://weldon.whipple.org/sendmail/dnsbl.html

But in this case it's IP based only not URI based. For URI (especially ones 
that you'll want to regex) SA may be more effective.


Thanks,
Alex

Re: URIBL/DNSBL from a database

[Marc Perkel]

On 02/12/16 05:39, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?

I'm familiar with how to create a uridnsbl, but is DNS the best
approach here? The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.

Is anyone else doing this, and are you just rejecting the IPs at the
SMTP level outright?

Thanks,
Alex




Yeah - unless you write your own SA module using DNS is the quick easy 
solution.


--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: URIBL/DNSBL from a database

[Martin Gregorie]

On Fri, 2016-02-12 at 07:30 -0800, Marc Perkel wrote:

> Yeah - unless you write your own SA module using DNS is the quick
> easy solution.
> 
If Alex already has a set of scripts that populate and maintain the
database that he's happy with, then the quick and easy way may be to
make a custom SA module by using my database access module as a
starting point. 

The benefits would be that he's already familiar the care and feeding
of his database and that he can update it any time without needing to
stop and restart anything.


Martin

Re: URIBL plugins are broken

[Kevin A. McGrail]

On 5/11/2015 9:46 AM, Reindl Harald wrote:

stripped down and anonymized sample attached

the real bad thing is that the part triggering the URIBL rules wrongly 
is the quote of the signature from the message replied to


Am 11.05.2015 um 15:13 schrieb Reindl Harald:

i face false positives where the links are just facebook.com with the
http-prefix in front and NOT com between the http-prefix and the real
facebook domain

the domain with com in front is indeed on both URIBL but it just don#t
exist in the messages at all - why does SA extract the domains wrong
from the mailsource when there is no comfacebook at all besides the SA
report?

URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]




Not a bug in SA.

The plain text version of the email contains: 
a...@sepashvili.comfacebook.com/ketevan.sepashvili


The subdomain sepashvili is dropped leaving comfacebook.com.

Regards,
KAM

Re: URIBL plugins are broken

[Kevin A. McGrail]

On 5/11/2015 9:13 AM, Reindl Harald wrote:
i face false positives where the links are just facebook.com with 
the http-prefix in front and NOT com between the http-prefix and the 
real facebook domain


the domain with com in front is indeed on both URIBL but it just 
don#t exist in the messages at all - why does SA extract the domains 
wrong from the mailsource when there is no comfacebook at all 
besides the SA report?


URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]

Don't know.  Are you using 3.4.1?  Can you provide a spample that 
reproduces the issue?


regards,
KAM

Re: URIBL plugins are broken

[Reindl Harald]

Am 11.05.2015 um 15:43 schrieb Kevin A. McGrail:

On 5/11/2015 9:13 AM, Reindl Harald wrote:

i face false positives where the links are just facebook.com with
the http-prefix in front and NOT com between the http-prefix and the
real facebook domain

the domain with com in front is indeed on both URIBL but it just
don#t exist in the messages at all - why does SA extract the domains
wrong from the mailsource when there is no comfacebook at all
besides the SA report?

URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]


Don't know.  Are you using 3.4.1?  Can you provide a spample that
reproduces the issue?


3.4.0, sample attached in my previous mail, sorry for not attach it in 
the first mail :-(





signature.asc
Description: OpenPGP digital signature

Re: uribl problem

[Nick Edwards]

Hi Karsten,

On 12/1/13, Karsten Bräckelmann guent...@rudersport.de wrote:
 On Fri, 2013-11-29 at 13:30 +1000, Nick Edwards wrote:
 Hi, have a problem with our internal uribl

 urirhsblINT_URI uri.int.lan. A
 bodyINT_URI eval:check_uridnsbl('INT_URI')
 describeINT_URI Contains a URI listed in internal URIBL
 tflags  INT_URI net
 score   INT_URI 3

 That's correct.


Thanks

 this rule performs lookups if in normal text of body, however, i we
 have inside html if does not lookup. eg

 hi see example.org  looks up example.org
 but
 hi see a href=http://example.org;example.net/a
 it will lookup example.net, not example.org

 How do you tell SA does not lookup the domain in the HTML anchor href?


I ran debug and viewed the scrollback (see below)

 The general SA method of verifying which domains are queried for, is to
 have a look at the debug output. In your case, you can also check your
 local DNSBL's logs.

   spamassassin -D uridnsbl   msg


Ahh ok, this produces output I missed in the 2000 lines of normal
debug output, it turns out it is seeing that host/domain for a lookup,
however in my case that prompted me to ask this question, it was not
looking up the domain in question because as your suggested debug
output easily shows, that domain is in a skip list, which explains why
it was not looking up.

Is there an easy way to say ignore this host/domain in a skip list? or
disable skip list altogether? closest I can find is skip rbl checks.


 To see more of the URIDNSBL plugin activity, including which DNSBLs are
 queried and what domains are looked up, you can use e.g.

   spamassassin -D   msg  21 | grep URI-DNSBL

 To limit that to your local DNSBL, grep for DNSBL:uri.int.lan.


right, added that to my cheats list :)


 Note: The absence of a rule match for the second domain in the Report
 header is NOT an indicator of a missing query. If more than one domain
 is listed in the DNSBL, the urirhsbl rule will still be triggered once
 only, showing one domain, not all listed domains:

   X-Spam-Report:
 *  3.0 INT_URI Contains a URI listed in internal URIBL
 *  [URIs: example.net]

 Despite the plural in the automatically added detail, it does list one
 domain only. Probably a bug in the URIDNSBL plugin, though might also be
 intended.

 Since the DNSBL lookups are asynchronous, it is likely undefined which
 listed domain will trigger the rule to hit and be reported, influenced
 by lookup time and the order they are parsed from the message.



Awesome, thank you.

Re: uribl problem

[Karsten Bräckelmann]

On Mon, 2013-12-02 at 07:58 +1000, Nick Edwards wrote:
 On 12/1/13, Karsten Bräckelmann guent...@rudersport.de wrote:

  The general SA method of verifying which domains are queried for, is to
  have a look at the debug output. In your case, you can also check your
  local DNSBL's logs.
 
spamassassin -D uridnsbl   msg
 
 Ahh ok, this produces output I missed in the 2000 lines of normal
 debug output, it turns out it is seeing that host/domain for a lookup,
 however in my case that prompted me to ask this question, it was not
 looking up the domain in question because as your suggested debug
 output easily shows, that domain is in a skip list, which explains why
 it was not looking up.

That's what grep or searching in less is for. :)  You would have quickly
noticed example.net being skipped by searching the output for your
test-case...


 Is there an easy way to say ignore this host/domain in a skip list? or
 disable skip list altogether? closest I can find is skip rbl checks.

See the URIDNSBL plugin documentation, (clear_)uridnsbl_skip_domain
options.
  http://spamassassin.apache.org/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html

Before dropping any domain from the default skip list in 25_uribl.cf,
keep in mind that affects all URI DNSBLs. Unless a domain in the skip
list turns severely rogue, they will never be listed by DNSBLs anyway.

  $ grep example.net 25_uribl.cf
  uridnsbl_skip_domain example.com example.net example.org
  $ grep uridnsbl_skip_domain 25_uribl.cf | wc -l -w
   51 254

The default skip list is about 200 domains, generated from URI DNSBL
data of domains frequently appearing in mail and thus (previously) being
looked up.


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: uribl problem

[Benny Pedersen]

Nick Edwards skrev den 2013-11-29 04:30:


urirhsblINT_URI uri.int.lan. A
bodyINT_URI eval:check_uridnsbl('INT_URI')
describeINT_URI Contains a URI listed in internal URIBL
tflags  INT_URI net
score   INT_URI 3


rule is okay as designed


this rule performs lookups if in normal text of body, however, i we
have inside html if does not lookup. eg

hi see example.org  looks up example.org


grep example.org msg | wc -l


but
hi see a href=http://example.org;example.net/a
it will lookup example.net, not example.org


grep example.org msg | wc -l
grep example.net msg | wc -l

is both is with one line ?

rule of thumps here is that how email ia designed, with means header is 
part of body testing, so if example.net exits in From: then it will be 
tested in uridnsbl aswell, basicly its a feature that is imho not meant 
to be so, but its a nice bug :=)


problem is here that it also test uridnsbl domains as sender domains, eg 
it unfiltered with type it is, testing that this domains have mx records 
would also make more fails, testing if domains have a or  is limted 
usable for testing spam



 is this correct or do I need some other lookup method in local.cf ?


you will need another plugin to make this specific test imho

Re: uribl problem

[Karsten Bräckelmann]

On Fri, 2013-11-29 at 13:30 +1000, Nick Edwards wrote:
 Hi, have a problem with our internal uribl
 
 urirhsblINT_URI uri.int.lan. A
 bodyINT_URI eval:check_uridnsbl('INT_URI')
 describeINT_URI Contains a URI listed in internal URIBL
 tflags  INT_URI net
 score   INT_URI 3

That's correct.

 this rule performs lookups if in normal text of body, however, i we
 have inside html if does not lookup. eg
 
 hi see example.org  looks up example.org
 but
 hi see a href=http://example.org;example.net/a
 it will lookup example.net, not example.org

How do you tell SA does not lookup the domain in the HTML anchor href?

The general SA method of verifying which domains are queried for, is to
have a look at the debug output. In your case, you can also check your
local DNSBL's logs.

  spamassassin -D uridnsbl   msg

will limit the debug output to the URIDNSBL plugin, which would look
like this:

  dbg: uridnsbl: domain example.net in skip list
  dbg: uridnsbl: domain example.com in skip list
  dbg: uridnsbl: domains to query: anchor-text.net anchor-href.net
  dbg: uridnsbl: domain anchor-text.net listed (INT_URI): 127.0.0.2
  dbg: uridnsbl: domain anchor-href.net listed (INT_URI): 127.0.0.2

Note the placeholder domains as found in the HTML anchor href and parsed
from the text. The example.(net|com) domains you used are perfect for
the HTML sample snippet, but won't work for actual debugging, since they
are in the default skip list.

To see more of the URIDNSBL plugin activity, including which DNSBLs are
queried and what domains are looked up, you can use e.g.

  spamassassin -D   msg  21 | grep URI-DNSBL

To limit that to your local DNSBL, grep for DNSBL:uri.int.lan.


Note: The absence of a rule match for the second domain in the Report
header is NOT an indicator of a missing query. If more than one domain
is listed in the DNSBL, the urirhsbl rule will still be triggered once
only, showing one domain, not all listed domains:

  X-Spam-Report:
*  3.0 INT_URI Contains a URI listed in internal URIBL
*  [URIs: example.net]

Despite the plural in the automatically added detail, it does list one
domain only. Probably a bug in the URIDNSBL plugin, though might also be
intended.

Since the DNSBL lookups are asynchronous, it is likely undefined which
listed domain will trigger the rule to hit and be reported, influenced
by lookup time and the order they are parsed from the message.


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: uribl test

[Kevin A. McGrail]

For those who have asked, the RBL I am testing is included in the rules 
from KAM.cf at http://www.pccc.com/downloads/SpamAssassin/contrib/KAM.cf


There will be some new tests coming as I'm working on more tests that 
require code changes.


regards,
KAM

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails

[Michael Scheidell]

On 3/2/12 11:36 AM, Benny Pedersen wrote:

just a note to whom it might concern :)


phisting?

OUCH.


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
*| *SECNAP Network Security Corporation

   * Best Mobile Solutions Product of 2011
   * Best Intrusion Prevention Product
   * Hot Company Finalist 2011
   * Best Email Security Product
   * Certified SNORT Integrator

__
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com/
__  
 

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails

[Jeremy McSpadden]

Ha. Nice


--
Jeremy McSpadden

On Mar 2, 2012, at 10:38 AM, Michael Scheidell michael.scheid...@secnap.com 
wrote:

 On 3/2/12 11:36 AM, Benny Pedersen wrote:
 just a note to whom it might concern :)
 
 phisting?
 
 OUCH.
 
 
 -- 
 Michael Scheidell, CTO
 o: 561-999-5000
 d: 561-948-2259
 *| *SECNAP Network Security Corporation
 
   * Best Mobile Solutions Product of 2011
   * Best Intrusion Prevention Product
   * Hot Company Finalist 2011
   * Best Email Security Product
   * Certified SNORT Integrator
 
 __
 This email has been scanned and certified safe by SpammerTrap(r). For 
 Information please see http://www.spammertrap.com/
 __   

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails

[Simon Loewenthal]

It was a last minute decision.

Jeremy McSpadden jer...@fluxlabs.net wrote:

Ha. Nice


--
Jeremy McSpadden

On Mar 2, 2012, at 10:38 AM, Michael Scheidell
michael.scheid...@secnap.com wrote:

 On 3/2/12 11:36 AM, Benny Pedersen wrote:
 just a note to whom it might concern :)
 
 phisting?
 
 OUCH.
 
 
 -- 
 Michael Scheidell, CTO
 o: 561-999-5000
 d: 561-948-2259
 *| *SECNAP Network Security Corporation
 
   * Best Mobile Solutions Product of 2011
   * Best Intrusion Prevention Product
   * Hot Company Finalist 2011
   * Best Email Security Product
   * Certified SNORT Integrator
 

__
 This email has been scanned and certified safe by SpammerTrap(r). For
Information please see http://www.spammertrap.com/

__ 
 

Re: uribl lastminute.com listed in uribl white and is now used for nordea phishiing mails

[Benny Pedersen]

Den 2012-03-02 17:40, Jeremy McSpadden skrev:

Ha. Nice


be nice to an old mand




--
Jeremy McSpadden

On Mar 2, 2012, at 10:38 AM, Michael Scheidell
michael.scheid...@secnap.com wrote:


On 3/2/12 11:36 AM, Benny Pedersen wrote:

just a note to whom it might concern :)


phisting?

OUCH.


--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
*| *SECNAP Network Security Corporation

  * Best Mobile Solutions Product of 2011
  * Best Intrusion Prevention Product
  * Hot Company Finalist 2011
  * Best Email Security Product
  * Certified SNORT Integrator


__
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com/


__

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails

[Axb]

On 03/02/2012 05:36 PM, Benny Pedersen wrote:

just a note to whom it might concern :)



why no pastebin a sample?

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails (SOLVED)

[Benny Pedersen]

Den 2012-03-02 17:50, Axb skrev:

On 03/02/2012 05:36 PM, Benny Pedersen wrote:

just a note to whom it might concern :)

why no pastebin a sample?


february had 29 days this yaer ?

its being resolved, sorry for the noice

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails (SOLVED)

[Jeremy McSpadden]

Leap Year


--
Jeremy McSpadden

On Mar 2, 2012, at 11:11 AM, Benny Pedersen m...@junc.org wrote:

 Den 2012-03-02 17:50, Axb skrev:
 On 03/02/2012 05:36 PM, Benny Pedersen wrote:
 just a note to whom it might concern :)
 why no pastebin a sample?
 
 february had 29 days this yaer ?
 
 its being resolved, sorry for the noice
 
 
 

Re: uribl lastminute.com listed in uribl whte and is now used for nordea phisting mails (SOLVED)

[Benny Pedersen]

Den 2012-03-02 18:15, Jeremy McSpadden skrev:

Leap Year


sure ?

#
# Copyright 2012 Nordea
#
body __COPYRIGHT_NORDEA /Copyright\ 201.\ Nordea/i
meta PHISHMAIL_NORDEA (__COPYRIGHT_NORDEA  !SPF_PASS)
describe PHISHMAIL_NORDEA Meta: __COPYRIGHT_NORDEA  !SPF_PASS
score PHISHMAIL_NORDEA 3.0

if senders begin using spf we need another rule

Re: URIBL blocked

[spamassassin]

On 23/01/12 12:22, Tom Kinghorn wrote:

 Resolving the block might be as simple as using your own caching
 nameserver to avoid being lumped together with other users queries;
 setting up your own mirror of the DNS-blocklist; or paying to use the
 blocklist. The choice is up to the DNS-Blocklist administrator.

 Okay, so my question is, How can I rectify this as we use our own
 caching servers already?

Contact the people who are blocking your lookups (URIBL) and ask them
what you need to do to get unblocked.

-- 
Mike Cardwell  https://grepular.com/ http://cardwellit.com/
OpenPGP Key35BC AF1D 3AA2 1F84 3DC3  B0CF 70A5 F512 0018 461F
XMPP OTR Key   8924 B06A 7917 AAF3 DBB1  BF1B 295C 3C78 3EF1 46B4



signature.asc
Description: OpenPGP digital signature

Re: URIBL blocked

[Giles Coochey]

 

I would look at getting a datafeed:
http://www.uribl.com/datafeed.shtml [7] 

Out of interest, how much
volume of email are you processing to experience this? 

Are you sharing
your external IP with any other of your ISP customers? Does your ISP do
anything strange with DNS queries? 

On 2012-01-23 12:22, Tom Kinghorn
wrote: 

 Good afternoon list.
 Please could someone explain the
following
 
 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to
URIBL was blocked.
 See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block [1] for
more information.
 
 The link shows:
 
 A: DNS-Blocklists often run
on the free for some model and/or they may limit the number of queries
you can perform to maximize resources. 
 
 If you were directed to
this link from a rule description, then you have a DNS-Blocklist that is
purposefully blocking your queries. 
 
 Resolving the block might be
as simple as using your own caching nameserver to avoid being lumped
together with other users queries; setting up your own mirror of the
DNS-blocklist; or paying to use the blocklist. The choice is up to the
DNS-Blocklist administrator. 
 
 SpamAssassin [2] supports the free
for some model since it works for the majority of SpamAssassin [3]
installations. However, we do not support methodologies that
purposefully return wrong answers and those DNS-Blocklists will be
disabled by default. 
 
 The following blocklist providers have
implemented a Block Notification Rule with SpamAssassin [4]: 
 
 * 


 URIBL http://www.uribl.com/ [5] 
 * 
 
 DNSWL
http://www.dnswl.org/ [6] 
 
 Okay, so my question is, How can I
rectify this as we use our own caching servers already?
 
 Thanks

Tom

-- 
Message sent via my webmail account.
 

Links:
--
[1]
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
[2]
http://wiki.apache.org/spamassassin/SpamAssassin
[3]
http://wiki.apache.org/spamassassin/SpamAssassin
[4]
http://wiki.apache.org/spamassassin/SpamAssassin
[5]
http://www.uribl.com/
[6] http://www.dnswl.org/
[7]
http://www.uribl.com/datafeed.shtml

Re: URIBL lookup count

[Karsten Bräckelmann]

On Tue, 2011-10-18 at 12:51 +0100, Martin Gregorie wrote:
 I've just been thinking about URIBL lookups, etc and realising that I
 don't know how many of these an SA configuration does or how to estimate
 it. 
 
 Is it correct to assume that every configured URIBL is sent a single
 lookup request for every message that SA scans?

No. This message has no URIs, thus no lookup.

Furthermore, since the BL lookups are DNS, each URI needs a lookup of
its own. The only thing safe to assume is, that with multiple aggregated
lists (or listings) per URI DNSBL, there is one DNS lookup per URI and
DNSBL -- e.g. SURBL (multiple lists) or URIBL (multiple listings).


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL lookup count

[Martin Gregorie]

On Tue, 2011-10-18 at 19:22 +0200, Karsten Bräckelmann wrote:
 On Tue, 2011-10-18 at 12:51 +0100, Martin Gregorie wrote:
  I've just been thinking about URIBL lookups, etc and realising that I
  don't know how many of these an SA configuration does or how to estimate
  it. 
  
  Is it correct to assume that every configured URIBL is sent a single
  lookup request for every message that SA scans?
 
 No. This message has no URIs, thus no lookup.
 
 Furthermore, since the BL lookups are DNS, each URI needs a lookup of
 its own. The only thing safe to assume is, that with multiple aggregated
 lists (or listings) per URI DNSBL, there is one DNS lookup per URI and
 DNSBL -- e.g. SURBL (multiple lists) or URIBL (multiple listings).
 
 
OK, so the answer is not straight forward: thanks for confirming it. 

OTOH, a fairly regular occurrence on this list is thread from people who
get problems from hitting usage limits set by various BL lookups. So, I
wonder if it would be useful for SA to log the number of BL lookups it
does: as it need only involve of writing a log message every hour or day
giving the accumulated count for the period, its performance hit should
be tiny and, of course, it should have an enable/disable configuration
parameter. Output would be a single log message containing a total for
all BL lookups or (deluxe version) a total for each configured BL.

It goes without saying that, if this is incompatible with the SA
internal structure, then forget about it.
  

Martin

Re: URIBL lookup count

[Karsten Bräckelmann]

On Tue, 2011-10-18 at 23:52 +0100, Martin Gregorie wrote:
 On Tue, 2011-10-18 at 19:22 +0200, Karsten Bräckelmann wrote:

  [...]  there is one DNS lookup per URI and
  DNSBL -- e.g. SURBL (multiple lists) or URIBL (multiple listings).
 
 OK, so the answer is not straight forward: thanks for confirming it. 
 
 OTOH, a fairly regular occurrence on this list is thread from people who
 get problems from hitting usage limits set by various BL lookups. So, I

From memory, these are typically no hits issues by private or SOHO
users, who aren't anywhere close to the free usage limits -- but using
the DNS by their (large) ISP, instead of running their own caching
nameserver. The ISPs DNS is blocked, or in one recent occurrence the
Google DNS.

I cannot remember any large system being mentioned here, whose admins
did not know they exceed the free usage limit. (The DNSBLs do tell the
admins, the limits aren't strictly enforced AFAIK, and most importantly
only the most aggressive abusers will ever get anything worse than no
responses.)

 wonder if it would be useful for SA to log the number of BL lookups it
 does: as it need only involve of writing a log message every hour or day
 giving the accumulated count for the period, its performance hit should
 be tiny and, of course, it should have an enable/disable configuration
 parameter. Output would be a single log message containing a total for
 all BL lookups or (deluxe version) a total for each configured BL.

Wouldn't grepping the DNS logs already tell the admin about it?

Keep in mind, the actual number of queries isn't relevant unless you're
at least in the general ball-park of 100,000 messages a day.


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL lookup count

[Karsten Bräckelmann]

On Wed, 2011-10-19 at 01:29 +0200, Karsten Bräckelmann wrote:
 On Tue, 2011-10-18 at 23:52 +0100, Martin Gregorie wrote:
  On Tue, 2011-10-18 at 19:22 +0200, Karsten Bräckelmann wrote:

  wonder if it would be useful for SA to log the number of BL lookups it
  does: as it need only involve of writing a log message every hour or day
  giving the accumulated count for the period, its performance hit should
  be tiny and, of course, it should have an enable/disable configuration
  parameter. Output would be a single log message containing a total for
  all BL lookups or (deluxe version) a total for each configured BL.

Oh, and of course, caching applies here, too.

The number of queries SA performs does NOT tell you the number of
queries actually hitting the URI DNSBL's infrastructure. SURBL has a TTL
of 3 minutes, URIBL even uses 30 minutes.

Thus, a spam run targeting lots of your users within a short time period
will result in more queries (sent by SA) than actually ending up at the
DNSBL's mirrors.

Similar for negative caching and not-blacklisted domains frequently
observed in your mail stream.


I probably should stop replying to self, though. ;)


 Wouldn't grepping the DNS logs already tell the admin about it?
 
 Keep in mind, the actual number of queries isn't relevant unless you're
 at least in the general ball-park of 100,000 messages a day.

-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL lookup count

[Martin Gregorie]

On Wed, 2011-10-19 at 01:29 +0200, Karsten Bräckelmann wrote:

 Keep in mind, the actual number of queries isn't relevant unless you're
 at least in the general ball-park of 100,000 messages a day.
 
Indeed: I'm not remotely near that. It was just an idea that I thought
might be useful provided it was a no-brainer to implement.

I like your log analysis though: should have thought of it myself. That
would take just a few lines of awk to implement or a little more Perl to
make it a logwatch plugin.


Martin

RE: uribl not working properly with .gg TLD

[DaveAtJLA]

Ah I understand now why they are treated differently.. I've never delved into
the details of that module.

Blacklisting might be a good idea!

Thanks

Dave


Giampaolo Tomassoni-2 wrote:
 
 What I am asking is why a reference to http://querty.ru.gg generates a
 URI
 lookup for ru.gg (ie missing out the first component) whereas a
 reference to
 http://qwerty.ru.com generates a URI lookup for qwerty.ru.com.
 
 Dave
 
 Because the ru.gg second level domain is not in the TWO_LEVEL_DOMAINS
 variable defined in Mail::SpamAssassin::Util::RegistrarBoundaries , while
 ru.com is.
 
 If you mean that ru.gg should be there too, please note that querty.ru.gg
 is
 a third-level domain of ru.gg, which is assigned to webme.com. So, I don't
 see any need to discriminate querty.ru.gg from ru.gg.
 
 Further, I would personally blacklist the whole .gg gTLD since their whois
 service is ridiculous.
 
 Giampaolo
  
 
 
 Giampaolo Tomassoni-2 wrote:
 
  I'm running SpamAssassin version 3.3.0 and we received some spam
  recently
  which contained a link to a .ru.gg domain. While investigating
 whether
  it
  was listed in any of the URIBLs I discovered that if a message
 contains
  a
  link to http://qwerty.ru.gg;, spamassassin only looks up the domain
  ru.gg
  - here's a snippet from the log:
 
  Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.026 .
  DNSBL:dob.sibl.support-intelligence.net:ru.gg
  Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.027 .
  DNSBL:multi.uribl.com.:ru.gg
 
  However if I edit the message, change the link to
  http://qwerty.ru.com; and
  run it through spamassassin again, then the URIBL lookups are done
 for
  the
  full domain name:
 
  Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.287 .
  DNSBL:dob.sibl.support-intelligence.net:qwerty.ru.com
  Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.290 .
  DNSBL:multi.uribl.com.:qwerty.ru.com
 
  This can't be right, can it? It looks like the gg top-level domain
  isn't
  being handled properly. Any ideas?
 
  I don't see why you believe querty.ru.gg == querty.ru.com .
 
  .gg is a gTLD (for the Bailiwick of Guernsey, according to
  http://en.wikipedia.org/wiki/.gg).
 
 
  Dave
 
  Giampaolo
 
 
 
 
 --
 View this message in context: http://old.nabble.com/uribl-not-working-
 properly-with-.gg-TLD-tp29159353p29159839.html
 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.
 
 
 

-- 
View this message in context: 
http://old.nabble.com/uribl-not-working-properly-with-.gg-TLD-tp29159353p29170299.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

RE: uribl not working properly with .gg TLD

[Giampaolo Tomassoni]

 I'm running SpamAssassin version 3.3.0 and we received some spam
 recently
 which contained a link to a .ru.gg domain. While investigating whether
 it
 was listed in any of the URIBLs I discovered that if a message contains
 a
 link to http://qwerty.ru.gg;, spamassassin only looks up the domain
 ru.gg
 - here's a snippet from the log:
 
 Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.026 .
 DNSBL:dob.sibl.support-intelligence.net:ru.gg
 Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.027 .
 DNSBL:multi.uribl.com.:ru.gg
 
 However if I edit the message, change the link to
 http://qwerty.ru.com; and
 run it through spamassassin again, then the URIBL lookups are done for
 the
 full domain name:
 
 Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.287 .
 DNSBL:dob.sibl.support-intelligence.net:qwerty.ru.com
 Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.290 .
 DNSBL:multi.uribl.com.:qwerty.ru.com
 
 This can't be right, can it? It looks like the gg top-level domain
 isn't
 being handled properly. Any ideas?

I don't see why you believe querty.ru.gg == querty.ru.com .

.gg is a gTLD (for the Bailiwick of Guernsey, according to
http://en.wikipedia.org/wiki/.gg).


 Dave

Giampaolo

RE: uribl not working properly with .gg TLD

[DaveAtJLA]

What I am asking is why a reference to http://querty.ru.gg generates a URI
lookup for ru.gg (ie missing out the first component) whereas a reference to
http://qwerty.ru.com generates a URI lookup for qwerty.ru.com.

Dave


Giampaolo Tomassoni-2 wrote:
 
 I'm running SpamAssassin version 3.3.0 and we received some spam
 recently
 which contained a link to a .ru.gg domain. While investigating whether
 it
 was listed in any of the URIBLs I discovered that if a message contains
 a
 link to http://qwerty.ru.gg;, spamassassin only looks up the domain
 ru.gg
 - here's a snippet from the log:
 
 Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.026 .
 DNSBL:dob.sibl.support-intelligence.net:ru.gg
 Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.027 .
 DNSBL:multi.uribl.com.:ru.gg
 
 However if I edit the message, change the link to
 http://qwerty.ru.com; and
 run it through spamassassin again, then the URIBL lookups are done for
 the
 full domain name:
 
 Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.287 .
 DNSBL:dob.sibl.support-intelligence.net:qwerty.ru.com
 Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.290 .
 DNSBL:multi.uribl.com.:qwerty.ru.com
 
 This can't be right, can it? It looks like the gg top-level domain
 isn't
 being handled properly. Any ideas?
 
 I don't see why you believe querty.ru.gg == querty.ru.com .
 
 .gg is a gTLD (for the Bailiwick of Guernsey, according to
 http://en.wikipedia.org/wiki/.gg).
 
 
 Dave
 
 Giampaolo
 
 
 

-- 
View this message in context: 
http://old.nabble.com/uribl-not-working-properly-with-.gg-TLD-tp29159353p29159839.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

RE: uribl not working properly with .gg TLD

[Giampaolo Tomassoni]

 What I am asking is why a reference to http://querty.ru.gg generates a
 URI
 lookup for ru.gg (ie missing out the first component) whereas a
 reference to
 http://qwerty.ru.com generates a URI lookup for qwerty.ru.com.
 
 Dave

Because the ru.gg second level domain is not in the TWO_LEVEL_DOMAINS
variable defined in Mail::SpamAssassin::Util::RegistrarBoundaries , while
ru.com is.

If you mean that ru.gg should be there too, please note that querty.ru.gg is
a third-level domain of ru.gg, which is assigned to webme.com. So, I don't
see any need to discriminate querty.ru.gg from ru.gg.

Further, I would personally blacklist the whole .gg gTLD since their whois
service is ridiculous.

Giampaolo
 


 Giampaolo Tomassoni-2 wrote:
 
  I'm running SpamAssassin version 3.3.0 and we received some spam
  recently
  which contained a link to a .ru.gg domain. While investigating
 whether
  it
  was listed in any of the URIBLs I discovered that if a message
 contains
  a
  link to http://qwerty.ru.gg;, spamassassin only looks up the domain
  ru.gg
  - here's a snippet from the log:
 
  Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.026 .
  DNSBL:dob.sibl.support-intelligence.net:ru.gg
  Jul 14 07:55:54.785 [3269] dbg: async: timing: 0.027 .
  DNSBL:multi.uribl.com.:ru.gg
 
  However if I edit the message, change the link to
  http://qwerty.ru.com; and
  run it through spamassassin again, then the URIBL lookups are done
 for
  the
  full domain name:
 
  Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.287 .
  DNSBL:dob.sibl.support-intelligence.net:qwerty.ru.com
  Jul 14 08:52:49.412 [16122] dbg: async: timing: 0.290 .
  DNSBL:multi.uribl.com.:qwerty.ru.com
 
  This can't be right, can it? It looks like the gg top-level domain
  isn't
  being handled properly. Any ideas?
 
  I don't see why you believe querty.ru.gg == querty.ru.com .
 
  .gg is a gTLD (for the Bailiwick of Guernsey, according to
  http://en.wikipedia.org/wiki/.gg).
 
 
  Dave
 
  Giampaolo
 
 
 
 
 --
 View this message in context: http://old.nabble.com/uribl-not-working-
 properly-with-.gg-TLD-tp29159353p29159839.html
 Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: URIBL Notice

[Brian]

On Fri, 2010-03-12 at 07:48 -0800, Ray Dzek wrote:
 I just received the dreaded URIBL “You send us to many DNS queries”
 notice.  This is fine.  We have been growing and I am sure our queries
 have gone up.  But when looking at their data feed service options the
 first thing I noticed was that there is no fee structure.  I don’t
 know about you, but that is always a red flag in my world.  Before I
 even get past the first paragraph it already smells like a
 “shakedown”.
 
  
 
 But…
 
  
 
 My real question is how badly is my SA environment going to be
 impacted by turning URIBL off?  What increase in spam should I expect?
 
  
 
 Ray
 
 
You'll see some difference but from experience it lets through more than
it blocks and is a bit of a 'shutting the stable door after the horse
has bolted' kind of list.

There is nothing to stop you setting up a simple BIND server and
creating your own local uri based block list customised to your own
needs and based on the links you most frequently see. I've done it and
I'm sure plenty of others have too.

Re: URIBL Notice

[Steve Freegard]

On 12/03/10 15:48, Ray Dzek wrote:

I just received the dreaded URIBL “You send us to many DNS queries”
notice.  This is fine. We have been growing and I am sure our queries
have gone up. But when looking at their data feed service options the
first thing I noticed was that there is no fee structure. I don’t know
about you, but that is always a red flag in my world. Before I even get
past the first paragraph it already smells like a “shakedown”.


Did you actually go through the effort to create a URIBL account and go 
to the process of requesting a feed?


There is a price structure; it's based on how often you want to do the 
rsync and how many users you have and it's all shown on that page IIRC.



But…

My real question is how badly is my SA environment going to be impacted
by turning URIBL off? What increase in spam should I expect?


I actually subscribe to URIBL and reject based on hits to the URIBL 
black list at the SMTP level.


214-2.0.0 169 spamd-connect=3985 33.17%
214-2.0.0 170 spamd-connect-error=0 0.00%
214-2.0.0 171 spamd-reject=559 4.65%
214-2.0.0 172 spamd-sender-marked-spam=9 0.07%
214-2.0.0 173 spamd-tag=100 0.83%

So 559 reject by SA as score 10 and just 100 tagged with score =5 10

214-2.0.0 178 uri-bl=731 6.09%

Whereas I've rejected 731 with URIBL.  Draw your own conclusions from that.

I find it URIBL works very well and if you subscribe you get access to a 
number of extra lists (e.g. URIBL Gold etc.) which also adds extra value 
and catch rate.


Hope that helps.

Kind regards,
Steve.

Re: URIBL Notice

[Yet Another Ninja]

On 2010-03-12 16:48, Ray Dzek wrote:

I just received the dreaded URIBL You send us to many DNS queries
notice.  This is fine.  We have been growing and I am sure our
queries have gone up.  But when looking at their data feed service
options the first thing I noticed was that there is no fee structure.
I don't know about you, but that is always a red flag in my world.
Before I even get past the first paragraph it already smells like a
shakedown.

But...

My real question is how badly is my SA environment going to be
impacted by turning URIBL off?  What increase in spam should I
expect?


These stats are for small trap box which only accepts mail from bots and 
rejects stuff listed by DNSWL and other public WLs. Since midnight CET-


These are only URI BL tats - so you woun't see other dnsbls like 
Spamcop, etc.


Some zones may sound familiar - others are private.

RANKRULE NAME   COUNT  %OFMAIL %OFSPAM  %OFHAM

   1URI_IN_MSG  4794393.48   95.22   53.92
   2ANY_URIBL_COM   4646088.38   92.270.09
   3URIBL_BLACK 4518685.95   89.740.00
   4ANY_SPAMHAUS4229980.46   84.010.05
   5URIBL_DBL   3955575.24   78.560.00
   6HTML_MESSAGE3940576.83   78.26   44.46
   7URIBL_SBL   3868073.58   76.820.00
   8CM_URI_DNSBL3863673.49   76.730.00
   9AXB_BLACK_NSIP  3843973.12   76.340.00
  10URIBL_SPAMEATINGMONKEY_RED  3825072.76   75.970.00
  11ANY_SURBL   3374264.24   67.011.31
  12URIBL_SC_SWINOG 3326563.28   66.070.00
  13URIBL_IVMURI3298762.75   65.520.00
  14RDNS_NONE   3173362.82   63.02   58.11
  15URIBL_JP_SURBL  3040857.84   60.390.00
  16URIBL_SPAMEATINGMONKEY_BLACK3033857.71   60.250.00
  17URIBL_DRS_BLACK 3027257.58   60.120.00
  18MIME_HTML_ONLY  2948556.13   58.561.08
  19URIBL_WS_SURBL  2895955.14   57.521.31
  20URIBL_AB_SURBL  2720051.74   54.020.00
  21AXB_BLACK_NS2507647.70   49.800.00
  22HK_NAME_DRUGS   1602730.49   31.830.00
  23RDNS_DYNAMIC1333426.09   26.48   17.25
  24URIBL_OB_SURBL  1270124.16   25.230.00
  25FSL_HELO_NON_FQDN_1 1268125.47   25.19   31.89

Re: URIBL Notice

[Rob McEwen]

Yet Another Ninja wrote:
 These stats are for small trap box which only accepts mail from bots
 and rejects stuff listed by DNSWL and other public WLs. Since midnight
 CET-
 These are only URI BL tats - so you woun't see other dnsbls like
 Spamcop, etc.

Alex,

about those stats...

(1) Do those include spams sent to non-existent users (i.e. dictionary
attack spams)?

(2) Was pre-filtering done, such as collecting stats only on messages
which made it past zen.spamhaus.org (etc.)? Or was there no pre-filtering?

-- 
Rob McEwen
http://dnsbl.invaluement.com/
r...@invaluement.com
+1 (478) 475-9032

Re: URIBL Notice

[Yet Another Ninja]

On 2010-03-12 20:23, Rob McEwen wrote:

Yet Another Ninja wrote:

These stats are for small trap box which only accepts mail from bots
and rejects stuff listed by DNSWL and other public WLs. Since midnight
CET-
These are only URI BL tats - so you woun't see other dnsbls like
Spamcop, etc.


Alex,

about those stats...

(1) Do those include spams sent to non-existent users (i.e. dictionary
attack spams)?


there are no users - its  trap domains which have never had any real 
users - ever.



(2) Was pre-filtering done, such as collecting stats only on messages
which made it past zen.spamhaus.org (etc.)? Or was there no pre-filtering?


no prefiltering except rejecting potential bounces and stuff leaking 
from whatever may be on DNSWL and a coupleof other WLs.

Re: URIBL Notice

[Rob McEwen]

Yet Another Ninja wrote:
 there are no users - its  trap domains which have never had any real
 users - ever.
 
 no prefiltering except rejecting potential bounces and stuff leaking
 from whatever may be on DNSWL and a coupleof other WLs. 

Alex,

Your stats are certainly valuable and illustrative... but not reflective
of the stats one would see in a MOST real world mail streams where:

(A) the spams were sent to actual users (which would be a distinctively
different mix of spams compared to a pure honeypot stream of spams--for
example, there'd be more can-spam spam/snowshoe spam in the real user
mix, as well as less spam easily block by other techniques)

--AND--

(B) where most of the messages have been prefiltered by FP-safe sender
IP blacklists like Zen (which would then also alter the makeup of the
spam stream--this would cause a lowered percentage of the easy stuff
left over for the URI lists to process... and a higher percentage of the
hard stuff left for the URI lists to process). Those two things would
alter those stats dramatically and would paint a very different picture
for some of those uri blacklists you compared.

BTW - don't get me wrong... URIBL would still fare VERY well either
way--so don't think I'm saying or implying ANYTHING bad about URIBL! (or
anything bad about ANY other list)

(fwiw)

-- 
Rob McEwen
http://dnsbl.invaluement.com/
r...@invaluement.com
+1 (478) 475-9032

Re: URIBL Notice

[Yet Another Ninja]

On 2010-03-13 0:50, Rob McEwen wrote:

Yet Another Ninja wrote:

there are no users - its  trap domains which have never had any real
users - ever.

no prefiltering except rejecting potential bounces and stuff leaking
from whatever may be on DNSWL and a coupleof other WLs. 


Alex,

Your stats are certainly valuable and illustrative... but not reflective
of the stats one would see in a MOST real world mail streams where:


was not the point, as your real world is yours, and not somebody elses.

I specified what those stats showed.. only bot spam. There is no ham, no 
users, no ESP traffic, no bounces, just trash  /dev/null

Re: URIBL Notice

[Karsten Bräckelmann]

On Fri, 2010-03-12 at 18:50 -0500, Rob McEwen wrote:
 Your stats are certainly valuable and illustrative... but not reflective
 of the stats one would see in a MOST real world mail streams where:
 
 (A) the spams were sent to actual users (which would be a distinctively
 different mix of spams compared to a pure honeypot stream of spams [...]

Just for comparison, below are some stats gathered quickly from 2
different and entirely unrelated systems. Real mail stream, real users
only, no traps.

RANKRULE NAME   COUNT  %OFMAIL %OFSPAM  %OFHAM
--
  11URIBL_BLACK  120262.73   63.940.00
   8URIBL_BLACK   57241.12   78.360.00

Unfortunately, both of these systems are still 3.2.5, so there is no
util_rb_3tld love either, which would drive up the %spam numbers quite a
bit.

Oh, and there are some custom rules in place, which take the highest
ranks in these lists, pushing URIBL and all other stock rules down. So
don't base on the rank...


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL Notice

[Chris Owen]

On Mar 12, 2010, at 6:17 PM, Karsten Bräckelmann wrote:

 Just for comparison, below are some stats gathered quickly from 2
 different and entirely unrelated systems. Real mail stream, real users
 only, no traps.

Here are mine from yesterday while we are at it:

--
RANKRULE NAME   COUNT  %OFMAIL %OFSPAM  %OFHAM
--
  1 URIBL_INVALUEMENT   1803736.19   78.780.81
  2 RCVD_IN_INVALUEMENT 1728834.44   75.510.31
  3 HTML_MESSAGE1620876.97   70.80   82.09
  4 BAYES_991542130.57   67.360.01
  5 RCVD_IN_JMF_BL  1464529.65   63.971.14
  6 URIBL_BLACK 1393428.17   60.861.00
  7 RCVD_IN_INVALUEMENT24   1338226.57   58.450.07
  8 MIME_HTML_ONLY  1050429.07   45.88   15.11
  9 URIBL_JP_SURBL   836416.59   36.530.02
 10 RCVD_IN_SORBS_HTTP   813621.39   35.549.63

Real users.Lots and lots of RBL greylisting in front of it.

Chris

-
Chris Owen - Garden City (620) 275-1900 -  Lottery (noun):
President  - Wichita (316) 858-3000 -A stupidity tax
Hubris Communications Inc  www.hubris.net
-

Re: URIBL Notice

[Karsten Bräckelmann]

On Sat, 2010-03-13 at 01:17 +0100, Karsten Bräckelmann wrote:
 RANKRULE NAME   COUNT  %OFMAIL %OFSPAM  %OFHAM
 --
8  URIBL_BLACK   57241.12   78.360.00
 
 Unfortunately, both of these systems are still 3.2.5, so there is no
 util_rb_3tld love either, which would drive up the %spam numbers quite a
 bit.
 
 Oh, and there are some custom rules in place, which take the highest
 ranks in these lists, pushing URIBL and all other stock rules down. So
 don't base on the rank...

To substantiate this:  Bayes_99, Razor (3 small rules, optional in stock
SA). Plus 3 other custom rules. URIBL.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL Removal [OT]

[Karsten Bräckelmann]

On Sat, 2008-12-13 at 21:23 -0800, Marc Perkel wrote:
 I'm trying to get collaborate.com off of the URIBL list and I've 
 submitted it for removal several times and nothing happens.

Log in to your URIBL account, then see the track link, more verbosely
named Track Your Submissions in the page content. There you can see
all your submissions, where if any they are listed, and if the
submission has been acknowledged or a brief reason for denial.

 Does anyone know why removal doesn't work?

My guess is that it *does* work -- your request probably just has been
denied. Likely will be de-listed once mail with this domain ceases to
hit the spam-traps.

  guenther

-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL Removal [OT]

[Karsten Bräckelmann]

On Sun, 2008-12-14 at 13:42 +0100, Karsten Bräckelmann wrote:
 On Sat, 2008-12-13 at 21:23 -0800, Marc Perkel wrote:
  I'm trying to get collaborate.com off of the URIBL list and I've 
  submitted it for removal several times and nothing happens.
 
 Log in to your URIBL account, then see the track link, more verbosely
 named Track Your Submissions in the page content. There you can see
 all your submissions, where if any they are listed, and if the
 submission has been acknowledged or a brief reason for denial.
 
  Does anyone know why removal doesn't work?

Oh, and btw, it is not listed currently.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URIBL Removal [OT]

[Marc Perkel]

My fault - never mind. I was doing something wrong.

Karsten Bräckelmann wrote:

On Sat, 2008-12-13 at 21:23 -0800, Marc Perkel wrote:
  
I'm trying to get collaborate.com off of the URIBL list and I've 
submitted it for removal several times and nothing happens.



Log in to your URIBL account, then see the track link, more verbosely
named Track Your Submissions in the page content. There you can see
all your submissions, where if any they are listed, and if the
submission has been acknowledged or a brief reason for denial.

  

Does anyone know why removal doesn't work?



My guess is that it *does* work -- your request probably just has been
denied. Likely will be de-listed once mail with this domain ceases to
hit the spam-traps.

  guenther

  

Re: Uribl for myself

[mouss]

Kris Deugau wrote:

[EMAIL PROTECTED] wrote:

Hello,

i want to start my own local uribl.

Spamassassin should read a raw-textfile for example 
/home/spamblack.txt where some url's are in


wunschurlaub.biz
euromillion.de

and another..

If match one of these entries, the Mail should marked with X Points.

How do i implement this?


Mmm.  To literally match URIs against a local flat text file, you'll 
need to write a plugin that does just that.


maybe by modifying the accessdb plugin. I didn't check this though.



However, it's probably easier to do one of two things:

1) Create a script that takes the flatfile and writes out a SpamAssassin 
config file with URI rules for each URI in a suitable location


2) Create a script to stuff the URIs into a DNS zone of your own, and 
duplicate a suitable configuration fragment from eg URIBL or SURBL.




and here, rbldnsd may be a good choice if the list becomes large.

I've done both of these;  the first for a while with SA2.5x, the second 
starting with SA2.63 shortly after a patch appeared to look up URIs in 
DNS instead of relying on great long lists of URI rules.


-kgd

Re: Uribl for myself

[Yet Another Ninja]

On 10/2/2008 10:18 PM, [EMAIL PROTECTED] wrote:

Hello,

i want to start my own local uribl.

Spamassassin should read a raw-textfile for example /home/spamblack.txt 
where some url's are in


wunschurlaub.biz
euromillion.de

and another..

If match one of these entries, the Mail should marked with X Points.

How do i implement this?


Using rbldnsd you can run a full featured RBL
http://www.corpit.ru/mjt/rbldnsd.html

or if its only a few URIs its simpler and cheaper to use a couple of uri 
rules

Re: Uribl for myself

[Kris Deugau]

[EMAIL PROTECTED] wrote:

Hello,

i want to start my own local uribl.

Spamassassin should read a raw-textfile for example /home/spamblack.txt 
where some url's are in


wunschurlaub.biz
euromillion.de

and another..

If match one of these entries, the Mail should marked with X Points.

How do i implement this?


Mmm.  To literally match URIs against a local flat text file, you'll 
need to write a plugin that does just that.


However, it's probably easier to do one of two things:

1) Create a script that takes the flatfile and writes out a SpamAssassin 
config file with URI rules for each URI in a suitable location


2) Create a script to stuff the URIs into a DNS zone of your own, and 
duplicate a suitable configuration fragment from eg URIBL or SURBL.


I've done both of these;  the first for a while with SA2.5x, the second 
starting with SA2.63 shortly after a patch appeared to look up URIs in 
DNS instead of relying on great long lists of URI rules.


-kgd

RE: URIBL

[Jeff Chan]

Quoting Rocco Scappatura [EMAIL PROTECTED]:


Maybe, now is the case to set up a copy of zone locally on my server.. I
ve about 1300K messages rejected per day!!


Yes, you should not query 1.3 million messages per day on the public  
nameservers.  That would be considered abusive.


Jeff C.

RE: URIBL

[Rocco Scappatura]

 Quoting Rocco Scappatura [EMAIL PROTECTED]:

 Maybe, now is the case to set up a copy of zone locally on my server.. I
 ve about 1300K messages rejected per day!!

 Yes, you should not query 1.3 million messages per day on the public
 nameservers.  That would be considered abusive.

Je suis desolee.. I will try to to implement the SURBL zone copy during
the next days.. Should this improve the performance of message scan?

rocsca

RE: URIBL

[Rocco Scappatura]

 I have to 
  enable only the plugin with loadPlugin.
 
 ... and it's enabled by default, so you should be all set. :)
 
  Then I have to use the command 'urirhssub' of the plugin 
 URIDNSBL to 
  specify that I want to use SURBLs:
 
 ... the rules exist by default, so you should be all set. :)

OK. So the SURBL on my gateway should already work.. But how could I
check this fact?

rocsca

RE: URIBL

[Jeff Chan]

Quoting Rocco Scappatura [EMAIL PROTECTED]:


I have to
 enable only the plugin with loadPlugin.

... and it's enabled by default, so you should be all set. :)

 Then I have to use the command 'urirhssub' of the plugin
URIDNSBL to
 specify that I want to use SURBLs:

... the rules exist by default, so you should be all set. :)


OK. So the SURBL on my gateway should already work.. But how could I
check this fact?

rocsca



You should see many spams with the rules named SURBL hitting.  You can  
also try:


  spamassassin -D  message

where message contains one of the testpoints:

  http://www.surbl.org/faq.html#test-uris

Jeff C.

RE: URIBL

[Rocco Scappatura]

 Quoting Rocco Scappatura [EMAIL PROTECTED]:

 I have to
  enable only the plugin with loadPlugin.

 ... and it's enabled by default, so you should be all set. :)

  Then I have to use the command 'urirhssub' of the plugin
 URIDNSBL to
  specify that I want to use SURBLs:

 ... the rules exist by default, so you should be all set. :)

 OK. So the SURBL on my gateway should already work.. But how could I
 check this fact?

 rocsca


 You should see many spams with the rules named SURBL hitting.  You can
 also try:

spamassassin -D  message

Infact..

X-Spam-Status: Yes, score=9.573 tag=2 tag2=6.2 kill=6.31
tests=[ALL_TRUSTED=-1.8, AWL=0.583, BAYES_80=2, HTML_MESSAGE=0.001,
URIBL_AB_SURBL=1.86, URIBL_BLACK=1.955, URIBL_JP_SURBL=1.501,
URIBL_OB_SURBL=1.5, URIBL_SBL=1.499, URIBL_SC_SURBL=0.474]

SURBL works!

Maybe, now is the case to set up a copy of zone locally on my server.. I
ve about 1300K messages rejected per day!!

Even though my customers complain a lot of false negative.. What I can do
more??

Thanks,

rocsca

Re: URIBL

[Dave Koontz]

I remember there was a period of time when dozens of  URI delist 
requests were submitted all together without any detail.  Could that 
have been the case with your reports?


Theo Van Dinter wrote:

FWIW, I used to report FP domains to URIBL daily until I was told to
stop because there were too many to deal with.

RE: URIBL

[Rocco Scappatura]

 From: Theo Van Dinter [mailto:[EMAIL PROTECTED] 
 Sent: Wednesday, February 20, 2008 8:08 PM
 To: users@spamassassin.apache.org
 Subject: Re: URIBL
 
 On Wed, Feb 20, 2008 at 06:52:14PM +, Nigel Frankcom wrote:
  Anyway I heard talking about URIBL, which as I have understod is a 
  quite different service (it blacklists 'domains' rather 
 'IPs'). But 
  is it maybe a dangerous practice to fight spam? Anyway, 
 does anyone 
  suggest me to use URIBL?
 
 URI black lists have been around for several years now, and 
 are generally very helpful at detecting spam.  URIBL is one 
 of the standard such black lists that are in use in SA, but 
 there are others: SURBL (the oldest and most well known
 IMO) as well as Razor (also does message hashing but largely 
 uses domain detection these days).  (I may be forgetting 
 someone else, sorry, these are just the ones that come to mind.)
 
 Here are my results for the past 60 days for the different groups:
 
 (you want the most spam% with the lowest ham%, aka: the 
 higher the S/O the
 better)
 
 OVERALLSPAM% HAM% S/ORANK   SCORE  NAME
   0   769001570130.931   0.000.00  (all messages)
 0.0  93.0978   6.90220.931   0.000.00  (all messages as %)
 
  65.312  70.1541   0.00531.000   1.000.00  URIBL_JP_SURBL
  54.979  59.0545   0.00181.000   0.990.00  URIBL_SC_SURBL
  33.513  35.9976   0.00181.000   0.980.00  URIBL_AB_SURBL
  58.407  62.7323   0.06670.999   0.940.00  URIBL_OB_SURBL
  43.120  46.3111   0.07370.998   0.930.00  URIBL_WS_SURBL
   1.385   1.4874   0.00350.998   0.870.00  URIBL_PH_SURBL
 
   0.758   0.8091   0.07020.920   0.780.00  URIBL_RED
  71.920  77.1604   1.23310.984   0.710.00  URIBL_BLACK
   1.545   1.4891   2.30470.393   0.520.00  URIBL_GREY
 
  69.598  74.7537   0.06140.999   0.950.00  
 RAZOR2_CF_RANGE_E8_51_100
 
 
 So URIBL is a bit more problematic than the others by itself, 
 due to the high ham hit rate, but given SA's method of using 
 multiple data sources to determine ham/spam, the false 
 positive issue is minimized.
 

I have looked at the SURBL site. If I have well understood I have to
enable only the plugin with loadPlugin.

Then I have to use the command 'urirhssub' of the plugin URIDNSBL to
specify that I want to use SURBLs:

urirhssub URIBL_JP_SURBL  multi.surbl.org.A   64  
body  URIBL_JP_SURBL  eval:check_uridnsbl('URIBL_JP_SURBL')
describe  URIBL_JP_SURBL  Has URI in JP at
http://www.surbl.org/lists.html
tflagsURIBL_JP_SURBL  net

score URIBL_JP_SURBL3.0

Indeed, I have not understood a number of things:

1. Why I have to use 'URIBL_JP_SURBL' as 'NAME_OF_RULE'? Is it an
arbitrary name or it exists a number of 'NAME_OF_RULE'?
2. Does the body command have to specify
'eval:check_uridnsbl('NAME_OF_RULE')' where 'NAME_OF_RULE' is the name
of the rule specified as parameter of the command 'urirhssub'?
3. tflags?
4. score?
5. Is there any simpler URIDNSBL plugin setting? Maybe a default one?

rocsca

RE: URIBL

[Rocco Scappatura]

  Anyway I heard talking about URIBL, which as I have understod is a 
  quite different service (it blacklists 'domains' rather 
 'IPs'). But is 
  it maybe a dangerous practice to fight spam? Anyway, does anyone 
  suggest me to use URIBL?
 
 Are you looking for a PRE QUEUE blacklist? Or a way to help 
 score SpamAssassin emails?
 
 URIBL (I think from spamcop/ironport/cisco) is already 
 included in modern SA builds.

I don't know what you mean for 'PRE QUEUE blacklist'.. Anyway I would
like to help SpamAssassin in scoring emails..

rocsca

Re: URIBL

[Luis Hernán Otegui]

HI, Rocco

2008/2/21, Rocco Scappatura [EMAIL PROTECTED]:
   Anyway I heard talking about URIBL, which as I have understod is a
quite different service (it blacklists 'domains' rather
   'IPs'). But is
it maybe a dangerous practice to fight spam? Anyway, does anyone
suggest me to use URIBL?
  
   Are you looking for a PRE QUEUE blacklist? Or a way to help
   score SpamAssassin emails?
  
   URIBL (I think from spamcop/ironport/cisco) is already
   included in modern SA builds.


 I don't know what you mean for 'PRE QUEUE blacklist'.. Anyway I would
  like to help SpamAssassin in scoring emails..


He means a blacklist which runs IN the MTA, not at SA level, when the
MTA has accepted the message. It rejects spammers as they connect,
mostly based on their IP. I run Zen, from Spamhaus here, with very
good results.
  rocsca


Regards,


Luis
-- 
-
GNU-GPL: May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-

RE: URIBL

[Jeff Chan]

Quoting Rocco Scappatura [EMAIL PROTECTED]:



I have looked at the SURBL site. If I have well understood I have to
enable only the plugin with loadPlugin.

Then I have to use the command 'urirhssub' of the plugin URIDNSBL to
specify that I want to use SURBLs:

urirhssub URIBL_JP_SURBL  multi.surbl.org.A   64
body  URIBL_JP_SURBL  eval:check_uridnsbl('URIBL_JP_SURBL')
describe  URIBL_JP_SURBL  Has URI in JP at
http://www.surbl.org/lists.html
tflagsURIBL_JP_SURBL  net

score URIBL_JP_SURBL3.0

Indeed, I have not understood a number of things:

1. Why I have to use 'URIBL_JP_SURBL' as 'NAME_OF_RULE'? Is it an
arbitrary name or it exists a number of 'NAME_OF_RULE'?
2. Does the body command have to specify
'eval:check_uridnsbl('NAME_OF_RULE')' where 'NAME_OF_RULE' is the name
of the rule specified as parameter of the command 'urirhssub'?
3. tflags?
4. score?
5. Is there any simpler URIDNSBL plugin setting? Maybe a default one?

rocsca




If you want to use SURBL and URIBL all you need to do is enable network tests:

  http://www.surbl.org/faq.html#nettest

URI checking is built into SpamAssassin.

Jeff C.

RE: URIBL

[Rocco Scappatura]

 HI, Rocco

Hi Luis,

  I don't know what you mean for 'PRE QUEUE blacklist'.. 
 Anyway I would  
  like to help SpamAssassin in scoring emails..
 
 
 He means a blacklist which runs IN the MTA, not at SA level, 
 when the MTA has accepted the message. It rejects spammers as 
 they connect, mostly based on their IP. I run Zen, from 
 Spamhaus here, with very good results.

Indeed, I'm using PRE QUEUE blacklist too (Zen from spamhaus, like you).

I get appreciable results, but during the last days I get an huge increase of 
rejected emails, but at the same time I get a major number of false negative.

So I want to lower the number of false negative.

rocsca

Re: URIBL

[Theo Van Dinter]

On Thu, Feb 21, 2008 at 09:57:17AM +0100, Rocco Scappatura wrote:
 I have looked at the SURBL site. If I have well understood I have to
 enable only the plugin with loadPlugin.

... and it's enabled by default, so you should be all set. :)

 Then I have to use the command 'urirhssub' of the plugin URIDNSBL to
 specify that I want to use SURBLs:

... the rules exist by default, so you should be all set. :)

 1. Why I have to use 'URIBL_JP_SURBL' as 'NAME_OF_RULE'? Is it an
 arbitrary name or it exists a number of 'NAME_OF_RULE'?

Rule names are arbitrary, but usually descriptive of what they do.
URIBL_JP_SURBL means it's a URIBL rule, using the SURBL JP information.

 3. tflags?

$ perldoc Mail::SpamAssassin::Conf

 4. score?

See tflags.  It's the score added to the message's total if the rule hits.

 5. Is there any simpler URIDNSBL plugin setting? Maybe a default one?

SURBL and URIBL are enabled by default.  If you want to add your own for some
other one, you can do that, but get your feet wet before you jump in. :)

-- 
Randomly Selected Tagline:
A Smith  Wesson beats four aces.


pgpNJtqS6Llt2.pgp
Description: PGP signature

RE: URIBL

[Rocco Scappatura]

 Quoting Rocco Scappatura [EMAIL PROTECTED]:
 
 
  I have looked at the SURBL site. If I have well understood 
 I have to 
  enable only the plugin with loadPlugin.
 
  Then I have to use the command 'urirhssub' of the plugin 
 URIDNSBL to 
  specify that I want to use SURBLs:
 
  urirhssub URIBL_JP_SURBL  multi.surbl.org.A   64
  body  URIBL_JP_SURBL  eval:check_uridnsbl('URIBL_JP_SURBL')
  describe  URIBL_JP_SURBL  Has URI in JP at 
  http://www.surbl.org/lists.html
  tflagsURIBL_JP_SURBL  net
 
  score URIBL_JP_SURBL3.0
 
  Indeed, I have not understood a number of things:
 
  1. Why I have to use 'URIBL_JP_SURBL' as 'NAME_OF_RULE'? Is it an 
  arbitrary name or it exists a number of 'NAME_OF_RULE'?
  2. Does the body command have to specify 
  'eval:check_uridnsbl('NAME_OF_RULE')' where 'NAME_OF_RULE' 
 is the name 
  of the rule specified as parameter of the command 'urirhssub'?
  3. tflags?
  4. score?
  5. Is there any simpler URIDNSBL plugin setting? Maybe a 
 default one?
 
  rocsca
 
 
 
 If you want to use SURBL and URIBL all you need to do is 
 enable network tests:
 
http://www.surbl.org/faq.html#nettest
 
 URI checking is built into SpamAssassin.

$sa_local_tests_only = 0;

I have already set in /etc/amavisd.conf:

$sa_local_tests_only = 0;

So you say that SURBL is already set?

rocsca

Re: URIBL

[Nigel Frankcom]

On Wed, 20 Feb 2008 16:40:33 +0100, Rocco Scappatura
[EMAIL PROTECTED] wrote:

During last days I have noticed an increasing of 'rejected' messages.

I'm currently using 'zen.spamhaus.org' and 'list.dsbl.org' as reputation
servers.

At the same time, the number of false negative is growth.

I would like to know if is there any better reputation server that
anyone know (of course, it would be nice if it is a free service :-)).

Anyway I heard talking about URIBL, which as I have understod is a quite
different service (it blacklists 'domains' rather 'IPs'). But is it
maybe a dangerous practice to fight spam? Anyway, does anyone suggest me
to use URIBL?

Thanks,

rocsca

Hi,

Try Googling spamassassin backscatter or take a look at
http://www.rulesemporium.com/rules.htm there's some handy stuff there
but READ THE DOCS...

For what it's worth I'm seeing an escalation here in the UK and on US
and AUS servers so it's not isolated. Admittedly it's not a large
proportion but it is a rise.

HTH

Nigel

Re: URIBL

[Theo Van Dinter]

On Wed, Feb 20, 2008 at 06:52:14PM +, Nigel Frankcom wrote:
 Anyway I heard talking about URIBL, which as I have understod is a quite
 different service (it blacklists 'domains' rather 'IPs'). But is it
 maybe a dangerous practice to fight spam? Anyway, does anyone suggest me
 to use URIBL?

URI black lists have been around for several years now, and are generally very
helpful at detecting spam.  URIBL is one of the standard such black lists that
are in use in SA, but there are others: SURBL (the oldest and most well known
IMO) as well as Razor (also does message hashing but largely uses domain
detection these days).  (I may be forgetting someone else, sorry, these are
just the ones that come to mind.)

Here are my results for the past 60 days for the different groups:

(you want the most spam% with the lowest ham%, aka: the higher the S/O the
better)

OVERALLSPAM% HAM% S/ORANK   SCORE  NAME
  0   769001570130.931   0.000.00  (all messages)
0.0  93.0978   6.90220.931   0.000.00  (all messages as %)

 65.312  70.1541   0.00531.000   1.000.00  URIBL_JP_SURBL
 54.979  59.0545   0.00181.000   0.990.00  URIBL_SC_SURBL
 33.513  35.9976   0.00181.000   0.980.00  URIBL_AB_SURBL
 58.407  62.7323   0.06670.999   0.940.00  URIBL_OB_SURBL
 43.120  46.3111   0.07370.998   0.930.00  URIBL_WS_SURBL
  1.385   1.4874   0.00350.998   0.870.00  URIBL_PH_SURBL

  0.758   0.8091   0.07020.920   0.780.00  URIBL_RED
 71.920  77.1604   1.23310.984   0.710.00  URIBL_BLACK
  1.545   1.4891   2.30470.393   0.520.00  URIBL_GREY

 69.598  74.7537   0.06140.999   0.950.00  RAZOR2_CF_RANGE_E8_51_100


So URIBL is a bit more problematic than the others by itself, due to the
high ham hit rate, but given SA's method of using multiple data sources
to determine ham/spam, the false positive issue is minimized.

-- 
Randomly Selected Tagline:
I'm looking for a Linux equivilant to PC Magazine.   - Brian Dudek
 Unfortunately, this isn't available. Linux-centric magazines tend to
 actually contain useful information. - Chris Saunderson


pgpixW6c8ENKx.pgp
Description: PGP signature

RE: URIBL

[Chris Santerre]

  71.920  77.1604   1.23310.984   0.710.00  URIBL_BLACK

You've always surprised me with your Ham rates Theo. I'm guessing these are
prbly good sites that fell into the affiliate spam category and got
listed. Anyway to pull out the top hitters of Ham and let us know. I'd like
to find out if we overlooked something. 

I'd like to correct this if it is an issue. 

Thanks, 

--Chris

Re: URIBL

[Theo Van Dinter]

On Wed, Feb 20, 2008 at 02:41:09PM -0500, Chris Santerre wrote:
   71.920  77.1604   1.23310.984   0.710.00  URIBL_BLACK
 
 Anyway to pull out the top hitters of Ham and let us know. I'd like
 to find out if we overlooked something. 
 
 I'd like to correct this if it is an issue. 

FWIW, I used to report FP domains to URIBL daily until I was told to
stop because there were too many to deal with.

-- 
Randomly Selected Tagline:
When you say 'I wrote a program that crashed Windows,' people just stare
 at you blankly and say 'Hey, I got those with the system, *for free*.'
  - Linus Torvalds


pgpnSfMjzllKV.pgp
Description: PGP signature

Re: URIBL

[Michael Scheidell]

 From: Rocco Scappatura [EMAIL PROTECTED]
 Date: Wed, 20 Feb 2008 16:40:33 +0100
 To: users@spamassassin.apache.org
 Conversation: URIBL
 Subject: URIBL
 
 
 Anyway I heard talking about URIBL, which as I have understod is a quite
 different service (it blacklists 'domains' rather 'IPs'). But is it
 maybe a dangerous practice to fight spam? Anyway, does anyone suggest me
 to use URIBL?

Are you looking for a PRE QUEUE blacklist? Or a way to help score
SpamAssassin emails?

URIBL (I think from spamcop/ironport/cisco) is already included in modern SA
builds.

-- 
Michael Scheidell, CTO
|SECNAP Network Security
Winner 2008 Network Products Guide Hot Companies
FreeBsd SpamAssassin Ports maintainer
Charter member, ICSA labs anti-spam consortium

_
This email has been scanned and certified safe by SpammerTrap(tm). 
For Information please see http://www.spammertrap.com
_

Re: URIBL

[Theo Van Dinter]

On Wed, Feb 20, 2008 at 07:03:58PM -0500, Dave Koontz wrote:
 I remember there was a period of time when dozens of  URI delist 
 requests were submitted all together without any detail.  Could that 
 have been the case with your reports?

I'm not sure if it was specifically what you're thinking of, but yes,
for various reasons, I was granted the ability to submit domains as FPs
without providing details about the message (unless specifically asked for
them)..

-- 
Randomly Selected Tagline:
Before marriage, a man yearns for the woman he loves.  After marriage,
 the 'Y' becomes silent.- Unknown


pgpoX719ANFNu.pgp
Description: PGP signature

RE: URIBL

[Rocco Scappatura]

 For what it's worth I'm seeing an escalation here in the UK 
 and on US and AUS servers so it's not isolated. Admittedly 
 it's not a large proportion but it is a rise.

How do you have inferred this?

rocsca

Re: URIBL

[Matt Kettler]

Jason Bertoch wrote:
   Lately I've been trying to report links in spam to uribl.com, obviously
 hoping to increase the hit rate for messages coming my way.  However, I've 
 found
 several occasions where that URL was already listed but the rule didn't 
 trigger.
 Upon further review, I'm not seeing URIBL_BLACK in my mail logs at all.  I
 double checked that I'm loading the plug-in via init.pre, and lint verifies 
 that
 for me.  At this point, I'm not entirely sure where to go next.
   Currently, the following line from 25_uribl.cf has my attention:

 urirhssub   URIBL_BLACK multi.uribl.com.A   2

 If I do a dig on a known listed url at multi.uribl.com, it reports 127.0.0.2.
 Will the above line not match the returned value and cause my issues, or am I
 looking in the wrong place?
   

Do you have Net::DNS installed and working?

try spamassassin -D sample-spam.txt

Does the debug output indicate that DNS is available and working?
 For what it's worth, I'm running SA 3.2.0 with daily sa-updates.
   

RE: URIBL

[Jason Bertoch]

On Wednesday, May 30, 2007 10:05 AM Matt Kettler wrote:

 
 Do you have Net::DNS installed and working?
 
 try spamassassin -D sample-spam.txt
 
 Does the debug output indicate that DNS is available and working?

Yes, Net::DNS is installed and debug output says it's working.  Other DNS-based
tests, such as SPF, are functioning correctly as well.


Jason A. Bertoch
Network Administrator
[EMAIL PROTECTED]
ElectroNet Intermedia Consulting
3411 Capital Medical Blvd.
Tallahassee, FL 32308
(V) 850.222.0229 (F) 850.222.8771

RE: URIBL

[Jason Bertoch]

On Wednesday, May 30, 2007 10:16 AM John Wilcock wrote:

 Jason Bertoch wrote:
 Yes, Net::DNS is installed and debug output says it's working. 
 Other DNS-based tests, such as SPF, are functioning correctly as
 well. 
 
 Is Mail::SpamAssassin::Plugin::URIDNSBL enabled in your init.pre file?
 ISTR it is commented out in a default 3.2.0 installation.
 
 John.

Yes, it is being loaded via init.pre.  There is another part of the 25_uribl.cf
that has caught my attention:

 Note that this plugin defines a new config setting, 'uridnsbl',
 which lists the zones to look up in advance.  The rules will
 not hit unless each rule has a corresponding 'uridnsbl' line.


The only uridnsbl line in 25_uribl.cf refers to sbl.spamhaus.org.  Throughout
the entire default ruleset there is no uridnsbl line corresponding to
multi.surbl.org.  The debug output below seems to confirm that SA is not going
to query multi.surbl.org.


[25188] dbg: uridnsbl: domains to query: 
[25188] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
[25188] dbg: dns: checking RBL combined.njabl.org., set njabl
[25188] dbg: dns: checking RBL bl.spamcop.net., set spamcop
[25188] dbg: dns: checking RBL dob.sibl.support-intelligence.net., set dob
[25188] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-lastexternal
[25188] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs
[25188] dbg: dns: checking RBL zen.spamhaus.org., set zen-lastexternal
[25188] dbg: dns: checking RBL sa-accredit.habeas.com., set habeas-firsttrusted
[25188] dbg: dns: checking RBL combined-HIB.dnsiplists.completewhois.com., set
whois
[25188] dbg: dns: checking RBL list.dsbl.org., set dsbl-lastexternal
[25188] dbg: dns: checking RBL sa-trusted.bondedsender.org., set
bsp-firsttrusted
[25188] dbg: dns: checking RBL combined-HIB.dnsiplists.completewhois.com., set
whois-lastexternal
[25188] dbg: dns: checking RBL zen.spamhaus.org., set zen
[25188] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted




Jason A. Bertoch
Network Administrator
[EMAIL PROTECTED]
ElectroNet Intermedia Consulting
3411 Capital Medical Blvd.
Tallahassee, FL 32308
(V) 850.222.0229 (F) 850.222.8771

Re: URIBL

[Theo Van Dinter]

On Wed, May 30, 2007 at 10:52:09AM -0400, Jason Bertoch wrote:
 multi.surbl.org.  The debug output below seems to confirm that SA is not going
 to query multi.surbl.org.

Of course not...

 [25188] dbg: uridnsbl: domains to query: 

There are no domains to query for, so it doesn't.

-- 
Randomly Selected Tagline:
I... I'm touched.  I fear you're a bit touched as well. - Benjy Feen


pgpx4CCpimt0K.pgp
Description: PGP signature

RE: URIBL

[Jason Bertoch]

Dangit...wish replies were sent back to the list.  Resending for everyone else
to see...

On Wednesday, May 30, 2007 11:02 AM Theo Van Dinter wrote:

 On Wed, May 30, 2007 at 10:52:09AM -0400, Jason Bertoch wrote:
 multi.surbl.org.  The debug output below seems to confirm that SA is
 not going to query multi.surbl.org.
 
 Of course not...
 
 [25188] dbg: uridnsbl: domains to query:
 
 There are no domains to query for, so it doesn't.

Fair enough, the sample-spam.txt file doesn't contain a URL.  The problem still
stands, though.  Again, I'd like to refer to the line from 25_uribl.cf:

urirhssub   URIBL_BLACK multi.uribl.com.A   2

If I change the 2 to 127.0.0.2 in 25_uribl.cf URIBL_BLACK begins to hit.

Filing a bug report.

Jason A. Bertoch
Network Administrator
[EMAIL PROTECTED]
ElectroNet Intermedia Consulting
3411 Capital Medical Blvd.
Tallahassee, FL 32308
(V) 850.222.0229 (F) 850.222.8771

Re: URIBL

[Daniel J McDonald]

On Wed, 2007-05-30 at 11:02 -0400, Theo Van Dinter wrote:
 On Wed, May 30, 2007 at 10:52:09AM -0400, Jason Bertoch wrote:
  multi.surbl.org.  The debug output below seems to confirm that SA is not 
  going
  to query multi.surbl.org.
 
 Of course not...
 
  [25188] dbg: uridnsbl: domains to query: 
 
 There are no domains to query for, so it doesn't.

Ok, here's one that does fail:
under 3.2.0:
[16543] dbg: uridnsbl: domain theauthenticmemento.com listed
(URIBL_RHS_URIBL_BLACK): 127.0.0.2
[16543] dbg: uridnsbl: query for theauthenticmemento.com took 2 seconds
to look up (multi.uribl.com.:theauthenticmemento.com)
[16543] dbg: async: queries completed: 1 started: 0
[16543] dbg: async: queries active: DNSBL-A=7 DNSBL-TXT=3 URI-DNSBL=3
URI-NS=1 at Wed May 30 11:25:11 2007
[16543] dbg: async: select found 1 socks ready
[16543] dbg: uridnsbl: domain theauthenticmemento.com listed
(URIBL_OB_SURBL): 127.0.0.16
[16543] dbg: dns: URIBL_OB_SURBL lookup finished
[16543] dbg: uridnsbl: query for theauthenticmemento.com took 2 seconds
to look up (multi.surbl.org.:theauthenticmemento.com)
...
[16543] dbg: check:
tests=DKIM_POLICY_SIGNSOME,HTML_IMAGE_RATIO_04,HTML_MESSAGE,INVALID_DATE,L_P0F_W,MIME_HTML_ONLY,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RELAY_US,SARE_UNA,URIBL_OB_SURBL
[16543] dbg: check:
subtests=__CD,__CT,__CTE,__CTYPE_HTML,__DOS_HAS_ANY_URI,__DOS_RCVD_WED,__DOS_SINGLE_EXT_RELAY,__EXCLAIM_SUBJ,__FB_MA,__FB_S_PRICE,__FM_MY_PRICE,__HAS_ANY_URI,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HTML_LINK_IMAGE,__MIME_HTML,__MIME_VERSION,__MISSING_REF,__MSGID_OK_HOST,__NAKED_TO,__NONEMPTY_BODY,__SANE_MSGID,__SARE_HAS_BG_COLOR,__SARE_HAS_FG_COLOR,__SARE_HTML_HAS_A,__SARE_HTML_HAS_BR,__SARE_HTML_HAS_DIV,__SARE_HTML_HAS_FONT,__SARE_HTML_HAS_IMG,__SARE_HTML_HAS_P,__SARE_HTML_HAS_TITLE,__SARE_URI_ANY,__SARE_WHITE_BG_COLOR,__SUBJ_3DIGIT,__TAG_EXISTS_BODY,__TAG_EXISTS_CENTER,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS


Debug says URIBL BLACK matched, but it does not get scored

Under 3.1.8:
[19829] dbg: uridnsbl: domain theauthenticmemento.com listed
(URIBL_OB_SURBL): 127.0.0.16
[19829] dbg: uridnsbl: query for theauthenticmemento.com took 2 seconds
to look up (multi.surbl.org.:theauthenticmemento.com)
[19829] dbg: uridnsbl: queries completed: 1 started: 0
[19829] dbg: uridnsbl: queries active: A=4 DNSBL=1 at Wed May 30
11:35:28 2007
[19829] dbg: uridnsbl: select found 1 socks ready
[19829] dbg: uridnsbl: domain theauthenticmemento.com listed
(URIBL_BLACK): 127.0.0.2
[19829] dbg: uridnsbl: query for theauthenticmemento.com took 2 seconds
to look up (multi.uribl.com.:theauthenticmemento.com)
...
[19829] dbg: check:
tests=HTML_MESSAGE,HTML_TAG_EXIST_TBODY,INVALID_DATE,MANY_EXCLAMATIONS,MIME_HTML_ONLY,RAZOR2_CF_RANGE_51_100,RAZOR2_CF_RANGE_E4_51_100,RAZOR2_CF_RANGE_E8_51_100,RAZOR2_CHECK,RELAYCOUNTRY_US,SARE_UNA,SPF_HELO_PASS,URIBL_BLACK,URIBL_OB_SURBL
[19829] dbg: check:
subtests=__CD,__CT,__CTE,__CTYPE_HTML,__ENV_AND_HDR_FROM_MATCH,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HTML_LINK_IMAGE,__MANY_EXCLS,__MIME_HTML,__MIME_VERSION,__MSGID_OK_HOST,__NAKED_TO,__NONEMPTY_BODY,__SANE_MSGID,__SARE_HAS_BG_COLOR,__SARE_HAS_FG_COLOR,__SARE_HTML_BEHTML2,__SARE_HTML_HAS_A,__SARE_HTML_HAS_BR,__SARE_HTML_HAS_DIV,__SARE_HTML_HAS_FONT,__SARE_HTML_HAS_IMG,__SARE_HTML_HAS_P,__SARE_HTML_HAS_TITLE,__SARE_URI_ANY,__SARE_WHITE_BG_COLOR,__TAG_EXISTS_BODY,__TAG_EXISTS_CENTER,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS

Debug says URIBL BLACK matched, and it is scored.

-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
Austin Energy
http://www.austinenergy.com