Re: heads up for false uribl black hits

[Sidney Markowitz]

Benny Pedersen wrote on 21/05/21 4:59 am:

only place i find it https://spameatingmonkey.com/lookup/libehat



Spameatingmonkey lists it as "This domain was first registered within 
the last 30 days  Listings automatically expire in less than 30 days"


It was registered on April 23. Maybe see how it looks next week in case 
that's the problem with the other listings.

Re: heads up for false uribl black hits

[Sidney Markowitz]

John Hardin wrote on 21/05/21 2:28 am:


Odd, the URIBL website lookup tool says libera (.chat) is not listed,
and didn't yesterday when you first posted this.

https://admin.uribl.com/


Lookup Results (obfuscated just in case)
Domain  Status
libera_chat NOT Listed on URIBL


Is that not working correctly?




I just tried again, both on https://uribl.com that I used before and 
admin.uribl.com like you did, both with identical results, same as I got 
yesterday with the addition of a second "pending removal from block"


Lookup Results
Domain  Status  Manage
libra_chat  Listed on URIBL black

Pending Removal from black Pending Removal from black (details)
Pending Removal from black Pending Removal from black (details)

I wonder if their removal requests are processed automatically and their 
adding to the block list also happens automatically from the 
sbc.spamhaus list? So it will keep popping on and off?

Re: heads up for false uribl black hits

[John Hardin]

On Thu, 20 May 2021, Riccardo Alfieri wrote:


On 20/05/21 18:59, Benny Pedersen wrote:




Is that not working correctly?


only place i find it https://spameatingmonkey.com/lookup/libera.chat


Hi,

by checking: http://multirbl.valli.org/lookup/libera.chat.html

it looks like that is indeed listed on URIBL too: 
http://lookup.uribl.com/?domain=libera.chat


Ot at least it is *now* , maybe it comes and goes for some reasons


...and now it's listed at https://admin.uribl.com/ as well.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.org pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  To be civilized is to restrain the ability to commit mayhem.
  To be incapable of committing mayhem is not the mark of the
  civilized, merely the domesticated.-- Trefor Thomas
---
 355 days since the first private commercial manned orbital mission (SpaceX)

Re: heads up for false uribl black hits

[Riccardo Alfieri]

On 20/05/21 18:59, Benny Pedersen wrote:




Is that not working correctly?


only place i find it https://spameatingmonkey.com/lookup/libera.chat


Hi,

by checking: http://multirbl.valli.org/lookup/libera.chat.html

it looks like that is indeed listed on URIBL too: 
http://lookup.uribl.com/?domain=libera.chat


Ot at least it is *now* , maybe it comes and goes for some reasons

--
Best regards,
Riccardo Alfieri

Spamhaus Technology
https://www.spamhaustech.com/

Re: heads up for false uribl black hits

[Benny Pedersen]

On 2021-05-20 16:28, John Hardin wrote:

On Thu, 20 May 2021, Noel Butler wrote:



Odd, the URIBL website lookup tool says libera (.chat) is not listed,
and didn't yesterday when you first posted this.



Is that not working correctly?


only place i find it https://spameatingmonkey.com/lookup/libera.chat

Re: heads up for false uribl black hits

[John Hardin]

On Thu, 20 May 2021, Noel Butler wrote:


On 20/05/2021 11:58, Bill Cole wrote:


On 2021-05-19 at 21:13:41 UTC-0400 (Thu, 20 May 2021 11:13:41 +1000)
Noel Butler 
is rumored to have said:

By now most of you are aware of the hostile takeover of freenode and the 
mass exodus that's currently underway (if not  see kline.sh for more) [1]


Interestingly it seems uribl.com has the replacement, Im going to 
obfuscate it else you wont likely see this :)  just replace digits with 
their alpha  lib3ra dott ch4t


in their listings, interesting because they dont seem to list new domains 
that way and that one is new, heh maybe andrew lee controls that too, who 
knows...


The new domain was NOT listed in any RHSBL at 13:55 UTC.

OTOH, they didn't like something about my usual single-venue address 
pattern so I had to register with an alternative tagging pattern.


still listed in URI
Domain Status Manage
libe.cxxx   Listed on URIBL black


Odd, the URIBL website lookup tool says libera (.chat) is not listed, 
and didn't yesterday when you first posted this.


  https://admin.uribl.com/


  Lookup Results (obfuscated just in case)
  DomainStatus
  libera_chat   NOT Listed on URIBL


Is that not working correctly?


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.org pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
 355 days since the first private commercial manned orbital mission (SpaceX)

Re: heads up for false uribl black hits

[Sidney Markowitz]

Bill Cole wrote on 20/05/21 1:58 pm:

The new domain was NOT listed in any RHSBL at 13:55 UTC.


The first of its four ip addresses, 185.199.108.153, is on 
sbl.spamhaus.org but not the domain name.


That is the only match that shows up in the list of RBLs checked at 
ant-abuse.org Multi-RBL Check, which seems to convert to ip address for 
its checks, or maybe checks each by both name and ip address.


The domain, but not the ip address, does show up at uribl.com. They 
don't provide any explanation for entries on the list.

Re: heads up for false uribl black hits

[Noel Butler]

On 20/05/2021 11:58, Bill Cole wrote:


On 2021-05-19 at 21:13:41 UTC-0400 (Thu, 20 May 2021 11:13:41 +1000)
Noel Butler 
is rumored to have said:

By now most of you are aware of the hostile takeover of freenode and 
the mass exodus that's currently underway (if not  see kline.sh for 
more) [1]


Interestingly it seems uribl.com has the replacement, Im going to 
obfuscate it else you wont likely see this :)  just replace digits 
with their alpha  lib3ra dott ch4t


in their listings, interesting because they dont seem to list new 
domains that way and that one is new, heh maybe andrew lee controls 
that too, who knows...


The new domain was NOT listed in any RHSBL at 13:55 UTC.

OTOH, they didn't like something about my usual single-venue address 
pattern so I had to register with an alternative tagging pattern.


still listed in URI
Domain Status Manage
libe.cxxx   Listed on URIBL black

at 02:46 UTC

someone has made a delist request about 8 hours ago though

strange  that a service that has a policy of not saying why they list is 
included in default SA


(btw - I have no affiliation with either party - I'm just mentioning it 
here since its where I found my confirm request)


--
Regards,
Noel Butler

This Email, including attachments, may contain legally privileged 
information, therefore at all times remains confidential and subject to 
copyright protected under international law. You may not disseminate 
this message without the authors express written authority to do so.   
If you are not the intended recipient, please notify the sender then 
delete all copies of this message including attachments immediately. 
Confidentiality, copyright, and legal privilege are not waived or lost 
by reason of the mistaken delivery of this message.

Re: heads up for false uribl black hits

[Bill Cole]

On 2021-05-19 at 21:13:41 UTC-0400 (Thu, 20 May 2021 11:13:41 +1000)
Noel Butler 
is rumored to have said:

By now most of you are aware of the hostile takeover of freenode and 
the mass exodus that's currently underway (if not  see kline.sh for 
more) [1]


Interestingly it seems uribl.com has the replacement, Im going to 
obfuscate it else you wont likely see this :)  just replace digits 
with their alpha  lib3ra dott ch4t


in their listings, interesting because they dont seem to list new 
domains that way and that one is new, heh maybe andrew lee controls 
that too, who knows...


The new domain was NOT listed in any RHSBL at 13:55 UTC.

OTOH, they didn't like something about my usual single-venue address 
pattern so I had to register with an alternative tagging pattern.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: uribl result not triggering meta rule

[Wolfgang Breyha]

On 02/04/2021 13:46, Wolfgang Breyha wrote:

Hi!

It seems that 3.4.5 changed the behavior of URIBL lookups in a quite bad 
way compared to 3.4.4.


Just as a pointer:
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7897

Greetings,
Wolfgang

uribl result not triggering meta rule

[Wolfgang Breyha]

Hi!

It seems that 3.4.5 changed the behavior of URIBL lookups in a quite bad 
way compared to 3.4.4.


I have I urirhs lookup defined like:

ifplugin Mail::SpamAssassin::Plugin::URIDNSBL
urirhssub   URIBL_DENY  uribl.local.A   8
bodyURIBL_DENY  eval:check_uridnsbl('URIBL_DENY')
describeURIBL_DENY  Contains an URL listed in the URIBL blacklist
tflags  URIBL_DENY  net
score   URIBL_DENY  1
endif   # Mail::SpamAssassin::Plugin::URIDNSBL


and in a test E-Mail I see that it triggers on listed domains on 3.4.4 and 
3.4.5 as well.


But if I use a meta rule like:

meta   DENYTEST( URIBL_DENY || )

I see the rule triggering on 3.4.4 but not on 3.4.5 anymore.

I tested this on RHEL6 with 3.4.4, RHEL8 3.4.4 and 3.4.5, RHEL7 3.4.5.

Is this a know bug?

Greetings,
Wolfgang
--
Wolfgang Breyha  | https://www.blafasel.at/
Vienna University Computer Center | Austria

Re: using URIBL on other headers

[Kevin A. McGrail]

On 9/26/2018 10:59 AM, Pedro David Marco wrote:
>
> On Sunday, September 23, 2018, 12:55:28 AM GMT+2, Kevin A. McGrail
>  wrote:
>
> >It's fractured.  There are various lookups in various states in
> various plugins.
>
> >From, Reply-to, Received, nameservers, rdns, webmail server headers,
> >etc. are all enhancements I want to add for RBL lookups.  Some sort of
> >generic Header lookup would be best.  I can't remember if I have a
> >bugzilla for this but I have a lot of private notes about it.
>
> "Generic header", Kevin... would be much better so SA can check URLs
> added by any external software in a specific header
> that is removed before email delivery...
Agreed.

Re: using URIBL on other headers

[Pedro David Marco]

 
On Sunday, September 23, 2018, 12:55:28 AM GMT+2, Kevin A. McGrail 
 wrote:  
 >It's fractured.  There are various lookups in various states in various 
 >plugins.
>From, Reply-to, Received, nameservers, rdns, webmail server headers,
>etc. are all enhancements I want to add for RBL lookups.  Some sort of
>generic Header lookup would be best.  I can't remember if I have a
>bugzilla for this but I have a lot of private notes about it.

"Generic header", Kevin... would be much better so SA can check URLs added by 
any external software in a specific headerthat is removed before email 
delivery...
-PedroD  

Re: using URIBL on other headers

[Rob McEwen]

On 9/22/2018 5:55 PM, Michael Grant wrote:

The URIBL plugin looks for URLs in the subject and message body.
Is there some way to coax it to look in the other headers as well, for 
example the From: Reply-to: or the Received headers?



Michael,

This reminds me of that saying, "just because you can, doesn't mean you 
should" - and along those lines, I have some interesting observations 
about this:


(1) some URI/domain blacklists are ONLY intended for blocking on the 
domain or IP that is at the base of clickable links inside the body of 
the message. These will often have a small (but critical) uptick in 
false positives if used to check against domains found in the SMTP 
envelope (FROM, PTR record, HELO), with typically a very small increase 
in additional spams blocked. SO BE CAREFUL -AND- if you use a URI/domain 
blacklist in that way and they don't prescribe that type of usage, don't 
complain to them or anyone about any resulting false positives - because 
it would then be your MIS-usage off their list that caused those false 
positive.


(2) Even so, there really are SOME series of spams that can be safely 
blocked based on domains that are in the SMTP envelope (FROM, PTR 
record, HELO). In some cases, these are snowshoe spammers who are 
sending from their own spammy domain - but where this domain is NOT 
found in a clickable link inside the body of the message - they really 
are trying to get the user to hit "reply". So there really is a purpose 
for this, even if it is is a very small percentage of all spam


(3) However, even with that being a very small percentage of all - LARGE 
mail hosters LOVE THIS IDEA? Why? Because it is SO EFFICIENT for them to 
be able to block MORE spam based on information in the SMTP envelope - 
BEFORE the "data" command. Sometimes, this helps block messages where 
the domain was in a clickable link inside the body of the message - but 
it is still MORE EFFICIENT to block that based on the domain also being 
in the SMTP envelop.


(4) ABOUT THOSE FALSE POSITIVES: One of the main reasons that this is so 
risky for False Positives... is because two things are epidemic in 
recent years: (a) web site gets hijacked by criminal spammer, who 
installed pages there that redirect to pornographic dating sites or pill 
spam websites -AND/OR- (b) email account on the mail server gets 
credentials hijacked and starts spewing spam. HERE IS THE PROBLEM: 
*MOST* of the time, one or the other happens, (a or b) but not both. 
Therefore, if (a) happens, they are sure to land on traditional URI 
blacklists like SURBL, URIBL, and ivmURI. But this company - whose web 
site was hacked - might not have a single spam coming from their mail 
server. Yet, if you do the SMTP envelope checking against such URI 
blacklists - you're going to have a substantially higher amount of false 
positives due to blocking ALL of those emails that merely have a "FROM" 
address ending in that domain name - even though NONE of THOSE messages 
are spam.


(5) So which lists *DO* support blocking on the SMTP envelope? Spamhaus' 
DBL list is designed for this. However, invaluement's ivmURI list is NOT 
supposed to be used in this manner. SURBL and URIBL were originally 
designed to not be used in this way - but that might have changed in 
recent years? I recommend checking on that. In the meantime, I recommend 
*ONLY* using Spamhaus' DBL list in this way. (possibly SURBL or URIBL 
too? - but double check on that!)


(6) QUESTION: So why would a list not support both blocking methods? For 
example, why wouldn't ivmURI support this method?


ANSWER: What Spamhaus did with DBL, while interesting, put them at a 
strategic disadvantage, and there isn't a thing they can do about that 
without making fundamental changes to their strategy. Recall that false 
positive scenario mentioned earlier, where a hacked web site causing a 
URI-list blacklisting can lead to substantially more false positives due 
to only hitting on legit mails when blocking based on this domain being 
in the SMTP envelope? Well.. the OPPOSITE situation ALSO causes more 
false positives. When their email system has a hijacked email account, 
but their web site was NOT hacked - then domain blacklists that 
prescribe BOTH blocking methods and blacklist that domain... are going 
to then start blocking ALL messages that have that domain as a hyperlink 
inside the body of the message, even if THOSE messages are legit. This 
will then cause a substantial number of false positives that were not 
part of those hijacked outbound messages. So this works both ways. The 
problem with such domain blacklists that prescribe both uses... is that 
they either have to settle for (a) more false positives -OR- (b) more 
false negatives. In other words, the higher collateral damage potential 
means that there is going to be more collateral damage when they "take 
the bait" and blacklist the domain -OR- their desire 

Re: using URIBL on other headers

[RW]

On Sun, 23 Sep 2018 20:37:48 +0100
Michael Grant wrote:


> I tried to read through the plugin.  I'm not a spamassassin plugin
> developer, I didn't have much luck trying to figure out how to do it
> myself.  I know this plugin only does subject and body but I saw
> nothing in the plugin itself that referenced the subject header.
> arbitrary header through this like the subject and body.

The subject text is the first paragraph of the normalized body which
is parsed for domains.

> I am not sure you need to do that.  Why not just run all the headers
> or rather the entire message including headers through this plugin
> just like the body, in fact, just extend it's scope to look at the
> entire message rather than just the body & subject.

Most emails don't have a domain in the body, so if you start adding
a lot of domains from the headers, the number of look-ups could increase
dramatically. It could push some mail servers beyond the usage limits.

The main point of URI blocklists is to catch the website that's the
point of contact with the spammer. I think it going to be pretty
rare for a listed domain to appear in the headers without its being in
the body. That was my experience with my askdns rules.

The from header is already largely covered by the parse_dkim_uris
option. Reply-to might be worth trying, but  most of the interesting
reply-to addresses are Freemail.

Re: using URIBL on other headers

[Michael Grant]

On Sat, 22 Sep 2018 at 23:55, Kevin A. McGrail  wrote:

> On 9/22/2018 5:55 PM, Michael Grant wrote:
> > The URIBL plugin looks for URLs in the subject and message body.
> >
> > Is there some way to coax it to look in the other headers as well, for
> > example the From: Reply-to: or the Received headers?
> >
> >
> It's fractured.  There are various lookups in various states in various
> plugins.
>
> From, Reply-to, Received, nameservers, rdns, webmail server headers,
> etc. are all enhancements I want to add for RBL lookups.  Some sort of
> generic Header lookup would be best.  I can't remember if I have a
> bugzilla for this but I have a lot of private notes about it.
>
>
Thanks Kevin, good to hear other folks and yourself wants this too, it sees
to make sense!

I tried to read through the plugin.  I'm not a spamassassin plugin
developer, I didn't have much luck trying to figure out how to do it
myself.  I know this plugin only does subject and body but I saw nothing in
the plugin itself that referenced the subject header.  So I am gathering
it's more complex than simply running the output of an arbitrary header
through this like the subject and body.

Is this difficult because you feel you need to parse out domain names from
all these fields?

I am not sure you need to do that.  Why not just run all the headers or
rather the entire message including headers through this plugin just like
the body, in fact, just extend it's scope to look at the entire message
rather than just the body & subject.

Just a thought.  Hopefully if it's really that easy or if you can tell me
how to extend the scope of this to encompass the entire message, we could
do this sooner than later!

Thanks for your excellent plugin by the way!

Michael Grant

Re: using URIBL on other headers

[RW]

On Sat, 22 Sep 2018 22:55:49 +0100
Michael Grant wrote:

> The URIBL plugin looks for URLs in the subject and message body.
> 
> Is there some way to coax it to look in the other headers as well, for
> example the From: Reply-to: or the Received headers?

You can create individual rules for "From:" like:

 askdns  AUTHOR_IN_URIBL_BLACK  _AUTHORDOMAIN_.multi.uribl.com  A 2

However since I wrote this I've had 25 hits compared with 940 for
URIBL_BLACK. Only 4 spams hit AUTHOR_IN_URIBL_BLACK without URIBL_BLACK.

Re: using URIBL on other headers

[Kevin A. McGrail]

On 9/22/2018 5:55 PM, Michael Grant wrote:
> The URIBL plugin looks for URLs in the subject and message body.
>
> Is there some way to coax it to look in the other headers as well, for
> example the From: Reply-to: or the Received headers?
>
>
It's fractured.  There are various lookups in various states in various
plugins.

From, Reply-to, Received, nameservers, rdns, webmail server headers,
etc. are all enhancements I want to add for RBL lookups.  Some sort of
generic Header lookup would be best.  I can't remember if I have a
bugzilla for this but I have a lot of private notes about it.

Regards,

KAM

-- 
Kevin A. McGrail
VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

using URIBL on other headers

[Michael Grant]

The URIBL plugin looks for URLs in the subject and message body.

Is there some way to coax it to look in the other headers as well, for
example the From: Reply-to: or the Received headers?

Re: stackexchange.com in URIBL (false positive?)

[John Hardin]

On Sun, 29 Jul 2018, Daniele Duca wrote:


On 29/07/2018 09:53, Yves Goergen wrote:

No I can't because it's a locked system. I'd need an account for that. And 
I'm not going to register just for saving another admin's system. So either 
stackexchange admins repair their entry themselves, or the blacklist 
operator needs a review.


-Yves
A third option would be for you to use uridnsbl_skip_domain and don't bother 
anymore ;)


As of right now URIBL does not report stackexchange.com as being listed.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Look at the people at the top of both efforts. Linus Torvalds is a
  university graduate with a CS degree. Bill Gates is a university
  dropout who bragged about dumpster-diving and using other peoples'
  garbage code as the basis for his code. Maybe that has something to
  do with the difference in quality/security between Linux and
  Windows.   -- anytwofiveelevenis on Y! SCOX
---
 6 days until the 283rd anniversary of John Peter Zenger's acquittal

Re: stackexchange.com in URIBL (false positive?)

[Daniele Duca]

On 29/07/2018 09:53, Yves Goergen wrote:

No I can't because it's a locked system. I'd need an account for that. 
And I'm not going to register just for saving another admin's system. 
So either stackexchange admins repair their entry themselves, or the 
blacklist operator needs a review.


-Yves
A third option would be for you to use uridnsbl_skip_domain and don't 
bother anymore ;)


Daniele

Re: stackexchange.com in URIBL (false positive?)

[Yves Goergen]

No I can't because it's a locked system. I'd need an account for that. 
And I'm not going to register just for saving another admin's system. So 
either stackexchange admins repair their entry themselves, or the 
blacklist operator needs a review.


-Yves



Von: Dave Wreski
Gesendet: Sa, 2018-07-28 21:29 +0200


    5.7 URIBL_BLACK    Contains an URL listed in the URIBL blacklist
   [URIs: stackexchange.com]

I guess that's not supposed to be like that. I can't change anything at
it, just for information for somebody in the position to fix that.


It is indeed listed, and listed for a reason.

The default score for URIBL_BLACK is 1.7 with bayes. Why have you
changed it?

You can request that it be delisted here:

https://admin.uribl.com/

Regards,
Dave

Re: stackexchange.com in URIBL (false positive?) *** Spam 5.7

[Yves Goergen]

Oh I can surely change anything I want. But I don't want to weaken my 
spam filter. It's weak enough already. Spam is getting more and more 
through. It got to the point where I have to reconsider my complete mail 
receiving strategy with subaddresses, filters and a set of inbox 
subfolders to keep anything unknown away from me and only put in my 
inbox what I already know.


-Yves



Von: Reindl Harald
Gesendet: Sa, 2018-07-28 21:23 +0200


Am 28.07.2018 um 21:20 schrieb Yves Goergen:

I've received a notification e-mail from stackexchange.com
(stackoverflow.com) with a high spam score. It has this line in its report:

   5.7 URIBL_BLACK    Contains an URL listed in the URIBL blacklist
  [URIs: stackexchange.com]

I guess that's not supposed to be like that. I can't change anything at
it, just for information for somebody in the position to fix that


why in the world do you think you can't change anything as admin of your
server?

/etc/mail/spamassassin/local-06-uridnsbl-skip-domain.cf
uridnsbl_skip_domain stackexchange.com

Re: stackexchange.com in URIBL (false positive?)

[Yves Goergen]

Yes, I have changed the value of this rule long ago. It seemed to be 
better. I may have to turn it down a little.


And I am the admin myself but I'm no expert in spam fighting. Especially 
what the reason or source of that blacklisting is. I just see the rule 
matched and I consider that wrong because stackexchange is a service I 
use often and it never sent my anything unexpected.


So what is the reason for this host being listed?

-Yves


Von: RW
Gesendet: Sa, 2018-07-28 21:35 +0200
On Sat, 28 Jul 2018 21:20:49 +0200
Yves Goergen wrote:


Hello,

I've received a notification e-mail from stackexchange.com
(stackoverflow.com) with a high spam score. It has this line in its
report:

5.7 URIBL_BLACKContains an URL listed in the URIBL
blacklist [URIs: stackexchange.com]

I guess that's not supposed to be like that.


The default is 1.7, 5.7 is extremely aggressive for that rule,
particular when there's no BAYES_* result in the report.



  I can't change anything
at it, just for information for somebody in the position to fix that.


It's a very indirect way of getting to your local admin.

Re: stackexchange.com in URIBL (false positive?)

[RW]

On Sat, 28 Jul 2018 21:20:49 +0200
Yves Goergen wrote:

> Hello,
> 
> I've received a notification e-mail from stackexchange.com 
> (stackoverflow.com) with a high spam score. It has this line in its
> report:
> 
>5.7 URIBL_BLACKContains an URL listed in the URIBL
> blacklist [URIs: stackexchange.com]
> 
> I guess that's not supposed to be like that.

The default is 1.7, 5.7 is extremely aggressive for that rule,
particular when there's no BAYES_* result in the report. 


>  I can't change anything
> at it, just for information for somebody in the position to fix that.

It's a very indirect way of getting to your local admin.

Re: stackexchange.com in URIBL (false positive?)

[Dave Wreski]

   5.7 URIBL_BLACK    Contains an URL listed in the URIBL blacklist
  [URIs: stackexchange.com]

I guess that's not supposed to be like that. I can't change anything at 
it, just for information for somebody in the position to fix that.


It is indeed listed, and listed for a reason.

The default score for URIBL_BLACK is 1.7 with bayes. Why have you 
changed it?


You can request that it be delisted here:

https://admin.uribl.com/

Regards,
Dave

stackexchange.com in URIBL (false positive?)

[Yves Goergen]

Hello,

I've received a notification e-mail from stackexchange.com 
(stackoverflow.com) with a high spam score. It has this line in its report:


  5.7 URIBL_BLACKContains an URL listed in the URIBL blacklist
 [URIs: stackexchange.com]

I guess that's not supposed to be like that. I can't change anything at 
it, just for information for somebody in the position to fix that.


Here's the complete report:

 -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at 
http://www.dnswl.org/, no

 trust
 [198.252.206.125 listed in list.dnswl.org]
  5.7 URIBL_BLACKContains an URL listed in the URIBL blacklist
 [URIs: stackexchange.com]
 -0.0 SPF_PASS   SPF: Senderechner entspricht SPF-Datensatz
  0.0 HTML_MESSAGE   BODY: Nachricht enthält HTML
 -0.1 DKIM_VALID_AU  Message has a valid DKIM or DK signature 
from author's

 domain
  0.1 DKIM_SIGNEDMessage has a DKIM or DK signature, not 
necessarily valid
 -0.1 DKIM_VALID Message has at least one valid DKIM or DK 
signature



-Yves

Re: Fwd: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Bowie Bailey]

On 2/14/2017 10:01 AM, Emin Akbulut wrote:


-- Forwarded message --
From: *Bowie Bailey* <bowie_bai...@buc.com
<mailto:bowie_bai...@buc.com>>
Date: Tue, Feb 14, 2017 at 5:44 PM
Subject: Re: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to
URIBL was blocked.
To: users@spamassassin.apache.org
<mailto:users@spamassassin.apache.org>

That page is suggesting that you find the authoritative server for
blacklist domains and force those domain queries to go to those
servers.  This will fix the problem, but it is a bit fragile since
your lookups will start failing if those domains ever change their
DNS setup.
A better idea is to have your server stop forwarding altogether. 
Let your DNS server query the root servers and figure out the

authoritative DNS servers for the domains itself.  This is how DNS
servers were designed to work and there are few reasons not to do
it this way.  Unfortunately, I have no idea where those settings
are in the Windows DNS server.


That was the problem. I couldn't find the correct IP addresses. That's 
why I asked here how to configure conditional forwarders correctly, I 
mean IP addresses for uribl.com <http://uribl.com>, etc.


The page you referenced actually showed how to do that.

C:\> nslookup -querytype=ns uribl.com

uribl.com   nameserver = v.uribl.net
uribl.com   nameserver = o.icudp.com
uribl.com   nameserver = c.sarules.net
uribl.com   nameserver = p.icudp.net

c.sarules.net   internet address = 52.9.94.53
o.icudp.com internet address = 54.149.125.143
p.icudp.net internet address = 94.228.131.217
v.uribl.net internet address = 52.71.102.73

The IP addresses listed are all nameservers for uribl.com.

Now my DNS server runs like a DNS server, uses root DNS servers to 
resolve names.


A much better idea.


fbb
I think I should "subscibe" to uribl's paid system if any.


You don't need to unless you continue to get blocked.  Or if you just 
want to support them.


Before you think about paying, make absolutely sure that you are 
querying them directly.  The paid service still won't work (afaik) if 
you are using forwarding.


--
Bowie

Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Benny Pedersen]

Emin Akbulut skrev den 2017-02-14 16:03:

It's Gmail. When I hit the reply button, it only sends the last
poster, -in this reply, it's you and I manually added users@-


gmail ignores List-* headers, leading to much more problems then users 
using gmail


if you need more support on there broken gmail ask them

Re: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Benny Pedersen]

Emin Akbulut skrev den 2017-02-14 15:27:

I'm confused a bit. Should I use forwarders or not?


no stop any forward dns


I was trying to follow that guide:


i do not care of windows problems here

use spamasassin docs on how to use specific ip as dns server, but not 
global, only for spamassassin you should stay at 127.0.0.1, you windows 
problemativ dns server should do the rest for you, if it still not 
working ask where thay know more about windows then here

Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Emin Akbulut]

It's Gmail. When I hit the reply button, it only sends the last poster,
-in this reply, it's you and I manually added users@-

On Tue, Feb 14, 2017 at 5:57 PM, Reindl Harald <h.rei...@thelounge.net>
wrote:

> what is wrong with your mailprogram that it appearently is lacking a
> "reply" button and so you seem to need forward messages which breaks
> threading in any sane mail-client and list-archive?
>
> Am 14.02.2017 um 15:43 schrieb Emin Akbulut:
>
>>
>> -- Forwarded message --
>> From: *David Jones* <djo...@ena.com <mailto:djo...@ena.com>>
>> Date: Tue, Feb 14, 2017 at 5:33 PM
>> Subject: Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL
>> was blocked.
>> To: "users@spamassassin.apache.org
>> <mailto:users@spamassassin.apache.org>"
>> <users@spamassassin.apache.org <mailto:users@spamassassin.apache.org
>> >>
>>
>>
>> Note that if your mail volume is high enough, you may
>> still hit their free usage limit even after doing this.
>> Dave
>>
>>
>>
>> I've got plenty of inboxes. I've read SpamAssassin's info page about the
>> block and it says:
>>
>> Resolving the block might be as simple as using your
>> own non-forwarding
>> <https://wiki.apache.org/spamassassin/CachingNameserver#Non-
>> forwarding> caching
>> nameserver
>> <https://wiki.apache.org/spamassassin/CachingNameserver> to avoid
>> being lumped together with other users queries; setting up your own
>> mirror of the DNS-blocklist; or paying to use the blocklist. The
>> choice is up to the DNS-Blocklist administrator.
>>
>>
>>
>> Then I found myself at configuring DNS cond. forwarder because of an
>> incorrect advise
>>
>

Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Benny Pedersen]

Emin Akbulut skrev den 2017-02-14 14:21:


How can I set the DNS conditional forwarders properly?


setup spamasassin to use 127.0.0.1 as dns server, not any remote ips

i dont know anything on how windows works :=)

Fwd: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Emin Akbulut]

> -- Forwarded message --
> From: Bowie Bailey <bowie_bai...@buc.com>
> Date: Tue, Feb 14, 2017 at 5:44 PM
> Subject: Re: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL
> was blocked.
> To: users@spamassassin.apache.org
>
> That page is suggesting that you find the authoritative server for
> blacklist domains and force those domain queries to go to those servers.
> This will fix the problem, but it is a bit fragile since your lookups will
> start failing if those domains ever change their DNS setup.
> A better idea is to have your server stop forwarding altogether.  Let your
> DNS server query the root servers and figure out the authoritative DNS
> servers for the domains itself.  This is how DNS servers were designed to
> work and there are few reasons not to do it this way.  Unfortunately, I
> have no idea where those settings are in the Windows DNS server.


That was the problem. I couldn't find the correct IP addresses. That's why
I asked here how to configure conditional forwarders correctly, I mean IP
addresses for uribl.com, etc.

Now my DNS server runs like a DNS server, uses root DNS servers to resolve
names.

I think I should "subscibe" to uribl's paid system if any.

Re: Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Bowie Bailey]

On 2/14/2017 9:27 AM, Emin Akbulut wrote:

I'm confused a bit. Should I use forwarders or not?
I was trying to follow that guide:

-

As your issue with UTIBL_BLOCKED is a well-known one

I would like to point you the FAQ section of  our homepage:


http://www.jam-software.com/spamassassin_in_a_box/online_manual/EN/configuredns.html



Here you will find detailed information on how to configure

a Microsoft Windows DNS server to do a conditional forwarding.



That page is a bit confusing since it shows screenshots of the DNS query 
results, but never actually shows a screenshot of the setting you are 
supposed to be changing.


That page is suggesting that you find the authoritative server for 
blacklist domains and force those domain queries to go to those 
servers.  This will fix the problem, but it is a bit fragile since your 
lookups will start failing if those domains ever change their DNS setup.


A better idea is to have your server stop forwarding altogether. Let 
your DNS server query the root servers and figure out the authoritative 
DNS servers for the domains itself.  This is how DNS servers were 
designed to work and there are few reasons not to do it this way.  
Unfortunately, I have no idea where those settings are in the Windows 
DNS server.


--
Bowie

Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Emin Akbulut]

> -- Forwarded message --
> From: David Jones <djo...@ena.com>
> Date: Tue, Feb 14, 2017 at 5:33 PM
> Subject: Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was
> blocked.
> To: "users@spamassassin.apache.org" <users@spamassassin.apache.org>
>
>
> Note that if your mail volume is high enough, you may
> still hit their free usage limit even after doing this.
> Dave



I've got plenty of inboxes. I've read SpamAssassin's info page about the
block and it says:

Resolving the block might be as simple as using your own non-forwarding
> <https://wiki.apache.org/spamassassin/CachingNameserver#Non-forwarding> 
> caching
> nameserver <https://wiki.apache.org/spamassassin/CachingNameserver> to
> avoid being lumped together with other users queries; setting up your own
> mirror of the DNS-blocklist; or paying to use the blocklist. The choice is
> up to the DNS-Blocklist administrator.
>


Then I found myself at configuring DNS cond. forwarder because of an
incorrect advise.

Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[David Jones]

>From: RW <rwmailli...@googlemail.com>
>Sent: Tuesday, February 14, 2017 7:51 AM
>To: users@spamassassin.apache.org
>Subject: Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was 
>blocked.
    
>On Tue, 14 Feb 2017 16:21:04 +0300
>Emin Akbulut wrote:

>> Hi
>> 
>> URIBL checks are blocked. I think bec. of so many queries. I'm
>> advised to set up conditional forwarder on Windows DNS Server.>

>If you mean that you should *stop* forwarding this traffic than that
>is correct. You need to be doing your own look-ups to the
>whitelist/blacklist servers from your own IP address, forwarding to a
>shared server is what causes the problem.

This is a common problem and has been discussed on this list
many times before.  I wish SpamAssassin had a better way to
handle this rule hit and explaining to the server admin but I
don't think this is possible.

Basically you need to point to a DNS server that you manage
or know for sure that it's not forwarding to another DNS server.
It's not required to have a local DNS server on your SA box but
it's the best way to know for sure that it's doing full recursive
lookups, not forwarding to other DNS servers that will
consolidate your queries with others pushing you over the
free usage limits and thus hitting this rule.

Note that if your mail volume is high enough, you may
still hit their free usage limit even after doing this.

Dave

Fwd: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Emin Akbulut]

I'm confused a bit. Should I use forwarders or not?
I was trying to follow that guide:

-

As your issue with UTIBL_BLOCKED is a well-known one
>
> I would like to point you the FAQ section of  our homepage:
>
>
>
> http://www.jam-software.com/spamassassin_in_a_box/online_
> manual/EN/configuredns.html
>
>
>
> Here you will find detailed information on how to configure
>
> a Microsoft Windows DNS server to do a conditional forwarding.
>


-

Re: URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[RW]

On Tue, 14 Feb 2017 16:21:04 +0300
Emin Akbulut wrote:

> Hi
> 
> URIBL checks are blocked. I think bec. of so many queries. I'm
> advised to set up conditional forwarder on Windows DNS Server.

If you mean that you should *stop* forwarding this traffic than that
is correct. You need to be doing your own look-ups to the
whitelist/blacklist servers from your own IP address, forwarding to a
shared server is what causes the problem.



> How can I set the DNS conditional forwarders properly?

This is a question about Windows.

URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.

[Emin Akbulut]

Hi

URIBL checks are blocked. I think bec. of so many queries. I'm advised to
set up conditional forwarder on Windows DNS Server.

I've added uribl.com as DNS zone and  54.149.125.143 as IP.

SA still tags the messages.

How can I set the DNS conditional forwarders properly?

Re: How to create a URIBL

[Kris Deugau]

Alex wrote:
> Hi,
> 
> I've collected a bunch of URIs that I'd like to incorporate into my
> rulebase. I know how to create a DNSBL, but I don't specifically know
> how to create a URIBL. Can I use rbldnsd for this? Or would I have to
> extract the IP or hostname from the URL, then also use a bunch of uri
> rules? If so, is there a way of automating this, given a list of URIs?
> 
> For example, I have URIs like:
> 
> http://109.73.134.241/dgq01px
> http://51steel1.org/s4b5ztgcx
> http://amessofblues1.com/m0dqfx

Do you want to use the full URI (including the /dgq01px or /s4b5ztgcx
parts), or just the domain names?

If you want the full URI, I think you're pretty much stuck collecting
them up in a huge list of uri rules, unless you want to write a custom
plugin to do a custom DNS lookup.  (Not sure some of the new DNS lookup
widgets will go quite far enough to support something like this directly.)

If you only want the domain name, you can feed those into a local DNSBL.

> I'm also then not sure which of uri* rule definition should be used.
> I've used urirhsbl before for a local host blocklist, but now after
> reading the man page again for the first time in a while, I'm not even
> sure that's correct.

"uri" rules are standard SA regular expression rules that only look at
things that SA has extracted from the message as a URI.

The others are DNSBL lookup rules, with a lot of variations on how the
lookup should be done, and the results broken down.  The
Mail::SpamAssassin::Plugin::URIDNSBL man page has all the details, but
my experience has been that for local use, you generally only need
uridnsbl and/or uridnssub.

> I'm also unclear about rbldnsd config for dnset, where hostnames would
> be used. Here is my current command-line:

Other responses have gone into more detail on this, which I probably
tested for myself at one point when I set up local DNS blacklists.

I also wrote some basic tools to feed both relay IP and URI domain data
into these local lists;  I've published them at
https://secure.deepnet.cx/trac/dnsbl.  Note that these are mainly
data-entry/export utilities, and they're a little rough around the
edges, but these are substantially what I've been using in production
for quite a few years now.

-kgd

Re: How to create a URIBL

[Rob McEwen]

On 10/18/2016 9:09 PM, Alex wrote:

How do you then enter ranges? For example, one of the rbldnsd zone
examples I've seen have entries such as:
1.168.160.0-255
That does not look to be in reverse order, as the host octet is still last.


while there may be a more complicated and unusual answer for this.. the 
short answer is... you don't, and you shouldn't have to.


(1) IPs at the base of clickable links inside the body of the message in 
spams... is still a little rare... comprising roughly 2% of all such 
listings.


(2) This means that (a) those IPs aren't taking up a lot of space in the 
dnset files, when compared to the domains and host names there, and (b) 
of that ~2% of IPs, extremely few of those are even in the same /24 
block - so you don't get much mileage out of trying to list ranges


having said that... sending-IP lists that use ipset DO have the 
functionality that you desire. ipset actually has quite a number of 
acceptable formats to list blocks or ranges of IPs.


iptset... not so much. iptset is built for EXTRA speed and EXTRA 
low-memory usage, but isn't as flexible and generally requires one 
single IP per line.


Based on your question, it could be that you're trying to merge your 
sending IP blacklist, with your URI/domain blacklists... all into one 
single dnset rbldnsd file? if so, that is NOT recommended. It causes 
problems and removes some of rbldnsd best features/strengths.



Your service is great, btw.


Thanks. Please send me a note off-list as you how/why you think that. 
I'm not looking for praise... just curious if you're one of my clients 
(such as at your dayjob?) or if we've crossed paths somewhere and I 
forgot about it?... or if you have ever testing invaluement? etc (though 
I know you're a frequent SA discussion participant)



--
Rob McEwen
http://www.invaluement.com
+1 (478) 475-9032

Re: How to create a URIBL

[Rob McEwen]

On 10/19/2016 3:51 AM, Matus UHLAR - fantomas wrote:

are you REALLY sure the IP has to be reversed?
rbldns parses IP and reverses them by itself, if used in ip4* dataset.
When used in dnset, it should not be reversed.


Your most valid points do not apply to "dnset". they apply to ip4tset 
and ip4set for sending-IP blacklists.


Let me explain... but before I explain, let me say that I'm not arguing 
for any of this. These standards were put in place long before my time 
(and are followed by SURBL and URIBL, too). Or, at least I didn't set 
these standards. I MIGHT have been involved in some of the discussions 
about this circa 2004, in internal discussions at SURBL - and in SA 
discussions - but I think this was all set just a little before my time 
in those forums.


So basically, if you look at the anatomy of a domain name... from left 
to right, you get into a higher hierarchy.


So in "foo.example.com"

"foo" is drilling into detail. while "example.com" is the bigger 
picture. And then ".com" is an even bigger picture! In a domain, as you 
get FURTHER to the right, you go to a HIGHER hierarchy or level.


But IPs are the opposite. For an IPv4 IP, the leftmost number is the 
highest in the hierarchy, and you drill down into more detail as you 
move to the right.


For this reason, it was decided a long time ago... that for URI DNSBL 
blacklists that use "dnset", the IP should be reversed in the source file.


Therefore, in the data file, the test point IP:

127.0.0.1

shows up as

1.0.0.127

And then when the client queries that IP, the query is formatted as follows:

1.0.0.127.example.com

(where example.com is the URI blacklist's host name)

And, likewise, ALL of the major anti-spam software, (such as 
SpamAssassin), automatically reverses the IP when that (forward-ordered) 
IP is extracted from a base of a URL found in the body of a spam, and 
then this is appended to the beginning of a URI blacklist's hostname, 
for checking against a URIBL blacklists (such as SURBL, URIBL, or my own 
ivmURI list)


This decision to do it this way PROBABLY had something to do with trying 
to get rbldnsd engine to NOT have to internally treat IPs and 
domains/host-names differently. otherwise, it would have had to "know" 
to reverse IPs, but yet know to NOT reverse domains or host names. (and 
who knows what TLDs could be coming up in the future?)


In contrast, IPs found in sending IP data files (for ip4tset and ip4set) 
don't have this inconsistency problem. So it make sense to just leave 
them in forward-order, for EASY readability... and then just allow 
rbldnsd to reverse order them on-the-fly. (thank God - I'd go nuts if my 
ip4tset and ip4set were all in reverse order! meanwhile, IPs in URIBL 
data files are usually a TINY percentage of the listings!)


--

Having said all of that, for regular sending0IP blacklists, (just as you 
said) the IP is NOT in reverse order in the file. But rbldnsd "knows" to 
reverse order it in memory, before it is compared to the reverse-ordered 
query that comes in from the client.


So you're correct when you say, "rbldns parses IP and reverses them by 
itself" ... but that only applies to sending-IP blacklists, set up with 
ip4tset and ip4set in rbldnsd.


As shown, dnset operates differently for IP addresses found in URIBL 
blacklists.


--

This was a trip down memory lane for me.

--
Rob McEwen
invaluement

Re: How to create a URIBL

[Axb]

On 10/19/2016 09:51 AM, Matus UHLAR - fantomas wrote:

On 18.10.16 20:03, Rob McEwen wrote:

So your three examples:

109 .73 .134 .241



would like like this:

.241 .134 .73 .109



NOTICE 2 things:



(2) the fact that the IP is in reverse order. The great part about
rbldnsd is that a lookup on either


are you REALLY sure the IP has to be reversed?
rbldns parses IP and reverses them by itself, if used in ip4* dataset.
When used in dnset, it should not be reversed.



in the rbldnsd zone the ip does NOT have to reversed
the query reverses the IP

Re: How to create a URIBL

[Matus UHLAR - fantomas]

On 18.10.16 20:03, Rob McEwen wrote:

So your three examples:

109 .73 .134 .241



would like like this:

.241 .134 .73 .109



NOTICE 2 things:


(2) the fact that the IP is in reverse order. The great part about 
rbldnsd is that a lookup on either


are you REALLY sure the IP has to be reversed?
rbldns parses IP and reverses them by itself, if used in ip4* dataset.
When used in dnset, it should not be reversed.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Eagles may soar, but weasels don't get sucked into jet engines. 

Re: How to create a URIBL

[Alex]

Hi,

> (2) the fact that the IP is in reverse order.

How do you then enter ranges? For example, one of the rbldnsd zone
examples I've seen have entries such as:

1.168.160.0-255

That does not look to be in reverse order, as the host octet is still last.

> foo.example.com:127.0.0.2:Blocked System
>
> in my experience, I haven't been able to get this to work unless I put a
> space just before the first colon, as follows
>
> foo.example.com :127.0.0.2:Blocked System

That was my exact problem that caused me to write this post. It was
frustrating that ip4set worked fine, but dnset always failed because
of that.

> But sometimes you don't need that and can simply use just the domain or IP
> on each line, since much of that can be accomplished with a single line
> at/near the top of the file, such as this one that I use for the invaluement
> URI list:
>
> :127.0.0.2:Blocked by ivmURI - see http://www.invaluement.com/lookup/?item=$

Yes, this is what I've settled on for now.

> of course, the most difficult part is not collecting spammy IPs and
> domains... that part is easy. The most difficult part is knowing when NOT to
> blacklist a domain--which would be a decoy domain found in a spam, that
> wasn't the actual "payload" for the spam and is instead an innocent
> bystander's domain -- and/or generally keeping FPs super low. THAT is the
> hard part.

Yeah, absolutely. That's a large part of what's been delaying my
progress with my honeypots. It's still in progress, but one thing I've
been doing is checking my entries against existing whitelists, and
other ways such as seeing how long they've been around, etc.

> But try this and blacklist:
>
> .blogspot.com
>
> ...and trigger massive FPs... when you should have listed:
>
> .somehorrificspammerfromhell.blogspot.com

Yes, exactly. I've just been doing specific hostnames.

I appreciate that this is slightly off-topic, but it's an extension of
SA. Thanks so much for your help. Your service is great, btw.

Re: How to create a URIBL

[Rob McEwen]

Alex,

here are some suggestions:

In your rbldnsd-formatted file, put a dot at the beginning, which serves 
as a wildcard.


So your three examples:

109 .73 .134 .241
51steel1 .org
amessofblues1 .com

(I added spaces here to evade spam filtering, but those spaces shouldn't 
actually be there)


would like like this:

.241 .134 .73 .109
.51steel1 .org
.amessofblues1 .com

(again, the extra spaces shouldn't be there)

NOTICE 2 things:

(1) The extra dot at the beginning
-and-
(2) the fact that the IP is in reverse order. The great part about 
rbldnsd is that a lookup on either


example.com
OR
www.example.com
OR
foo.bar.foo.example.com

ALL of those will get a "hit" when the rbldnsd file has

.example.com



When it comes to formatting the rbldnsd-formatted file, in addition to 
my suggestions above, it comes down to a choice... make it a simply list 
of the domains and (reverse-ordered) IPs? Or provide more information 
for each individual IP, such as a custom text response, as you did here:


foo.example.com:127.0.0.2:Blocked System

in my experience, I haven't been able to get this to work unless I put a 
space just before the first colon, as follows


foo.example.com :127.0.0.2:Blocked System

But sometimes you don't need that and can simply use just the domain or 
IP on each line, since much of that can be accomplished with a single 
line at/near the top of the file, such as this one that I use for the 
invaluement URI list:


:127.0.0.2:Blocked by ivmURI - see http://www.invaluement.com/lookup/?item=$

...which then causes all following lines of just domains and IPs... to 
use this line above as if it were on every single line. - and the "$" 
causes the actual listed item to show up in the SMTP text message. That 
"$" feature can be very informative and helpful!


of course, the most difficult part is not collecting spammy IPs and 
domains... that part is easy. The most difficult part is knowing when 
NOT to blacklist a domain--which would be a decoy domain found in a 
spam, that wasn't the actual "payload" for the spam and is instead an 
innocent bystander's domain -- and/or generally keeping FPs super low. 
THAT is the hard part.


There are other issues as to WHERE to divide the domain.

For example, if you listed

.foo.bar.foo.bar.foo.bar.foo.bar.example.com

... but foo.bar.foo.bar.foo.bar.foo.bar. was just decoy material added 
by the spammer... then...


foo.bar.example.com comes in and guess what? your lookup fails to find 
it. Yet all such variations would be listed if you had simply blacklisted:


.example.com
(again, with the dot in front)

But try this and blacklist:

.blogspot.com

...and trigger massive FPs... when you should have listed:

.somehorrificspammerfromhell.blogspot.com

so that either

www.somehorrificspammerfromhell.blogspot.com
OR
somehorrificspammerfromhell.blogspot.com
foo.bar.foo.bar.somehorrificspammerfromhell.blogspot.com

would ALL return listing, but

blogspot.com

...wouldn't.

So it also takes some work determining those boundaries. Some of those 
are simple domains... while others like blogspot.com or wordpress.com, 
are more "artificial" (but still critically important).



--
Rob McEwen
invaluement.com

Re: How to create a URIBL

[Joe Quinn]

On 10/18/2016 6:21 PM, Alex wrote:

Hi,

I've collected a bunch of URIs that I'd like to incorporate into my
rulebase. I know how to create a DNSBL, but I don't specifically know
how to create a URIBL. Can I use rbldnsd for this? Or would I have to
extract the IP or hostname from the URL, then also use a bunch of uri
rules? If so, is there a way of automating this, given a list of URIs?

For example, I have URIs like:

http://109.73.134.241/dgq01px
http://51steel1.org/s4b5ztgcx
http://amessofblues1.com/m0dqfx

I'm also then not sure which of uri* rule definition should be used.
I've used urirhsbl before for a local host blocklist, but now after
reading the man page again for the first time in a while, I'm not even
sure that's correct.

I'm also unclear about rbldnsd config for dnset, where hostnames would
be used. Here is my current command-line:

/usr/sbin/rbldnsd -n -srbldnsd.stats -r/var/lib/rbldnsd -f -n -b
66.123.123.106/53 uri.example.com:dnset:urilist

My urilist file looks like this:

:127.0.0.2:Blocked System: http://example.com/bl?$
$NS 1w uri.example.com
$SOA 1w uri.example.com admin.uri.example.com 0 2h 2h 1w 1h
@ A 66.123.123.106
@ MX 10 uri.example.com
@ TXT "example hostname blocklist"
25z5g623wpqpdwis.onion1.to:127.0.0.2:Blocked System, Last-Attack: 1476825181
27lelchgcvs2wpm7.3lhjyx1.top:127.0.0.2:Blocked System, Last-Attack: 1476825181
27lelchgcvs2wpm7.7jiff71.top:127.0.0.2:Blocked System, Last-Attack: 1476825181

Using the following (and variations, including dig +short) fail with NXDOMAIN
# host 25z5g623wpqpdwis.onion1.to.uri.example.com 66.123.123.106

Can someone show me an example zone file using the dnset option?

I'm guessing my first attempt at this message being received by the
list was due to the domain samples I've included, so they've been
modified.

Any ideas greatly appreciated.
Thanks,
Alex


rbldnsd is still suitable for this, as the DNS lookups are fundamentally 
just mapping strings to IPs. Getting too deep into it is outside SA's 
scope, but the only real difference between an IP rbl and a domain rbl 
is that IP rbls tend to reverse the IP so the most significant octet is 
the most significant subdomain.


On the rules side of things there's multiple different ways to write uri 
rules that match against a dns lookup. Some of them are looking for 
nxdomain vs anything else, some of them can look for particular IPs, 
etc. Just look for the existing RBL that's most similar to what you are 
looking to create.

How to create a URIBL

[Alex]

Hi,

I've collected a bunch of URIs that I'd like to incorporate into my
rulebase. I know how to create a DNSBL, but I don't specifically know
how to create a URIBL. Can I use rbldnsd for this? Or would I have to
extract the IP or hostname from the URL, then also use a bunch of uri
rules? If so, is there a way of automating this, given a list of URIs?

For example, I have URIs like:

http://109.73.134.241/dgq01px
http://51steel1.org/s4b5ztgcx
http://amessofblues1.com/m0dqfx

I'm also then not sure which of uri* rule definition should be used.
I've used urirhsbl before for a local host blocklist, but now after
reading the man page again for the first time in a while, I'm not even
sure that's correct.

I'm also unclear about rbldnsd config for dnset, where hostnames would
be used. Here is my current command-line:

/usr/sbin/rbldnsd -n -srbldnsd.stats -r/var/lib/rbldnsd -f -n -b
66.123.123.106/53 uri.example.com:dnset:urilist

My urilist file looks like this:

:127.0.0.2:Blocked System: http://example.com/bl?$
$NS 1w uri.example.com
$SOA 1w uri.example.com admin.uri.example.com 0 2h 2h 1w 1h
@ A 66.123.123.106
@ MX 10 uri.example.com
@ TXT "example hostname blocklist"
25z5g623wpqpdwis.onion1.to:127.0.0.2:Blocked System, Last-Attack: 1476825181
27lelchgcvs2wpm7.3lhjyx1.top:127.0.0.2:Blocked System, Last-Attack: 1476825181
27lelchgcvs2wpm7.7jiff71.top:127.0.0.2:Blocked System, Last-Attack: 1476825181

Using the following (and variations, including dig +short) fail with NXDOMAIN
# host 25z5g623wpqpdwis.onion1.to.uri.example.com 66.123.123.106

Can someone show me an example zone file using the dnset option?

I'm guessing my first attempt at this message being received by the
list was due to the domain samples I've included, so they've been
modified.

Any ideas greatly appreciated.
Thanks,
Alex

Re: URIBL randomly not triggered for the same message

[Benny Pedersen]

On 2016-07-26 11:39, Reindl Harald wrote:


sadly it don't work as expected
https://bugzilla.redhat.com/show_bug.cgi?id=1360222


add forward-first: yes to forward zone

without you are qquery stale data in unbound

no i do not use bind9 now :=)

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 17:40 schrieb Reindl Harald:

Am 06.07.2016 um 17:35 schrieb John Hardin:

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?


sounds good and at leat i don't get any error by using
unbound-1.5.8-2.fc23.x86_64 and the follwoing settings

cache-min-ttl: 600
cache-max-ttl: 43200
cache-max-negative-ttl: 100

when it works as expected it should lead in not so often expire heavily
used crap domains without take too long for realize new listings and at
least makes the problem nit so big as now


sadly it don't work as expected
https://bugzilla.redhat.com/show_bug.cgi?id=1360222



signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 17:35 schrieb John Hardin:

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?


sounds good and at leat i don't get any error by using 
unbound-1.5.8-2.fc23.x86_64 and the follwoing settings


cache-min-ttl: 600
cache-max-ttl: 43200
cache-max-negative-ttl: 100

when it works as expected it should lead in not so often expire heavily 
used crap domains without take too long for realize new listings and at 
least makes the problem nit so big as now


thanks god then normal DNSBL/DNSWL are not affected becaus ethey are 
used also in prostscreen for weighting and so at the moment SA is using 
them the cache is always hot





signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Paul Stead wrote:


On 06/07/16 16:16, John Hardin wrote:

 Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
 configure different TTL for NXDOMAIN (relatively low) and positive
 results (relatively high)?


For this cache-max-negative-ttl exists :)


:) It's obvious I don't use unbound...

Reindl, does that approach help?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[John Hardin]

On Wed, 6 Jul 2016, Reindl Harald wrote:




Am 06.07.2016 um 14:36 schrieb RW:

 On Tue, 5 Jul 2016 14:01:17 +0200
 Reindl Harald wrote:

>  since there is a local unbound-cache with
> 
>cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i observe is 
that after TTL is reached and unbound needs to query again the at least first 
question leads to a negativeresult in spamassassin while the next cache hit 
correctly has URIBL_BLACK again


Does that cache-min-ttl also affect NXDOMAIN? Is it possible to configure 
different TTL for NXDOMAIN (relatively low) and positive results 
(relatively high)?


If not, you might want to file a bug with unbound to ask them to make that 
possible.




--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  So Microsoft's invented the ASCII equivalent to ugly ink spots that
  appear on your letter when your pen is malfunctioning.
 -- Greg Andrews, about Microsoft's way to encode apostrophes
---
 Tomorrow: Robert Heinlein's 109th birthday

Re: URIBL randomly not triggered for the same message

[Paul Stead]

On 06/07/16 16:16, John Hardin wrote:

Does that cache-min-ttl also affect NXDOMAIN? Is it possible to
configure different TTL for NXDOMAIN (relatively low) and positive
results (relatively high)?


For this cache-max-negative-ttl exists :)

Paul
--
Paul Stead
Systems Engineer
Zen Internet

Re: URIBL randomly not triggered for the same message

[Reindl Harald]

Am 06.07.2016 um 14:36 schrieb RW:

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:


since there is a local unbound-cache with

  cache-min-ttl: 300


thanks for the hint, but look at
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335#c8

reduce the value would make the problem even worser because what i 
observe is that after TTL is reached and unbound needs to query again 
the at least first question leads to a negativeresult in spamassassin 
while the next cache hit correctly has URIBL_BLACK again


so at the moment there is a tradeoff between get new domains fast enough 
and don't miss already known hits *and* that also affects SPF and so 
whitelist_auth in a bad way



You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.




signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered for the same message

[RW]

On Tue, 5 Jul 2016 14:01:17 +0200
Reindl Harald wrote:

> since there is a local unbound-cache with
> 
>   cache-min-ttl: 300

You might want to review that. From http://uribl.com

  July 8, 2015: Reduction in list time latency

  The spam trend of late has been to use short lived, high-volume
  campaigns in order to capitalize on the reactive nature of blacklist
  services. In the past, it could take up to 4 minutes for us to
  identify, list, rebuild, and syncronize the update. Recent campaigns
  we have investigated have sent 80-90% of their payload within 3
  minutes.

  Because of this, we have made a handful of enhancements to improve
  our identification speed and reduce the list time latency. As a
  result, we have reduced identification times by up to 100 seconds for
  new spam campaigns, by improving the speed at which we deliver live
  query data into our system. All users should see immediate results
  from these changes.

Re: URIBL randomly not triggered (and SPF too)

[Reindl Harald]

see also https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7335

BTW: the bugtracker has also a major bug - click on "My Bugs" leads to 
the URL below listing a ton of bugreports back to the year 2011 and 
pretends they are reported by me


https://bz.apache.org/SpamAssassin/buglist.cgi?bug_status=UNCONFIRMED_status=NEW_status=ASSIGNED_status=REOPENED_to1=1=1=exact&%20%20%20%20%20%20%20%20%20email1=h.reindl%40thelounge.net=bug_status=notequals=UNCONFIRMED=reporter=equals=h.reindl%40thelounge.net

Am 05.07.2016 um 14:10 schrieb Reindl Harald:

Am 05.07.2016 um 14:01 schrieb Reindl Harald:

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in
the ABUSE SURBL blocklist

50% of all tries against spamd it does NOT hit while the scantime for
the whole message is arounnd 3 seconds - since there is a local
unbound-cache with

 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe
the same behavior randomly with URIBL_LOCAL where the unbound dns cache
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053

that smells why ever very unrelieable and frankly i observed similar
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same
message and one get USER_IN_SPF_WHITELIST while the other goes through
all tests


that below too MUST NOT happen because one triggers
USER_IN_SPF_WHITELIST and the other don't have any SPF test and given
that there is a python-policyd-spf waiting 20 seconds for the response
in 'smtpd_recipient_restrictions' long before the contentfilters the
dns-results are cached

Jul  4 11:34:51 mail-gw postfix/smtpd[13648]: 3rjhgb71LVzB47:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:34:52 mail-gw spamd[12535]: spamd: processing message
<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:34:56 mail-gw spamd[12535]: spamd: result: . -4 -
BAYES_00,CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,HTML_MESSAGE,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD
scantime=4.2,size=18438,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>,bayes=0.00,autolearn=disabled,shortcircuit=no


Jul  4 11:57:01 mail-gw postfix/smtpd[14837]: 3rjj993Bk8zB7P:
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: processing message
<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>
for sa-milt:189
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: result: . -100 -
CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,CUST_SHORTCIRCUIT,RCVD_IN_MSPIKE_H2,SHORTCIRCUIT,USER_IN_SPF_WHITELIST
scantime=0.1,size=15685,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>,autolearn=disabled,shortcircuit=spam




signature.asc
Description: OpenPGP digital signature

Re: URIBL randomly not triggered (and SPF too)

[Reindl Harald]

Am 05.07.2016 um 14:01 schrieb Reindl Harald:

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in
the ABUSE SURBL blocklist

50% of all tries against spamd it does NOT hit while the scantime for
the whole message is arounnd 3 seconds - since there is a local
unbound-cache with

 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe
the same behavior randomly with URIBL_LOCAL where the unbound dns cache
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053

that smells why ever very unrelieable and frankly i observed similar
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same
message and one get USER_IN_SPF_WHITELIST while the other goes through
all tests


that below too MUST NOT happen because one triggers 
USER_IN_SPF_WHITELIST and the other don't have any SPF test and given 
that there is a python-policyd-spf waiting 20 seconds for the response 
in 'smtpd_recipient_restrictions' long before the contentfilters the 
dns-results are cached


Jul  4 11:34:51 mail-gw postfix/smtpd[13648]: 3rjhgb71LVzB47: 
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:34:52 mail-gw spamd[12535]: spamd: processing message 
<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail> 
for sa-milt:189
Jul  4 11:34:56 mail-gw spamd[12535]: spamd: result: . -4 - 
BAYES_00,CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,DKIM_SIGNED,DKIM_VALID,HTML_MESSAGE,RCVD_IN_MSPIKE_H2,RP_MATCHES_RCVD 
scantime=4.2,size=18438,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a2da06a20d_63ca5ed30013218...@delayedjobs-17aj6hbldm9spghikobe88v7k.wetransfer.com.mail>,bayes=0.00,autolearn=disabled,shortcircuit=no


Jul  4 11:57:01 mail-gw postfix/smtpd[14837]: 3rjj993Bk8zB7P: 
client=o3.email.wetransfer.com[192.254.123.42]
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: processing message 
<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail> 
for sa-milt:189
Jul  4 11:57:02 mail-gw spamd[14302]: spamd: result: . -100 - 
CUST_DNSWL_2_SENDERSC_L,CUST_DNSWL_3_JEF_L,CUST_DNSWL_5_ORG_N,CUST_DNSWL_8_TL_N,CUST_SHORTCIRCUIT,RCVD_IN_MSPIKE_H2,SHORTCIRCUIT,USER_IN_SPF_WHITELIST 
scantime=0.1,size=15685,user=sa-milt,uid=189,required_score=5.5,rhost=localhost,raddr=127.0.0.1,rport=/run/spamassassin/spamassassin.sock,mid=<577a32e8f35bb_671c116b30813485...@delayedjobs-16gux7nsdp9xgp69boio5hcsg.wetransfer.com.mail>,autolearn=disabled,shortcircuit=spam





signature.asc
Description: OpenPGP digital signature

URIBL randomly not triggered for the same message

[Reindl Harald]

i have here a message with URIBL_ABUSE_SURBL Contains an URL listed in 
the ABUSE SURBL blocklist


50% of all tries against spamd it does NOT hit while the scantime for 
the whole message is arounnd 3 seconds - since there is a local 
unbound-cache with


 cache-min-ttl: 300
 cache-max-ttl: 10800

it's impossible that there are happening dns timeouts and i can observe 
the same behavior randomly with URIBL_LOCAL where the unbound dns cache 
on 127.0.0.1:53 talks to rblsdnsd on 127.0.0.1:1053


that smells why ever very unrelieable and frankly i observed similar 
with SPF_PASS / SHORTCIRCUIT where people within 5 seconds get the same 
message and one get USER_IN_SPF_WHITELIST while the other goes through 
all tests




signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[Reindl Harald]

Am 14.06.2016 um 14:33 schrieb RW:

On Tue, 14 Jun 2016 12:40:34 +0200
Reindl Harald wrote:

when "uridnsbl" is wrong and don#t work the first paragraph just
needs to be removed


It's not wrong, uridnsbl and urirhsbl are different types of lookup. The
former targets spammer controlled web & dns servers, and looks-up IP
addresses. The latter is for actual domain look-ups.


which is not clear when it is called "uridnsbl" and don't change the 
fact the AXB's "the moaner is using a outdated doc" is bullshit by 
excellence



The name uribl.thelounge.net suggests that you do need urirhsbl[sub].

The two errors logged appear to be syntax errors:


  skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net. A
  127.0.0.2


i know - as you can see in the thread


I think this should be urirhssub or uridnssub to support the 127.0.0.2.


skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.


this is missing A or TXT


i know - as you can see in the thread



signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[RW]

On Tue, 14 Jun 2016 12:40:34 +0200
Reindl Harald wrote:


>  use
> 
>  urirhsbl BLAH uribl.thelounge.net. A
>  or
>  urirhssub BLAH uribl.thelounge.net. A 127.0.0.2
> 
>  instead of
>  uridnsbl
> 
>  so no "as said the syntax seems to be correct" it is NOT  
> >>
> >> again: what about fix
> >> https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html
> >> which says still:
> >>  
> >
> > Please suggest an proved documentation text so the devs can add it  
> 
> when "uridnsbl" is wrong and don#t work the first paragraph just
> needs to be removed

It's not wrong, uridnsbl and urirhsbl are different types of lookup. The
former targets spammer controlled web & dns servers, and looks-up IP
addresses. The latter is for actual domain look-ups. 

The name uribl.thelounge.net suggests that you do need urirhsbl[sub].


The two errors logged appear to be syntax errors:

>   skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net. A 
>   127.0.0.2

I think this should be urirhssub or uridnssub to support the 127.0.0.2.

> skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.

this is missing A or TXT.

Re: local uribl is not called

[Reindl Harald]

Am 14.06.2016 um 12:34 schrieb Tom Hendrikx:



On 14-06-16 11:47, Reindl Harald wrote:


Am 13.06.2016 um 22:53 schrieb Reindl Harald:

Am 13.06.2016 um 22:10 schrieb Axb:

HA! take a look into list and first thing you find is the moaner needing
help coz he so smart he looks at ANCIENT /3.2.x/doc instead of



https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html

use

urirhsbl BLAH uribl.thelounge.net. A
or
urirhssub BLAH uribl.thelounge.net. A 127.0.0.2

instead of
uridnsbl

so no "as said the syntax seems to be correct" it is NOT


again: what about fix
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html
which says still:



Please suggest an proved documentation text so the devs can add it


when "uridnsbl" is wrong and don#t work the first paragraph just needs 
to be removed - i wonder why the smartass (which is a dev) don't fix it 
when he talks about "coz he so smart he looks at ANCIENT /3.2.x/doc"




signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[Tom Hendrikx]

On 14-06-16 11:47, Reindl Harald wrote:
> 
> Am 13.06.2016 um 22:53 schrieb Reindl Harald:
>> Am 13.06.2016 um 22:10 schrieb Axb:
>>> HA! take a look into list and first thing you find is the moaner needing
>>> help coz he so smart he looks at ANCIENT /3.2.x/doc instead of
>>
>>> https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html
>>>
>>> use
>>>
>>> urirhsbl BLAH uribl.thelounge.net. A
>>> or
>>> urirhssub BLAH uribl.thelounge.net. A 127.0.0.2
>>>
>>> instead of
>>> uridnsbl
>>>
>>> so no "as said the syntax seems to be correct" it is NOT
> 
> again: what about fix
> https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html
> which says still:
> 

Please suggest an proved documentation text so the devs can add it.

Kind regards,
Tom

Re: local uribl is not called

[Reindl Harald]

Am 13.06.2016 um 22:53 schrieb Reindl Harald:

Am 13.06.2016 um 22:10 schrieb Axb:

HA! take a look into list and first thing you find is the moaner needing
help coz he so smart he looks at ANCIENT /3.2.x/doc instead of



https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html
use

urirhsbl BLAH uribl.thelounge.net. A
or
urirhssub BLAH uribl.thelounge.net. A 127.0.0.2

instead of
uridnsbl

so no "as said the syntax seems to be correct" it is NOT


again: what about fix 
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html 
which says still:


RULE DEFINITIONS AND PRIVILEGED SETTINGS

uridnsbl NAME_OF_RULE dnsbl_zone lookuptype

Specify a lookup. NAME_OF_RULE is the name of the rule to be used, 
dnsbl_zone is the zone to look up IPs in, and lookuptype is the type of 
lookup (TXT or A). Note that you must also define a body-eval rule 
calling check_uridnsbl() to use this.


This works by collecting domain names from URLs and querying DNS 
blocklists with an IP address of host names found in URLs or with IP 
addresses of their name servers, according to tflags as follows.


If the corresponding body rule has a tflag 'a', the DNS blocklist 
will be queried with an IP address of a host found in URLs.


If the corresponding body rule has a tflag 'ns', DNS will be 
queried for name servers (NS records) of a domain name found in URLs, 
then these name server names will be resolved to their IP addresses, 
which in turn will be sent to DNS blocklist.


Tflags directive may specify either 'a' or 'ns' or both flags. In 
absence of any of these two flags, a default is a 'ns', which is 
compatible with pre-3.4 versions of SpamAssassin.


The choice of tflags must correspond to the policy and expected use 
of each DNS blocklist and is normally not a local decision. As an 
example, a blocklist expecting queries resulting from an 'a' tflag is a 
"black_a.txt" ( http://www.uribl.com/datasets.shtml ).


Example:

 uridnsblURIBL_SBLXBLsbl-xbl.spamhaus.org.   TXT
 bodyURIBL_SBLXBLeval:check_uridnsbl('URIBL_SBLXBL')
 describeURIBL_SBLXBLContains a URL listed in the 
SBL/XBL blocklist

 tflags  URIBL_SBLXBLnet ns




signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[Reindl Harald]

Am 13.06.2016 um 22:10 schrieb Axb:

On 06/13/2016 09:12 PM, Reindl Harald wrote:


Am 13.06.2016 um 20:49 schrieb David B Funk:

On Mon, 13 Jun 2016, Reindl Harald wrote:


* the syntax seems to be correct
* domain listet and dig answers correctly on the sa-machine
* spamassassin -D < sample.eml 2> out.txt
* grep for the uribl don't show any call

uridnsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only



HA! take a look into list and first thing you find is the moaner needing
help coz he so smart he looks at ANCIENT /3.2.x/doc instead of


would you be so much smarter than me you would at least look into your 
link-target *before* post since then you had a chance to fix it before 
talking about smart so nobody seens that it's a incompatible change and 
nobody cared to fix the docs



https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html


says also "uridnsbl" so fix it or at least *read* what you are linking 
before play smartass instead become abusive

quoted from full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html

 uridnsblURIBL_SBLXBLsbl-xbl.spamhaus.org.   TXT
 bodyURIBL_SBLXBLeval:check_uridnsbl('URIBL_SBLXBL')
 describeURIBL_SBLXBLContains a URL listed in the SBL/XBL


use

urirhsbl BLAH uribl.thelounge.net. A
or
urirhssub BLAH uribl.thelounge.net. A 127.0.0.2

instead of
uridnsbl

so no "as said the syntax seems to be correct" it is NOT


so why it's mentioned in the docs you linked (in the hope they are newer 
than the radnom one i linked to) and "spamassassin --lint" nor 
"spamassassin -D" nor any logs don't complain and why the change at all 
when other sites refer to "uridnsbl"


http://uribl.com/usage.shtml
scroll down to "Datafeed users have the ability to load the 
black_nsip.txt zone locally to utilize this rule"


anyways, thanks, now it works

urirhsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only





signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[Axb]

On 06/13/2016 09:12 PM, Reindl Harald wrote:



Am 13.06.2016 um 20:49 schrieb David B Funk:

On Mon, 13 Jun 2016, Reindl Harald wrote:


* the syntax seems to be correct
* domain listet and dig answers correctly on the sa-machine
* spamassassin -D < sample.eml 2> out.txt
* grep for the uribl don't show any call

uridnsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only


with that two variants errors appear in the maillog while i don't get
what's wrong with tell the return-code here - anyways, that confirms
that the rule above seems not to be wrong

Jun 13 00:19:17 mail-gw spamd[5953]: config: SpamAssassin failed to
parse line, "URIBL_LOCAL uribl.thelounge.net. A 127.0.0.2" is not
valid for "uridnsbl", skipping: uridnsbl URIBL_LOCAL
uribl.thelounge.net. A 127.0.0.2

Jun 13 00:20:03 mail-gw spamd[5953]: config: SpamAssassin failed to
parse line, "URIBL_LOCAL uribl.thelounge.net." is not valid for
"uridnsbl", skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.


Did you "--lint" check the rules before you tried testing them?


yes, as said the syntax seems to be correct

also according to
https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html


tried also with TXT, no difference, while i would prefer A since it's
shorter in responses and in doubt with a defined response code, but
that's a nice-to-have


That 'SpamAssassin failed to parse line' error sounds like you've got a
syntax error in there


please read again: "failed to parse line" shows different tries as the
rule above (which is listed in the error message) but besides that lint
is fine with the current one it shows that the config seems to be
recognized

why "A 127.0.0.2" is wrong instead jsut "A" is a different issue and
don't matter as long as i would not have different respnse codes for
different rules / scores with a single rbldnsd zone

ignore anything below the line for the real problem


HA! take a look into list and first thing you find is the moaner needing 
help coz he so smart he looks at ANCIENT /3.2.x/doc instead of


https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html


use

urirhsbl BLAH uribl.thelounge.net. A
or
urirhssub BLAH uribl.thelounge.net. A 127.0.0.2

instead of
uridnsbl

so no "as said the syntax seems to be correct" it is NOT

Re: local uribl is not called

[Reindl Harald]

Am 13.06.2016 um 20:49 schrieb David B Funk:

On Mon, 13 Jun 2016, Reindl Harald wrote:


* the syntax seems to be correct
* domain listet and dig answers correctly on the sa-machine
* spamassassin -D < sample.eml 2> out.txt
* grep for the uribl don't show any call

uridnsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only


with that two variants errors appear in the maillog while i don't get
what's wrong with tell the return-code here - anyways, that confirms
that the rule above seems not to be wrong

Jun 13 00:19:17 mail-gw spamd[5953]: config: SpamAssassin failed to
parse line, "URIBL_LOCAL uribl.thelounge.net. A 127.0.0.2" is not
valid for "uridnsbl", skipping: uridnsbl URIBL_LOCAL
uribl.thelounge.net. A 127.0.0.2

Jun 13 00:20:03 mail-gw spamd[5953]: config: SpamAssassin failed to
parse line, "URIBL_LOCAL uribl.thelounge.net." is not valid for
"uridnsbl", skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.


Did you "--lint" check the rules before you tried testing them?


yes, as said the syntax seems to be correct

also according to 
https://spamassassin.apache.org/full/3.2.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html


tried also with TXT, no difference, while i would prefer A since it's 
shorter in responses and in doubt with a defined response code, but 
that's a nice-to-have



That 'SpamAssassin failed to parse line' error sounds like you've got a
syntax error in there


please read again: "failed to parse line" shows different tries as the 
rule above (which is listed in the error message) but besides that lint 
is fine with the current one it shows that the config seems to be recognized


why "A 127.0.0.2" is wrong instead jsut "A" is a different issue and 
don't matter as long as i would not have different respnse codes for 
different rules / scores with a single rbldnsd zone


ignore anything below the line for the real problem



signature.asc
Description: OpenPGP digital signature

Re: local uribl is not called

[David B Funk]

On Mon, 13 Jun 2016, Reindl Harald wrote:


* the syntax seems to be correct
* domain listet and dig answers correctly on the sa-machine
* spamassassin -D < sample.eml 2> out.txt
* grep for the uribl don't show any call

uridnsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only


with that two variants errors appear in the maillog while i don't get what's 
wrong with tell the return-code here - anyways, that confirms that the rule 
above seems not to be wrong


Jun 13 00:19:17 mail-gw spamd[5953]: config: SpamAssassin failed to parse 
line, "URIBL_LOCAL uribl.thelounge.net. A 127.0.0.2" is not valid for 
"uridnsbl", skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net. A 127.0.0.2


Jun 13 00:20:03 mail-gw spamd[5953]: config: SpamAssassin failed to parse 
line, "URIBL_LOCAL uribl.thelounge.net." is not valid for "uridnsbl", 
skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.


Did you "--lint" check the rules before you tried testing them?

That 'SpamAssassin failed to parse line' error sounds like you've got a syntax 
error in there.



--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

smime.p7s
Description: S/MIME Cryptographic Signature

local uribl is not called

[Reindl Harald]

* the syntax seems to be correct
* domain listet and dig answers correctly on the sa-machine
* spamassassin -D < sample.eml 2> out.txt
* grep for the uribl don't show any call

uridnsbl   URIBL_LOCAL  uribl.thelounge.net.  A
body   URIBL_LOCAL  eval:check_uridnsbl('URIBL_LOCAL')
describe   URIBL_LOCAL  Contains an URL listed in the URIBL blacklist
score  URIBL_LOCAL  0.1
tflags URIBL_LOCAL  net domains_only


with that two variants errors appear in the maillog while i don't get 
what's wrong with tell the return-code here - anyways, that confirms 
that the rule above seems not to be wrong


Jun 13 00:19:17 mail-gw spamd[5953]: config: SpamAssassin failed to 
parse line, "URIBL_LOCAL uribl.thelounge.net. A 127.0.0.2" is not valid 
for "uridnsbl", skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net. A 
127.0.0.2


Jun 13 00:20:03 mail-gw spamd[5953]: config: SpamAssassin failed to 
parse line, "URIBL_LOCAL uribl.thelounge.net." is not valid for 
"uridnsbl", skipping: uridnsbl URIBL_LOCAL uribl.thelounge.net.




signature.asc
Description: OpenPGP digital signature

URIBL dependency failures

[Alex]

Hi,

I've noticed a few of my meta rules involving URIBL_AB_SURBL and
others are failing with dependency errors because apparently the SURBL
rule names have changed, or are changing very soon:

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7279

This includes some rules in the KAM.cf as well as others:

Apr 22 23:15:32.932 [19346] dbg: rules: meta test SHORT_URIBL has
undefined dependency 'URIBL_AB_SURBL'
Apr 22 23:15:32.932 [19346] dbg: rules: meta test SHORT_URIBL has
undefined dependency 'URIBL_JP_SURBL'
Apr 22 23:15:32.932 [19346] dbg: rules: meta test SHORT_URIBL has
undefined dependency 'URIBL_SC_SURBL'

This is somewhat of a public service announcement for those of you who
may also be affected. It appears to me that the URIBL rules above that
are failing have all been replaced with the one URIBL_ABUSE_SURBL
rule.

Regards,
Alex

Re: URIBL/DNSBL from a database

[Alex]

Hi,

>> Is there any reason to not use the bl.score.sendrescore.com with
>> postscreen? I don't understand the distinction
>
> why?
>
> postscreen is supposed to be configured with sensible scoring to reject most
> spam without false positives long before it reachs smtpd or even expesnive
> contentfilters
>
> hence the scoring and any sensible setup would use postscreen combined with
> several whitelists
>
> that way your contentfilter has only to deal with the remaining 10% of junk
> and when you optimize postscreen to use a honeypot-MX (backup mx on a second
> IP with a postscreen whitelist_veto) and enforce pre-greet tests with a
> larger wait time there is not much for SpamAssasin to deal with

No, no, no. That's not at all what I mean. I know what the purpose and
benefit of postscreen is.

My issue relates to why is score.senderscore.com used with postscreen,
and not bl.score.senderscore.com as it is with SA?

Perhaps it should be as well?

The postscreen weights for score.senderscore.com are such that they
are relative to the threshold, so a reputation of say, 70 would
receive a higher score than a reputation of say, 90. In fact, 90
removes points.

And why is only bl.score.senderscore.com used with SA, and not the
reputation system?

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 03.03.2016 um 02:44 schrieb Alex:

Is there any reason to not use the bl.score.sendrescore.com with
postscreen? I don't understand the distinction


why?

postscreen is supposed to be configured with sensible scoring to reject 
most spam without false positives long before it reachs smtpd or even 
expesnive contentfilters


hence the scoring and any sensible setup would use postscreen combined 
with several whitelists


that way your contentfilter has only to deal with the remaining 10% of 
junk and when you optimize postscreen to use a honeypot-MX (backup mx on 
a second IP with a postscreen whitelist_veto) and enforce pre-greet 
tests with a larger wait time there is not much for SpamAssasin to deal with




signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Alex]

Hi,

Some time ago, David Jones wrote:
> In a related note, I have found that using the senderscore.org score combined
> with postscreen's weighting is very effective in quickly catching new 
> spammers.
>
> postscreen_dnsbl_sites =
>   score.senderscore.com=127.0.4.[60..69]*2
>   score.senderscore.com=127.0.4.[50..59]*4
>   score.senderscore.com=127.0.4.[30..49]*6
>   score.senderscore.com=127.0.4.[0..29]*8
>   score.senderscore.com=127.0.4.[90..100]*-6
>   score.senderscore.com=127.0.4.[80..89]*-4
>   score.senderscore.com=127.0.4.[70..79]*-2

This has been quite effective, but there have also been some
false-positives which I've had to whitelist. I've lowered the 0-29
result a bit so as to not make it a poison pill in my case.

I also probably should have asked at the time what your
postscreen_dnsbl_threshold is? Mine is 8.

Can someone explain how this differs from the bl.score.senderscore.com
that's used in the RCVD_IN_RP_RNBL rule?

Is there any reason to not use the bl.score.sendrescore.com with
postscreen? I don't understand the distinction.

Does anyone know where the return result codes are defined? I've
looked all over the senderscore website and can't find them.

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Noel Butler]

On 16/02/2016 01:08, Shawn Bakhtiar wrote:



There are A LOT more people out there, far greater than just the
Googles and Yahoos of the world, and to block IP addresses/subnets
without an automated system using definable metric (that usually is
enterprise specific), invariably IT will be inundated with complaints
about users not receiving legitimate vendor emails.




Thats the entire point though, as it has been for over 20 years.

admins shrug off badguy-complaints, badguy complaints go to rbl, rbl 
blocks, rbl gets notified badguy uses more resources,  rbl blocks wider 
range due to other IPs used


It's much much harder for admins to shrug off their own customers 
complaints, so admin gets off lazy useless arse and sorts out the badguy 
like should have in first place, rbl then removes blocks... life goes 
on..


--
If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[Shawn Bakhtiar]

I use to spend a lot of time blocking hosts and subnets, using IP tables, of 
malicious providers who would let any tom, dick, and Harry (no pun intended) to 
host spam hosts/relays on their servers. What I ended up doing is also blocking 
a lot SMB vendors from sending legitimate emails to users because most SMBs 
outsource their services without really comprehending the consequences of the 
provider they choose, this is especially true for low tech industries such as 
toll and process manufacturing companies, and frankly led to a management 
nightmare.

There are A LOT more people out there, far greater than just the Googles and 
Yahoos of the world, and to block IP addresses/subnets without an automated 
system using definable metric (that usually is enterprise specific), invariably 
IT will be inundated with complaints about users not receiving legitimate 
vendor emails.

It is much more effective to use existing RBLs, and supplementing it with your 
own honeypot RBL that uses metrics developed in house that can react to what 
your organization will consider the critical mass of spam it can take. That, 
along with the proper training of SA, is perhaps the best defense you can have. 
Using metric like last seen, total count, and frequency seem to provide the 
best metrics for me, my private RBL (based on honeypot addresses) can react 
faster than the big guys, on both ends of the equation (to block and to 
release), It's not that Google doesn't sometimes land on my RBL, it's that it 
also drops off fast as they remedy the issue, and the time outs are reached and 
they drop off my list.



> On Feb 14, 2016, at 10:19 PM, Noel Butler  wrote:
> 
> On 15/02/2016 09:02, Reindl Harald wrote:
>> Am 14.02.2016 um 23:34 schrieb Noel Butler:
>>> On 14/02/2016 01:46, Alex wrote:
 rejecting outright at the SMTP level for IPs reaching my honeypots
 could be dangerous if not checked.
>>> how so? if your honey pots use specific non human used (ever) addresses,
>>> then there should never ever be a genuine mail destined for it.
>>> I dont care who the connector is, be it foobar.com or gmail.com if they
>>> relay it, they are listed, its where spamhaus and I always disagreed,
>>> because what they are doing is sending a clear message to spammers to
>>> simply "use gmail" to avoid being listed in spamhaus.
>>> You are never too big to be stuffed into a dnsbl, there are a number of
>>> well known bl's that have been around for over ten years that also take
>>> that approach.
>> you missed to say that you are the type RBL operator which lists whole
>> subnets (in not only personal RBL's) because you don't like specific
>> people on mailing-lists
> 
> 
> Ohh, so you wanna bring this up again in public do you, fine by me... lets 
> have some history though shall we Harry...
> 
> Most DNSBL's blacklist spam *and* abusive hosts, there is no question about 
> you spamming, I know you don't and would never do that, but you are/were a 
> very very aggressively abusive person - this is supported by all those 
> mailing lists bannings/moderations you've copped over recent years which we 
> need both hands to count, the listing I placed on you was not just because of 
> the abuse and blackmailing you leveled at me, but number of complaints we 
> received also.
> 
> Further more, most people who've had interactions with you over the past 
> couple of years, espeically those that you've disagreed with also know how 
> you used to act, and occasionally still come close to, because you think you 
> are always right and anyone who disagrees with you is the anti christ or 
> something.
> 
> Ordinarily this does just warrant a /32 listing, however as a system 
> administrator with access to at least a /24, and evidence of your mailing 
> list ghost accounts, including at least one I recall from another IP in that 
> /24 a while back, yes, I took the step to block your /24.
> 
> 
>> also you don't realize that this don't stop any single mail from a
>> list sent by that person but just harms other domains using the SMTP
>> server
> 
> I realise a lot more than you think, as I've told you, and told you, and told 
> you, its up to lists what DNSBL's if any they use, but you are known to, on 
> the lists youve been moderated on, send abusive messages to recipients 
> directly since you can't via the lists
> so it does have a catching effect of those who use it.
> 
>> so *you* are hardly in the position for education about RBL's since
>> you don't care about any collateral damage but only your ego
> 
> You are entitled to your opinion, I care about valid collateral damage, if 
> you abuse an employers resources and your employers customers are caught up 
> on it, your employer, if they care, would take appropriate action, it is no 
> different than blocking a domain for spamming, forcing the host to clean up 
> its act and get rid of its spamming clients, of course at no time did I wish 
> to see your employment 

Re: URIBL/DNSBL from a database

[Noel Butler]

On 15/02/2016 09:02, Reindl Harald wrote:

Am 14.02.2016 um 23:34 schrieb Noel Butler:

On 14/02/2016 01:46, Alex wrote:


rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.


how so? if your honey pots use specific non human used (ever) 
addresses,

then there should never ever be a genuine mail destined for it.

I dont care who the connector is, be it foobar.com or gmail.com if 
they

relay it, they are listed, its where spamhaus and I always disagreed,
because what they are doing is sending a clear message to spammers to
simply "use gmail" to avoid being listed in spamhaus.

You are never too big to be stuffed into a dnsbl, there are a number 
of
well known bl's that have been around for over ten years that also 
take

that approach.


you missed to say that you are the type RBL operator which lists whole
subnets (in not only personal RBL's) because you don't like specific
people on mailing-lists




Ohh, so you wanna bring this up again in public do you, fine by me... 
lets have some history though shall we Harry...


Most DNSBL's blacklist spam *and* abusive hosts, there is no question 
about you spamming, I know you don't and would never do that, but you 
are/were a very very aggressively abusive person - this is supported by 
all those mailing lists bannings/moderations you've copped over recent 
years which we need both hands to count, the listing I placed on you was 
not just because of the abuse and blackmailing you leveled at me, but 
number of complaints we received also.


Further more, most people who've had interactions with you over the past 
couple of years, espeically those that you've disagreed with also know 
how you used to act, and occasionally still come close to, because you 
think you are always right and anyone who disagrees with you is the anti 
christ or something.


Ordinarily this does just warrant a /32 listing, however as a system 
administrator with access to at least a /24, and evidence of your 
mailing list ghost accounts, including at least one I recall from 
another IP in that /24 a while back, yes, I took the step to block your 
/24.




also you don't realize that this don't stop any single mail from a
list sent by that person but just harms other domains using the SMTP
server



I realise a lot more than you think, as I've told you, and told you, and 
told you, its up to lists what DNSBL's if any they use, but you are 
known to, on the lists youve been moderated on, send abusive messages to 
recipients directly since you can't via the lists

so it does have a catching effect of those who use it.


so *you* are hardly in the position for education about RBL's since
you don't care about any collateral damage but only your ego


You are entitled to your opinion, I care about valid collateral damage, 
if you abuse an employers resources and your employers customers are 
caught up on it, your employer, if they care, would take appropriate 
action, it is no different than blocking a domain for spamming, forcing 
the host to clean up its act and get rid of its spamming clients, of 
course at no time did I wish to see your employment terminated, just 
actions reigned in, resulting in cleaner transmissions, allowing for 
removal of blocking, just like networks that clean up spam.


I have seen you have remarkable behaved yourself in past 6 months 
compared to how you used to carry on, your still no saint, but no one 
including me is either.


This list is also off topic and I apologise to Gunther and co for 
replying to it on list, but some things needed to be said. No doubt 
Harry will rant and rave and carry on trollbaiting me, but I will try 
with-hold any further responses since, we are, well and truly OT.


Have a nice day.

--


If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 14.02.2016 um 23:34 schrieb Noel Butler:

On 14/02/2016 01:46, Alex wrote:


rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.


how so? if your honey pots use specific non human used (ever) addresses,
then there should never ever be a genuine mail destined for it.

I dont care who the connector is, be it foobar.com or gmail.com if they
relay it, they are listed, its where spamhaus and I always disagreed,
because what they are doing is sending a clear message to spammers to
simply "use gmail" to avoid being listed in spamhaus.

You are never too big to be stuffed into a dnsbl, there are a number of
well known bl's that have been around for over ten years that also take
that approach.


you missed to say that you are the type RBL operator which lists whole 
subnets (in not only personal RBL's) because you don't like specific 
people on mailing-lists


also you don't realize that this don't stop any single mail from a list 
sent by that person but just harms other domains using the SMTP server


so *you* are hardly in the position for education about RBL's since you 
don't care about any collateral damage but only your ego





signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Noel Butler]

On 14/02/2016 01:46, Alex wrote:




rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.




how so? if your honey pots use specific non human used (ever) addresses, 
then there should never ever be a genuine mail destined for it.


I dont care who the connector is, be it foobar.com or gmail.com if they 
relay it, they are listed, its where spamhaus and I always disagreed, 
because what they are doing is sending a clear message to spammers to 
simply "use gmail" to avoid being listed in spamhaus.


You are never too big to be stuffed into a dnsbl, there are a number of 
well known bl's that have been around for over ten years that also take 
that approach.



--
If you have the urge to reply to all rather than reply to list, you best
first read  http://members.ausics.net/qwerty/

Re: URIBL/DNSBL from a database

[John Hardin]

On Sun, 14 Feb 2016, Allen Chen wrote:


On 2/12/2016 8:48 AM, Axb wrote:

 On 02/12/2016 02:39 PM, Alex wrote:
>  For some time now I've been cycling URLs and IPs through  a mariadb
>  database gathered from incoming mail on a honeypot I've created.
>  Surprising how many are received ahead of spamhaus/barracuda.
> 
>  I'm looking for ideas on how to now make this information available to

>  spamassassin on my production system. I'd like to somehow export the
>  IPs, any URLs in the body, and email addresses to spamassassin.
> 
>  Is it possible for spamassassin to query a database directly?


Did you try iptables to block/allow IPs?


If you're getting that much abuse from specific IPs and you're sure that 
it's all spam, then set up a TCP tarpit.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Ignorance is no excuse for a law.
---
 8 days until George Washington's 284th Birthday

Re: URIBL/DNSBL from a database

[Allen Chen]

On 2/12/2016 8:48 AM, Axb wrote:

On 02/12/2016 02:39 PM, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?

Did you try iptables to block/allow IPs?



You'd need a custom plugin query the DB directly.



I'm familiar with how to create a uridnsbl, but is DNS the best
approach here?

DNS is cheap/reliable and simple to deploy / load balance.


The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


rbldnsd can check and load fresh data instantly within seconds.
If your dataset is not HUGE (loading 100MB zones is slow) rbldnspy 
will take inmemory updates so instant listings...

https://github.com/gryphius/rbldnspy






--
Allen Chen
Network Administrator
IT

Harbourfront Centre

235 Queens Quay West, Toronto, ON
M5J 2G8, Canada | harbourfrontcentre.com 
Office: +1 416 973 7973
Cell: +1 416 556 2493

Re: URIBL/DNSBL from a database

[David Jones]

>> DNS is very effective to block at the MTA level.  I setup my own private
>> RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
>> rbldnsd formatted zone file and setup your private RBL zone (doesn't
>> have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
>> will detect changes to it's zone files and reload them automatically to
>> keep current.

>Do you have some kind of whitelist that includes gmail, yahoo, etc?

Yes. My database query excludes FREEMAIL hits.   I also use/parse SPF
records of many of the large FREEMAIL domains to allow these in before
RBL checks.  You also have to whitelist many of these from greylisting too
and let SA score them.

>I'm not looking to compete with spamhaus, just compliment it, but
>rejecting outright at the SMTP level for IPs reaching my honeypots
>could be dangerous if not checked.

I don't have any honeypots so I can't speak from experience but I
would think you would need to filter these differently -- much more
relaxed than real user domains and mailboxes.   If your honeypot
addresses are on a different domain, send them through a different
MTA config that doesn't have all of these RBL checks.

>I've now got rbldnsd implemented. I've also known for a while it's
>faster/better than bind, but bind has always been in place.

>I have rbldnsd running on port 530, alongside bind on 53. How do I
>specify a urirhsbl in spamassassin to query the DNS server running on
>530 instead of 53?

You setup BIND to forward that zone of your own RBL to localhost:530.
http://www.surbl.org/setup-local-rbl-mirror  (toward the bottom)
rbldnsd only has to be listening on 127.0.0.1:530

>> In a related note, I have found that using the senderscore.org score combined
>> with postscreen's weighting is very effective in quickly catching new 
>> spammers.
>>
>> postscreen_dnsbl_sites =
>>   score.senderscore.com=127.0.4.[60..69]*2
>>   score.senderscore.com=127.0.4.[50..59]*4
>>   score.senderscore.com=127.0.4.[30..49]*6
>>   score.senderscore.com=127.0.4.[0..29]*8
>>   score.senderscore.com=127.0.4.[90..100]*-6
>>   score.senderscore.com=127.0.4.[80..89]*-4
>>   score.senderscore.com=127.0.4.[70..79]*-2
>>
>> You should monitor your own outbound IPs for their sender score.  If your
>> IP goes below 90, it's a good indication that you have been sending spam
>> and that your users are going to start experiencing delivery issues to the
>> Internet.

>Do you use this on inbound mail as well?

Yes.  Definitely use this primarily on inbound email.  I also use
some RBLs on outbound email to help detect compromised
accounts but make sure you have your internal_networks and
trusted_networks properly so SA will work with external IPs
properly.

>How does it fit with the other postscreen dnsbls? I already have at
>least six various dnsbls with varying weights...

I have more than a dozen in addition to the ones above.  You simply
list as many RBLs as you want with the proper weighting you think
based on their reliability/trustworthiness for your environment.
Negative numbers are used for reliable RBLs that show a good reputation
for the sending mail server IP.  Positive numbers go higher toward
the threshold number (I use 8 like many examples I have seen).  Set
your own private RBL at or slightly above your threshold along with
other trustworthy RBLs like zen.spamhaus.org.  Only use negative
number weighting for those RBLs that you have confirmed to be
good sources for good reputation.

Re: URIBL/DNSBL from a database

[Alex]

Hi,

> DNS is very effective to block at the MTA level.  I setup my own private
> RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
> rbldnsd formatted zone file and setup your private RBL zone (doesn't
> have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
> will detect changes to it's zone files and reload them automatically to
> keep current.

Do you have some kind of whitelist that includes gmail, yahoo, etc?

I'm not looking to compete with spamhaus, just compliment it, but
rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked.

I've now got rbldnsd implemented. I've also known for a while it's
faster/better than bind, but bind has always been in place.

I have rbldnsd running on port 530, alongside bind on 53. How do I
specify a urirhsbl in spamassassin to query the DNS server running on
530 instead of 53?

> In a related note, I have found that using the senderscore.org score combined
> with postscreen's weighting is very effective in quickly catching new 
> spammers.
>
> postscreen_dnsbl_sites =
>   score.senderscore.com=127.0.4.[60..69]*2
>   score.senderscore.com=127.0.4.[50..59]*4
>   score.senderscore.com=127.0.4.[30..49]*6
>   score.senderscore.com=127.0.4.[0..29]*8
>   score.senderscore.com=127.0.4.[90..100]*-6
>   score.senderscore.com=127.0.4.[80..89]*-4
>   score.senderscore.com=127.0.4.[70..79]*-2
>
> You should monitor your own outbound IPs for their sender score.  If your
> IP goes below 90, it's a good indication that you have been sending spam
> and that your users are going to start experiencing delivery issues to the
> Internet.

Do you use this on inbound mail as well?

How does it fit with the other postscreen dnsbls? I already have at
least six various dnsbls with varying weights...

Thanks,
Alex

Re: URIBL/DNSBL from a database

[Reindl Harald]

Am 13.02.2016 um 16:46 schrieb Alex:

DNS is very effective to block at the MTA level.  I setup my own private
RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
rbldnsd formatted zone file and setup your private RBL zone (doesn't
have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
will detect changes to it's zone files and reload them automatically to
keep current.


Do you have some kind of whitelist that includes gmail, yahoo, etc?

I'm not looking to compete with spamhaus, just compliment it, but
rejecting outright at the SMTP level for IPs reaching my honeypots
could be dangerous if not checked


something PTR based like below is a good start

snippet of our in PHP written honeypot daemon at the bottom, and yes you 
can write a proper network service in PHP listening not only on port 25

_

 /** chroot to runtime directory and change basedir for later operations */
 if(chroot(__DIR__))
 {
  $chroot_basedir = '/honeypot-chroot';
 }
 else
 {
  $chroot_basedir = __DIR__;
 }

 /** drop privileges to 'nobody' */
 if(!posix_initgroups('nobody', $nobody_group) || 
!posix_setgid($nobody_group) || !posix_setuid($nobody_user))

 {
  error_log('ERROR: Drop privileges failed (' . $port . ')');
  exit('ERROR: Drop privileges failed (' . $port . ')' . "\n");
 }
_

 /**
  * Grosse Provider und offensichtliche Mailserver von automatischem
  * Blacklisting ausnehmen Basis ist der Reverse-DNS
  *
  * Gibt 'true' zurueck wenn die IP zu ignorieren ist
  * Honeypot speichert somit nur die Spam-Samples
  *
  * @param  string $ptr
  * @return boolean
  * @access public
 */
 function ignore_blacklist_ptr($ptr)
 {
  /** Sonderbehandlung */
  if(strpos($ptr, 'smtp') !== false || strpos($ptr, 'mail') !== false 
|| strpos($ptr, 'mxout') !== false)

  {
   return true;
  }
  /** Zu ignorierende PTR-Ends */
  $ignored = array
  (
   '.ac.at',
   '.apple.com',
   '.ebay.com',
   '.eyepin.com',
   '.facebook.com',
   '.gmx.at',
   '.gmx.com',
   '.gmx.de',
   '.gmx.net',
   '.google.com',
   '.gv.at',
   '.itronic.at',
   '.itronic.at',
   '.kundenserver.de',
   '.microsoft.com',
   '.mx.aol.com',
   '.mx.aol.com',
   '.observer.at',
   '.office-vienna.at',
   '.orf.at',
   '.outlook.com',
   '.paylife.at',
   '.paypal.com',
   '.phx3.secureserver.net',
   '.pinterest.com',
   '.skype.com',
   '.smtp-out.amazonses.com',
   '.thelounge.net',
   '.twitter.com',
   '.web.de',
   '.wetransfer.com',
   '.xing.com',
   '.yahoo.co.jp',
   '.yahoo.com',
   'taro.utanet.at',
   'tatiana.utanet.at',
  );
  /** Durchlaufen und gegen PTR testen */
  foreach($ignored as $test)
  {
   if(strpos($ptr, $test) !== false)
   {
$xtest = substr($ptr, strlen($ptr)-strlen($test));
if($xtest == $test)
{
 return true;
 break;
}
   }
  }
  /** Wenn nicht gelistet 'false' zurueckgeben */
  false;
 }



signature.asc
Description: OpenPGP digital signature

Re: URIBL/DNSBL from a database

[Dave Funk]

On Sat, 13 Feb 2016, Alex wrote:


I've now got rbldnsd implemented. I've also known for a while it's
faster/better than bind, but bind has always been in place.

I have rbldnsd running on port 530, alongside bind on 53. How do I
specify a urirhsbl in spamassassin to query the DNS server running on
530 instead of 53?


One way to do this is to set up a "forward only" zone in your bind config.

For example, assume you're authoritative for "example.com" and you've got
your rbldnsd set up to serve up your data as zone "mybl.example.com" and
it's bound to 192.168.124.23/530

Then in your bind config file create a zone:

zone "mybl.example.com" {
type forward;
forward only;
forwarders {
192.168.124.23 port 530;
};
};

Then when your clients (spamd or regular dns tools) query
"blah.com.mybl.example.com" it will hit your bind and then
get passed on to your rbldnsd for an answer.

If you want to hide that resource from the world put that zone
in a private 'view' in your bind. You could control access via an
ACL but by putting it inside a private view they'll never even see it
to try pounding on it.

To provide fault tolerance, you can set up rbldnsd's on multiple
machines and put multiple addresses in that 'forwarders' stanza.
You will need to put that zone definition in your primary bind and
each secondary.

--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: URIBL/DNSBL from a database

[Martin Gregorie]

On Fri, 2016-02-12 at 08:39 -0500, Alex wrote:
> Is it possible for spamassassin to query a database directly?
> 
Yes, with a plugin. 

I've been doing the opposite for some years now: I archive all my
outgoing mail and most of my non-spam incoming mail in a Postgres
database and use this as a whitelist: incoming mail from anybody that
I've sent mail to gets whitelisted. I use a plugin to query the
database via a view: the view is there to present the list of addresses
to which I've sent mail to the plugin's SQL query: its needed for
performance reasons because the database uses a many-to-many structure
to associate addresses with the messages they send or receive. 

It should be simple enough to change my plugin's query to work with
your database, particularly if you already have a table containing the
addresses you'd like to blacklist. Likewise, its probably fairly simple
to extend it to deal with the URLs and IPs from message bodies. 

If you'd like a copy of the plugin plus the associated .cf file[*],
contact me offlist.


Martin

[*] this loads and configures the plugin with database login details
and defines the rule that whitelists hits.

URIBL/DNSBL from a database

[Alex]

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?

I'm familiar with how to create a uridnsbl, but is DNS the best
approach here? The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.

Is anyone else doing this, and are you just rejecting the IPs at the
SMTP level outright?

Thanks,
Alex

Re: URIBL/DNSBL from a database

[David Jones]

>
>From: Alex 

>For some time now I've been cycling URLs and IPs through  a mariadb
>database gathered from incoming mail on a honeypot I've created.
>Surprising how many are received ahead of spamhaus/barracuda.

Major RBLs like that keep up with lots of data points for IP reputation
over time so that can give a little extra time for normally reputable IPs
that happen to have a compromised account -- which happens to us
all.  But if you don't detect compromised accounts on your system
through feedback loops and abuse reports, then a reputable IP can
eventually get listed on those major RBLs.

>Is anyone else doing this, and are you just rejecting the IPs at the
>SMTP level outright?

DNS is very effective to block at the MTA level.  I setup my own private
RBL on the DNS servers my SA boxes point to.  Dump your IPs into a
rbldnsd formatted zone file and setup your private RBL zone (doesn't
have to be a real zone on the Internet) to forward to rbldnsd.  Rbldnsd
will detect changes to it's zone files and reload them automatically to
keep current.

Then I have a nightly script that goes through my list of IPs in my private
RBL to remove them if they show up in another major RBL that I use.  This
prevents my list from becoming stale in the event that the IP becomes
delisted from the public RBLs.

In a related note, I have found that using the senderscore.org score combined
with postscreen's weighting is very effective in quickly catching new spammers.

postscreen_dnsbl_sites =
  score.senderscore.com=127.0.4.[60..69]*2
  score.senderscore.com=127.0.4.[50..59]*4
  score.senderscore.com=127.0.4.[30..49]*6
  score.senderscore.com=127.0.4.[0..29]*8
  score.senderscore.com=127.0.4.[90..100]*-6
  score.senderscore.com=127.0.4.[80..89]*-4
  score.senderscore.com=127.0.4.[70..79]*-2

You should monitor your own outbound IPs for their sender score.  If your
IP goes below 90, it's a good indication that you have been sending spam
and that your users are going to start experiencing delivery issues to the
Internet.

Dave

Re: URIBL/DNSBL from a database

[Axb]

On 02/12/2016 02:39 PM, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?


You'd need a custom plugin query the DB directly.



I'm familiar with how to create a uridnsbl, but is DNS the best
approach here?

DNS is cheap/reliable and simple to deploy / load balance.


The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


rbldnsd can check and load fresh data instantly within seconds.
If your dataset is not HUGE (loading 100MB zones is slow) rbldnspy will 
take inmemory updates so instant listings...

https://github.com/gryphius/rbldnspy

Re: URIBL/DNSBL from a database

[Shawn Bakhtiar]

On Feb 12, 2016, at 5:39 AM, Alex 
> wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spam assassin.

DNSBLs are very effective at this task, and I would recommend using before you 
filter the email with SA, unless you specifically want to score, due to 
uncertainty.


Is it possible for spamassassin to query a database directly?

It is:
https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_URIDNSBL.html

But even than I find it more effective having the server running the DNSBL 
manage the the block list using some metrics such as number of times the IP 
address appears, and/or not recording ip addresses in a whitelist table etc... 
Once (either via blacklist or metric) the IP gets into the DNSBL there is no 
need for me to worry about SA, simply reject. I find URI tend to change A LOT, 
so IP based blocking can be much more effective. But I think that's more of a 
preference.


I'm familiar with how to create a uridnsbl, but is DNS the best
approach here? The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.


That's the way I do it. using bind DLZ http://bind-dlz.sourceforge.net/
We have a delegated subdomain off our main domain that hosts a DNS exclusively 
used for block list, created from incoming mail sent to honeypot email address 
(ones that are no never were/or are no longer valid). Again I tend to focus on 
the IP address not the URI as a find that URI are dime a dozen and change quite 
frequently.

Is anyone else doing this, and are you just rejecting the IPs at the
SMTP level outright?

We use sendmail features to reject long before it gets to SA. It works better 
(IMHO) since there is much lower over head for sendmail doing a quick DNS 
lookup than engaging the milter that runs the email through it's passes with SA.

http://weldon.whipple.org/sendmail/dnsbl.html

But in this case it's IP based only not URI based. For URI (especially ones 
that you'll want to regex) SA may be more effective.


Thanks,
Alex

Re: URIBL/DNSBL from a database

[Marc Perkel]

On 02/12/16 05:39, Alex wrote:

Hi,

For some time now I've been cycling URLs and IPs through  a mariadb
database gathered from incoming mail on a honeypot I've created.
Surprising how many are received ahead of spamhaus/barracuda.

I'm looking for ideas on how to now make this information available to
spamassassin on my production system. I'd like to somehow export the
IPs, any URLs in the body, and email addresses to spamassassin.

Is it possible for spamassassin to query a database directly?

I'm familiar with how to create a uridnsbl, but is DNS the best
approach here? The info needs to be updated and reloaded rapidly, and
not all the info (URLs, emails) are conducive to being in DNS.

Is anyone else doing this, and are you just rejecting the IPs at the
SMTP level outright?

Thanks,
Alex




Yeah - unless you write your own SA module using DNS is the quick easy 
solution.


--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: URIBL/DNSBL from a database

[Martin Gregorie]

On Fri, 2016-02-12 at 07:30 -0800, Marc Perkel wrote:

> Yeah - unless you write your own SA module using DNS is the quick
> easy solution.
> 
If Alex already has a set of scripts that populate and maintain the
database that he's happy with, then the quick and easy way may be to
make a custom SA module by using my database access module as a
starting point. 

The benefits would be that he's already familiar the care and feeding
of his database and that he can update it any time without needing to
stop and restart anything.


Martin

Re: shortcircuit dnsbl/uribl

[Reindl Harald]

Am 02.06.2015 um 16:30 schrieb RW:

On Tue, 02 Jun 2015 14:36:07 +0200
Reindl Harald wrote:


given that USER_IN_SPF_WHITELIST score with -100 here there is no
real point to fire up all the other tests, it's clear anyways that
this message will pass


As far as possible spamassassin does network test in parallel with each
other and with the local tests. Making one set of network tests
conditional on another set slows down the scanning of all mail


i doubt it would slow down since i could skip around 50 dns requests for 
30% of all scanned messages if i just could order whitelist_auth with a 
priority before other network tests


so even if the remaining 70% would get a small slowdown it won't matter, 
90% of all inbound mail don't touch SA at all




signature.asc
Description: OpenPGP digital signature

Re: shortcircuit dnsbl/uribl

[RW]

On Wed, 03 Jun 2015 11:22:42 +0200
Reindl Harald wrote:

 
 Am 02.06.2015 um 16:30 schrieb RW:
  On Tue, 02 Jun 2015 14:36:07 +0200
  Reindl Harald wrote:
 
  given that USER_IN_SPF_WHITELIST score with -100 here there is no
  real point to fire up all the other tests, it's clear anyways that
  this message will pass
 
  As far as possible spamassassin does network test in parallel with
  each other and with the local tests. Making one set of network tests
  conditional on another set slows down the scanning of all mail
 
 i doubt it would slow down since i could skip around 50 dns requests
 for 30% of all scanned messages 

Skipping those DNS requests would save some network traffic, but not
much else. Within SA itself sending requests and processing responses
are both cheap; the expensive part is waiting for slow responses and
time-outs and that will be short-circuited. 


 so even if the remaining 70% would get a small slowdown it won't
 matter, 90% of all inbound mail don't touch SA at all

Which means that any savings made to network traffic by short-circuiting
would be diluted by the DNS look-ups on the 90%.  Also whether or not
the slowdown is small, comparing it with a hypothetical load that's 10
times your actual load is bogus.

Re: shortcircuit dnsbl/uribl

[RW]

On Tue, 02 Jun 2015 14:36:07 +0200
Reindl Harald wrote:


 given that USER_IN_SPF_WHITELIST score with -100 here there is no
 real point to fire up all the other tests, it's clear anyways that
 this message will pass

As far as possible spamassassin does network test in parallel with each
other and with the local tests. Making one set of network tests
conditional on another set slows down the scanning of all mail. 

shortcircuit dnsbl/uribl

[Reindl Harald]

is there a way to skip DNSBL/URIBL if a message hits the rule below, i 
tried to define dnsbl-rules with priority CUST_DNSBL -450 but that 
don't change anything


given that USER_IN_SPF_WHITELIST score with -100 here there is no real 
point to fire up all the other tests, it's clear anyways that this 
message will pass


meta SHORTCIRCUIT_NET_HAM (USER_IN_DKIM_WHITELIST || 
USER_IN_SPF_WHITELIST)

priority SHORTCIRCUIT_NET_HAM -500
shortcircuit SHORTCIRCUIT_NET_HAM on
scoreSHORTCIRCUIT_NET_HAM -0.001
describe SHORTCIRCUIT_NET_HAM Skip tests for SPF/DKIM whitelisted 
senders





signature.asc
Description: OpenPGP digital signature

URIBL plugins are broken

[Reindl Harald]

i face false positives where the links are just facebook.com with the 
http-prefix in front and NOT com between the http-prefix and the real 
facebook domain


the domain with com in front is indeed on both URIBL but it just don#t 
exist in the messages at all - why does SA extract the domains wrong 
from the mailsource when there is no comfacebook at all besides the SA 
report?


URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]



signature.asc
Description: OpenPGP digital signature

Re: URIBL plugins are broken

[Kevin A. McGrail]

On 5/11/2015 9:46 AM, Reindl Harald wrote:

stripped down and anonymized sample attached

the real bad thing is that the part triggering the URIBL rules wrongly 
is the quote of the signature from the message replied to


Am 11.05.2015 um 15:13 schrieb Reindl Harald:

i face false positives where the links are just facebook.com with the
http-prefix in front and NOT com between the http-prefix and the real
facebook domain

the domain with com in front is indeed on both URIBL but it just don#t
exist in the messages at all - why does SA extract the domains wrong
from the mailsource when there is no comfacebook at all besides the SA
report?

URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]




Not a bug in SA.

The plain text version of the email contains: 
a...@sepashvili.comfacebook.com/ketevan.sepashvili


The subdomain sepashvili is dropped leaving comfacebook.com.

Regards,
KAM

Re: URIBL plugins are broken

[Kevin A. McGrail]

On 5/11/2015 9:13 AM, Reindl Harald wrote:
i face false positives where the links are just facebook.com with 
the http-prefix in front and NOT com between the http-prefix and the 
real facebook domain


the domain with com in front is indeed on both URIBL but it just 
don#t exist in the messages at all - why does SA extract the domains 
wrong from the mailsource when there is no comfacebook at all 
besides the SA report?


URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]

Don't know.  Are you using 3.4.1?  Can you provide a spample that 
reproduces the issue?


regards,
KAM

Re: URIBL plugins are broken

[Reindl Harald]

Am 11.05.2015 um 15:43 schrieb Kevin A. McGrail:

On 5/11/2015 9:13 AM, Reindl Harald wrote:

i face false positives where the links are just facebook.com with
the http-prefix in front and NOT com between the http-prefix and the
real facebook domain

the domain with com in front is indeed on both URIBL but it just
don#t exist in the messages at all - why does SA extract the domains
wrong from the mailsource when there is no comfacebook at all
besides the SA report?

URIBL_DBL_SPAM Contains a spam URL
[URIs: com__facebook.com]

URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: com__facebook.com]


Don't know.  Are you using 3.4.1?  Can you provide a spample that
reproduces the issue?


3.4.0, sample attached in my previous mail, sorry for not attach it in 
the first mail :-(





signature.asc
Description: OpenPGP digital signature

The query to URIBL was blocked

[Chris]

Seeing this in most of the markups

0.0 URIBL_BLOCKED  ADMINISTRATOR NOTICE: The query to URIBL was
blocked.
   See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
   
I installed Bind9 as a caching name server and AFAICT it's running
correctly. If I go to the URIBL.com site it has a test to see which DNS
server is being blocked. I ran the test and the result is:

chris@localhost:~$ host -tTXT 2.0.0.127.multi.uribl.com
2.0.0.127.multi.uribl.com descriptive text permanent testpoint

The output doesn't show any blocked DNS servers. If that's the case then
why am I still seeing the output in my markup?

Chris

Note, I sent this is the 3rd time I've sent this and it hasn't made it to the 
list. I guess
possibly the URI listed caused it to be trashed. 


-- 
Chris
KeyID 0xE372A7DA98E6705C
31.11°N 97.89°W (Elev. 1092 ft)
19:53:44 up 4 days, 3:48, 3 users, load average: 0.23, 0.26, 0.22
Ubuntu 14.04.2 LTS, kernel 4.0.0-997-generic #201503310205 SMP Tue Mar
31 02:07:04 UTC 2015