> On Aug 13, 2015, at 2:15 AM, Thomas Eckardt <thomas.ecka...@thockar.com> > wrote: > >> Isn’t the “fix” simple? > > No, it isn't. > > These are the simplyfied steps of the URIBL engine. > > 1. calculate regular expressions for URI's based on the configuration > (shortening is defined here) > 2. decode the content to check in to UTF8 > 3. check the content against the regex - collect all URI's (shortening is > done here) > 4.1 loop: check each URI against the URIBLCache > 4.2. if no cache entry is found, call the central RBL/URIBL/RWL-engine - > means check the URI with all URIBL-providers in one step > 5. interpret the result summary > > If hostname shortening would not be done for any URIBL-provider, the > number cache entries will grow to a very large number. > IMHO the URIBL-cache would become inefficent for SURBL. > > I hav'nt found any changes in the guidlines of SURBL > http://www.surbl.org/guidelines. Step 2 requests still the shortening of > the hostname. They are still request the usage of the three TLD-lists.
Guess they haven’t updated the page yet. Their public resolvers have been cut over for a couple of months. > The files to download are stll the same. So I think, if SURBL accepts/uses > wildcards, they are shortening the hostname before doing the check - I'm > right? No. In the DNS world its the difference between (in Bind terms) example.com <http://example.com/> A 127.0.0.1 (before wildcarding) and *. example.com <http://example.com/> A 127.0.0.1 (after wildcarding) > If I do an online check at SURBL, I see exactly this behavior. For > example: if I checked for 'www.google.com' , I got back the result for > 'google.com'. For 'this-is-a-test.google.com' , I got the same result. > Yes BUT on the following domain whose DNS has been cracked and that host is feeding pillz (using example.com <http://example.com/>) So http://bad.sub.domain.example.com/canadianpharma.php <http://bad.sub.domain.example.com/canadianpharma.php> DNSbl entries: example.com <http://example.com/> NXDOMAIN (before wildcarding) and *. sub.domain.example.com <http://bad.sub.domain.example.com/> A 127.0.0.1 (after wildcarding) If you shorten to example.com <http://example.com/> you would not get a listing because SURBL cannot list example .com but if you query with the whole host (bad.sub.domain.example.com <http://bad.sub.domain.example.com/>) you would find it listed. This is true with domains that cannot be listed for various reasons and also for those site appear inbetween 2nd and 3rd level file update cycles and thus would be missed. Additionally I found out that SURBL has a 4th level domain list necessary to list abusable amazonaws and other cloud and k12 sub domains that is not published. Using the host for lookup provides for catching these. Just as a side note Spamhaus’ DBL is also a wildcarded list Tom > Thomas > > > > > > Von: TR Shaw <ts...@oitc.com> > An: ASSP development mailing list <assp-test@lists.sourceforge.net> > Datum: 12.08.2015 14:26 > Betreff: Re: [Assp-test] SURBL changes > > > > Isn’t the “fix” simple? In URIBL checking, test if you are querying SURBL > or DBL and if so do not shorten the host name. > >> On Aug 12, 2015, at 5:16 AM, Thomas Eckardt <thomas.ecka...@thockar.com> > wrote: >> >>> A possible tweak may be writing an ASSP module to deal with SURBL >> >> First I had the same nice idea. But this is very complicated because of >> two things. >> >> 1. as a level 1 plugin (after SMTP handshake, after DATA) it gets no > mail >> data - as a level 2 plugin (complete mail) , the plugin would cause assp > >> to queue the complete mail, even it is not nessesary for checking URI's > in >> 'maxBytes' >> >> 2. the OCR plugin forces an URIBL check for the extracted data - in case > >> this would force a call from a plugin to a plugin. The plugins and the >> assp.pl plugin check code does not support recursion. This would break > too >> many things. >> >> There is another issue I have to deal with. At this time URI's to check >> must have an IP as hostpart or a hostname which ends with a TLD of a > valid >> level (1,2,3). >> Yes - cracked DNS will currently bypass the URIBL check in assp (in case > >> the local DNS server is hacked - nothing will ever help). >> But removing the TLD check will cause assp to collect and check all the >> possible (looks like URI) parts in the header lines (most of them are >> useless). Also several misspelled parts of the mail text will be > detected >> as an URI - like 'This is a test.But this is another test' - here >> 'test.But' looks like a hostname (ignoring TLD), because the space after > >> the dot is missing. >> OK - there is no link behind the possible host name. But it is common to > >> make such a nonsense hostname looking like a link - with the hint: "use >> this link, if does not work copy and paste it in to your browser". Or >> another example: >> ... this.is.a.cracked.dns ... >> Now the spammer makes a valid link behind the URI , like > 'maps.google.com' >> , which will fail because the ISP-DNS is hacked - but it looks innocent >> for everyone - now the hint 'use this ....' . >> It can be possible, that assp reached the count limit of URI's , because > >> TLD is ignored, before this dangerous part. >> >> My question is: Can we assume, that we will get correct answers for our >> URIBL DNS-queries, if a local or ISP DNS-server is hacked? - IMHO, NO! >> >> The only way to step in to such a trap is, if the DNS-servers of the >> domain, where an URI points to, are hacked. >> Is it worth the effort to deal with such rarely cases? Where the effort > is >> querying 500%-1000% of the current URI's nearly completely in vain - not > >> to forget the coding and testing. >> >> Thomas >> >> >> >> >> >> >> >> >> >> >> >> >> Von: grayhat <gray...@gmx.net> >> An: assp-test@lists.sourceforge.net >> Datum: 12.08.2015 09:29 >> Betreff: Re: [Assp-test] SURBL changes >> >> >> >> It was Tue, 11 Aug 2015 08:47:55 +0200 when >> Thomas Eckardt <thomas.ecka...@thockar.com> wrote: >> >>> Thank you for the information -Tom. At this time I'm unable to use >>> these very nice new features of SURBL in assp. Implementing them in >>> the current URIBL-code, will make the code too complex. >>> The current code has to be redesigned, or a new code and logic must >>> be written for SURBL. >>> I'll put it on the TODO list. >> >> A possible tweak may be writing an ASSP module to deal with SURBL >> >> > ------------------------------------------------------------------------------ >> _______________________________________________ >> Assp-test mailing list >> Assp-test@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/assp-test >> >> >> >> >> >> >> DISCLAIMER: >> ******************************************************* >> This email and any files transmitted with it may be confidential, > legally >> privileged and protected in law and are intended solely for the use of > the >> >> individual to whom it is addressed. >> This email was multiple times scanned for viruses. There should be no >> known virus in this email! >> ******************************************************* >> >> > ------------------------------------------------------------------------------ >> _______________________________________________ >> Assp-test mailing list >> Assp-test@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/assp-test > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test > > > > > DISCLAIMER: > ******************************************************* > This email and any files transmitted with it may be confidential, legally > privileged and protected in law and are intended solely for the use of the > > individual to whom it is addressed. > This email was multiple times scanned for viruses. There should be no > known virus in this email! > ******************************************************* > > > ------------------------------------------------------------------------------ > _______________________________________________ > Assp-test mailing list > Assp-test@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/assp-test ------------------------------------------------------------------------------ _______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test
