Re: consolidating DNSBLs into a single query (was Spam Eating Monkey?)
On 07/10/2009 05:19, Rob McEwen wrote: Also, this loses the ability to *score* on multiple lists... unless you use a bitmasked scoring system whereby one list gets assigned .2, another .4, another .8, on to .128. But that leaves a maximum of only 7 lists. Sure, you can add more than 7 by employing other octets in the answer IP, but that only severely complicates matters. And as it stands, you'd also have the complexity of getting the spam filter to parse, understand, and react properly to those bitmasks. I don't understand the logic of that. Ie, why you'd need to use bitmasking? zen.spamhaus.org is a combination of various different lists and returns multiple values like this: m...@haven:~$ host -t a 2.0.0.127.zen.spamhaus.org 2.0.0.127.zen.spamhaus.org A 127.0.0.4 2.0.0.127.zen.spamhaus.org A 127.0.0.10 2.0.0.127.zen.spamhaus.org A 127.0.0.2 m...@haven:~$ It's perfectly easy for SpamAssassin to see that three different values have been returned, so 127.0.0.2 is on three separate lists and that an extra score should be applied for each of those three. It's also quite easy to do it in Exim, eg if I wanted to block an email in Exim if the sending ip is on both sbl.spamhaus.org and xbl.spamhaus.org I could either do two dns lookups like this: deny dnslists = sbl.spamhaus.org dnslists = xbl.spamhaus.org Or I could do it with a single dns lookup like this: deny dnslists = zen.spamhaus.org=127.0.0.2 dnslists = zen.spamhaus.org=127.0.0.4 You can be 100% backwards compatible by leaving all of your lists as they are, but then adding another one which is a combined version of all of them... -- Mike Cardwell - IT Consultant and LAMP developer Cardwell IT Ltd. (UK Reg'd Company #06920226) http://cardwellit.com/
Re: consolidating DNSBLs into a single query (was Spam Eating Monkey?)
Mike Cardwell wrote: I don't understand the logic of that. Ie, why you'd need to use bitmasking? zen.spamhaus.org is a combination of various different lists and returns multiple values like this:SNIP If every list is an outright block list, then you are correct. My point applies to situations where some lists are used in scoring mode, and where there is a desire to be able to calculate a score based on exactly which lists hit on a particular sending IP. But even if someone tries this with all outright block lists, and uses rbldnsd's built in ability to consolidate lists, then there are still two problems: (a) for auditing purposes, there'd be no way to tell *which* lists hit on that IP since many use the same return codes (b) some hundreds-of-MB-large lists which previously could have used the lower-memory ip4tset would have to revert back to slower and higher-memory-usage ip4set, fwiw Again, not saying these problems can't be solved, only pointing them out so that anyone who cares to try can know what they need to do, or need to expect. -- Rob McEwen http://dnsbl.invaluement.com/ r...@invaluement.com +1 (478) 475-9032
Re: consolidating DNSBLs into a single query (was Spam Eating Monkey?)
On Tue, Oct 6, 2009 at 8:19 PM, Rob McEwen r...@invaluement.com wrote: Warren Togami wrote: You are misunderstanding the question. A single DNS query could respond different numbers meaning they are hits on different lists. Your lists that are subsets or supersets of other lists can easily use this. The querying software need only to know what each result means. Not saying that this is a bad idea, but it does have its limitations. For example, some lists are into the hundreds of megabytes large, and getting the whole file rsncned and updated can take more than several minutes. Often, such lists update only once or twice per hour, if even that often. Hmm ... interesting. If implemented via rbldnsd, each list could be maintained in a separate file, and since rbldnsd can be configured to build a single zone using multiple files on the back end, different lists could be refreshed at different rates. Your comments about tradeoffs and bitmasking still stand, of course. Royce
