Re: thoughts about a new module called check_spammer_connect
On Thu, 28 Jul 2011, Ask Bjørn Hansen wrote: On Jul 28, 2011, at 11:28, Charlie Brady wrote: I use dnscache on my home network and would like to know why anybody decide to move from dnscache. Performance, Could you supply some details, please? Off the top of my head the two things I'd run into are the outstanding requests limit Chris mentioned and how it wants to resolve all the NS records into A records before continuing the chain (or do I remember that wrong?) You don't remember wrong - if there is missing required glue in referrals, dnscache persues an endless search for the missing glue, rather than resolving using the name servers it has already located: http://comments.gmane.org/gmane.network.djbdns/12132 Also the occasional (though rare) incompatibility when someone has some extraordinary odd or broken DNS data or servers is annoying. It's their fault! doesn't really get your time debugging it back. This is the same issue, is it not?
Re: thoughts about a new module called check_spammer_connect
On Jul 27, 2011, at 16:41, Aleksandar Lazic wrote: I use dnscache on my home network and would like to know why anybody decide to move from dnscache. Performance, IPv6 support, ease of maintenance (when using dnscache I often end with subtly differently patched versions; frustrating). Also: Memory is much more plentiful now than ten years ago, so dnscache's excellent memory management is less important now than then. - ask -- http://develooper.com/ - http://askask.com/
Re: thoughts about a new module called check_spammer_connect
On Thu, 28 Jul 2011, Ask Bj?rn Hansen wrote: On Jul 27, 2011, at 16:41, Aleksandar Lazic wrote: I use dnscache on my home network and would like to know why anybody decide to move from dnscache. Performance, Could you supply some details, please? IPv6 support, ease of maintenance (when using dnscache I often end with subtly differently patched versions; frustrating). Also: Memory is much more plentiful now than ten years ago, so dnscache's excellent memory management is less important now than then. - ask
Re: thoughts about a new module called check_spammer_connect
On Jul 28, 2011, at 11:28, Charlie Brady wrote: I use dnscache on my home network and would like to know why anybody decide to move from dnscache. Performance, Could you supply some details, please? Off the top of my head the two things I'd run into are the outstanding requests limit Chris mentioned and how it wants to resolve all the NS records into A records before continuing the chain (or do I remember that wrong?) Also the occasional (though rare) incompatibility when someone has some extraordinary odd or broken DNS data or servers is annoying. It's their fault! doesn't really get your time debugging it back. - ask -- http://develooper.com/ - http://askask.com/
Re: thoughts about a new module called check_spammer_connect
On 7/28/2011 12:09 PM, Chris Lewis wrote: I'd not use dnscache in corporate/enterprise/high reliability environments. Unbound is nice and hiccup-free. Bind9 is reasonable enough. I hear good things about PowerDNS too. As a FYI, on a machine running qpsmtpd, handling up to 10M emails per day, with 4-5 DNSBL queries per email, rDNS etc., Unbound has been running since Feb (probably a manual restart testing unbound's init script, we've never had ANY issue with Unbound itself), and has consumed about 27 hours of CPU, and has a memory footprint of 45Mb. Could probably up the cache size, but I don't think it would make a lot of difference. rbldnsd has been up for 300 days, has consumed about the same amount of CPU, and the memory footprint is 900Mb. Which, no surprise, is about the size of the zone file it serves.
Re: thoughts about a new module called check_spammer_connect
On 7/26/2011 10:41 PM, Jared Johnson wrote: We discussed doing this once at my organization and someone astutely pointed out that considering that this is DNS, the better solution would really be to run a local caching DNS server, e.g. bind9 and point resolv.conf to 127.0.0.1 with forwarders to your ISP's nameservers. Well, what they actually pointed out was that since we already do this everywhere, we're already getting in-memory caching that's designed expressly for DNS results, for free :) running a local caching DNS server is basically a good idea anyway, and obsoletes the need for caching DNS results in any particular software. Agreed. Because one of the things that many people will want to do is customize where the DNS cache gets its answers, and that makes developing a cache more complicated. In our case, we run split-horizon DNS (where some of our DNS queries for our _own_ space has to be split between internal and external resolvers), AND we run our own rbldnsd instances for most of our DNSBLs that can't be published externally thru the normal chain of DNS queries. For a while, we had one rbldnsd and used corporate DNS caching (which knew how to route DNSBL queries) for our (at one point 11) qpsmtpd instances. The loading (at 30+ million emails/day) caused the DNS guys to yell at me for overloading our bind9 caches. Now each of our qpsmtpd instances has their own rbldnsd (serving ~700Mb zone consisting of almost all the DNSBLs we use as one merged zone) plus the Unbound DNS server package. This is probably overkill even at 30+ million emails/day, but it works extremely well. The unbounds are configured to route DNSBL queries to the co-resident rbldnsds (they can failover to other server rbldnsds), DNS queries of our infrastructure to the corporate bind9 instances, and external queries through normal recursive resolution (involving the root servers if necessary), and unbound caches all of it. [Note if you are running qpsmtpd-async, as we do, it's not really possible to route DNS queries differently for DNSBLs versus other DNS queries qpsmtpd does. ParaDNS doesn't handle paralleled DNS queries to different servers well.] Unbound is easy to install and configure. I chose not to use Bind9 partially because of previous experiences with much older versions. I did run DJB's dnscache for a while instead, but unbound works so much better. Avoid using dnscache for this. I can explain why if people are interested. Before qpsmtpd, we were using Lyris Mailshield which had its own internal cache for some reason. It was quite awkward because it's capabilities of routing queries via configuration were highly limited.
Re: thoughts about a new module called check_spammer_connect
On Wed, 27 Jul 2011, Jared Johnson wrote: That sounds like a pretty sweet configuration! [Note if you are running qpsmtpd-async, as we do, it's not really possible to route DNS queries differently for DNSBLs versus other DNS queries qpsmtpd does. ParaDNS doesn't handle paralleled DNS queries to different servers well.] when I first mucked around with the uribl plugin (see other threads), I switched the non-async plugin to use Net::DNS::Async, but as far as I could tell, N::D::A wasn't truly async, so I left the async plugin using ParaDNS. As it turns out, one of my associates since determined a way to use Net::DNS::Async in a truly async fashion. We're still using the prefork daemon, but when we inevitably switch to async I'll probably try to switch the async to N::D::A. We also have a consolidated feed so the multi-feed thing isn't too much of a concern, but N::D::A just seems more straightforward. If anyone is interested in switching to N::D::A now, feel free to ping me and I can get the details on how to use it in a way that doesn't break the true async plugin... I'm also curious if anyone knows of reasons why this switch would not be such a good idea ;) I have an XS version of ParaDNS too, which may be useful to people. It's in the ParaDNS subversion repository. We use it at work and it seems stable now. However I'm unlikely to maintain much on Qpsmtpd now that Haraka has taken off. Matt.
Re: thoughts about a new module called check_spammer_connect
On Wed, Jul 27, 2011 at 2:05 PM, Matt Sergeant m...@sergeant.org wrote: However I'm unlikely to maintain much on Qpsmtpd now that Haraka has taken off. Matt. how about a plugin adapter, so Haraka can use Qpsmtpd plugins or v/v? That probably implies either a node.js -- perl integration layer, a V8 -- perl integration layer, or use of a more arm's-length plugin architecture (like sendmail's Milter) around both parts.
Re: thoughts about a new module called check_spammer_connect
On Wed, 27 Jul 2011, David Nicol wrote: On Wed, Jul 27, 2011 at 2:05 PM, Matt Sergeant m...@sergeant.org wrote: However I'm unlikely to maintain much on Qpsmtpd now that Haraka has taken off. Matt. how about a plugin adapter, so Haraka can use Qpsmtpd plugins or v/v? That probably implies either a node.js -- perl integration layer, a V8 -- perl integration layer, or use of a more arm's-length plugin architecture (like sendmail's Milter) around both parts. Haraka ships with both smtp_forward and smtp_proxy plugins (with tradeoffs for each), so there's no reason you wouldn't just run both, if you needed qpsmtpd plugins too. Matt.
Re: thoughts about a new module called check_spammer_connect
Dear Chris, On Mit 27.07.2011 10:59, Chris Lewis wrote: On 7/26/2011 10:41 PM, Jared Johnson wrote: [snipp] Now each of our qpsmtpd instances has their own rbldnsd (serving ~700Mb zone consisting of almost all the DNSBLs we use as one merged zone) plus the Unbound DNS server package. This is probably overkill even at 30+ million emails/day, but it works extremely well. The unbounds are configured to route DNSBL queries to the co-resident rbldnsds (they can failover to other server rbldnsds), DNS queries of our infrastructure to the corporate bind9 instances, and external queries through normal recursive resolution (involving the root servers if necessary), and unbound caches all of it. [snipp] Unbound is easy to install and configure. I chose not to use Bind9 partially because of previous experiences with much older versions. I did run DJB's dnscache for a while instead, but unbound works so much better. Avoid using dnscache for this. I can explain why if people are interested. Please, yes. I use dnscache on my home network and would like to know why anybody decide to move from dnscache. Thank you for your detail infos. I will try to setup the http://unbound.net/ as caching server. BR Aleks
Re: thoughts about a new module called check_spammer_connect
I thought about a module which learns from the plugin dnsbl. Maybe we call it check_known_dnsbl_spammer ;-) and use the module http://search.cpan.org/~robm/Cache-FastMmap-1.39/lib/Cache/FastMmap.pm Hi Aleks, We discussed doing this once at my organization and someone astutely pointed out that considering that this is DNS, the better solution would really be to run a local caching DNS server, e.g. bind9 and point resolv.conf to 127.0.0.1 with forwarders to your ISP's nameservers. Well, what they actually pointed out was that since we already do this everywhere, we're already getting in-memory caching that's designed expressly for DNS results, for free :) running a local caching DNS server is basically a good idea anyway, and obsoletes the need for caching DNS results in any particular software. -Jared
