Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On 10 Jun 2015, at 10:26, Kevin A. McGrail wrote: On 6/10/2015 10:18 AM, Dianne Skoll wrote: I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. IMO, the primary goal of a caching-only nameserver is in fact, not the caching, but rather the unique source IP so as to avoid running into DNS limits placed on RBL queries from some BL providers that you can run afoul of when sharing a DNS server. Caching is really just icing on the cake coupled with the simplest way to get a local DNS server up and running, no? Not at all scales and styles of mail system. The MTA does lookups at connect and at each command that mostly block progress, and them if the message makes it to SA, virtually all of those lookups and often closely related ones will be done again, often in another process running as a different user which might (OS-dependent) mean that a record in the meager cache kept by the OS won't be used for the second lookup. I no longer have access to the data I gathered on this when I was handling a big-ish system with multiple then-hefty MX gateways doing spam filtering, but my memory is still sound enough that I can say the difference between (1) talking to The Official Enterprise DNS Server on the other side of a router that handled all recursive resolution and (2) using a machine-local caching forwarder on each MX forwarding to a shared caching recursive resolver on a common LAN was most of the median SMTP session life. My recollection is that (1) meant most sessions took ~7 seconds or longer, (2) dropped it to near 3 seconds. A number of things have changed in the past decade that might substantially change that effect even in a similar site, but I think most of the effect (proliferation of DNS-based tactics like SPF & DKIM and many more usable DNSBLs and particularly URIBLs) can only make a cache more helpful, even if the help is marginal. On the other hand, even legitimate operations seem to think every DNS record should expire before today's close of business, and that makes caching less possible. Also, a smaller site gets less benefit at all from a DNS cache. If you've got a few dozen users getting the same mail simultaneously in parallel, you win. If you don't HAVE a few dozen users and most of your users get no spam and little mail, you have a cache that's pretty sparse. You still avoid the problems of looking like part of an abusive behemoth when you forward to Google or of getting self-serving lies from the local ISP browser-aid resolver, so it remains worthwhile.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On 11/06/2015 00:18, Dianne Skoll wrote: > On Wed, 10 Jun 2015 13:56:49 + > David Jones wrote: > > [One should run a caching DNS server on a mail server.] > >> We are giving you solid advice based on real experiences where we >> ran into problems and worked around them. Just try to enable RBLs >> and see how it works for you. > > I'm not disputing that running a caching DNS server is a good idea, but > you may be quite surprised at the low cache hit rate for IP-based DNSBLs. > Spamhaus, for example, has a TTL of 1 minute on its A records. (Check > out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) > > Quite a number of years ago, I ran an analysis of the mail logs on a > very busy server and found an abysmally low cache hit rate (about 30%) > and that was in the day when Spamhaus had a 15-minute TTL. 30% is an excellent hit rate, however - The longer the TTL the higher the cache hit The longer the TTL the higher the collateral damage It's why most run 1-10 min TTL's, might not seem much, but take for example in the mid 90's when AOL was useless at dealing with spam issues, a listing of 10 mins could deny thousands of messages back then, and that helped "prompt" them into getting their act together, especially when a number of DNSBL's were doing it, so they kicked off their user (who often retuned 30 mins later courtesy of AOL's world wide flood of freebie CD's), and blocks where removed quick enough to minimise more innocents getting caught up.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Am 11.06.2015 um 03:33 schrieb Dianne Skoll: On Thu, 11 Jun 2015 01:00:45 +0200 Reindl Harald wrote: cache-min-ttl: 600 Even a 10-minute cache time buys you very little. My original analysis assumed a 15-minute TTL calling 32% cache hits on a single day "very little" is questionable server stats for thread 0: 436986 queries, 140106 answers from cache, 296880 recursions, 4202 prefetch signature.asc Description: OpenPGP digital signature
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Thu, 11 Jun 2015 01:00:45 +0200 Reindl Harald wrote: > cache-min-ttl: 600 Even a 10-minute cache time buys you very little. My original analysis assumed a 15-minute TTL. Regards, Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
Am 10.06.2015 um 16:18 schrieb Dianne Skoll: On Wed, 10 Jun 2015 13:56:49 + David Jones wrote: [One should run a caching DNS server on a mail server.] We are giving you solid advice based on real experiences where we ran into problems and worked around them. Just try to enable RBLs and see how it works for you. I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. Spamhaus, for example, has a TTL of 1 minute on its A records. (Check out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) yes, to exceed the volume quicker and only if your resolver has a bad configuration and that's even one reason more to use a local cache msg-cache-size: 96m neg-cache-size: 96m rrset-cache-size: 192m cache-min-ttl: 600 cache-max-ttl: 10800 signature.asc Description: OpenPGP digital signature
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015, David Jones wrote:
[One should run a caching DNS server on a mail server.]
My point was that running a local caching server is the only way one
can know exactly how the lookups are happening. If you point to a
DNS server that you don't manage, it could be forwarding to an ISP's
DNS caches which will aggregate your queries in with others and could
cause unexpected results for those RBLs that limit queries.
One other technical benefit to running a local caching server is that if
SA is configured to talk to it va the localhost (loopback) interface there
are MTU advantages.
Most loopback interfaces have a MTU of 16K (or bigger) and will handle large
UDP packets without fragementation. In general DNS transactions are fastest
via UDP if you don't have fragementation issues.
--
Dave Funk University of Iowa
College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include
Better is not better, 'standard' is better. B{
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015 14:56:40 + David Jones wrote: > My point was that running a local caching server is the only way one > can know exactly how the lookups are happening. Ah, true. I missed that point I guess. Regards, Dianne.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
>[One should run a caching DNS server on a mail server.] >> We are giving you solid advice based on real experiences where we >> ran into problems and worked around them. Just try to enable RBLs >> and see how it works for you. >I'm not disputing that running a caching DNS server is a good idea, but >you may be quite surprised at the low cache hit rate for IP-based DNSBLs. >Spamhaus, for example, has a TTL of 1 minute on its A records. (Check >out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) >Quite a number of years ago, I ran an analysis of the mail logs on a >very busy server and found an abysmally low cache hit rate (about 30%) >and that was in the day when Spamhaus had a 15-minute TTL. My point was that running a local caching server is the only way one can know exactly how the lookups are happening. If you point to a DNS server that you don't manage, it could be forwarding to an ISP's DNS caches which will aggregate your queries in with others and could cause unexpected results for those RBLs that limit queries. I have 8 mail filters that run a local caching DNS server which forward to a pair of DNS caches running rbldnsd for a local copy of a number of RBL zones including my own private RBL. This configuration has to provide some caching benefits when I get blasted by mass marketing campaigns. Postfix should keep my local cache populated so when SA asks for the same DNS information it would be a few milliseconds response. I should take some time to do some real analysis as you have done. Thanks for the info and link.
Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On 6/10/2015 10:18 AM, Dianne Skoll wrote: I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. IMO, the primary goal of a caching-only nameserver is in fact, not the caching, but rather the unique source IP so as to avoid running into DNS limits placed on RBL queries from some BL providers that you can run afoul of when sharing a DNS server. Caching is really just icing on the cake coupled with the simplest way to get a local DNS server up and running, no? Regards, KAM
DNSBLs and cache hit rate (was Re: Must-Have Plugins?)
On Wed, 10 Jun 2015 13:56:49 + David Jones wrote: [One should run a caching DNS server on a mail server.] > We are giving you solid advice based on real experiences where we > ran into problems and worked around them. Just try to enable RBLs > and see how it works for you. I'm not disputing that running a caching DNS server is a good idea, but you may be quite surprised at the low cache hit rate for IP-based DNSBLs. Spamhaus, for example, has a TTL of 1 minute on its A records. (Check out "host -v 2.0.0.127.sbl.spamhaus.org" if you don't believe me.) Quite a number of years ago, I ran an analysis of the mail logs on a very busy server and found an abysmally low cache hit rate (about 30%) and that was in the day when Spamhaus had a 15-minute TTL. Anyway, run through the exercise yourself; it's eye-opening. My original post was here (back when I used to be David, so don't let the signature confuse you...) http://spamassassin.1065346.n5.nabble.com/Fwd-Asrg-draft-levine-iprangepub-01-tp28778p28802.html Regards, Dianne.
