Hi Oliver, On 5/15/20 1:51 PM, labs--- via Pdns-dev wrote: > I have a discussion with some of our developers about the experimental > feature KVS in dnsdist. > > > We use dnsdist in front of PowerDNS (with MySQL) backends. With KVS we > could check if a domain or record exists before we forward the request > to PowerDNS. > > > Changes in our zones or database will happen quite often, I think that > we have changes every minute. We have appr. 300k zones with >3 million > records in our database. > > > One idea is to use KVS with CDB files just in case of an attack, e.g. > use the KVS lookup in combination with MaxQPSRule. The other idea is to > create new a CDB file everytime when a change of a zone happens or a new > zone was added. > > In the first case we would generate CDB files every 15 or 30 minutes. In > case of an attack perhaps some new zones or new records would fail to > resolve. We would use a refreshDelay with 300 seconds. > > In the later case dnsdist has to reload the CDB file quite often, we > would set refreshDelay to 1 second. Even if we would just generate the > CDB file e.g. once a minute, dnsdist would have to reload it every minute. > > > As dnsdist was made for forwarding DNS requests and not for reading CDB > files, my fear is that dnsdist will loose performance when we reload the > CDB file that often.
In itself, opening the CDB file is not an expensive operation, it mostly consists of one stat() call to see if the file exists, one open() call to open it then one mmap() call to map the content of the file into the memory of the process. Doing that once per minute should not be too bad, it might be perceptible on the latency on the request triggering the reload but that should not kill your performance since dnsdist does not parse the whole file at once. Note however that the content of the new file might not be in the file system cache, so there might a noticeable cost when subsequent queries trigger a cache miss, asking the content of the file to be loaded from the disk into memory. Whether that cost will make a noticeable impact mostly depends on the size of your file and the memory pressure of your system, a small file (~a few MBytes) will likely still have its content loaded in cache if it was very recently generated or copied. Is there any reason you are not considering KVS with LMDB instead? For a database whose content changes that often, that might be a better option. Best regards, -- Remi Gacogne PowerDNS.COM BV - https://www.powerdns.com/
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Pdns-dev mailing list Pdns-dev@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-dev
