To our customers and partners -- Today ISC is issuing an operational notification concerning a new feature that was introduced in BIND 9.12 that we have discovered may have a significant performance impact on some servers, depending on the size and composition of their caches.
This feature (synth-from-dnssec) is enabled by default in BIND 9.12, 9.13 (both now EOL) and in 9.14 and 9.15. Later this month we will be issuing new maintenance releases of BIND 9.14 and 9.15 that include a change to reverse the default so that this synth-from-dnssec is inactive unless explicitly enabled. We plan to improve this feature in future maintenance releases, but until then, our recommendation is to disable it unless you are certain that it is providing a clear benefit in your production environment. Servers with large caches and which respond with a high proportion of NXDOMAINs are most likely to be affected. If you are experiencing intermittent periods of poor resolver responsiveness and/or unexplained spikes in CPU consumption that do not correspond with a similar increase in client traffic, it is possible that synth-from-dnssec processing is the underlying root cause. Cathy Almond ISC Support ----- Posting date: 11 November 2019 Program Impacted: BIND Versions affected: 9.12.0 -> 9.12.4-P2, 9.14.0 -> 9.14.7. Also versions 9.13.0 -> 9.13.7 of the 9.13 development branch and versions 9.15.0 -> 9.15.5 of the 9.15 development branch. Description: "synth-from-dnssec" is a feature, introduced in BIND 9.12.0, intended to mitigate the impact of "random subdomain" attacks on recursive resolvers by allowing them to avoid recursive lookups when NSEC information already present in the resolver's cache would be sufficient to conclude the nonexistence of a requested record. Unfortunately, it has been discovered that on resolvers that have large caches with a high proportion of negative answers, synth-from- dnssec can be noticeably slower than simply performing recursion to obtain an answer from an authoritative server. Because of the computational expense of traversing a large number of cache data structures in order to reach a conclusion, using synth-from-dnssec on a large cache can cause high CPU consumption and delays while the system tries to determine whether a synthesized answer can be provided. While the intent of the feature is to modestly improve performance, under some cache conditions performance can instead be significantly reduced; whether or not the feature improves performance as intended depends considerably on the size and contents of the cache. Impact: Under certain cache circumstances synth-from-dnssec may significantly slow performance of recursive resolution, irrespective of whether or not the servers have enabled DNSSEC-validation. Workarounds: Until replacement versions of BIND are made available which contain an improvement to the synth-from-dnssec feature, operators can choose to disable it by including "synth-from-dnssec no;" in the global options section of named.conf. Synth-from-dnssec is currently enabled by default in versions which support the feature so it must be turned off explicitly if operators wish to avoid its use. Beginning with November 2019 maintenance releases the default behavior will be reversed (so that synth-from-dnssect defaults to off) until further notice. Solution: ISC plans to improve the behavior of the synth-from-dnssec feature in future maintenance releases. Until then, we will be changing the default behavior, beginning with the November 2019 maintenance releases. However, until the feature has been corrected to prevent potential significant performance impact we recommend disabling the feature (using the configuration syntax provided in the "Workarounds" section above) if you suspect that it is negatively affecting resolver performance. Do you still have questions? Questions regarding this advisory should go to security-offi...@isc.org. To report a new issue, please encrypt your message using security-offi...@isc.org's PGP key which can be found here: https://www.isc.org/downloads/software-support-policy/openpgp-key/. If you are unable to use encrypted email, you may also report new issues at: https://www.isc.org/community/report-bug/. Note: ISC patches only currently supported versions. When possible we indicate EOL versions affected. (For current information on which versions are actively supported, please see https://www.isc.org/downloads/). ISC Security Vulnerability Disclosure Policy: Details of our current security advisory policy and practice can be found here: https://kb.isc.org/docs/aa-00861 This Knowledgebase article is the complete and official operational notification document: https://kb.isc.org/docs/operational-notification-synth-from-dnssec-may-cause-slow-resolution-on-resolvers-under-certain-cache-conditions Legal Disclaimer: Internet Systems Consortium (ISC) is providing this notice on an "AS IS" basis. No warranty or guarantee of any kind is expressed in this notice and none should be implied. ISC expressly excludes and disclaims any warranties regarding this notice or materials referred to in this notice, including, without limitation, any implied warranty of merchantability, fitness for a particular purpose, absence of hidden defects, or of non-infringement. Your use or reliance on this notice or materials referred to in this notice is at your own risk. ISC may change this notice at any time. A stand-alone copy or paraphrase of the text of this document that omits the document URL is an uncontrolled copy. Uncontrolled copies may lack important information, be out of date, or contain factual errors. _______________________________________________ bind-announce mailing list bind-announce@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-announce