Missing DNSSEC key causes BIND process overload

[Raymond Drew Walker]

Running BIND 9.9.0

Upon having some DNSSEC keys run out of activity with no active
replacements, we noticed some interesting behavior with the named
process...

When a zone signing key enters it's Inactive phase, the zone still loads
on startup:

19-Jun-2012 09:54:10.176 general: zone_timer: zone badzone.nau.edu/IN:
enter
19-Jun-2012 09:54:10.176 general: zone_maintenance: zone
badzone.nau.edu/IN: enter
19-Jun-2012 09:54:10.176 notify: zone badzone.nau.edu/IN: sending notifies
(serial 91416)
19-Jun-2012 09:54:10.177 general: zone badzone.nau.edu/IN: Key
badzone.nau.edu/RSASHA1/11985 missing or inactive and has no replacement:
retaining signatures.
19-Jun-2012 09:54:10.177 general: zone_settimer: zone badzone.nau.edu/IN:
enter
19-Jun-2012 09:54:10.177 general: zone_settimer: zone badzone.nau.edu/IN:
enter

Eventually we'll see failures on updating the zone:

Jun 17 04:06:58 diamond named[19951]: client 134.114.X.X#52804: updating
zone 'badzone.nau.edu/IN': found no active private keys, unable to
generate any signatures
Jun 17 04:06:58 diamond named[19951]: client 134.114.X.X#52804: updating
zone 'badzone.nau.edu/IN': RRSIG/NSEC/NSEC3 update failed: not found


This occurred to a few zones, but then something odd started happening...

The named process ramped up to +%100 of processor. Nothing in the named
logs indicated why this was happening... This caused SERVFAIL and other
timeouts on all kinds of operations on the machine.

Our initial solution was to make new keys available (keys were actually
created, just not put in place,) and the zones at issue should recover.

The zones at issue ended up requiring a manual re-sign to completely
resolve the issue.


Anyone have an explanation of why this would happen (named gobbling up
CPU, and also requiring manual resigning of the zones)?

Thanks in advance,

Raymond Walker
Software Systems Engineer Sr.
ITS Northern Arizona University




___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

dig +sigchase looping

[Raymond Drew Walker]

I’m experiencing an interesting issue where sometimes when performing a 
sigchase on a valid signed zone the command loops indefinitely when an expired 
RRSIG exists:

Live example:
dig +sigchase +trusted-key=./trusted.keys aa.nau.edu A

Notes:
There is currently a valid RRSIG for this zone.
dig compiled with -DDIG_SIGCHASE=1
BIND 9.9.4

Roughly %50 of the time it returns as expected, while other times looping in 
such a fashion:

;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired

Any particular reason this should be expected or is it bug worthy (or fixed in 
9.9.5, as I didn’t see anything in the change log referring to it)?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: dig +sigchase looping

[Raymond Drew Walker]

I have verified that this also happens intermittently with dig in BIND 9.9.5 
built/configured with:

STD_CDEFINES="-DDIG_SIGCHASE=1"
export STD_CDEFINES
./configure --enable-threads --enable-largefile
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University

From: Ray Walker mailto:ray.wal...@nau.edu>>
Date: Friday, February 21, 2014 at 4:28 PM
To: "bind-users@lists.isc.org" 
mailto:bind-users@lists.isc.org>>
Subject: dig +sigchase looping

I’m experiencing an interesting issue where sometimes when performing a 
sigchase on a valid signed zone the command loops indefinitely when an expired 
RRSIG exists:

Live example:
dig +sigchase +trusted-key=./trusted.keys aa.nau.edu A

Notes:
There is currently a valid RRSIG for this zone.
dig compiled with -DDIG_SIGCHASE=1
BIND 9.9.4

Roughly %50 of the time it returns as expected, while other times looping in 
such a fashion:

;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired
;; OK a DS valids a DNSKEY in the RRset
;; Now verify that this DNSKEY validates the DNSKEY RRset
;; VERIFYING DNSKEY RRset for aa.nau.edu. with DNSKEY:25159: RRSIG has expired

Any particular reason this should be expected or is it bug worthy (or fixed in 
9.9.5, as I didn’t see anything in the change log referring to it)?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

DNSSEC not populating parent zone files with DS records

[Raymond Drew Walker]

In our initial implementation of DNSSEC, we chose to try out the "auto"
functionalities in version 9.8.0 P4 ie. using "auto-dnssec maintain" in
all master zones.

When going live, we found that though all zones that we are acting as
master for would populate their own DS records, but there would be no
population of a child zone's DS record in the corresponding parent master
zone file. 

This means upon go-live, any DNSSEC validation of our children zones
(X.nau.edu, Y.X.nau.edu etc.) would fail, though our root master zone
(nau.edu) would validate fine.

We have since backed out DNSSEC until we can get a resolution of the issue.

After much research, I'm not sure why this is happening... Any suggestions
or ideas?

Raymond Walker
Software Systems Engineer Sr.
ITS Northern Arizona University





___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNSSEC not populating parent zone files with DS records

[Raymond Drew Walker]

-Original Message-

From: Tony Finch 
Date: Mon, 3 Oct 2011 14:59:38 +0100
To: Michael Sinatra 
Cc: , , Raymond Walker

Subject: Re: DNSSEC not populating parent zone files with DS records

>Michael Sinatra  wrote:
>>
>> There are ways of getting the DS records into the zone(s).  Here are
>>some
>> steps that I took on some test zones:
>
>Alternatively, set "update-policy local;" on your parent zone and use this
>little pipeline on the master server. Substitute $parent and $child as
>necessary:
>
>  dig +noall +answer dnskey $child |
>  dnssec-dsfromkey -f /dev/stdin $child |
>  (echo "zone $parent"; sed 's/^/update add /'; echo "send") |
>  nsupdate -l

In testing, this pipe sets up the following for nsupdate which fails:

zone nautest.edu
update add test3.nautest.edu. IN DS 35113 5 1
4D27C35B0F638218659F740252604980CE445F16
update add test3.nautest.edu. IN DS 35113 5 2
843544D4F01EE147257FBDB92D9AC3C51129DEF0FC7D972D57EB6E20 550E4161
Send



The error is:
ttl 'IN': not a valid number
syntax error


I have been unable to determine the correct method to add a DS record by
hand. The ultimate goal would be the automation of this process.

Am I also missing somewhere in the RFC where NS records of children zones
need be populated in the parent? Is this something that has changed with
the addition of DNSSEC?

Raymond Walker
Software Systems Engineer Sr.
ITS Northern Arizona University



___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: DNSSEC not populating parent zone files with DS records

[Raymond Drew Walker]

-Original Message-

From: Tony Finch 
Date: Tue, 4 Oct 2011 20:30:43 +0100
To: Raymond Walker 
Cc: "bind-users@lists.isc.org" 
Subject: Re: DNSSEC not populating parent zone files with DS records

>Raymond Drew Walker  wrote:
>
>> In testing, this pipe sets up the following for nsupdate which fails:
>
>Sorry, I forgot the TTL command. Adjust its value as you require...
>
>  dig +noall +answer dnskey $child |
>  dnssec-dsfromkey -f /dev/stdin $child |
>  (echo "zone $parent"; echo "ttl 3600"; sed 's/^/update add /'; echo
>"send") |
>  nsupdate -l

Thanks much, this makes much more sense.

>
>> Am I also missing somewhere in the RFC where NS records of children
>>zones
>> need be populated in the parent? Is this something that has changed with
>> the addition of DNSSEC?
>
>No, it has always been an error. See RFC 2181 section 6. DNSSEC just makes
>the breakage more obvious.


After reading this, RFC1034, and conferring with the original implementor
of DNS at our institution, I have a better wrangle on the NS issue. Child
zone NS records were never populated in the parent because all zones were
under the same name servers, and "it just worked" (circa the early 90's.)
I mistakenly inherited on this understanding... until now.

As for better automation of DNSSEC, my research lends little to no
information on proper child/parent DS record population. I am curious
about how to properly deploy in the future.

My assumptions are the following:
-Sign a zone, then insert it's corresponding DS data into it's parent by
hand (nsupdate).
-Keep track of & update DS record data on a regular schedule. Potentially
via nsupdate, by deleting all DS record data in the parent zone for said
child, then inserting new DS records.

Yikes, I was hoping auto-dnssec would handle these tasks. ;) Am I missing
any elegant solutions?

Much thanks to the list for their insightful comments...

Raymond Walker
Software Systems Engineer Sr.
ITS Northern Arizona University


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Bad owner name on hidden primary

[Raymond Drew Walker]

Running BIND 9.9.5:

On moving to a hidden primary setup, dynamic updates to zones we are master for 
with “unallowed characters” (underscores in our case) have started to fail with 
the error "bad owner name (check-names)” In the past (pre hidden primary) they 
did not fail.

In the past we have not used the ‘check-names’ option, so behavior should be 
default… odd since the default behavior is to fail for master zones.

Could this have something to do with the SOA of the zone no longer being the 
actual primary?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Bad owner name on hidden primary

[Raymond Drew Walker]

Our current workaround is to add the following to NAMED configuration:

  check-names master ignore;

Is there a more preferred solution?

…or perhaps a different way of looking at this issue?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University

From: Ray Walker mailto:ray.wal...@nau.edu>>
Date: Monday, June 9, 2014 at 11:47 AM
To: "bind-users@lists.isc.org" 
mailto:bind-users@lists.isc.org>>
Subject: Bad owner name on hidden primary

Running BIND 9.9.5:

On moving to a hidden primary setup, dynamic updates to zones we are master for 
with “unallowed characters” (underscores in our case) have started to fail with 
the error "bad owner name (check-names)” In the past (pre hidden primary) they 
did not fail.

In the past we have not used the ‘check-names’ option, so behavior should be 
default… odd since the default behavior is to fail for master zones.

Could this have something to do with the SOA of the zone no longer being the 
actual primary?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Bad owner name on hidden primary

[Raymond Drew Walker]

Apologies,

Our workaround was actually the addition of 2 lines:

   check-names master ignore;
   check-names response ignore;

Without the second ‘response’ clause, the update does not error, but does not 
get applied to the record.
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University

From: Ray Walker mailto:ray.wal...@nau.edu>>
Date: Monday, June 9, 2014 at 3:18 PM
To: "bind-users@lists.isc.org" 
mailto:bind-users@lists.isc.org>>
Subject: Re: Bad owner name on hidden primary

Our current workaround is to add the following to NAMED configuration:

  check-names master ignore;

Is there a more preferred solution?

…or perhaps a different way of looking at this issue?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University

From: Ray Walker mailto:ray.wal...@nau.edu>>
Date: Monday, June 9, 2014 at 11:47 AM
To: "bind-users@lists.isc.org" 
mailto:bind-users@lists.isc.org>>
Subject: Bad owner name on hidden primary

Running BIND 9.9.5:

On moving to a hidden primary setup, dynamic updates to zones we are master for 
with “unallowed characters” (underscores in our case) have started to fail with 
the error "bad owner name (check-names)” In the past (pre hidden primary) they 
did not fail.

In the past we have not used the ‘check-names’ option, so behavior should be 
default… odd since the default behavior is to fail for master zones.

Could this have something to do with the SOA of the zone no longer being the 
actual primary?
—
Raymond Walker
Software Systems Engineer StSp.
ITS - Northern Arizona University
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Bad owner name on hidden primary

[Raymond Drew Walker]

On 6/9/14, 9:05 PM, "Mark Andrews"  wrote:


>
>In message , Raymond Drew Walker
>writes:
>>
>> Apologies,
>>
>> Our workaround was actually the addition of 2 lines:
>>
>>check-names master ignore;
>>check-names response ignore;
>
>"check-names master ignore;" or "check-names ignore;" at the zone
>level, is all that is required to have updates that would be block
>by check-names accepted and returned.  "check-names response ignore;"
>is the default and has been in all release of BIND back to the
>initial release where check-names was added in BIND 8.
>
>I suspect you were running a very out of date version of named on
>the old master (pre-9.5.0).

We¹ve been running current versions of NAMED on our master for at least
the past year: 9.5.4 since Nov 2013 upgraded to 9.5.5 in April.
Underscores in dynamic updates were not having any issue until our move to
a hidden primary was just earlier this month. This is why I thought this
behavior was interesting and worth getting to the bottom of. Currently if
I remove either, updating fails for underscore hosts.

>The correct fix is to stop using non-compliant names.

As much as we¹d all love to live in a 1 RFC bubble, our customers do not
allow us to. On the other hand, I don¹t like the idea of moving away from
NAMED! I¹ll shortly tangent to explain:

DNS-SD (DNS-Based Service Discovery): widely used here & requires
underscores. ( RFC-6763 http://tools.ietf.org/html/rfc6763 ) Use of DNS-SD
is quickly becoming inevitable for the modern enterprise DNS environment,
as it gains support from many, many services that are becoming commonplace
(SIP (VoIP), MS, Apple, etc.): http://www.dns-sd.org/ServiceTypes.html

Coming full circle, my interest is more focused in what factors play into
check-names behavior (why we were able to get away with this behavior for
years,) and more importantly what best practices should be used for
supporting DNS-SD. I¹m not too comfortable with the catch-all of not
checking names being part of our main options section (or should I be?).
I¹ve included some relevant information for debugging: (feel free to ask
for more if applicable.)

- Our old master (ns3.nau.edu) was reconfigured to run as an authoritative
slave as the new hidden master was brought live.
- The SOA MNAME record has stayed the same for zones we master
(ns3.nau.edu).
- Dynamic updates are made directly to the master, and are only granted
via TSIG key based allow-update.
- From my research, and how we do dynamic updates, we have not added any
"allow-update-forwarding² clauses.

Thanks much for your attention to this matter.

>
>Mark
>
>> Without the second `response' clause, the update does not error, but
>>does
>> not get applied to the record.
>> -
>> Raymond Walker
>> Software Systems Engineer StSp.
>> ITS - Northern Arizona University
>>
>> From: Ray Walker mailto:ray.wal...@nau.edu>>
>> Date: Monday, June 9, 2014 at 3:18 PM
>> To: "bind-users@lists.isc.org<mailto:bind-users@lists.isc.org>"
>> mailto:bind-users@lists.isc.org>>
>> Subject: Re: Bad owner name on hidden primary
>>
>> Our current workaround is to add the following to NAMED configuration:
>>
>>   check-names master ignore;
>>
>> Is there a more preferred solution?
>>
>> ...
>> or perhaps a different way of looking at this issue?
>> -
>> Raymond Walker
>> Software Systems Engineer StSp.
>> ITS - Northern Arizona University
>>
>> From: Ray Walker mailto:ray.wal...@nau.edu>>
>> Date: Monday, June 9, 2014 at 11:47 AM
>> To: "bind-users@lists.isc.org<mailto:bind-users@lists.isc.org>"
>> mailto:bind-users@lists.isc.org>>
>> Subject: Bad owner name on hidden primary
>>
>> Running BIND 9.9.5:
>>
>> On moving to a hidden primary setup, dynamic updates to zones we are
>> master for with "unallowed characters" (underscores in our case) have
>> started to fail with the error "bad owner name (check-names)" In the
>>past
>> (pre hidden primary) they did not fail.
>>
>> In the past we have not used the `check-names' option, so behavior
>>should
>> be default...
>>  odd since the default behavior is to fail for master zones.
>>
>> Could this have something to do with the SOA of the zone no longer being
>> the actual primary?
>> -
>> Raymond Walker
>> Software Systems Engineer StSp.
>> ITS - Northern Arizona University
>>
>
>-- 
>Mark Andrews, ISC
>1 Seymour St., Dundas Valley, NSW 2117, Australia
>PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

What to report for "refresh: failure trying master ... operation canceled" bug?

[Raymond Drew Walker]

Howdy,

We’ve noticed the error message "refresh: failure trying master ...: operation 
canceled” in our logs debugged from some slaves not updating DS records in some 
zones.

Looking into this error over at: 
https://deepthought.isc.org/article/AA-01213/0/What-causes-refresh:-failure-trying-master-...:-operation-canceled-error-messages.html

So far we have updated the RHEL6 kernel on the slaves which did nothing.

We have disabled the netfilter module which does seem to resolve the issue in 
our limited testing, but our sysadmins would like to continue use of this 
module for other reasons.

My question:
What information would be most useful in our incoming bug report?

—
Raymond Walker
Software Systems Engineer StSp
ITS Northern Arizona University

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users