Re: Jostle logic seems to randomly stop working

[Tuomo Soini via Unbound-users]

On Wed, 23 May 2018 08:11:09 +0200
"W.C.A. Wijngaards via Unbound-users"  wrote:

> Hi Dmitry,
> 
> On 19/05/18 03:59, Dmitri Kourennyi via Unbound-users wrote:
> > More investigation results:
> > 
> > I think the issue appears when unbound is trying to probe the master
> > servers for
> > the auth_zone section. The logs show unbound doing lookups on all
> > the auth_zone
> > domain names in my config, and I think unbound is answering them
> > from its own
> > cache. After the lookups, I get the following in the logs:  
> 
> Can you show the work that it does for looking up one of the root
> servers?  Not getting an address must be causing it to not have
> content. But it does work (eventually), you say, once the long list
> appears, that means the AXFR has completed.  Meanwhile you should
> have normal service, because the zone is loaded (the file that is
> configured has content, right?)?  When a normal query arrives, it
> should just be answered with the auth-zone?
> 
> The bug that was fixed in 1.7.1 (causes problem now?), supposedly
> fixes behaviour with respect to the forward-zones configured.  Is
> that still not right somehow?  Note that having a forward zone for
> "." and also an auth-zone 7706 for the root, in 1.7.1 answers only
> queries for the root itself from the root (only domain ".") and other
> queries from the forward-zone.  Where in 1.7.0 it would pick the
> auth-zone referral and go make authoritative queries (and that was a
> bug and fixed).  So, if 1.7.1 does not work, perhaps authoritative
> queries work, but the forward-zone does not work so well.  And if you
> remove that, then unbound starts making authoritative queries again.
> 
> That the root zone is downloaded every half hour is normal, that is
> exactly the AXFR of the root zone that the auth-zone is supposed to
> do. So that seems to be working fine and is keeping the root zone up
> to date.

I did hit this same issue with 1.7.1rc1 with just root zone and without
any forward zone. In my small installation unbound stopped answering
queries after a day or two. And fix for the issue is removing auth-zone
7706. And this worked on 1.7.0.

-- 
Tuomo Soini 
Foobar Linux services
+358 40 5240030
Foobar Oy 


pgpE5DJwcJGSI.pgp
Description: OpenPGP digital signature

Re: Response Policy Zone Support

[Paul Vixie via Unbound-users]

as before, we have code that implements rpz for unbound. however, it is 
not open-source licensed. any unbound recursive server that operates a 
passive dns sensor and thus sends its cache miss traffic to SIE, is 
automatically licensed to be linked against and run alongside "fastrpz" 
which is our name for the rpz implementation for unbound (and also bind9 
though that's rarely used.)


vixie

re:

Matthew Stith via Unbound-users wrote:

Hello,

Unbound does not currently provide support for Response Policy Zone
(RPZ) but it has been stated in the past on the list that support for it
is on the roadmap of development. Is there any update on when RPZ will
be implemented and if there is any alpha/beta version of Unbound with
RPZ that needs some testing done?

Regards,
Matt




--
P Vixie

Re: Some sites not resolving (DNSSEC?)

[Petr Špaček via Unbound-users]

On 23.5.2018 15:58, Petr Špaček via Unbound-users wrote:

On 23.5.2018 15:46, W.C.A. Wijngaards via Unbound-users wrote:

Hi Hank,

On 23/05/18 15:23, Hank Barta via Unbound-users wrote:

Hi all,
I use pfsense for my firewall and have selected the unbound resolver for
DNS on my home LAN. I have configured this to use Cloudflare DNS with
DNSSEC enabled.  In addition to checking the "Enable DNSSEC Support"
checkbox on the DNS Resolver configuration page I have added the custom
options


The 1.1.1.1 server responds without DNSSEC for coder.show DS queries.
And for an insecure referral it needs DS denial information for type DS,
eg. the NSEC or NSEC3 from the .show TLD.

Without the forward to 1.1.1.1 it works fine for me.  So it doesn't seem
to be the .show TLD or coder.show site, but the 1.1.1.1 unsigned CNAME
for qtype DS.

A workaround is domain-insecure: "coder.show" in unbound.conf


This is most likely a bug in Knot Resolver and we are working on fix:
https://gitlab.labs.nic.cz/knot/knot-resolver/issues/359


For the record:
We found out that domain coder.show is misconfigured in a way which 
breaks even 30 years old DNS standards.


See this:

$ dig +dnssec @ns2.hover.com. coder.show DS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50641
;; flags: qr rd; QUERY: 1, ANSWER: 1, AUTHORITY: 13, ADDITIONAL: 0

;; QUESTION SECTION:
;coder.show.IN  DS

;; ANSWER SECTION:
coder.show. 900 IN  CNAME   hosted.fireside.fm.


$ dig +dnssec @ns2.hover.com. coder.show NS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24968
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;coder.show.IN  NS

;; ANSWER SECTION:
coder.show. 900 IN  NS  ns2.hover.com.
coder.show. 900 IN  NS  ns1.hover.com.


I.e. this domain has CNAME at the apex which is violation of DNS 
standards, namely

https://tools.ietf.org/html/rfc1034#section-3.6.2

Please contact domain owner and ask for a fix. (It seems that all the 
domains mentioned in the ticket have the same issue.)


It might work elsewhere but this is not guaranteed (i.e. works 
accidentally).


Thank you for understanding.

--
Petr Špaček  @  CZ.NIC

Re: Some sites not resolving (DNSSEC?)

[Havard Eidnes via Unbound-users]

> This generally seems to work except for several hosts from which I try to
> fetch podcasts. One of these is coder.show.

Just a note,

 http://dnsviz.net/d/coder.show/dnssec/

shows several warnings related to coder.show -- apparently the
auth name servers reply with CNAME *and* other data for the zone
apex, and they also fail to respond with an EDNS0 OPT record.

Regards,

- Håvard

Re: Some sites not resolving (DNSSEC?)

[Hank Barta via Unbound-users]

Thanks for looking into this. I have added some other sites that also
present this problem to the issue.

best,
hank

On Wed, May 23, 2018 at 8:58 AM, Petr Špaček via Unbound-users <
unbound-users@unbound.net> wrote:

> On 23.5.2018 15:46, W.C.A. Wijngaards via Unbound-users wrote:
>
>> Hi Hank,
>>
>> On 23/05/18 15:23, Hank Barta via Unbound-users wrote:
>>
>>> Hi all,
>>> I use pfsense for my firewall and have selected the unbound resolver for
>>> DNS on my home LAN. I have configured this to use Cloudflare DNS with
>>> DNSSEC enabled.  In addition to checking the "Enable DNSSEC Support"
>>> checkbox on the DNS Resolver configuration page I have added the custom
>>> options
>>>
>>
>> The 1.1.1.1 server responds without DNSSEC for coder.show DS queries.
>> And for an insecure referral it needs DS denial information for type DS,
>> eg. the NSEC or NSEC3 from the .show TLD.
>>
>> Without the forward to 1.1.1.1 it works fine for me.  So it doesn't seem
>> to be the .show TLD or coder.show site, but the 1.1.1.1 unsigned CNAME
>> for qtype DS.
>>
>> A workaround is domain-insecure: "coder.show" in unbound.conf
>>
>
> This is most likely a bug in Knot Resolver and we are working on fix:
> https://gitlab.labs.nic.cz/knot/knot-resolver/issues/359
>
> --
> Petr Špaček  @  CZ.NIC
>



-- 
'03 BMW F650CS - hers
'98 Dakar K12RS - "BABY K" grew up.
'93 R100R w/ Velorex 700 (MBD starts...)
'95 Miata - "OUR LC"
polish visor: apply squashed bugs, rinse, repeat
Beautiful Sunny Winfield, Illinois

Tuning for survey workloads

[Viktor Dukhovni via Unbound-users]

My workload sends lots of queries to various TLDs and public suffix
2LDs (.co.uk, ...), but non-infrastructure queries to leaf domains
are almost not repeated sufficiently often to be found in the cache.

How should I tune the cache?  Ideally, (but unbound likely can't
do this), the NS/A//DNSKEY records of domains that have delegated
sub-domains would be cached in a separate cache (maybe even a
separate max-ttl) from the cache that handles "leaf" domains.

In the mean time I probably need a medium-sized infra cache and a
small data cache?  Not quite sure how to tune a nameserver whose
dominant client walks ~6 or more million domains scattered across
various TLDs and 2LDs, getting a few records from each domain
(DNSKEY, MX, A + TLSA for each MX stopping early if same MX already
seen at some other domain or is in an unsigned zone) and moves on
to the next domain, visiting each delegated domain "once" (a few
related queries in quick succession, in parallel with a few hundred
similar domains).

As mentioned originally, 3.5 billion queries, 1.4 million cache
hits!  Any advice on tuning for "surveys"?

-- 
Viktor.

Re: Unbound on FreeBSD 11, uses just one of 8 threads?

[Viktor Dukhovni via Unbound-users]

On Wed, May 23, 2018 at 07:56:42AM +0200, W.C.A. Wijngaards wrote:

> > I have 8 threads configured, anyone know why unbound would
> > do all the work in just one thread?
> 
> Previously people that asked this, had a usage that one thread could
> satisfy.  Perhaps the other cpu cores are running some other process.

Or it seems that on FreeBSD (and perhaps other BSDs) SO_REUSEPORT
does not dispatch to multiple threads.  One thread gets all the
traffic.

> It it the systems scheduler for delivering packets to the listening
> network socket that determines which thread gets the content.
> so-reuseport: yes reportedly improves distribution between threads on Linux.

On FreeBSD it seems to cause all the traffic to go one thread.  I
turned it off, restarted unbound, and now all the threads are busy.
The throughput has not however changed significantly.  Perhaps as
you say one thread is enough...

-- 
Viktor.

Re: Response Policy Zone Support

[Matthew Stith via Unbound-users]

On 5/23/2018 10:03 AM, A. Schulze via Unbound-users wrote:

>
> Matthew Stith via Unbound-users:
>
>> Unbound does not currently provide support for Response Policy Zone
>> (RPZ) but it has been stated in the past on the list that support for it
>> is on the roadmap of development. Is there any update on when RPZ will
>> be implemented and if there is any alpha/beta version of Unbound with
>> RPZ that needs some testing done?
>
> unbound source come with contrib/fastrpz.patch
> That's a patch that you may apply if you compile unbound yourself.
>
> It you do that, than unbound is able to talk to a fastrpz daemon
> provided as commercial product by farsightsecurity.com
>
> I currently build unbound with that patch enabled but don't use fastrpzd
> on any resolver. So all I can say: it compiles and don't hurt.
>
> Andreas
>
Andreas,

Thank you for the detail here regarding the fastrpz implementation.

Does anyone know if there is a plan for native support without the need
for a commercial license?

~Matt

Re: Response Policy Zone Support

[A. Schulze via Unbound-users]

Matthew Stith via Unbound-users:


Unbound does not currently provide support for Response Policy Zone
(RPZ) but it has been stated in the past on the list that support for it
is on the roadmap of development. Is there any update on when RPZ will
be implemented and if there is any alpha/beta version of Unbound with
RPZ that needs some testing done?


unbound source come with contrib/fastrpz.patch
That's a patch that you may apply if you compile unbound yourself.

It you do that, than unbound is able to talk to a fastrpz daemon
provided as commercial product by farsightsecurity.com

I currently build unbound with that patch enabled but don't use fastrpzd
on any resolver. So all I can say: it compiles and don't hurt.

Andreas

Re: Some sites not resolving (DNSSEC?)

[Petr Špaček via Unbound-users]

On 23.5.2018 15:46, W.C.A. Wijngaards via Unbound-users wrote:

Hi Hank,

On 23/05/18 15:23, Hank Barta via Unbound-users wrote:

Hi all,
I use pfsense for my firewall and have selected the unbound resolver for
DNS on my home LAN. I have configured this to use Cloudflare DNS with
DNSSEC enabled.  In addition to checking the "Enable DNSSEC Support"
checkbox on the DNS Resolver configuration page I have added the custom
options


The 1.1.1.1 server responds without DNSSEC for coder.show DS queries.
And for an insecure referral it needs DS denial information for type DS,
eg. the NSEC or NSEC3 from the .show TLD.

Without the forward to 1.1.1.1 it works fine for me.  So it doesn't seem
to be the .show TLD or coder.show site, but the 1.1.1.1 unsigned CNAME
for qtype DS.

A workaround is domain-insecure: "coder.show" in unbound.conf


This is most likely a bug in Knot Resolver and we are working on fix:
https://gitlab.labs.nic.cz/knot/knot-resolver/issues/359

--
Petr Špaček  @  CZ.NIC

Re: Some sites not resolving (DNSSEC?)

[W.C.A. Wijngaards via Unbound-users]

Hi Hank,

On 23/05/18 15:23, Hank Barta via Unbound-users wrote:
> Hi all,
> I use pfsense for my firewall and have selected the unbound resolver for
> DNS on my home LAN. I have configured this to use Cloudflare DNS with
> DNSSEC enabled.  In addition to checking the "Enable DNSSEC Support"
> checkbox on the DNS Resolver configuration page I have added the custom
> options

The 1.1.1.1 server responds without DNSSEC for coder.show DS queries.
And for an insecure referral it needs DS denial information for type DS,
eg. the NSEC or NSEC3 from the .show TLD.

Without the forward to 1.1.1.1 it works fine for me.  So it doesn't seem
to be the .show TLD or coder.show site, but the 1.1.1.1 unsigned CNAME
for qtype DS.

A workaround is domain-insecure: "coder.show" in unbound.conf

Best regards, Wouter


;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION:
coder.show. IN  DS

;; ANSWER SECTION:
coder.show. 437 IN  CNAME   hosted.fireside.fm.

;; AUTHORITY SECTION:
fireside.fm.3600IN  SOA cory.ns.cloudflare.com.
dns.cloudflare.com. 2027772252 1 2400 604800 3600

;; ADDITIONAL SECTION:
;; MSG SIZE  rcvd: 122


>     server:
>     forward-zone:
>     name: "."
>     forward-ssl-upstream: yes
>     forward-addr: 1.1.1.1@853
>     forward-addr: 1.0.0.1@853
> 
> (full configuration at the link below.)
> 
> This generally seems to work except for several hosts from which I try
> to fetch podcasts. One of these is coder.show. I have bumped logging for
> unbound one level and collected the log for this host and which can be
> viewed at
> https://docs.google.com/document/d/1oPUpRzIdANfuUuU7ljXNts1cR79FxBul099lbcBwE54/edit?usp=sharing
> 
> 
> The last several lines are (oldest last)
> May 20 10:34:52 info: Could not establish a chain of trust to keys for
> coder.show. DNSKEY IN
> May 20 10:34:52info: query response was nodata ANSWER
> May 20 10:34:52 info: reply from <.> 1.1.1.1#853
> 
> Other information: Even though none of the other hosts on my LAN can
> resolve this name, it is resolved by the diagnostic page on pfsense.
> 
> If I check the name at https://dnslookup.org/coder.show/A/#dnssec it
> reports that the "Result is Insecure." However I get the same result for
> google.com  and it resolves w/out difficulty on my
> LAN. I'm not familiar with all of the information on this page but one
> thing that caught my attention was the query to ns2.hover.com
> . The AUTHORITY section seems to show a bunch of
> queries that return no data. Does this indicate a missing certificate?
> 
> Any suggestions for fixing this are most welcome!
> 
> thanks,
> hank
> 
> -- 
> '03 BMW F650CS - hers
> '98 Dakar K12RS - "BABY K" grew up.
> '93 R100R w/ Velorex 700 (MBD starts...)
> '95 Miata - "OUR LC"
> polish visor: apply squashed bugs, rinse, repeat
> Beautiful Sunny Winfield, Illinois




signature.asc
Description: OpenPGP digital signature

Some sites not resolving (DNSSEC?)

[Hank Barta via Unbound-users]

Hi all,
I use pfsense for my firewall and have selected the unbound resolver for
DNS on my home LAN. I have configured this to use Cloudflare DNS with
DNSSEC enabled.  In addition to checking the "Enable DNSSEC Support"
checkbox on the DNS Resolver configuration page I have added the custom
options

server:
forward-zone:
name: "."
forward-ssl-upstream: yes
forward-addr: 1.1.1.1@853
forward-addr: 1.0.0.1@853

(full configuration at the link below.)

This generally seems to work except for several hosts from which I try to
fetch podcasts. One of these is coder.show. I have bumped logging for
unbound one level and collected the log for this host and which can be
viewed at
https://docs.google.com/document/d/1oPUpRzIdANfuUuU7ljXNts1cR79FxBul099lbcBwE54/edit?usp=sharing

The last several lines are (oldest last)
May 20 10:34:52 info: Could not establish a chain of trust to keys for
coder.show. DNSKEY IN
May 20 10:34:52 info: query response was nodata ANSWER
May 20 10:34:52 info: reply from <.> 1.1.1.1#853

Other information: Even though none of the other hosts on my LAN can
resolve this name, it is resolved by the diagnostic page on pfsense.

If I check the name at https://dnslookup.org/coder.show/A/#dnssec it
reports that the "Result is Insecure." However I get the same result for
google.com and it resolves w/out difficulty on my LAN. I'm not familiar
with all of the information on this page but one thing that caught my
attention was the query to ns2.hover.com. The AUTHORITY section seems to
show a bunch of queries that return no data. Does this indicate a missing
certificate?

Any suggestions for fixing this are most welcome!

thanks,
hank

-- 
'03 BMW F650CS - hers
'98 Dakar K12RS - "BABY K" grew up.
'93 R100R w/ Velorex 700 (MBD starts...)
'95 Miata - "OUR LC"
polish visor: apply squashed bugs, rinse, repeat
Beautiful Sunny Winfield, Illinois

Response Policy Zone Support

[Matthew Stith via Unbound-users]

Hello,

Unbound does not currently provide support for Response Policy Zone
(RPZ) but it has been stated in the past on the list that support for it
is on the roadmap of development. Is there any update on when RPZ will
be implemented and if there is any alpha/beta version of Unbound with
RPZ that needs some testing done?

Regards,
Matt

Re: Jostle logic seems to randomly stop working

[W.C.A. Wijngaards via Unbound-users]

Hi Dmitry,

On 19/05/18 03:59, Dmitri Kourennyi via Unbound-users wrote:
> More investigation results:
> 
> I think the issue appears when unbound is trying to probe the master
> servers for
> the auth_zone section. The logs show unbound doing lookups on all the
> auth_zone
> domain names in my config, and I think unbound is answering them from
> its own
> cache. After the lookups, I get the following in the logs:

Can you show the work that it does for looking up one of the root
servers?  Not getting an address must be causing it to not have content.
 But it does work (eventually), you say, once the long list appears,
that means the AXFR has completed.  Meanwhile you should have normal
service, because the zone is loaded (the file that is configured has
content, right?)?  When a normal query arrives, it should just be
answered with the auth-zone?

The bug that was fixed in 1.7.1 (causes problem now?), supposedly fixes
behaviour with respect to the forward-zones configured.  Is that still
not right somehow?  Note that having a forward zone for "." and also an
auth-zone 7706 for the root, in 1.7.1 answers only queries for the root
itself from the root (only domain ".") and other queries from the
forward-zone.  Where in 1.7.0 it would pick the auth-zone referral and
go make authoritative queries (and that was a bug and fixed).  So, if
1.7.1 does not work, perhaps authoritative queries work, but the
forward-zone does not work so well.  And if you remove that, then
unbound starts making authoritative queries again.

That the root zone is downloaded every half hour is normal, that is
exactly the AXFR of the root zone that the auth-zone is supposed to do.
So that seems to be working fine and is keeping the root zone up to date.

Best regards, Wouter

> 
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: mesh_run: end
> 13 recursion states (462 with reply, 13 detached), 13 waiting replies,
> 1444 recursion replies sent, 1 replies dropped, 0 states jostled out
> [*snip* histogram]
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 0RDd mod0 
> b.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 1RDd mod0 
> c.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 2RDd mod0 
> f.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 3RDd mod0 
> g.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 4RDd mod0 
> k.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 5RDd mod0 
> xfr.cjr.dns.icann.org. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 6RDd mod0 
> xfr.lax.dns.icann.org. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 7RDd mod0 
> b.root-servers.net.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 8RDd mod0 
> c.root-servers.net.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 9RDd mod0 
> g.root-servers.net.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 10RDd mod0 
> k.root-servers.net.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 11RDd mod0 
> xfr.cjr.dns.icann.org.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: 12RDd mod0 
> xfr.lax.dns.icann.org.  IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: mesh_run: end
> 12 recursion states (462 with reply, 12 detached), 12 waiting replies,
> 1444 recursion replies sent, 1 replies dropped, 0 states jostled out
> ⋮
> 
> The above pattern repeats, with the last entry in the RDd list dropping
> off, and
> the `detached` and `waiting reply` counters going down by one each time.
> Once
> the list is emptied I get:
> 
> ⋮
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: mesh_run: end 1
> recursion states (462 with reply, 1 detached), 1 waiting replies, 1444
> recursion replies sent, 1 replies dropped, 0 states jostled out
> [*snip* histogram]
> 0RDd mod0  b.root-servers.net. A IN
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] info: mesh_run: end 1
> recursion states (462 with reply, 0 detached), 0 waiting replies, 1444
> recursion replies sent, 1 replies dropped, 0 states jostled out
> [*snip* histogram]
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] debug: auth zone .:
> soa probe serial is 2018051800
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] debug: auth_zone
> unchanged, new lease, wait
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] debug: auth zone .
> timeout in 1800 seconds
> May 18 14:59:11 homebrew unbound[30620]: [30620:0] debug: close fd 15
> 
> From this point onward, queries are rejected. Half an hour later, the
> auth-zone
> lookup attempt repeats, and it all looks like above. 4 hours later, I get a
> bunch of lines like this:
> 
> May 18 16:59:11 homebrew unbound[30620]: [30620:0] debug: comm point
> start listening 15
> May 18 16:59:11 homebrew unbound[30620]: [30620:0] debug: Reading tcp
> query of length