Re: [DNSOP] raising the bar: requiring implementations

[神明達哉]

At Fri, 6 Apr 2018 04:35:44 +,
Evan Hunt  wrote:

> > At Thu, 5 Apr 2018 13:46:29 -0400,
> > tjw ietf  wrote:
> >
> > > What is work: An "informational" document being used as standard is people
> > > taking a submitted (expired) draft as "standard"?
> >
> > I'm not sure how to interpret it (not even sure if it's a question to
> > me)...
>
> I suspect Tim meant to type "What is worse: An 'informational' document
> being used as standard, or people taking a submitted (expired) draft as
> 'standard'?"

Ah, okay, it makes sense, thanks.  (Correcting this was far beyond my
ability of dealing with the language of English:-).  I still don't
know if it was meant to be a counter argument to my previous message
or an unrelated followup, but in any case that's a different topic
than my main concern.  This is much closer to the point:

> ECS, though, was published before it was fully cooked, and continuing to
> iterate and update the drafts would've been better.

and, at least in retrospect, my observation was that the intended
informational status was (ab)used to have this result, and IMO we
should be careful not to repeat it.

--
JINMEI, Tatuya

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Evan Hunt]

On Thu, Apr 05, 2018 at 01:31:37PM -0700, 神明達哉 wrote:
> At Thu, 5 Apr 2018 13:46:29 -0400,
> tjw ietf  wrote:
> 
> > What is work: An "informational" document being used as standard is people
> > taking a submitted (expired) draft as "standard"?
> 
> I'm not sure how to interpret it (not even sure if it's a question to
> me)...

I suspect Tim meant to type "What is worse: An 'informational' document
being used as standard, or people taking a submitted (expired) draft as
'standard'?"

To answer, I think which of those is worse depends on the implementation
status. I don't have any problem with an informational document to explain
the details of something that's already widely deployed, but which for
whatever reason can't go through the standards process.

Consider RPZ, for example: it's been implemented several times, there's
lots and lots of real-world deployment experience. I'd be happy to see an
informational RFC describing it; I'd be confident in its stability.
Relying on old expired drafts would be disappointing.

ECS, though, was published before it was fully cooked, and continuing to
iterate and update the drafts would've been better.

-- 
Evan Hunt -- e...@isc.org
Internet Systems Consortium, Inc.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Andrew Sullivan]

On Wed, Mar 28, 2018 at 05:24:33PM +0200, bert hubert wrote:
> allow the one remaining closed source DNS implementation



Really?  I'm so pleased we have not only candidate censors of what is
going to be published by the WG but also census-takers who have
determined then number and types of DNS implemntations in the
universe.

In case it isn't clear from the above, I am deeply uncomfortable with
the entire DNS-police-cum-government latent in this thread.  I think
interoperability is important and valuable.  I do not think that
creating a clubhouse of kool kids who decide what "is" a DNS
implementation is wise, however.

A

-- 
Andrew Sullivan
a...@anvilwalrusden.com

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[神明達哉]

At Thu, 5 Apr 2018 13:46:29 -0400,
tjw ietf  wrote:

> What is work: An "informational" document being used as standard is people
> taking a submitted (expired) draft as "standard"?

I'm not sure how to interpret it (not even sure if it's a question to
me)...but I guess what I think is the most important point of my
previous message is "for future dnsop documents, we shouldn't use the
intended informational status as an excuse to casually dismiss serious
questions/suggestions from implementers".

--
JINMEI, Tatuya

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[tjw ietf]

What is work: An "informational" document being used as standard is people
taking a submitted (expired) draft as "standard"?

Tim





On Thu, Apr 5, 2018 at 1:39 PM, 神明達哉  wrote:

> At Wed, 4 Apr 2018 11:22:33 +0200,
> Petr Špaček  wrote:
>
> > >> This is actually quite a good example of another problem:
> > >> Client-subnet was documented for Informational purposes; it was
> > >> already in wide use in the public internet and had an EDNS opt code
> > >> codepoint allocated from the IANA registry.
> > >>
> > >> Nothing that happened in DNSOP or the IETF changed that, and I
> > >> strongly suspect nothing that happened in DNSOP or the IETF could have
> > >> changed it.
> > >
> > > We found issues in the client-subnet description (in the draft stages)
> > > that were corrected before it became RFC - this involved some
> behavioral
> > > changes. However, the opportunity was not given to address fundamental
> > > design issues, so what's in the RFC largely documents the
> > > implementations preceding it and still has issues. I didn't think
> > > client-subnet was in wide use when it came to dnsop. Even today it
> > > doesn't look like it's in wide use.
>
> [...]
>
> > I tend to agree with Mukund. What's the point of doing standards, if
> > there is nothing to test against?
>
> I also agree here.  Especially in the case of client-subnet, I
> observed another (IMO) bad practice that seems to be abused quite
> often recently: use the word of "informational" as an excuse for
> dismissing suggestions.  The commonly seen logic is "this is just
> intended to be an information document, just describing an existing
> deployment in case that may be useful for others (and so any
> significant changes should be rejected)".  But in actuality such a
> spec is often used as a standard protocol spec that should be
> interoperable among various different implementations.  And, IMO, that
> was actually the case for ECS.
>
> Another rhetoric often (ab)used in such a case is to say "a more
> complete full standard will follow this informational document; we can
> make significant changes at that point."  But in reality such a
> followup task often doesn't even start.  Again, that's also the case
> for ECS after nearly two years since the publication of RFC7871.
>
> In the context of the camel discussion, I think we should use this
> lesson for rejecting this kind of abusing the "informational" status.
> We should not pretend it's really just for information when we
> actually expect it will be used a "de facto" standard that is likely
> to be implemented by various vendors.
>
> --
> JINMEI, Tatuya
>
> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop
>
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[神明達哉]

At Wed, 4 Apr 2018 11:22:33 +0200,
Petr Špaček  wrote:

> >> This is actually quite a good example of another problem:
> >> Client-subnet was documented for Informational purposes; it was
> >> already in wide use in the public internet and had an EDNS opt code
> >> codepoint allocated from the IANA registry.
> >>
> >> Nothing that happened in DNSOP or the IETF changed that, and I
> >> strongly suspect nothing that happened in DNSOP or the IETF could have
> >> changed it.
> >
> > We found issues in the client-subnet description (in the draft stages)
> > that were corrected before it became RFC - this involved some behavioral
> > changes. However, the opportunity was not given to address fundamental
> > design issues, so what's in the RFC largely documents the
> > implementations preceding it and still has issues. I didn't think
> > client-subnet was in wide use when it came to dnsop. Even today it
> > doesn't look like it's in wide use.

[...]

> I tend to agree with Mukund. What's the point of doing standards, if
> there is nothing to test against?

I also agree here.  Especially in the case of client-subnet, I
observed another (IMO) bad practice that seems to be abused quite
often recently: use the word of "informational" as an excuse for
dismissing suggestions.  The commonly seen logic is "this is just
intended to be an information document, just describing an existing
deployment in case that may be useful for others (and so any
significant changes should be rejected)".  But in actuality such a
spec is often used as a standard protocol spec that should be
interoperable among various different implementations.  And, IMO, that
was actually the case for ECS.

Another rhetoric often (ab)used in such a case is to say "a more
complete full standard will follow this informational document; we can
make significant changes at that point."  But in reality such a
followup task often doesn't even start.  Again, that's also the case
for ECS after nearly two years since the publication of RFC7871.

In the context of the camel discussion, I think we should use this
lesson for rejecting this kind of abusing the "informational" status.
We should not pretend it's really just for information when we
actually expect it will be used a "de facto" standard that is likely
to be implemented by various vendors.

--
JINMEI, Tatuya

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Petr Špaček]

On 29.3.2018 22:59, Phillip Hallam-Baker wrote:
> On Thu, Mar 29, 2018 at 3:18 PM, Suzanne Woolf  > wrote:
> 
> 
> Should we refuse to document such things because they’re not in
> well-known open source codebases, or have otherwise not passed tests
> of goodness?
> 
> It’s not a rhetorical question. Given that people are extending the
> protocol outside of the IETF or any other formal process, in order
> to solve their own technical and business problems, it makes sense
> to ask ourselves periodically whether we should be encouraging them,
> trying to stop them, or something in between.
> 
> 
> ​There are in fact precedents for doing something of the sort. But not
> ones that I think we should follow.
> 
> DNSSEC would have been implemented as part of the VeriSign ATLAS roll
> out. The reason it was not was it simply could not be deployed using the
> tech available with the spec as it was at the time without a major
> increase in cost and complexity.
> 
> A blanket requirement for open source implementations may well have the
> same effect.
> 
> Operating systems at Internet scale is vastly more complex than running
> systems at network scale. It is not just the genius of the original
> Internet designers that has allowed the Internet to scale from 100 nodes
> to 10 billion. There is a vast amount of unseen engineering work that is
> equally heroic and some of that infrastructure comes with some very
> specific constraints.
> 
> Some of the most useful work in IETF is making closed systems more open.
> Often those projects require some extra slots to carry information that
> is needed for some legacy system. I see no value to saying folk can't
> have a smooth transition path from proprietary to open.

I do not see a problem. First there needs to be running code and RFC
will follow. It IMHO does not make much sense the other way around, and
we will suffer for next decade because we did not follow this rule in
the past.

-- 
Petr Špaček  @  CZ.NIC

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Petr Špaček]

On 29.3.2018 22:05, Mukund Sivaraman wrote:
> On Thu, Mar 29, 2018 at 03:18:45PM -0400, Suzanne Woolf wrote:
>>
>> On Mar 28, 2018, at 11:19 AM, Mukund Sivaraman  wrote:
>>
>>> On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:
 I would say that most things we have adopted in the past few years do have
 some implementations to reference.
 Not when drafts are adopted, but generally before we head to WGLC I've
 always wanted to see someone
 who implemented the option in some manner.

 But yes, agree.
>>>
>>> I'd raise the bar even higher, to see complete implementation in a major
>>> open source DNS implementation when it applies. Sometimes implementation
>>> problems are very revealing (client-subnet should have gone through
>>> this).
>>
>> This is actually quite a good example of another problem:
>> Client-subnet was documented for Informational purposes; it was
>> already in wide use in the public internet and had an EDNS opt code
>> codepoint allocated from the IANA registry.
>>
>> Nothing that happened in DNSOP or the IETF changed that, and I
>> strongly suspect nothing that happened in DNSOP or the IETF could have
>> changed it.
> 
> We found issues in the client-subnet description (in the draft stages)
> that were corrected before it became RFC - this involved some behavioral
> changes. However, the opportunity was not given to address fundamental
> design issues, so what's in the RFC largely documents the
> implementations preceding it and still has issues. I didn't think
> client-subnet was in wide use when it came to dnsop. Even today it
> doesn't look like it's in wide use.
> 
> You have asked an interesting question, because what happens if
> something is already being used and there's no chance to update/correct
> problems because that would make it a different protocol?
> 
> IMO, we should not put anything through dnsop that does not allow
> reviewing and correcting problems. It is pointless for dnsop to shepherd
> a draft to RFC when there's no possibility to influence it.
> 
> During later stages of the draft via dnsop, we implemented client-subnet
> (resolver side) and found various problems. Some of these were addressed
> in the draft, but it was revealing how poor the state of the
> design/draft was. This is why I strongly suggest: A code contribution of
> a _complete implementation_ of everything described in the draft to one
> of the open source DNS implementations should come sometime during
> adoption, so that
> 
> (a) issues in production implementation are revealed
> 
> (b) it can be tried in the real world to understand how it behaves.
> 
> As you're a co-chair:
> 
> When Bert did that Camel presentation, I felt hooray, here's one of us
> who's talking about what the steady churn of ideas and RFCs is doing to
> DNS implementations. We finish implementing one draft and at almost
> breathless pace, a few more drafts queue up. It's not sustainable,
> esp. as all the existing features also need to be maintained for years.
> 
> Introducing a new RR type that doesn't involve additional processing is
> relatively trivial and nobody objects about it.
> 
> Introducing something like TCP pipelining, or rather, out-of-order query
> processing, may seem trivial when describing the change, but
> implementing it may need fundamental restructuring of the
> implementation. Proper implementation of client-subnet needs change
> everywhere within a nameserver from the client message parsing code to
> the data structures used for cache, and how cache lifetime is managed.
> 
> Implementations are being stretched and bent 5 different ways to adapt
> to the length and breadth of all these newfangled DNS items that
> literally are showing up at an alarming pace.
> 
> Bert really hit the spot at the right time. Something needs to be done
> to check what becomes an RFC. A good way is to require an open source
> implementation. If draft authors cannot supply that or convince an open
> source implementation to write support for it, it can go back to where
> it came from.

I tend to agree with Mukund. What's the point of doing standards, if
there is nothing to test against?

Let's imagine that e.g. Cisco and Google propose a brand new feature of
DNS protocol, but its implementation is available only for their
enterprise customers. Why should The Internet bother?

DNS is/was mostly open-source place, and I have a feeling that RFCs are
in the end closely followed only by open-source implementations anyway,
and that rest of the DNS players do whatever they want.
Examples:
- Empty non-terminals vs. Akamai
- EDNS vs. Cisco middleboxes
- EDNS ignorance (auth side) vs. Google
- 

So, why should we (as dnsop) bother "ratifying" something for the big
folks who simply do not care enough to follow what we already published?

I can see parallel with TLS group where "big guys" continually submit
various drafts to accommodate their needs while not paying attention to
impact on the rest of 

Re: [DNSOP] raising the bar: requiring implementations

[Matthijs Mekking]

I can agree with the argument that if implemented in a major open-source 
DNS implementation it would weigh in more to the discussion, but 
limiting to is far too restricting in my opinion.


Best regards,
  Matthijs


On 03/28/2018 09:48 PM, Mukund Sivaraman wrote:

On Wed, Mar 28, 2018 at 05:29:18PM +0200, Matthijs Mekking wrote:

As mentioned in the meeting, I am in favor of requiring implementations
before drafts become standards.

However, I would be opposed to limit acceptable implementations to the few
major open source DNS implementations (define major). It should be
acceptable for other organizations or just persons to contribute a reference
implementation.


It depends on the topic of the draft of course, esp. where in operations
it applies. If it is nameserver territory, I absolutely want to see an
implementation in *any* of the major DNS implementations. By major, I
mean the popular ones (e.g., PowerDNS, NSD, Unbound, Knot, etc.) This is
because:

* A full-fledged nameserver is somewhat different from a toy
   implementation in performance and scalability (this point is from
   experience with a bad implementation of a draft)

* The rest of us want to see proof that it can be implemented (not just
   a promise or mention of implementation) and play with it and observe
   operational characteristics _freely_, and determine whether a draft
   will really improve things in the way it says it will. E.g., take all
   the multiple-answer drafts that are making the rounds.. in Singapore
   there was a presentation of a grand matrix of them, but who knows how
   they actually perform?

It's 2018. We aren't living in the dark ages with a single DNS
implementation.

If a draft is for nameserver software to implement, and if the authors
cannot implement it by themselves, they can persuade one of the open
source vendors to do so. If they are unable to persuade any, that should
be enough consensus about how significant that draft is. Speaking for
myself, we in our DNS implementation add support for several drafts
early in the draft stage because they look necessary or interesting, or
because we want to know how they behave early on.

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop



___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Phillip Hallam-Baker]

On Thu, Mar 29, 2018 at 3:18 PM, Suzanne Woolf 
wrote:

>
> Should we refuse to document such things because they’re not in well-known
> open source codebases, or have otherwise not passed tests of goodness?
>
> It’s not a rhetorical question. Given that people are extending the
> protocol outside of the IETF or any other formal process, in order to solve
> their own technical and business problems, it makes sense to ask ourselves
> periodically whether we should be encouraging them, trying to stop them, or
> something in between.
>

​There are in fact precedents for doing something of the sort. But not ones
that I think we should follow.

DNSSEC would have been implemented as part of the VeriSign ATLAS roll out.
The reason it was not was it simply could not be deployed using the tech
available with the spec as it was at the time without a major increase in
cost and complexity.

A blanket requirement for open source implementations may well have the
same effect.

Operating systems at Internet scale is vastly more complex than running
systems at network scale. It is not just the genius of the original
Internet designers that has allowed the Internet to scale from 100 nodes to
10 billion. There is a vast amount of unseen engineering work that is
equally heroic and some of that infrastructure comes with some very
specific constraints.

Some of the most useful work in IETF is making closed systems more open.
Often those projects require some extra slots to carry information that is
needed for some legacy system. I see no value to saying folk can't have a
smooth transition path from proprietary to open.
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Mukund Sivaraman]

On Thu, Mar 29, 2018 at 03:18:45PM -0400, Suzanne Woolf wrote:
> 
> On Mar 28, 2018, at 11:19 AM, Mukund Sivaraman  wrote:
> 
> > On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:
> >> I would say that most things we have adopted in the past few years do have
> >> some implementations to reference.
> >> Not when drafts are adopted, but generally before we head to WGLC I've
> >> always wanted to see someone
> >> who implemented the option in some manner.
> >> 
> >> But yes, agree.
> > 
> > I'd raise the bar even higher, to see complete implementation in a major
> > open source DNS implementation when it applies. Sometimes implementation
> > problems are very revealing (client-subnet should have gone through
> > this).
> 
> This is actually quite a good example of another problem:
> Client-subnet was documented for Informational purposes; it was
> already in wide use in the public internet and had an EDNS opt code
> codepoint allocated from the IANA registry.
> 
> Nothing that happened in DNSOP or the IETF changed that, and I
> strongly suspect nothing that happened in DNSOP or the IETF could have
> changed it.

We found issues in the client-subnet description (in the draft stages)
that were corrected before it became RFC - this involved some behavioral
changes. However, the opportunity was not given to address fundamental
design issues, so what's in the RFC largely documents the
implementations preceding it and still has issues. I didn't think
client-subnet was in wide use when it came to dnsop. Even today it
doesn't look like it's in wide use.

You have asked an interesting question, because what happens if
something is already being used and there's no chance to update/correct
problems because that would make it a different protocol?

IMO, we should not put anything through dnsop that does not allow
reviewing and correcting problems. It is pointless for dnsop to shepherd
a draft to RFC when there's no possibility to influence it.

During later stages of the draft via dnsop, we implemented client-subnet
(resolver side) and found various problems. Some of these were addressed
in the draft, but it was revealing how poor the state of the
design/draft was. This is why I strongly suggest: A code contribution of
a _complete implementation_ of everything described in the draft to one
of the open source DNS implementations should come sometime during
adoption, so that

(a) issues in production implementation are revealed

(b) it can be tried in the real world to understand how it behaves.

As you're a co-chair:

When Bert did that Camel presentation, I felt hooray, here's one of us
who's talking about what the steady churn of ideas and RFCs is doing to
DNS implementations. We finish implementing one draft and at almost
breathless pace, a few more drafts queue up. It's not sustainable,
esp. as all the existing features also need to be maintained for years.

Introducing a new RR type that doesn't involve additional processing is
relatively trivial and nobody objects about it.

Introducing something like TCP pipelining, or rather, out-of-order query
processing, may seem trivial when describing the change, but
implementing it may need fundamental restructuring of the
implementation. Proper implementation of client-subnet needs change
everywhere within a nameserver from the client message parsing code to
the data structures used for cache, and how cache lifetime is managed.

Implementations are being stretched and bent 5 different ways to adapt
to the length and breadth of all these newfangled DNS items that
literally are showing up at an alarming pace.

Bert really hit the spot at the right time. Something needs to be done
to check what becomes an RFC. A good way is to require an open source
implementation. If draft authors cannot supply that or convince an open
source implementation to write support for it, it can go back to where
it came from.

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Suzanne Woolf]

On Mar 28, 2018, at 11:19 AM, Mukund Sivaraman  wrote:

> On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:
>> I would say that most things we have adopted in the past few years do have
>> some implementations to reference.
>> Not when drafts are adopted, but generally before we head to WGLC I've
>> always wanted to see someone
>> who implemented the option in some manner.
>> 
>> But yes, agree.
> 
> I'd raise the bar even higher, to see complete implementation in a major
> open source DNS implementation when it applies. Sometimes implementation
> problems are very revealing (client-subnet should have gone through
> this).

This is actually quite a good example of another problem: Client-subnet was 
documented for Informational purposes; it was already in wide use in the public 
internet and had an EDNS opt code codepoint allocated from the IANA registry.

Nothing that happened in DNSOP or the IETF changed that, and I strongly suspect 
nothing that happened in DNSOP or the IETF could have changed it.

Other documents we’ve considered on features or modifications to the protocol 
include refuse-any and, I believe, serve-stale, and the stated purpose of the 
localhost draft is to clarify the existing documentation on the reserved name 
“localhost”.

Should we refuse to document such things because they’re not in well-known open 
source codebases, or have otherwise not passed tests of goodness?

It’s not a rhetorical question. Given that people are extending the protocol 
outside of the IETF or any other formal process, in order to solve their own 
technical and business problems, it makes sense to ask ourselves periodically 
whether we should be encouraging them, trying to stop them, or something in 
between.


Suzanne

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Phillip Hallam-Baker]

On Wed, Mar 28, 2018 at 2:45 PM, Paul Vixie  wrote:

>
> i'm in general agreement with each of the assertions made at each layer of
> quoting above, but i have two quibbles.
>
> first, they aren't reference implementations. not even BIND, which for
> many years i called a reference implementation, is not one. a reference
> implementation is a special kind of beast, it's something that if you don't
> interoperate with it, you are in the wrong. we have a specification, and we
> judge the quality of that specification by the ease with which
> interoperable non-reference implementations can be made.
>
> second, i think it's 2018, and  can require that at least one of the
> demonstrated interoperable implementations be source-available. (not open
> source; we don't care about license, only transparency.)


​Quite, a reference implementation is not a production implementation. In
fact it may well not even be standards compliant because the most important
parts of an implementation to test are what happens when messages are not
as expected.

Production implementations should be forgiving in what they accept and
conservative in what they generate. Reference implementations should be the
opposite.

I generate my deployment and reference implementations from the same source
but they are not the same thing.
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Jan Komissar (jkomissa)]

Cisco

On 3/29/18, 8:12 AM, "DNSOP on behalf of Tony Finch"  wrote:

Frederico A C Neves  wrote:
> On Wed, Mar 28, 2018 at 04:46:52PM +0100, Tony Finch wrote:
> >
> >  ... I have probably missed several ...
>
> MS

D'oh!

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/  -  I xn--zr8h punycode
Lundy, Fastnet: Cyclonic 4 or 5. Moderate or rough, occasionally very rough 
in
southwest Fastnet. Thundery showers. Good, occasionally moderate.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop


___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Tony Finch]

Arsen STASIC  wrote:
>
> Quad9 is using powerdns dnsdist as loadbalancer and powerdns recursor and
> unbound [0], [1]
>
> [0] https://www.makeuseof.com/tag/quad9-dns-overview/
> [1] 
> https://www.theinquirer.net/inquirer/news/3021536/ibm-teams-with-global-cyber-alliance-to-launch-quad9-a-free-public-domain-name-service-system

Thanks for the correction!

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/  -  I xn--zr8h punycode
South Utsire, Forties, Cromarty: Easterly 5 to 7, occasionally gale 8 at first
except in Cromarty. Moderate or rough. Occasional sleet. Good, occasionally
moderate.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Tony Finch]

Frederico A C Neves  wrote:
> On Wed, Mar 28, 2018 at 04:46:52PM +0100, Tony Finch wrote:
> >
> >  ... I have probably missed several ...
>
> MS

D'oh!

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/  -  I xn--zr8h punycode
Lundy, Fastnet: Cyclonic 4 or 5. Moderate or rough, occasionally very rough in
southwest Fastnet. Thundery showers. Good, occasionally moderate.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Arsen STASIC]

* Tony Finch  [2018-03-28 16:46 (+0100)]:

bert hubert  wrote:


Well to allow the one remaining closed source DNS implementation some room,


authoritative services: Akamai Amazon Cloudflare Dyn Google Verisign
recursive services: Google OpenDNS Quad9
COTS: Nominum


Quad9 is using powerdns dnsdist as loadbalancer and powerdns recursor and 
unbound [0], [1]

[0] https://www.makeuseof.com/tag/quad9-dns-overview/
[1] 
https://www.theinquirer.net/inquirer/news/3021536/ibm-teams-with-global-cyber-alliance-to-launch-quad9-a-free-public-domain-name-service-system


-arsen

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Mukund Sivaraman]

Also, this "Camel" in the presentation may have been a mythological
beast, but the real and existing DNS Camels that struggle with their
backs breaking under the weight of DNS are the DNS implementations,
especially the open source ones that are underfunded and understaffed.

If someone wants to add more weight on top, do so by helping by
implementing it for one or more of the majorly used open source
implementations. I can't agree that a closed-source or an example
implementation of itself will suffice, when the burden eventually comes
onto us.

Other than that, the "Camel" that everyone feels sorry for doesn't
exist.

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Mukund Sivaraman]

On Thu, Mar 29, 2018 at 01:18:36AM +0530, Mukund Sivaraman wrote:
> it applies. If it is nameserver territory, I absolutely want to see an
> implementation in *any* of the major DNS implementations. By major, I
> mean the popular ones (e.g., PowerDNS, NSD, Unbound, Knot, etc.) This is

Sorry, that should read major *open source* DNS implementations! That's
the point.

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Mukund Sivaraman]

On Wed, Mar 28, 2018 at 05:29:18PM +0200, Matthijs Mekking wrote:
> As mentioned in the meeting, I am in favor of requiring implementations
> before drafts become standards.
> 
> However, I would be opposed to limit acceptable implementations to the few
> major open source DNS implementations (define major). It should be
> acceptable for other organizations or just persons to contribute a reference
> implementation.

It depends on the topic of the draft of course, esp. where in operations
it applies. If it is nameserver territory, I absolutely want to see an
implementation in *any* of the major DNS implementations. By major, I
mean the popular ones (e.g., PowerDNS, NSD, Unbound, Knot, etc.) This is
because:

* A full-fledged nameserver is somewhat different from a toy
  implementation in performance and scalability (this point is from
  experience with a bad implementation of a draft)

* The rest of us want to see proof that it can be implemented (not just
  a promise or mention of implementation) and play with it and observe
  operational characteristics _freely_, and determine whether a draft
  will really improve things in the way it says it will. E.g., take all
  the multiple-answer drafts that are making the rounds.. in Singapore
  there was a presentation of a grand matrix of them, but who knows how
  they actually perform?

It's 2018. We aren't living in the dark ages with a single DNS
implementation.

If a draft is for nameserver software to implement, and if the authors
cannot implement it by themselves, they can persuade one of the open
source vendors to do so. If they are unable to persuade any, that should
be enough consensus about how significant that draft is. Speaking for
myself, we in our DNS implementation add support for several drafts
early in the draft stage because they look necessary or interesting, or
because we want to know how they behave early on.

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Frederico A C Neves]

On Wed, Mar 28, 2018 at 04:46:52PM +0100, Tony Finch wrote:
> bert hubert  wrote:
> >
> > Well to allow the one remaining closed source DNS implementation some room,
> 
> authoritative services: Akamai Amazon Cloudflare Dyn Google Verisign
> recursive services: Google OpenDNS Quad9
> COTS: Nominum
> 
>  ... I have probably missed several ...

MS

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Frederico A C Neves]

Bert,

On Wed, Mar 28, 2018 at 05:24:33PM +0200, bert hubert wrote:
> On Wed, Mar 28, 2018 at 08:49:39PM +0530, Mukund Sivaraman wrote:
> > I'd raise the bar even higher, to see complete implementation in a major
> > open source DNS implementation when it applies. Sometimes implementation
> > problems are very revealing (client-subnet should have gone through
> > this).
> 
> Well to allow the one remaining closed source DNS implementation some room,
> I think we could live with a 'demo' from them if they'd want to. This would
> lead to an implementation report, much like is customary in the BGP WGs.
> 
> But otherwise, +100. 
> 
> This might go for MIXFR which we are discussing now btw.  It looks nice in
> theory, but I wonder about the practice, and if the people who want this
> (TLD operators I guess) would be willing to test it in simulated production
> to see if it fits their needs.

We'll definitely test this on real data with open-source clients and
with our own server implementation.

> 
>   Bert
> 

Fred

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Paul Vixie]

Matthijs Mekking wrote:



On 03/28/2018 05:19 PM, Mukund Sivaraman wrote:

On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:

I would say that most things we have adopted in the past few
years do have some implementations to reference. Not when drafts
are adopted, but generally before we head to WGLC I've always
wanted to see someone who implemented the option in some manner. >>>
But yes, agree.


I'd raise the bar even higher, to see complete implementation in a major
open source DNS implementation when it applies. Sometimes implementation
problems are very revealing (client-subnet should have gone through
this).


As mentioned in the meeting, I am in favor of requiring implementations
before drafts become standards.

However, I would be opposed to limit acceptable implementations to the
few major open source DNS implementations (define major). It should be
acceptable for other organizations or just persons to contribute a
reference implementation.


i'm in general agreement with each of the assertions made at each layer 
of quoting above, but i have two quibbles.


first, they aren't reference implementations. not even BIND, which for 
many years i called a reference implementation, is not one. a reference 
implementation is a special kind of beast, it's something that if you 
don't interoperate with it, you are in the wrong. we have a 
specification, and we judge the quality of that specification by the 
ease with which interoperable non-reference implementations can be made.


second, i think it's 2018, and we can require that at least one of the 
demonstrated interoperable implementations be source-available. (not 
open source; we don't care about license, only transparency.)


--
P Vixie

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Job Snijders]

On Wed, Mar 28, 2018 at 3:46 PM, Tony Finch  wrote:
> bert hubert  wrote:
>>
>> Well to allow the one remaining closed source DNS implementation some room,
>
> authoritative services: Akamai Amazon Cloudflare Dyn Google Verisign
> recursive services: Google OpenDNS Quad9
> COTS: Nominum

Those can just as well participate in implementation compatibility
matrices. While open source can make interop testing easier, it is
certainly not a requirement and shouldn't be treated as such.

Kind regards,

Job

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Tony Finch]

bert hubert  wrote:
>
> Well to allow the one remaining closed source DNS implementation some room,

authoritative services: Akamai Amazon Cloudflare Dyn Google Verisign
recursive services: Google OpenDNS Quad9
COTS: Nominum

 ... I have probably missed several ...

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/  -  I xn--zr8h punycode
Humber, Thames, Dover: Cyclonic becoming west or northwest 5 or 6. Slight or
moderate. Rain then showers. Moderate or poor, becoming good.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[tjw ietf]

An enterprise company with rather large zone which update often are "highly
interested" in MIXFR.
But we may be the exception rather than the rule.

Tim

On Wed, Mar 28, 2018 at 11:24 AM, bert hubert 
wrote:

> On Wed, Mar 28, 2018 at 08:49:39PM +0530, Mukund Sivaraman wrote:
> > I'd raise the bar even higher, to see complete implementation in a major
> > open source DNS implementation when it applies. Sometimes implementation
> > problems are very revealing (client-subnet should have gone through
> > this).
>
> Well to allow the one remaining closed source DNS implementation some room,
> I think we could live with a 'demo' from them if they'd want to. This would
> lead to an implementation report, much like is customary in the BGP WGs.
>
> But otherwise, +100.
>
> This might go for MIXFR which we are discussing now btw.  It looks nice in
> theory, but I wonder about the practice, and if the people who want this
> (TLD operators I guess) would be willing to test it in simulated production
> to see if it fits their needs.
>
> Bert
>
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Matthijs Mekking]

On 03/28/2018 05:19 PM, Mukund Sivaraman wrote:

On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:

I would say that most things we have adopted in the past few years do have
some implementations to reference.
Not when drafts are adopted, but generally before we head to WGLC I've
always wanted to see someone
who implemented the option in some manner.

But yes, agree.


I'd raise the bar even higher, to see complete implementation in a major
open source DNS implementation when it applies. Sometimes implementation
problems are very revealing (client-subnet should have gone through
this).


As mentioned in the meeting, I am in favor of requiring implementations 
before drafts become standards.


However, I would be opposed to limit acceptable implementations to the 
few major open source DNS implementations (define major). It should be 
acceptable for other organizations or just persons to contribute a 
reference implementation.


Best regards,

Matthijs





Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop



___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[bert hubert]

On Wed, Mar 28, 2018 at 08:49:39PM +0530, Mukund Sivaraman wrote:
> I'd raise the bar even higher, to see complete implementation in a major
> open source DNS implementation when it applies. Sometimes implementation
> problems are very revealing (client-subnet should have gone through
> this).

Well to allow the one remaining closed source DNS implementation some room,
I think we could live with a 'demo' from them if they'd want to. This would
lead to an implementation report, much like is customary in the BGP WGs.

But otherwise, +100. 

This might go for MIXFR which we are discussing now btw.  It looks nice in
theory, but I wonder about the practice, and if the people who want this
(TLD operators I guess) would be willing to test it in simulated production
to see if it fits their needs.

Bert

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Mukund Sivaraman]

On Wed, Mar 28, 2018 at 10:55:13AM -0400, tjw ietf wrote:
> I would say that most things we have adopted in the past few years do have
> some implementations to reference.
> Not when drafts are adopted, but generally before we head to WGLC I've
> always wanted to see someone
> who implemented the option in some manner.
> 
> But yes, agree.

I'd raise the bar even higher, to see complete implementation in a major
open source DNS implementation when it applies. Sometimes implementation
problems are very revealing (client-subnet should have gone through
this).

Mukund

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[tjw ietf]

I would say that most things we have adopted in the past few years do have
some implementations to reference.
Not when drafts are adopted, but generally before we head to WGLC I've
always wanted to see someone
who implemented the option in some manner.

But yes, agree.

On Wed, Mar 28, 2018 at 10:52 AM, Willem Toorop  wrote:

> I would love to see a hard requirement for implementations &
> implementation reports (like IDR has) in the charter or in the working
> group house rules.  Early implementations (perhaps even during the
> hackathon) can reveal implications that might have been missed while
> designing the draft.  In addition it is a good way to see if everybody
> interprets a draft the same way, by interoperability testing.
>
>
> Op 24-03-18 om 12:07 schreef Job Snijders:
> > Dear DNSOP,
> >
> > I'd like to share some pointers from the working group that governs the
> > BGP protocol, IDR, on requiring implementations before drafts can
> > advance towards RFC publication. Raising the bar for publication will
> > take weight off the camel's back.
> >
> > The IDR policy and rationale is outlined here:
> https://trac.ietf.org/trac/idr/wiki
> >
> > An example of an implementation report is available here:
> > https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-
> community%20implementations
> > Every normative (RFC 2119 / RFC 8174) term should expand into a
> > compliance test, and it is particularly good to document where
> > implementations deviate from what is required or suggested in the
> > proposed specification. The work listed on this wikipage was done
> > _before_ the draft was submitted to the IESG, and we intentionally
> > sought to create more than the minimum amount of implementations
> > required.
> >
> > In the case of the BGP large communities project the working group was
> > particularly lucky because Pier Carlo Chiodi created an open 'regression
> > testing'-style environment into which BGP speakers could be plugged in
> > to assess their compliance:
> > https://github.com/pierky/bgp-large-communities-playground
> > https://github.com/pierky/bgp-large-communities-playground/
> blob/master/tests/README.md
> >
> > The DNS world would benefit from such environments and automated
> > compliance testing. Manual testing for a specification (that is still
> > being worked on) can be tedious, having a 'playground' can pay huge
> > dividend. We did catch bugs in implementations thanks to this
> > environment, so it not only helped the specification, but increased the
> > quality of the participating implementations.
> >
> > RFC 7942 suggest that During the development of a draft people are
> > encouraged to document what implementations exist, an example is here:
> > https://tools.ietf.org/html/draft-ietf-idr-large-community-12#section-7
> >
> > Closed source implementations as part of such a list is not an issue
> > (just a little bit more work), at IETF meetings we'd meet up, plug
> > laptops with virtual machines into each other and run compliance tests.
> > Sidenote: when IANA codepoints are needed but not yet assigned, don't
> > forget to keep everything on separate beta/feature branches; don't ship
> > software with squatted codepoints :-)
> >
> > The DNSOP working group will have to figure out what constitutes a
> > meaningful implementation in what context. For example, a tcpdump or
> > wireshark decoder would hardly count as an implementation of a proposed
> > DNS feature, but those decoders are _incredibly_ useful to have
> > alongside real implementations, especially during development. DNS
> > people will probably want to see multiple implementation reports in
> > context of packet decoding, stub, resolver, and authoritative.
> >
> > Kind regards,
> >
> > Job
> >
> > ___
> > DNSOP mailing list
> > DNSOP@ietf.org
> > https://www.ietf.org/mailman/listinfo/dnsop
> >
>
> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop
>
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Willem Toorop]

I would love to see a hard requirement for implementations &
implementation reports (like IDR has) in the charter or in the working
group house rules.  Early implementations (perhaps even during the
hackathon) can reveal implications that might have been missed while
designing the draft.  In addition it is a good way to see if everybody
interprets a draft the same way, by interoperability testing.


Op 24-03-18 om 12:07 schreef Job Snijders:
> Dear DNSOP,
> 
> I'd like to share some pointers from the working group that governs the
> BGP protocol, IDR, on requiring implementations before drafts can
> advance towards RFC publication. Raising the bar for publication will
> take weight off the camel's back.
> 
> The IDR policy and rationale is outlined here: 
> https://trac.ietf.org/trac/idr/wiki
> 
> An example of an implementation report is available here:
> https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-community%20implementations
> Every normative (RFC 2119 / RFC 8174) term should expand into a
> compliance test, and it is particularly good to document where
> implementations deviate from what is required or suggested in the
> proposed specification. The work listed on this wikipage was done
> _before_ the draft was submitted to the IESG, and we intentionally
> sought to create more than the minimum amount of implementations
> required.
> 
> In the case of the BGP large communities project the working group was
> particularly lucky because Pier Carlo Chiodi created an open 'regression
> testing'-style environment into which BGP speakers could be plugged in
> to assess their compliance:
> https://github.com/pierky/bgp-large-communities-playground
> https://github.com/pierky/bgp-large-communities-playground/blob/master/tests/README.md
> 
> The DNS world would benefit from such environments and automated
> compliance testing. Manual testing for a specification (that is still
> being worked on) can be tedious, having a 'playground' can pay huge
> dividend. We did catch bugs in implementations thanks to this
> environment, so it not only helped the specification, but increased the
> quality of the participating implementations.
> 
> RFC 7942 suggest that During the development of a draft people are
> encouraged to document what implementations exist, an example is here:
> https://tools.ietf.org/html/draft-ietf-idr-large-community-12#section-7
> 
> Closed source implementations as part of such a list is not an issue
> (just a little bit more work), at IETF meetings we'd meet up, plug
> laptops with virtual machines into each other and run compliance tests.
> Sidenote: when IANA codepoints are needed but not yet assigned, don't
> forget to keep everything on separate beta/feature branches; don't ship
> software with squatted codepoints :-)
> 
> The DNSOP working group will have to figure out what constitutes a
> meaningful implementation in what context. For example, a tcpdump or
> wireshark decoder would hardly count as an implementation of a proposed
> DNS feature, but those decoders are _incredibly_ useful to have
> alongside real implementations, especially during development. DNS
> people will probably want to see multiple implementation reports in
> context of packet decoding, stub, resolver, and authoritative.
> 
> Kind regards,
> 
> Job
> 
> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop
> 

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] raising the bar: requiring implementations

[Petr Špaček]

On 24.3.2018 12:07, Job Snijders wrote:
> Dear DNSOP,
> 
> I'd like to share some pointers from the working group that governs the
> BGP protocol, IDR, on requiring implementations before drafts can
> advance towards RFC publication. Raising the bar for publication will
> take weight off the camel's back.
> 
> The IDR policy and rationale is outlined here: 
> https://trac.ietf.org/trac/idr/wiki
> 
> An example of an implementation report is available here:
> https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-community%20implementations
> Every normative (RFC 2119 / RFC 8174) term should expand into a
> compliance test, and it is particularly good to document where
> implementations deviate from what is required or suggested in the
> proposed specification. The work listed on this wikipage was done
> _before_ the draft was submitted to the IESG, and we intentionally
> sought to create more than the minimum amount of implementations
> required.

I very much support this approach.

BTW we already have test framework which was used to test BIND, Unbound,
Knot Resolver, and PowerDNS recursor! I.e. we have one tool which is
able to test all of these using the same test data, so the remaining
work here is writting tests, not creating test tools from scratch.

Petr Špaček  @  CZ.NIC


> In the case of the BGP large communities project the working group was
> particularly lucky because Pier Carlo Chiodi created an open 'regression
> testing'-style environment into which BGP speakers could be plugged in
> to assess their compliance:
> https://github.com/pierky/bgp-large-communities-playground
> https://github.com/pierky/bgp-large-communities-playground/blob/master/tests/README.md
> 
> The DNS world would benefit from such environments and automated
> compliance testing. Manual testing for a specification (that is still
> being worked on) can be tedious, having a 'playground' can pay huge
> dividend. We did catch bugs in implementations thanks to this
> environment, so it not only helped the specification, but increased the
> quality of the participating implementations.
> 
> RFC 7942 suggest that During the development of a draft people are
> encouraged to document what implementations exist, an example is here:
> https://tools.ietf.org/html/draft-ietf-idr-large-community-12#section-7
> 
> Closed source implementations as part of such a list is not an issue
> (just a little bit more work), at IETF meetings we'd meet up, plug
> laptops with virtual machines into each other and run compliance tests.
> Sidenote: when IANA codepoints are needed but not yet assigned, don't
> forget to keep everything on separate beta/feature branches; don't ship
> software with squatted codepoints :-)
> 
> The DNSOP working group will have to figure out what constitutes a
> meaningful implementation in what context. For example, a tcpdump or
> wireshark decoder would hardly count as an implementation of a proposed
> DNS feature, but those decoders are _incredibly_ useful to have
> alongside real implementations, especially during development. DNS
> people will probably want to see multiple implementation reports in
> context of packet decoding, stub, resolver, and authoritative.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

[DNSOP] raising the bar: requiring implementations

[Job Snijders]

Dear DNSOP,

I'd like to share some pointers from the working group that governs the
BGP protocol, IDR, on requiring implementations before drafts can
advance towards RFC publication. Raising the bar for publication will
take weight off the camel's back.

The IDR policy and rationale is outlined here: 
https://trac.ietf.org/trac/idr/wiki

An example of an implementation report is available here:
https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-community%20implementations
Every normative (RFC 2119 / RFC 8174) term should expand into a
compliance test, and it is particularly good to document where
implementations deviate from what is required or suggested in the
proposed specification. The work listed on this wikipage was done
_before_ the draft was submitted to the IESG, and we intentionally
sought to create more than the minimum amount of implementations
required.

In the case of the BGP large communities project the working group was
particularly lucky because Pier Carlo Chiodi created an open 'regression
testing'-style environment into which BGP speakers could be plugged in
to assess their compliance:
https://github.com/pierky/bgp-large-communities-playground
https://github.com/pierky/bgp-large-communities-playground/blob/master/tests/README.md

The DNS world would benefit from such environments and automated
compliance testing. Manual testing for a specification (that is still
being worked on) can be tedious, having a 'playground' can pay huge
dividend. We did catch bugs in implementations thanks to this
environment, so it not only helped the specification, but increased the
quality of the participating implementations.

RFC 7942 suggest that During the development of a draft people are
encouraged to document what implementations exist, an example is here:
https://tools.ietf.org/html/draft-ietf-idr-large-community-12#section-7

Closed source implementations as part of such a list is not an issue
(just a little bit more work), at IETF meetings we'd meet up, plug
laptops with virtual machines into each other and run compliance tests.
Sidenote: when IANA codepoints are needed but not yet assigned, don't
forget to keep everything on separate beta/feature branches; don't ship
software with squatted codepoints :-)

The DNSOP working group will have to figure out what constitutes a
meaningful implementation in what context. For example, a tcpdump or
wireshark decoder would hardly count as an implementation of a proposed
DNS feature, but those decoders are _incredibly_ useful to have
alongside real implementations, especially during development. DNS
people will probably want to see multiple implementation reports in
context of packet decoding, stub, resolver, and authoritative.

Kind regards,

Job

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop