Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Francis Dupont]

 In your previous mail you wrote:

>  If you want to make the connections full-duplex instead of
>  half-duplex, you need to negotiate connection teardown at the DNS
>  layer.  Otherwise, the TCP connection teardown will result in loss of
>  already-transmitted responses.

=> I don't understand: to close a TCP connection doesn't drop responses.
Perhaps you mean to abort (RST vs FIN) the TCP connection but there is
usually no reason to do this and BTW I am afraid most programmers
don't know how to abort a TCP connection...

Regards

francis.dup...@fdupont.fr

PS: set the SO_LINGER socket option to 0 before close().

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

> On 31 Jan 2015, at 16:56, Florian Weimer  wrote:
> 
> If you want to make the connections full-duplex instead of
> half-duplex, you need to negotiate connection teardown at the DNS
> layer.  Otherwise, the TCP connection teardown will result in loss of
> already-transmitted responses.

This is wrong. You imply that loss of response means complete failure, but that 
isn't true.

Regardless of any DNS-level signalling, there will be situations when you lose 
a connection with queries in progress. Maybe the server crashed. Maybe the 
client is on a train that went into a tunnel.

The client must have retry logic that copes with network lossage, UDP or TCP, 
even if from the server's point of view the response was sent.

I am puzzled by this discussion about extra TCP close signalling. I can't see 
any significant savings that will justify the extra complexity.

The worry seems to be entirely about the race between the client sending a 
query and the server closing. But in this situation it is easy for the client 
to recover fast: it gets an immediate error, and it knows exactly which queries 
to re-send. No timeouts like in UDP.

So why the fuss?

Can someone give me a description, a packet exchange timeline, that shows when 
the proposed extra DNS signalling will do better than a raw TCP close?

Tony.
-- 
f.anthony.n.finchhttp://dotat.at

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Delany]

> Why do you think this?  RFC 103[45] has client initiated shutdown.
> The client sends out x queries withe unique ids.  It waits for
> responses to all of them.  It then closes the connection.  The
> client still has to cope with the connection being closed early.

Indeed. Please let's not go down the SMTP QUIT path (and all the
flammage that produced). TCP FIN communicates intent just fine.

(As an aside, I've always wonder what a client should do if a server
responds 5XX to QUIT? Issue a SUDO QUIT command?).


Mark.


pgpMzjYnNRrXk.pgp
Description: PGP signature
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <878ugidh2g@mid.deneb.enyo.de>, Florian Weimer writes:
> * John Heidemann:
> 
> > DNS over TCP/53 is *already* persistent.  No *protocol* changes are
> > needed.
> 
> If you want to make the connections full-duplex instead of
> half-duplex, you need to negotiate connection teardown at the DNS
> layer.  Otherwise, the TCP connection teardown will result in loss of
> already-transmitted responses.

Why do you think this?  RFC 103[45] has client initiated shutdown.
The client sends out x queries withe unique ids.  It waits for
responses to all of them.  It then closes the connection.  The
client still has to cope with the connection being closed early.

This is the same regardless of whether the queries are sent in
lock step (half duplex) or pipelined (full duplex).

> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Florian Weimer]

* John Heidemann:

> DNS over TCP/53 is *already* persistent.  No *protocol* changes are
> needed.

If you want to make the connections full-duplex instead of
half-duplex, you need to negotiate connection teardown at the DNS
layer.  Otherwise, the TCP connection teardown will result in loss of
already-transmitted responses.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Francis Dupont 
> Wednesday, January 28, 2015 5:19 AM
>  In your previous mail you wrote:
>
>>  Francis, while my own thinking goes further-- an initiator should
>>  not leave a persistent TCP session idle, even for a microsecond,
>>  unless the responder has signaled its approval of such strategy--
>
> => it is what RFC 1035 said so a bit difficult to change in a drastic way.

if we're about to make a change, then we have an opportunity to make a
strong recommendation. the two will only be uptaken together, and at
that, very rarely. but at least we'll have met the burden of "first, do
no harm."
>
>>  I do also agree with your observation above.
>
> => thanks. BTW as pipelining initiators are very rare we have still time
> to fix it.

yes.
>
>>  Have you considered an appropriate signal path for negotiation of
>>  this kind?
>
> => IMHO the tcp-keepalive option is a good start point as it indicates
> the critical value and can be extended to be two ways (i.e., both client
> -> server and server -> client).

thanks. 


-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Francis Dupont]

 In your previous mail you wrote:

>  Francis, while my own thinking goes further-- an initiator should
>  not leave a persistent TCP session idle, even for a microsecond,
>  unless the responder has signaled its approval of such strategy--

=> it is what RFC 1035 said so a bit difficult to change in a drastic way.

>  I do also agree with your observation above.

=> thanks. BTW as pipelining initiators are very rare we have still time
to fix it.

>  Have you considered an appropriate signal path for negotiation of
>  this kind?

=> IMHO the tcp-keepalive option is a good start point as it indicates
the critical value and can be extended to be two ways (i.e., both client
-> server and server -> client).

Thanks

francis.dup...@fdupont.fr

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

John Heidemann  wrote:

> If adns does pipelining over its TCP connection, does it handle
> reordered replies?

Yes, it has done for ever.

> Do you have any comments on the successful use of TCP with adns
> initiators to other responders?  How many responders keep your TCP
> connection on and actually allow multiple queries?

I've only tested it with BIND which works OK (modulo the performance
issues we have already discussed).

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Humber, Thames, Dover, Wight, Portland, Plymouth: Northwest 4 or 5, backing
west or southwest 6 to gale 8 later, perhaps severe gale 9 later except in
Portland and Plymouth. Moderate or rough in Plymouth, otherwise slight or
moderate, becoming rough later. Occasional rain. Good, occasionally poor.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Francis Dupont 
> Tuesday, January 27, 2015 4:06 AM
> ...
>
> Now about the DNS over TCP scaling issue: as I said at the last meeting
> DNS over TCP won't scale until servers will be allowed to use any
> timeout they want, including a zero timeout (which means the connection
> is closed if there is no pending query to read).

francis, while my own thinking goes further-- an initiator should not leave a 
persistent TCP session idle, even for a microsecond, unless the responder has 
signaled its approval of such strategy-- i do also agree with your observation 
above. have you considered an appropriate signal path for negotiation of this 
kind?

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[John Heidemann]

On Mon, 26 Jan 2015 14:58:47 +, Tony Finch wrote: 
>Paul Vixie  wrote:
>>
>> the installed initiator base does not use pipelining, ever.
>
>adns has done since 1999.

Great!  Running code should say a lot.

Two questions for folks with adns experience:

If adns does pipelining over its TCP connection, does it handle
reordered replies?  (As is required by rfc5966, but this library may
pre-date that work.)  In other words, if responders go to out-of-order
TCP replies, will this break adns initiators?

Do you have any comments on the successful use of TCP with adns
initiators to other responders?  How many responders keep your TCP
connection on and actually allow multiple queries?  (There are concerns
about TCP port 53 being firewalled, or responders not handling TCP at
all.)

   -John Heidemann

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Francis Dupont]

 In your previous mail you wrote:

>  If you see a use case for the EDNS tcp-keepalive option as originally 
>  discussed, please say so, on the list, by February 4, 2015.

=> I have one (more later).

>  If you want to pursue the connection-close draft, please say so, on the 
>  list, by February 4, 2015, especially if you're willing to work on it.

=> I can't see what is the benefit of this proposal because:
 - RFC 1035 already said the connection close should be initiated by
  the client
 - almost all current clients send a single query and close the connection
  when they receive the response
 - a server is allowed to close a connection when it wants (more later)

BTW the 2 proposals are not incompatible, the first (tcp-keepalive)
is just more powerful and can provide the same function than the second
(connection-close) with a server sending a zero timeout option
(note the draft says 0 codes infinity, IMHO this must be changed).

(more details)

I don't understand some of the discussion about TCP and  out-of-order
responses: if a client is sophiticated enough to have more than one
query on the fly on a TCP connection I can't believe it doesn't know
to map responses to queries using the transaction ID. Nevertheless
the next bind code will provide an access-list to disable out-of-order
responses (proposed by Mark Andrews during code review) as stupidity
is a good example of something without bounds...

To come back to the tcp-keepalive option, I believe it is a better
alternative for a client than sending junk queries to keep the connection
not idle. And it can be used the other ways, i.e., by signaling a smaller
timeout, and for the from server to client signaling.

Now about the DNS over TCP scaling issue: as I said at the last meeting
DNS over TCP won't scale until servers will be allowed to use any
timeout they want, including a zero timeout (which means the connection
is closed if there is no pending query to read).

Regards

francis.dup...@fdupont.fr (and also fdup...@isc.org)

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Vixie  wrote:
>
> > adns has done since 1999.
>
> in glibc, adns provides the gethostby*() interface and similar,

No, adns has nothing to do with the glibc resolver.

> are you saying that some important apps use the former, or are you
> saying that adns has a socket pool shared amongst multiple app threads?

I don't know much about what apps use adns.

adns uses two sockets, for UDP and TCP, both are kept open persistently
and both handle concurrent queries and out-of-order responses.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Rockall, Malin: West backing southwest 5 to 7. Very rough, occasionally high
at first. Showers, occasional rain later. Good, occasionally moderate.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Tony Finch 
> Monday, January 26, 2015 1:42 AM
>
> ["transport encoding" of "deflate"] doesn't help with HTTP's header
> overhead, nor will it help with
> expansion due to textual instead of binary crypto blobs.

makes sense.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Tony Finch 
> Monday, January 26, 2015 6:58 AM
>
> adns has done since 1999.

in glibc, adns provides the gethostby*() interface and similar, but also
offers its own asynch API. i believe that most apps in the world use the
latter, which is synchronous. are you saying that some important apps
use the former, or are you saying that adns has a socket pool shared
amongst multiple app threads?
 
-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Vixie  wrote:
>
> the installed initiator base does not use pipelining, ever.

adns has done since 1999.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Lundy, Fastnet, Irish Sea: Northwest backing southwest 4 or 5, occasionally 6
at first in Irish Sea. Slight or moderate in Irish Sea, otherwise moderate or
rough. Mainly fair. Good.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Vixie  wrote:
> > Tony Finch 
> > Saturday, January 24, 2015 5:09 PM
> >
> > Sorry, I was being too terse. I meant extra latency due to the time
> > taken to transmit all that redundant data.
>
> isn't that what "transport encoding" of "deflate" is meant for?

That doesn't help with HTTP's header overhead, nor will it help with
expansion due to textual instead of binary crypto blobs.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Forties, Cromarty: West 5 or 6, veering northwest 6 or 7, occasionally gale 8.
Moderate or rough, occasionally very rough in Forties. Showers. Good.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

TL;DR: i'd like to only behave differently if the other side signals its
readiness for it. in a "big TCP" model where thousands or tens of
thousands of sessions remain open while idle (even if only for a few
seconds), we are asking for application, library, kernel, RAM, CPU, and
firewall conditions that are not pervasive in the installed base --
which includes tens of millions of responders who will never be
upgraded, and whose operators are not reading this mailing list, and
will not be reading any new RFCs on the topic.

if we want better TCP/53 behaviour than that required in RFC 1035 4.2.2,
then we should define signalling that requests it, and we should behave
differently if that request is granted.

that's what "first, do no harm" means in an installed base that's
literally the size and shape of The Internet.

longer version:

> John Heidemann 
> Sunday, January 25, 2015 9:10 PM
> ...
>
> We are, I think, in the lucky place of having a new feature (multiple
> DNS queries over TCP with pipelining and reordering) with SOME level of
> responder support and basically zero initiator use.
> Do we really need new signaling?

yes, i think so. you're only talking about old-style initiators here.
there are problems on the responder side that i worry more about,
because of the impact that new-style initiators could indirectly but
pervasively have on old-style initiators, due to the behaviour of
old-style responders.
> ... The other question is harm on the
> responder side.  That's why I was trying to get to the bottom of the
> assertion that DNS-over-TCP is inherently a DoS.

there may not be a bottom. existing responders who follow RFC 1035 4.2.2
are extremely weak, but are in the critical path for existing initiators
responding to TC=1 (or, in other cases where a UDP response is unusable
or untrustworthy, which i'm loathe to describe in public.)

if a new-style initiator prefers TCP and keeps a connection open longer
than the time it takes to send just the queries it has in hand, and if
the responder is old-style, then it causes significant problems for
old-style initiators. denying service to a by-the-book RFC 1035 4.2.2
TCP responder is childs play. we must not do it on purpose.

> I haven't seen
> evidence supporting that claim,

i am out of ideas as to what that might require.

> ... and I think we can all recognize the
> installed base of HTTP to show that at least someone can make TCP work
> at scale on the server side.

i have not, and i don't think anyone else has either, said that TCP
cannot be made to work at scale. however, TCP/53 as described in RFC
1035 4.2.2 is not part of making DNS-over-TCP work at scale; quite the
opposite.


>
>
>>> bind
>>> responders, since 4.8, has accepted pipelining, but with ordered
>>> responses until a currently unreleased patch was put in recently. bind
>>> responders through bind 8 did not read the next (potentially pipelined)
>>> request out of the tcp socket until after it had sent its response to
>>> the previous request, so, there was no parallelism of any resulting
>>> cache miss activities.
>
> Most implementations whose TCP we've examined (bind 9.9 and unbound)
> have performance problems when running over TCP.  But performance
> problems can be fixed incrementally and in place, unlike correctness
> issues where people fail.

the problems we must avoid involve servers whose source code you can't
get access to.
>
> Yes, there are definitely performance problems that will need to be
> fixed.  But performance has very different deployment issues
> than correctness does.

the problems we must avoid involve servers who will never be upgraded.
> ...
>
> I haven't seen anyone assert that TCP should become *manditory* for
> future DNS.  If it's encouraged, or at least not discouraged, then I
> suggest we can abide a multi-year rollout.

the problems we must avoid include those operated by people who do not
read this mailing list, or new RFC's.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Delany]

On 25Jan15, John Heidemann allegedly wrote:

> I think these statements are both overly strong.  They both suggest
> that careful signaling is required before deploying DNS over TCP with 
> pipelining or
> persistence.

If virtually no initiators send multiple requests then your conclusion
seems reasonable.

>   Only if you want 8% of your queries to fail.
>   http://users.isc.org/~marka/ts/alexa.optfail.html
> 
> 
> That's harm on the initiator side.

But the harm should be a lot less than the proportion of servers
exhibiting the problem, yes?

First off there is the matter of proportion of queries that actually
go to failing servers. Even with the top 1,000 domains, most of it is
long tail.

Second the cost should be amortized across all queries, not just the
first few to a given server.

I'm assuming that these yet-to-be-implemented out-of-order initiators
may well have heuristics that determine whether a TCP connection is
worth it or not. I'm also assuming that they'll have to track their
currently active TCP connections. With this sort of machinery in place
it's not a huge additional burden to track failed connections of
high-occurrence servers for a reasonably long time period.

In other words, the cost of detecting a non-compliant server could
reasonably be amortized across many queries. The net harm will never
be zero, but it should be approaching it.

Also, the motivation to improve the situation resides with the person
most capable of making changes - the owner of the domain. That's a
nice alignment of interests.


Mark.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[John Heidemann]

On Sun, 25 Jan 2015 09:44:24 +1100, Mark Andrews wrote: 
>
>In message <54c40d28.7050...@redbarn.org>, Paul Vixie writes:
>> > Mark Andrews 
>> > Thursday, January 22, 2015 6:29 PM
>> > In message <32707.1421975...@dash.isi.edu>, John Heidemann writes:
>> >> ...
>> >> I'm confused.  I thought we agreed the installed base doesn't do TCP
>> >> pipelining basically ever.
>> >
>> > The installed base has supported pipelining forever.  BIND 4.8
>> > supported it.
>> 
>> both statements above are partly correct.

My statement above ("the installed base doesn't do TCP
pipelining basically ever") was unclear.  I meant (in the context of
discussion that's now been cut), the installed based on INITIATORS
doesn't do pipelining.

Mark is correct that some of the installed base on responders DOES
generally handle pipelining incoming queries.  (I have not done a
careful survey and so cannot make a stronger statement.

Specifically, for all clients of which I'm aware (both
recursives and stubs), when they make DNS queries over TCP,
they make one query and close the connection.
Again, anecdotes should be taken with care, but I think published work
analyzing traces will bear this out.

>> the installed initiator base does not use pipelining, ever.

YES that was the point I was trying to make.

If we agree on that, then we need to review at Paul's concerns about old
clients causing failures.

>From other parts of the thread:

On 24Jan15, Paul Vixie allegedly wrote:

could violate older implementations' reasonable-at-the-time assumptions,
against the burden of choosing a non-interfering signal pattern, like a
new port number, or a new protocol verb.

And I think Mark Delay replied:

Does it have to be that drastic? Wouldn't an EDNS option "I understand
out-of-order" be enough? Once seen in a TCP session it would hold true
until closed. The non-presence of such an option could then entrench
the in-order assumptions that may exist in the installed base.

I think these statements are both overly strong.  They both suggest
that careful signaling is required before deploying DNS over TCP with 
pipelining or
persistence.

BUT, I think there is rough consensus that there are near-zero INTIATORS
that actually request this feature.  (I think both Mark Andrews and Paul
Vixie have said this more or less at in the thread; please correct me if
I'm wrong.)

If we can successfully demonstrate that the there are no (or virtually
no) DNS-TCP-pipelined or persistent initiators, then signaled seems
like overkill.

I asked up-thread:

It seems like you're arguing that all future responders must be
conservative to protect against old clients that are smart enough to do
pipelined TCP but not smart enough to track TXID.  Is there really a
non-negligible installed base base of such initiators?
(Are there any specific implementations that are known to behave this 
way?)

We are, I think, in the lucky place of having a new feature (multiple
DNS queries over TCP with pipelining and reordering) with SOME level of
responder support and basically zero initiator use.
Do we really need new signaling?

Put differently:  If we all accept Paul's  "first, do no harm"
principle, if there are no initiators that don't follow the spec, as we
think, then there is no harm to deploy.

And there IS harm in mandating new signaling, because options get
filtered, as Mark Andrews said:

[Wouldn't an EDNS option be enough?]

Only if you want 8% of your queries to fail.
http://users.isc.org/~marka/ts/alexa.optfail.html


That's harm on the initiator side.  The other question is harm on the
responder side.  That's why I was trying to get to the bottom of the
assertion that DNS-over-TCP is inherently a DoS.  I haven't seen
evidence supporting that claim, and I think we can all recognize the
installed base of HTTP to show that at least someone can make TCP work
at scale on the server side.


>>bind
>> responders, since 4.8, has accepted pipelining, but with ordered
>> responses until a currently unreleased patch was put in recently. bind
>> responders through bind 8 did not read the next (potentially pipelined)
>> request out of the tcp socket until after it had sent its response to
>> the previous request, so, there was no parallelism of any resulting
>> cache miss activities.

Most implementations whose TCP we've examined (bind 9.9 and unbound)
have performance problems when running over TCP.  But performance
problems can be fixed incrementally and in place, unlike correctness
issues where people fail.

Yes, there are definitely performance problems that will need to be
fixed.  But performance has very different deployment issues
than correctness does.


(on to installed base)

Mark Andrews wrote: 
We should also stop thinking of the installed base as something
that cannot be changed.  This is partic

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Tony Finch 
> Saturday, January 24, 2015 5:09 PM
>
> Sorry, I was being too terse. I meant extra latency due to the time taken
> to transmit all that redundant data.

isn't that what "transport encoding" of "deflate" is meant for?

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <54c424f2.4020...@redbarn.org>, Paul Vixie writes:
> 
> > Mark Delany 
> > Saturday, January 24, 2015 2:09 PM
> > On 24Jan15, Paul Vixie allegedly wrote:
> >
> >> could violate older implementations' reasonable-at-the-time assumptions,
> >> against the burden of choosing a non-interfering signal pattern, like a
> >> new port number, or a new protocol verb.
> >
> > Does it have to be that drastic? Wouldn't an EDNS option "I understand
> > out-of-order" be enough? Once seen in a TCP session it would hold true
> > until closed. The non-presence of such an option could then entrench
> > the in-order assumptions that may exist in the installed base.
> 
> i'd be fine with that. note that the "ask" and "answer" codes must
> differ, to avoid parrots.

To tell the server "you can send out of order responses" it does
not matter if the server is broken in that way or not.

EDNS cookies also doesn't care if the server incorrectly echos back
the option.  You still get all the signaling the client needs to
detect spoofed responses in the echos.

That said the behaviour should be stomped on with extreme prejudice.

> but i know that any new signal is seen as a bad bargain for those who
> wish to aggressively embrace TCP, simply because they won't be able to
> start pipelining until after they've heard their first answer (and then,
> only if that first answer contains an "i agree to persistency"
> indication.) perhaps since this information can be cached once
> discovered, i'm wrong to expect objections on this topic.

Of course they can start pipe lining without waiting.  The existing
servers don't break when we do it today without signaling.  The
real problem is that servers misbehave when presented with a unknown
EDNS option.

> this is a balancing act. but because RFC 1035 4.2.2 is so weak, and
> because TCP is in the current installed base a fallback not a primary or
> preferred transport, it's necessary to limit a client's occupancy time
> of a server's TCP slot to an extremely small time window unless you have
> current or recently-cached knowledge that the server does "big TCP". (by
> which i mean, the server has the new logic, and the server has
> discovered that its kernel can tolerate vast numbers of open sessions,
> and the server has been configured to operate this way by some operator
> who knows he won't be killing off his stateful firewall again.)
> 
> it's a "first, do no harm" situation. any damage incurred must be upon
> the newest revision of the protocol.

There is a difference between "reusing a existing socket because
it is already open rather than opening a new socket" (BIND 9.11
will do this for query processing) or "we will use tcp for this set
of queries because we expect we will need to use tcp anyway" (update
forwarding in BIND uses TCP and reuses the socket if it is open,
the later is new in BIND 9.11) and "holding a socket open longer
because perhaps we may re-use it."

The first two use TCP for the minimal amount of time required and
out of order processsing reduces that amount of time the socket is
held open.

netstat held the socket open and used tcp because it was asking lots
of queries.  Even then it wasn't pipelining queries.

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Vixie  wrote:
>
> i don't think there's any real time (measurable) difference in
> serialization (and deserialization) latency.

Sorry, I was being too terse. I meant extra latency due to the time taken
to transmit all that redundant data.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Biscay: North 4 or 5, becoming variable 3 or 4. Moderate or rough. Mainly
fair. Good.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <54c4267a.9030...@redbarn.org>, Paul Vixie writes:
> > Mark Andrews 
> > Saturday, January 24, 2015 2:44 PM
> > In message <54c40d28.7050...@redbarn.org>, Paul Vixie writes:
> > Pipeling over UDP has been standard practice between nameservers for
> > 25 years. Why are we even worrying about whether it should be
> > permitted over TCP?
> 
> because tcp has been a fallback-only for all these years, and someone
> who wasn't pipelining or wasn't checking txid would never have been
> penalized for it.
> 
> > ...
> >
> > We should also stop thinking of the installed base as something
> > that cannot be changed.  This is particularly true of authoritative
> > servers.  We can identify broken servers.  We can inform their
> > operators that they are broken.
> 
> mark, you and i know better than anybody that this approach doesn't
> work. it didn't work for lame delegation checking, it hasn't worked for
> EDNS, and it's so much of a risk in DNSSEC that we're now discussing
> ways that an RDNS operator can turn off validation for signed zones
> rather than signal failures on failed lookups.

Actually we don't know that it won't work.  It has not been tried
at scale (and it works at the tld level) and delegations require
the operator to know what they are doing.

Installing a fixed server provided by your DNS vendor is a different
type of request.  Package managers have made upgrading nameserver
versions a pretty trivial operation these days regardless of which
vendor you use.  This is more about telling the operator that they
need to run the command to do the upgrade.

> >   RFC 1033 even detailed how to do
> > this.
> 
> RFC 1033's complaint process contemplated a network of about the size of
> the pre-NSFnet "ARPAnet", and could have scaled anyway as far as the
> size of the pre-commercial "NSFnet". it can work for Internet2. but it's
> not going to work on the big-eye Internet as we know it today.
> 
> paul
> 
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <20150124220926.82207.qm...@f5-external.bushwire.net>, "Mark Delany" 
writes:
> On 24Jan15, Paul Vixie allegedly wrote:
> 
> > could violate older implementations' reasonable-at-the-time assumptions,
> > against the burden of choosing a non-interfering signal pattern, like a
> > new port number, or a new protocol verb.
> 
> Does it have to be that drastic? Wouldn't an EDNS option "I understand
> out-of-order" be enough? Once seen in a TCP session it would hold true
> until closed. The non-presence of such an option could then entrench
> the in-order assumptions that may exist in the installed base.

Only if you want 8% of your queries to fail.
http://users.isc.org/~marka/ts/alexa.optfail.html

Now it can be much better.  If .ZA and .NP fixed the one failing
server they each have we would be down to just packet loss at the
TLD level.
http://users.isc.org/~marka/ts/tld.optfail.html
http://users.isc.org/~marka/tld-report.html

> > i expected that DNS-over-HTTP would work the same as WWW-over-HTTP,
> > which is to open multiple parallel TCP/80 (or TCP/443) sessions if
> > parallelism is required.
> 
> If DNS over HTTP is really being considered, would it be better to
> start with HTTP/2.0 as the base protocol rather than 1.* then you get
> parallel for "free".
> 
> 
> Mark.
> 
> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Mark Andrews 
> Saturday, January 24, 2015 2:44 PM
> In message <54c40d28.7050...@redbarn.org>, Paul Vixie writes:
> Pipeling over UDP has been standard practice between nameservers for
> 25 years. Why are we even worrying about whether it should be
> permitted over TCP?

because tcp has been a fallback-only for all these years, and someone
who wasn't pipelining or wasn't checking txid would never have been
penalized for it.

> ...
>
> We should also stop thinking of the installed base as something
> that cannot be changed.  This is particularly true of authoritative
> servers.  We can identify broken servers.  We can inform their
> operators that they are broken.

mark, you and i know better than anybody that this approach doesn't
work. it didn't work for lame delegation checking, it hasn't worked for
EDNS, and it's so much of a risk in DNSSEC that we're now discussing
ways that an RDNS operator can turn off validation for signed zones
rather than signal failures on failed lookups.

>   RFC 1033 even detailed how to do
> this.

RFC 1033's complaint process contemplated a network of about the size of
the pre-NSFnet "ARPAnet", and could have scaled anyway as far as the
size of the pre-commercial "NSFnet". it can work for Internet2. but it's
not going to work on the big-eye Internet as we know it today.

paul
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Mark Delany 
> Saturday, January 24, 2015 2:09 PM
> On 24Jan15, Paul Vixie allegedly wrote:
>
>> could violate older implementations' reasonable-at-the-time assumptions,
>> against the burden of choosing a non-interfering signal pattern, like a
>> new port number, or a new protocol verb.
>
> Does it have to be that drastic? Wouldn't an EDNS option "I understand
> out-of-order" be enough? Once seen in a TCP session it would hold true
> until closed. The non-presence of such an option could then entrench
> the in-order assumptions that may exist in the installed base.

i'd be fine with that. note that the "ask" and "answer" codes must
differ, to avoid parrots.

but i know that any new signal is seen as a bad bargain for those who
wish to aggressively embrace TCP, simply because they won't be able to
start pipelining until after they've heard their first answer (and then,
only if that first answer contains an "i agree to persistency"
indication.) perhaps since this information can be cached once
discovered, i'm wrong to expect objections on this topic.

this is a balancing act. but because RFC 1035 4.2.2 is so weak, and
because TCP is in the current installed base a fallback not a primary or
preferred transport, it's necessary to limit a client's occupancy time
of a server's TCP slot to an extremely small time window unless you have
current or recently-cached knowledge that the server does "big TCP". (by
which i mean, the server has the new logic, and the server has
discovered that its kernel can tolerate vast numbers of open sessions,
and the server has been configured to operate this way by some operator
who knows he won't be killing off his stateful firewall again.)

it's a "first, do no harm" situation. any damage incurred must be upon
the newest revision of the protocol.
>
>
>> i expected that DNS-over-HTTP would work the same as WWW-over-HTTP,
>> which is to open multiple parallel TCP/80 (or TCP/443) sessions if
>> parallelism is required.
>
> If DNS over HTTP is really being considered, would it be better to
> start with HTTP/2.0 as the base protocol rather than 1.* then you get
> parallel for "free".

i don't know. it sounds great, but i'd want to know that the design team
on HTTP/2.0 had very carefully seen to backward compatibility with the
billion-or-so transparent middleboxes that think they know every element
TCP/80 can contain and the meaning of all possible permutations thereof.
in other words i'd like DNS not to be the test case for it. but if it
has parallelism and it's well designed and it's coming anyway, then,
sure, why not?

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <54c40d28.7050...@redbarn.org>, Paul Vixie writes:
> > Mark Andrews 
> > Thursday, January 22, 2015 6:29 PM
> > In message <32707.1421975...@dash.isi.edu>, John Heidemann writes:
> >> ...
> >> I'm confused.  I thought we agreed the installed base doesn't do TCP
> >> pipelining basically ever.
> >
> > The installed base has supported pipelining forever.  BIND 4.8
> > supported it.
> 
> both statements above are partly correct.
> 
> the installed initiator base does not use pipelining, ever. bind
> responders, since 4.8, has accepted pipelining, but with ordered
> responses until a currently unreleased patch was put in recently. bind
> responders through bind 8 did not read the next (potentially pipelined)
> request out of the tcp socket until after it had sent its response to
> the previous request, so, there was no parallelism of any resulting
> cache miss activities.
> >
> > What the installed base didn't do was out of order reponses /
> > parallel processing.  That said doing this is permitted by RFC
> > 103[45].  A client that pipelined queries needed to check the txid
> > of responses.
> 
> while that's common sense (if you have more than one request outstanding
> on a tcp session, you should check txid on each response), the fact that
> all responders in the installed base up to this present day answered in
> order meant that a pipelining initiator could lack this txid check and
> not have had any problems -- for 25 years or so. that's not a concern to
> be dismissed as "not our fault, let's move on."

It is clearly the clients fault.  Add to that there are extremely
few pipelining clients that I'm willing to bet that despite adding
a acl to force in order reponses for broken clients it will never
need to be set other than for testing.

This is a bit like improving the case preservation in responses by
using case sensitive compression when constructing responses.  There
is one set top manufacture that does case sensitive matching of the
answer section against the original query and a broken nagios test.
There is a acl to disable case sensitive compression but it is
enabled by default and almost all clients did not care.

> > [Pipelining] itself is supported by the protcol whereas SMTP didn't
> > support pipelining as it is a lock step protocol.
> 
> pipelining wasn't specified, but it did usually work, which is why
> spammers abused it, which is why a common check for "am i talking to a
> spammer" today is to deny pipelining and then if the initiator pipelines
> anyway, reset the connection. so, "supported by the protocol" isn't a
> perfectly clear standard guiding discussion.

Pipeling over UDP has been standard practice between nameservers
for 25 years.  Why are we even worrying about whether it should be
permitted over TCP?  If the socket is open when you need to send
another query just use it.

> ===
> 
> we've been "rebuilding the airplane in-flight", as suz put it, not just
> with dnssec, but with dns itself. i wrote about this ("DNS Complexity")
> and included this text:
> 
> > From this overview, it is possible to conclude that DNS is a poorly
> > specified protocol, but that would be unfair and untrue. DNS was
> > specified loosely, on purpose. This protocol design is a fine example
> > of what M.A. Padlipsky meant by "descriptive rather than prescriptive"
> > in his 1984 thriller, The Elements of Networking Style (Prentice
> > Hall). Functional interoperability and ease of implementation were the
> > goals of the DNS protocol specification, and from the relative ease
> > with which DNS has grown from its petri dish into a world-devouring
> > monster, it's clear to me that those goals were met. A stronger
> > document set would have eliminated some of the "gotchas" that DNS
> > implementers face, but the essential and intentional looseness of the
> > specification has to be seen as a strength rather than a weakness.
> >
> > That having been said, a stronger document set written today would not
> > be able to put all of the DNS genies back into their bottles. Too many
> > implementations have guessed differently when presented with a loose
> > specification, and interoperability today is a moving, organic target.
> > When I periodically itch to rewrite the specification from scratch, I
> > know there are too many things that must be said that also cannot be
> > said. It's as though, in a discussion of the meaning of some bit
> > pattern, a modern description of the protocol---written with full
> > perspective on all that has been done in the DNS field---would have to
> > say, "It could mean x but some implementations will think it means y
> > so you must be cautious."
> >
> > If the objective meaning of a set of potential states and conditions
> > and patterns has a complexity index that is a function, somehow, of
> > the combinations and permutations of those states, conditions, and
> > patterns, as well as the multiple interpretations and deliberate
> > unce

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Delany]

On 24Jan15, Paul Vixie allegedly wrote:

> could violate older implementations' reasonable-at-the-time assumptions,
> against the burden of choosing a non-interfering signal pattern, like a
> new port number, or a new protocol verb.

Does it have to be that drastic? Wouldn't an EDNS option "I understand
out-of-order" be enough? Once seen in a TCP session it would hold true
until closed. The non-presence of such an option could then entrench
the in-order assumptions that may exist in the installed base.


> i expected that DNS-over-HTTP would work the same as WWW-over-HTTP,
> which is to open multiple parallel TCP/80 (or TCP/443) sessions if
> parallelism is required.

If DNS over HTTP is really being considered, would it be better to
start with HTTP/2.0 as the base protocol rather than 1.* then you get
parallel for "free".


Mark.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Tony Finch 
> Friday, January 23, 2015 10:35 AM
>
> Paul Vixie  wrote:
>
> > why aren't we preferring a TCP/80 (and perhaps TCP/443) solution
>
> Inefficient encoding
> -> wastes battery
> -> greater serialization latency

i see your point about battery, and i hadn't considered that the common
configuration today and for the foreseeable future is mobile devices. i
don't know if that's a compelling argument, but it is an argument.

i don't think there's any real time (measurable) difference in
serialization (and deserialization) latency. it's been a long time since
processors were faster at accessing random memory locations using
offsets (in the sense of C structure variables) than they were at
accessing memory in order using auto-incrementing pointers. that's why
BIND8's "dns_parsemsg()" steps through the message in order, just as a
JSON parser would have to do.
>
> In-order responses
> -> head-of-line blocking
i expected that DNS-over-HTTP would work the same as WWW-over-HTTP,
which is to open multiple parallel TCP/80 (or TCP/443) sessions if
parallelism is required. either TCP sessions are cheap enough that this
doesn't matter, or, TCP sessions are expensive enough that we can't
leave them open for long (many seconds) periods of time.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Mark Andrews 
> Thursday, January 22, 2015 6:29 PM
> In message <32707.1421975...@dash.isi.edu>, John Heidemann writes:
>> ...
>> I'm confused.  I thought we agreed the installed base doesn't do TCP
>> pipelining basically ever.
>
> The installed base has supported pipelining forever.  BIND 4.8
> supported it.

both statements above are partly correct.

the installed initiator base does not use pipelining, ever. bind
responders, since 4.8, has accepted pipelining, but with ordered
responses until a currently unreleased patch was put in recently. bind
responders through bind 8 did not read the next (potentially pipelined)
request out of the tcp socket until after it had sent its response to
the previous request, so, there was no parallelism of any resulting
cache miss activities.
>
> What the installed base didn't do was out of order reponses /
> parallel processing.  That said doing this is permitted by RFC
> 103[45].  A client that pipelined queries needed to check the txid
> of responses.

while that's common sense (if you have more than one request outstanding
on a tcp session, you should check txid on each response), the fact that
all responders in the installed base up to this present day answered in
order meant that a pipelining initiator could lack this txid check and
not have had any problems -- for 25 years or so. that's not a concern to
be dismissed as "not our fault, let's move on."
>
> [Pipelining] itself is supported by the protcol whereas SMTP didn't
> support pipelining as it is a lock step protocol.

pipelining wasn't specified, but it did usually work, which is why
spammers abused it, which is why a common check for "am i talking to a
spammer" today is to deny pipelining and then if the initiator pipelines
anyway, reset the connection. so, "supported by the protocol" isn't a
perfectly clear standard guiding discussion.

===

we've been "rebuilding the airplane in-flight", as suz put it, not just
with dnssec, but with dns itself. i wrote about this ("DNS Complexity")
and included this text:

> From this overview, it is possible to conclude that DNS is a poorly
> specified protocol, but that would be unfair and untrue. DNS was
> specified loosely, on purpose. This protocol design is a fine example
> of what M.A. Padlipsky meant by "descriptive rather than prescriptive"
> in his 1984 thriller, The Elements of Networking Style (Prentice
> Hall). Functional interoperability and ease of implementation were the
> goals of the DNS protocol specification, and from the relative ease
> with which DNS has grown from its petri dish into a world-devouring
> monster, it's clear to me that those goals were met. A stronger
> document set would have eliminated some of the "gotchas" that DNS
> implementers face, but the essential and intentional looseness of the
> specification has to be seen as a strength rather than a weakness.
>
> That having been said, a stronger document set written today would not
> be able to put all of the DNS genies back into their bottles. Too many
> implementations have guessed differently when presented with a loose
> specification, and interoperability today is a moving, organic target.
> When I periodically itch to rewrite the specification from scratch, I
> know there are too many things that must be said that also cannot be
> said. It's as though, in a discussion of the meaning of some bit
> pattern, a modern description of the protocol---written with full
> perspective on all that has been done in the DNS field---would have to
> say, "It could mean x but some implementations will think it means y
> so you must be cautious."
>
> If the objective meaning of a set of potential states and conditions
> and patterns has a complexity index that is a function, somehow, of
> the combinations and permutations of those states, conditions, and
> patterns, as well as the multiple interpretations and deliberate
> uncertainties, then DNS is a very complex system, even though its
> rules are simple and few, and even though a new DNS protocol agent can
> be constructed using only a few thousand lines of software code.
>
> (at )

so:

sometimes the installed based can be simply and completely damned, as i
did with BIND when microsoft noticed that we only accepted AXFR with
ANCOUNT=1, which was drastically wasteful of network capacity, as well
as damned inconvenient for other implementors. in that case we fixed all
responders instantly, and gave initiators the option to use the old
(one-answer) or "new" (many-answer) format, first defaulting to the old
way, and then defaulting to the new way. stuff broke, and got fixed, and
we moved on. notably, that was in 1995 or so, and BIND was the only
widely installed DNS server, and there were only 30,000 or so
installations world wide. now that we have dozens of DNS implementations
many of which are inside security appliances and many of which are not
open-source, and there are 30,00

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Vixie  wrote:
>
> why aren't we preferring a TCP/80 (and perhaps TCP/443) solution

Inefficient encoding
 -> wastes battery
 -> greater serialization latency

In-order responses
 -> head-of-line blocking

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Wight, Portland: Southwest veering northwest 5 to 7, decreasing 4 later.
Moderate or rough, occasionally slight later. Occasional rain. Moderate or
good.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Mark Andrews]

In message <32707.1421975...@dash.isi.edu>, John Heidemann writes:
> 
> [Warning to rubberneckers: seems like at least two topics here: about
> the spec and about TCP as a DoS.]
> 
> On Wed, 21 Jan 2015 14:29:22 -0800, Paul Vixie wrote: 
> ># 
> >John Heidemann
> >Wednesday, January 21, 2015 1:53 PM
> >On Wed, 21 Jan 2015 09:30:44 +, Ray Bellis wrote:
> >   
> >I want to restate this, because people often confuse current practice
> >with current specifications:
> >   
> >DNS over TCP/53 is *already* persistent. No *protocol* changes are 
> > needed.
> >
> >i disagree with your use of terms here. see below.
> >
> >*Implementation* changes, however, are needed:
> >   
> >- clients need to not blindly close the connection after one request
> >   
> >- clients and servers need to use well known implementation techniques
> >(from HTTP) to get good performance---pipelining, processing requests
> >in parallel, sending replies out-of-order (rfc5966), handling TCP
> >fastopen (newly minited rfc7413).
> >
> >at the moment, a tcp/53 initiator that assumes in-order response and does 
> >not check the
> >TXID works. the protocol neither permits nor forbids this. the burden is 
> >either on the
> >responder to be conservative in what it generates in the presence of the 
> >ambiguous
> >specification, or on end-users to be able to upgrade their tcp/53 initiation 
> >software to be
> >more cautious in how it treats the answers.
> 
> I'm confused.  I thought we agreed the installed base doesn't do TCP
> pipelining basically ever.

The installed base has supported pipelining forever.  BIND 4.8
supported it.

What the installed base didn't do was out of order reponses /
parallel processing.  That said doing this is permitted by RFC
103[45].  A client that pipelined queries needed to check the txid
of responses.

> Presumably NEW initiators that choose to do pipelining to improve
> performance will do so aware of reordering, as is required by the 5-year
> old RFC5966.  Not to speak for Mr. Bellis (on the cc), but RFC5966 says:
> 
>[section 6, response reordering for DNS-over-TCP:]
> 
>Client resolvers MUST be able to process responses that arrive in a
>different order to that in which the requests were sent, regardless
>of the transport protocol in use.
> 
> You're correct that implementations that ignore this part of the
> specification may misbehave, but I stand by the protocol being clear
> about what is expected.
> 
> It seems like you're arguing that all future responders must be
> conservative to protect against old clients that are smart enough to do
> pipelined TCP but not smart enough to track TXID.  Is there really a
> non-negligible installed base base of such initiators?
> (Are there any specific implementations that are known to behave this way?)

Not that I am aware of.  That said we will have a acl that will force in-order
processing if matched (default no match).
 
> >similarly, a responder who aborts the tcp/53 session when pipelining is seen 
> >(which is a
> >common technique for spam defense on tcp/25, so, unimaginable here) would 
> >have neither
> >caused nor experienced any operational problems related to that 
> >implementation choice from
> >1985 to the present day, even though the specification was silent on the 
> >matter of
> >pipelining, neither forbidding it nor requiring that it be supported.
> 
> Let's break this out:
> 
> a- some responders may reply to the first query in the pipeline and
> abort on the second
> 
> b- others may abort before replying to either
> 
> Isn't case (a) a performance issue, not a correctness issue?
> 
> Case (b) is, I would say, an non-compliant and overly strict responder.
> But you frame it as applying a common spam defense technique from tcp/25
> to dns.  What is the installed base of such responders?
> (Are there any specific implementations that are known to behave this
> way?)

DNS doesn't need this defence.  You are sending repsonses back to
the initiator.  If there are too many new upstream queries this is
no different to lots of UDP queries.  The only difference is you
return SERVFAIL/REFUSED rather than dropping the query.  Alternatively
you can drop and close the connection.

Pipeling itself is supported by the protcol whereas SMTP didn't
support pipelining as it is a lock step protocol.

> If there are, couldn't an initiator work around them in the same way
> they deal with other non-compliant responders, by falling back to UDP?
> (I believe only new initiators will send pipelined, so we can MUST them
> into falling back, I think.)
> 
> >historically we say we don't want to break working configurations even if 
> >their
> >interpretation of the ambiguous specification is not to our liking. and we 
> >treat the
> >tightening up of the specification as a protocol change even though by some 
> >reading the new
> >behaviour was also acceptable under the older more ambiguous specification

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[John Heidemann]

[Warning to rubberneckers: seems like at least two topics here: about
the spec and about TCP as a DoS.]

On Wed, 21 Jan 2015 14:29:22 -0800, Paul Vixie wrote: 
># 
>John Heidemann
>Wednesday, January 21, 2015 1:53 PM
>On Wed, 21 Jan 2015 09:30:44 +, Ray Bellis wrote:
>   
>I want to restate this, because people often confuse current practice
>with current specifications:
>   
>DNS over TCP/53 is *already* persistent. No *protocol* changes are needed.
>
>i disagree with your use of terms here. see below.
>
>*Implementation* changes, however, are needed:
>   
>- clients need to not blindly close the connection after one request
>   
>- clients and servers need to use well known implementation techniques
>(from HTTP) to get good performance---pipelining, processing requests
>in parallel, sending replies out-of-order (rfc5966), handling TCP
>fastopen (newly minited rfc7413).
>
>at the moment, a tcp/53 initiator that assumes in-order response and does not 
>check the
>TXID works. the protocol neither permits nor forbids this. the burden is 
>either on the
>responder to be conservative in what it generates in the presence of the 
>ambiguous
>specification, or on end-users to be able to upgrade their tcp/53 initiation 
>software to be
>more cautious in how it treats the answers.

I'm confused.  I thought we agreed the installed base doesn't do TCP
pipelining basically ever.

Presumably NEW initiators that choose to do pipelining to improve
performance will do so aware of reordering, as is required by the 5-year
old RFC5966.  Not to speak for Mr. Bellis (on the cc), but RFC5966 says:

   [section 6, response reordering for DNS-over-TCP:]

   Client resolvers MUST be able to process responses that arrive in a
   different order to that in which the requests were sent, regardless
   of the transport protocol in use.

You're correct that implementations that ignore this part of the
specification may misbehave, but I stand by the protocol being clear
about what is expected.

It seems like you're arguing that all future responders must be
conservative to protect against old clients that are smart enough to do
pipelined TCP but not smart enough to track TXID.  Is there really a
non-negligible installed base base of such initiators?
(Are there any specific implementations that are known to behave this way?)

>similarly, a responder who aborts the tcp/53 session when pipelining is seen 
>(which is a
>common technique for spam defense on tcp/25, so, unimaginable here) would have 
>neither
>caused nor experienced any operational problems related to that implementation 
>choice from
>1985 to the present day, even though the specification was silent on the 
>matter of
>pipelining, neither forbidding it nor requiring that it be supported.

Let's break this out:

a- some responders may reply to the first query in the pipeline and
abort on the second

b- others may abort before replying to either

Isn't case (a) a performance issue, not a correctness issue?

Case (b) is, I would say, an non-compliant and overly strict responder.
But you frame it as applying a common spam defense technique from tcp/25
to dns.  What is the installed base of such responders?
(Are there any specific implementations that are known to behave this
way?)

If there are, couldn't an initiator work around them in the same way
they deal with other non-compliant responders, by falling back to UDP?
(I believe only new initiators will send pipelined, so we can MUST them
into falling back, I think.)

>historically we say we don't want to break working configurations even if their
>interpretation of the ambiguous specification is not to our liking. and we 
>treat the
>tightening up of the specification as a protocol change even though by some 
>reading the new
>behaviour was also acceptable under the older more ambiguous specification.

I agree that we must consider reality and installed base, even when it
is non-compliant.  And I'm sure you've seen and tried to work around
far, far more horrible clients and servers than I do.

But before we throw in the towel to HTTP, I'd sure feel better if we
could point to specific broken implementations and have a sense of the
scope of the problem.

Maybe another crazy experiment by Warren or two?

Or maybe I don't want to be just another web programmer. :-)

>
>you can see this principle in use here 
>:
>
>5.2. It is strongly urged that the DNS specification be amended to
>require that the question section from the request MUST be copied,
>exactly, bit for bit, into the question section of the response.  The
>DNS specification is silent on the matter of altering 0x20 bits in the
>question name when copying it from the request to the response, so, this
>change is "within the spirit."
>   
>A change to the specification is necessary because while such bit for
>bit copying "happen

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[John Heidemann]

On Wed, 21 Jan 2015 16:58:32 -0500, Christopher Morrow wrote: 
>On Wed, Jan 21, 2015 at 4:53 PM, John Heidemann  wrote:
>> I don't see how DoS is an argument against TCP for DNS.  (Unless one
>> assumes hardware and software at the servers is fixed to something like
>> 2004 standards.)  What am I missing?
>
>What's the average client load expected (number of unique clients in
>the timeout of the tcp connection expected) for an authoritative
>server today? (say an enterprise hosted server, and then someone that
>is a large domain aggregator)
>
>What is the same curve for a recursive server? (again, a small
>isp/enterprise vs a large provider)
>
>What impact will changing to longer lived persistent tcp connections
>have on hardware and network capacity planning?

Those are good questions, and take some time to answer.  We try to speak
to them in a tech report
at 

It doesn't seem useful copy and past long quotes from that here, but the 
pointers are:

>What's the average client load expected (number of unique clients in
>the timeout of the tcp connection expected) for an authoritative
>server today? (say an enterprise hosted server, and then someone that
>is a large domain aggregator)
>
>What is the same curve for a recursive server? (again, a small
>isp/enterprise vs a large provider)

[Zhu14b], figure 3a and 3b, with discussion in section 5.3.

>What impact will changing to longer lived persistent tcp connections
>have on hardware and network capacity planning?

See section 5.2 about memory usage, and appendix H for a long discussion
about deployment issues.

   -John Heidemann

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> John Heidemann 
> Wednesday, January 21, 2015 1:53 PM
> On Wed, 21 Jan 2015 09:30:44 +, Ray Bellis wrote:
>
> I want to restate this, because people often confuse current practice
> with current specifications:
>
> DNS over TCP/53 is *already* persistent. No *protocol* changes are needed.

i disagree with your use of terms here. see below.
>
> *Implementation* changes, however, are needed:
>
> - clients need to not blindly close the connection after one request
>
> - clients and servers need to use well known implementation techniques
> (from HTTP) to get good performance---pipelining, processing requests
> in parallel, sending replies out-of-order (rfc5966), handling TCP
> fastopen (newly minited rfc7413).

at the moment, a tcp/53 initiator that assumes in-order response and
does not check the TXID works. the protocol neither permits nor forbids
this. the burden is either on the responder to be conservative in what
it generates in the presence of the ambiguous specification, or on
end-users to be able to upgrade their tcp/53 initiation software to be
more cautious in how it treats the answers.

similarly, a responder who aborts the tcp/53 session when pipelining is
seen (which is a common technique for spam defense on tcp/25, so,
unimaginable here) would have neither caused nor experienced any
operational problems related to that implementation choice from 1985 to
the present day, even though the specification was silent on the matter
of pipelining, neither forbidding it nor requiring that it be supported.

historically we say we don't want to break working configurations even
if their interpretation of the ambiguous specification is not to our
liking. and we treat the tightening up of the specification as a
protocol change even though by some reading the new behaviour was also
acceptable under the older more ambiguous specification.

you can see this principle in use here
:

> 5.2. It is strongly urged that the DNS specification be amended to
> require that the question section from the request MUST be copied,
> exactly, bit for bit, into the question section of the response.  The
> DNS specification is silent on the matter of altering 0x20 bits in the
> question name when copying it from the request to the response, so, this
> change is "within the spirit."
>
> A change to the specification is necessary because while such bit for
> bit copying "happens to be" nearly universal practice today, we must
> warn all future responder implementors that the 0x20 bits, while not
> significant for name matching, are now in use as a "covert channel" by
> the requestor, to itself. 

since this draft did not go forward, any responder which does not copy
the 0x20 bits of the QNAME is compliant, and any initiator who depends
on the copying of 0x20 bits being copied has to allow for this perfectly
reasonable interpretation of the existing, somewhat ambiguous,
specification.

in other words the installed base gets a seat at the table, and if
you're going to behave very differently in a way that older
implementations could reasonably refuse to interoperate with, it's your
problem, not theirs. and under those circumstances, a mandatory change
to how the other end interprets the older ambiguous specification is, in
effect, a protocol change.

>
> Paul Vixie replies:
> } if they did,
> [that is: if clients take advange of persitent TCP over port 53]
> } then those initiators would either be a DoS from the responder's
> point of view, or a
> } DoS from other initiators' points of view. we are a prisoner to the
> reasonable expectations of
> } the billions of devices that were created in the decades-long era of
> RFC 1034 section 4.2.2.
>
> You're saying TCP is inherently a DoS when used for DNS?

yes. well, TCP/53 is. TCP/80, with an appropriate RESTful/JSON API,
would be fine. as i've said every time someone has said "to avoid
$problem, let's use TCP/53 as the primary transport", RFC 1035 section
4.2.2 is a mine field, and anyone at all can deny service to any
initiator who depends on tcp/53 service from anyone else at all. that's
why the fix to the Kaminsky bug was source port randomization, rather
than TCP/53.
>
> I don't get it. Some how the web community tolerates persistent TCP
> without
> falling over. And you've suggested DNS-over-HTTP is desirable. Won't
> that also create any DoS problems that stem from TCP?

no, it won't. i believe i spoke to this question in detail, down-thread:

> but, more importantly, persistent tcp is all we've got. RFC 6013
> failed, in the sense that the tcp-m WG chose not to give it the IANA
> resources it would have needed. google's tcp-fastopen is at best
> unsecure. SCTP seems to have jammed in the breach.
>
> if we want (and we do want) to keep a hot path open between a dns
> initiator and its pool of dns responders, then we need persistent tcp
> in the HTTP/1.1 style, and we need a large number of 

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Christopher Morrow]

On Wed, Jan 21, 2015 at 4:53 PM, John Heidemann  wrote:
> I don't see how DoS is an argument against TCP for DNS.  (Unless one
> assumes hardware and software at the servers is fixed to something like
> 2004 standards.)  What am I missing?

What's the average client load expected (number of unique clients in
the timeout of the tcp connection expected) for an authoritative
server today? (say an enterprise hosted server, and then someone that
is a large domain aggregator)

What is the same curve for a recursive server? (again, a small
isp/enterprise vs a large provider)

What impact will changing to longer lived persistent tcp connections
have on hardware and network capacity planning?

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[John Heidemann]

On Wed, 21 Jan 2015 09:30:44 +, Ray Bellis wrote: 
>
>> i realize that "no" votes aren't counted. but that's going to be my input if 
>> any document along the lines of adding persistence to tcp/53 is adopted by 
>> the WG. so, for full disclosure, i wanted to weigh in at this stage.
>
>TCP/53 is already persistent, it just happens most clients don't take 
>advantage of that feature.
>
>The point of my draft is to permit signalling that the current
>connection should _not_ be persisted ;-)

I want to restate this, because people often confuse current practice
with current specifications:

DNS over TCP/53 is *already* persistent.  No *protocol* changes are needed.

*Implementation* changes, however, are needed:

- clients need to not blindly close the connection after one request

- clients and servers need to use well known implementation techniques
  (from HTTP) to get good performance---pipelining, processing requests
  in parallel, sending replies out-of-order (rfc5966), handling TCP
  fastopen (newly minited rfc7413).

(We've measured and reported the performance differences here before.)

Paul Vixie replies:
} if they did,
[that is: if clients take advange of persitent TCP over port 53]
} then those initiators would either be a DoS from the responder's point of 
view, or a
} DoS from other initiators' points of view. we are a prisoner to the 
reasonable expectations of
} the billions of devices that were created in the decades-long era of RFC 1034 
section 4.2.2.

You're saying TCP is inherently a DoS when used for DNS?

I don't get it.  Some how the web community tolerates persistent TCP without
falling over.  And you've suggested DNS-over-HTTP is desirable.  Won't
that also create any DoS problems that stem from TCP?

I don't see how DoS is an argument against TCP for DNS.  (Unless one
assumes hardware and software at the servers is fixed to something like
2004 standards.)  What am I missing?

   -John Heidemann

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Tony Finch]

Paul Wouters  wrote:
>
> responders do not need to be upgraded for this, as we found out on this
> list about two years ago when Mark Andrews patched dig and I ran a test
> with that.

Not entirely true. Persistent TCP works but it needs some performance
engineering.

Responders need to be upgraded to handle queries concurrently and send
replies out-of-order, so that TCP performance is as good as UDP
performance. Both Unbound and BIND suffer from this (though BIND is being
fixed.)

They also need some web-server-style attention to TCP connection
scalability, e.g. by default BIND is limited to only 100 connections. It
should be reasonable to set it to 1 on servers that are relatively
modest by today's standards.

Tony.
-- 
f.anthony.n.finchhttp://dotat.at/
Tyne, Dogger, Fisher, German Bight: Southeast 4 or 5, occasionally 3 later.
Slight or moderate. Showers. Good, occasionally moderate.

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Paul Wouters 
> Wednesday, January 21, 2015 8:38 AM
> On Wed, 21 Jan 2015, Paul Vixie wrote:
>
>> even if changing TCP/53's connection semantics could be done without
>> creating new DoS vectors, the small number of DNS TCP initiators and
>> responders who will ever be upgraded
>
> responders do not need to be upgraded for this, as we found out on this
> list about two years ago when Mark Andrews patched dig and I ran a test
> with that.

a responder with a small file descriptor limit who ignores keepalive
signalling can easily see all of its tcp slots occupied, either by
persistent initiators, or by any extremely unskilled, low-investment
attacker.
>
>> , would be able to adopt TCP/80
>> faster. many middleboxes assume that DNS is UDP-only, and a few no doubt
>> proxy the transaction in a way that hijacks the connection management
>> semantics in a way that would (a) pass your new signalling along, but
>> (b) not follow it.
>
> What is the problem with "if you are behind broken middleware, do DNS
> like it it 1999" ? I don't see how that is a reason to start moving to
> port 80.

dnssec.

but, more importantly, persistent tcp is all we've got. RFC 6013 failed,
in the sense that the tcp-m WG chose not to give it the IANA resources
it would have needed. google's tcp-fastopen is at best unsecure. SCTP
seems to have jammed in the breach.

if we want (and we do want) to keep a hot path open between a dns
initiator and its pool of dns responders, then we need persistent tcp in
the HTTP/1.1 style, and we need a large number of tcp slots on the
responder, in the style of HTTP/1.1 responders.

the t-shirt is wrong. it's not "cross out 'lets take it to the ietf' and
write 'just put it into dns'". rather, it's "cross out 'lets assume that
our initiator has an internet connection' and write 'lets assume that
our initiator has a web connection'". the internet is older than the
web, but no longer larger than the web. just as ethernet was the RS232
of the 1990's, so now TCP/80 is the RS232 of the new century. i do not
love this fact, but i do recognize it.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Wouters]

On Wed, 21 Jan 2015, Paul Vixie wrote:


even if changing TCP/53's connection semantics could be done without
creating new DoS vectors, the small number of DNS TCP initiators and
responders who will ever be upgraded


responders do not need to be upgraded for this, as we found out on this
list about two years ago when Mark Andrews patched dig and I ran a test
with that.


, would be able to adopt TCP/80
faster. many middleboxes assume that DNS is UDP-only, and a few no doubt
proxy the transaction in a way that hijacks the connection management
semantics in a way that would (a) pass your new signalling along, but
(b) not follow it.


What is the problem with "if you are behind broken middleware, do DNS
like it it 1999" ? I don't see how that is a reason to start moving to
port 80.

Paul

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Ray Bellis 
> Wednesday, January 21, 2015 1:30 AM
>
> TCP/53 is already persistent, it just happens most clients don't take
> advantage of that feature.

if they did, then those initiators would either be a DoS from the
responder's point of view, or a DoS from other initiators' points of
view. we are a prisoner to the reasonable expectations of the billions
of devices that were created in the decades-long era of RFC 1034 section
4.2.2.
>
> The point of my draft is to permit signalling that the current
> connection should _not_ be persisted ;-)

i know. but the arrow of change does not point in that direction.
HTTP/0.9 was responder-close, and was thus able to be changed in
HTTP/1.1 to initiator-close unless and only unless "Connection: close"
was specified.

even if changing TCP/53's connection semantics could be done without
creating new DoS vectors, the small number of DNS TCP initiators and
responders who will ever be upgraded, would be able to adopt TCP/80
faster. many middleboxes assume that DNS is UDP-only, and a few no doubt
proxy the transaction in a way that hijacks the connection management
semantics in a way that would (a) pass your new signalling along, but
(b) not follow it.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Wouters]

On Tue, 20 Jan 2015, Paul Vixie wrote:


my input is not a direct answer to either question, but, may be relevant.

my view is: we can't reliably signal this capability, so any option we
pursue will create a DoS vector for either new or old initiators or
responders, and the right answer is to pursue DNS-over-HTTP as an
alternate transport that already has TCP persistence built into it. i
also note that we've got a JSON layout for DNS messages now, thanks to
bortzmeyer and hoffman; and we've got a reasonably portable and high
quality DNS shim layer now, thanks to the getdns team. so: adding
DNS-over-HTTP would happen faster than revising TCP/53.


It would be really sad to relegate all ports but 80/443 to the realm of CHAOS.
And inevitably, the transparancy of port 80/443 would suffer as it
becomes the default demuxing point. But I know you know this, and still
you think it is the best way, so therefor sad.

tcp/53 is already out there and working on the recursors. Adding client
side support that proactively uses this should, for example in unbound
and bind, should be easier than implementing dns-in-xml-over-port-80-in-new-api
(just like I'm not a fan of dns-over-dbus-into-your-vm-and-back approach
of systemd-resolved)

I also think that draft-ietf-edns-querychain is a much simpler api than
the new getdns api, but I'm biased. And speaking of that draft, I will
process the reviews received on it and push out a an updated version.

Paul

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Wessels, Duane]

I agree with Paul Hoffman.  While I think draft-ietf-dnsop-edns-tcp-keepalive is
good, even the simpler draft-bellis-dnsop-connection-close would be much
better than the current situation, so I support its adoption.

DW


On Jan 20, 2015, at 11:21 AM, Paul Hoffman  wrote:

> On Jan 20, 2015, at 7:37 AM, Tim Wicinski  wrote:
> 
> draft-ietf-dnsop-edns-tcp-keepalive is a reasonable document, but 
> draft-bellis-dnsop-connection-close achieves a great deal at a very low cost. 
> If we drop draft-ietf-dnsop-edns-tcp-keepalive (which seems likely if the 
> authors don't want to pursue it), we should adopt 
> draft-bellis-dnsop-connection-close and answer the many questions in the 
> draft.
> 
> --Paul Hoffman
> ___
> DNSOP mailing list
> DNSOP@ietf.org
> https://www.ietf.org/mailman/listinfo/dnsop



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Ray Bellis]

> i realize that "no" votes aren't counted. but that's going to be my input if 
> any document along the lines of adding persistence to tcp/53 is adopted by 
> the WG. so, for full disclosure, i wanted to weigh in at this stage.

TCP/53 is already persistent, it just happens most clients don't take advantage 
of that feature.

The point of my draft is to permit signalling that the current connection 
should _not_ be persisted ;-)

kind regards,

Ray

___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Vixie]

> Tim Wicinski 
> Tuesday, January 20, 2015 7:37 AM
> ...
> The chairs are wondering:
> 1) if their is still have a need for such an option,  and
>
> 2) if there is consensus on competing proposals.
>
>
> If you see a use case for the EDNS tcp-keepalive option as originally
> discussed, please say so, on the list, by February 4, 2015.
>
> If you want to pursue the connection-close draft, please say so, on
> the list, by February 4, 2015, especially if you're willing to work on
> it.
>
> If we don't hear anything about either, we drop them both.

my input is not a direct answer to either question, but, may be relevant.

my view is: we can't reliably signal this capability, so any option we
pursue will create a DoS vector for either new or old initiators or
responders, and the right answer is to pursue DNS-over-HTTP as an
alternate transport that already has TCP persistence built into it. i
also note that we've got a JSON layout for DNS messages now, thanks to
bortzmeyer and hoffman; and we've got a reasonably portable and high
quality DNS shim layer now, thanks to the getdns team. so: adding
DNS-over-HTTP would happen faster than revising TCP/53.

i realize that "no" votes aren't counted. but that's going to be my
input if any document along the lines of adding persistence to tcp/53 is
adopted by the WG. so, for full disclosure, i wanted to weigh in at this
stage.

-- 
Paul Vixie
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop

Re: [DNSOP] Followup Discussion on TCP keepalive proposals

[Paul Hoffman]

On Jan 20, 2015, at 7:37 AM, Tim Wicinski  wrote:

draft-ietf-dnsop-edns-tcp-keepalive is a reasonable document, but 
draft-bellis-dnsop-connection-close achieves a great deal at a very low cost. 
If we drop draft-ietf-dnsop-edns-tcp-keepalive (which seems likely if the 
authors don't want to pursue it), we should adopt 
draft-bellis-dnsop-connection-close and answer the many questions in the draft.

--Paul Hoffman
___
DNSOP mailing list
DNSOP@ietf.org
https://www.ietf.org/mailman/listinfo/dnsop