Re: [aqm] Experimental vs informational vs standards track

[Polina Goltsman]

On 02/05/2016 03:18 AM, Dave Täht wrote:

Pie itself is proposed as standards track, despite the lack of field
data, a 15 page criticism from bob briscoe of the public implementation,
and other open issues like that.
if "the public implementation" refers to Linux kernel module sch_pie, 
then it is no longer standard-conform


(http://www.ietf.org/mail-archive/web/aqm/current/msg01667.html, 
somewhere at the end)


___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Toke Høiland-Jørgensen]

Wesley Eddy  writes:

> IMHO, Standards Track carries more weight to say that there are no
> sharp corners, and the IETF is pretty sure this works well.
> Experimental is more cautious saying this looks pretty useful, and you
> should consider trying it out, but it might have some rough edges
> (e.g. like open research questions, identified in several of the AQM
> drafts).

Well, for the FQ-CoDel algorithm the caveats are more of the form
"fairness queueing in some cases have limitations", not "this particular
algorithm has limitations". So if we assume that people know what a
fairness queueing algorithm is, I think that FQ-CoDel can most certainly
be considered to "work well". I think the fact that it is now on *by
default* in a whole range of Linux distributions should attest to that.

-Toke

___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Dave Täht]

On 2/4/16 5:30 PM, Wesley Eddy wrote:
> On 2/4/2016 8:26 PM, Wesley Eddy wrote:
>>
>> There is IESG explanation of the distinction here:
>> https://www.ietf.org/iesg/informational-vs-experimental.html
>>
> 
> Quoting from that, I think this is the criteria that makes it most clear
> Informational is appropriate for DOCSIS-PIE:
> """
> 
>  1. If it's not going to be changed no matter what the result is, it's
> Informational. This is typically the case with vendor protocols; the
> vendor will publish it for the good of the community, but retains
> full change control, and gives no promises about listening to
> community feedback. Case in point: "Microsoft Point-To-Point
> Encryption (MPPE) Protocol" (RFC 3078).
> 
> """
> 
> It's simply what was done elsewhere, and not going to be changed.  Greg
> was kind enough to write it up, and it's useful information for the
> community, so the WG had decided to put it forward as Informational.
> 
> Does that make sense?

Sure. I'm OK with informational status for docsis-pie.

But: It is roughly the same box that the Linux community would put
fq_codel in (at this point), if you consider them a "vendor".
Informational was the status of the draft for the past several years,
til it changed this morning.

Can you address the rest of my message? (notably: what effect does the
status of the document have on other standards that might rely on it?).

We've not discussed "Standards track" for fq_codel. By what criteria
would it qualify or not? I figured it was mostly blocked on an
implementation from the draft.

Pie itself is proposed as standards track, despite the lack of field
data, a 15 page criticism from bob briscoe of the public implementation,
and other open issues like that. Personally I've been waiting for an
actual modem to test on before bothering to explore more deeply results
from pie than we already have. (There is a study starting up soon where
I hope to finally A/B the stuff)

And:

I've only recently discovered the pain that "experimental" can cause
when other ietf standards are attempted to be layered on top of it (in
the nascent babel wg). It didn't sound like informational would cost the
same pain. Am I wrong in that assumption?


> 
> 
> 
> 
> 
> ___
> aqm mailing list
> aqm@ietf.org
> https://www.ietf.org/mailman/listinfo/aqm
> 

___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Wesley Eddy]

On 2/4/2016 8:22 PM, Dave Täht wrote:

I do not really understand how this criterion promotes docsis-pie from
experimental to informational (or the reverse: demotes fq_codel from
informational to experimental, which happened this morning...


Hi Dave, I'm not ignoring the rest of your message, but do need to 
correct one misconception first.


There is no higher or lower relationship between Informational and 
Experimental.


There is IESG explanation of the distinction here:
https://www.ietf.org/iesg/informational-vs-experimental.html


___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Wesley Eddy]

On 2/4/2016 9:18 PM, Dave Täht wrote:

Pie itself is proposed as standards track, despite the lack of field
data, a 15 page criticism from bob briscoe of the public implementation,
and other open issues like that. Personally I've been waiting for an
actual modem to test on before bothering to explore more deeply results
from pie than we already have. (There is a study starting up soon where
I hope to finally A/B the stuff)


Before going to the IESG, we need to make sure there is consensus on 
that publication track for PIE.  As you have seen, I'm trying to address 
that for each document as all of the technical WGLC comments are 
addressed and closed out.  I think the PIE editors would like Standards 
Track, if I understand correctly.  I do feel like the working group 
needs to speak up about whether they agree, because as a chair, I 
haven't heard much feedback about it.




And:

I've only recently discovered the pain that "experimental" can cause
when other ietf standards are attempted to be layered on top of it (in
the nascent babel wg). It didn't sound like informational would cost the
same pain. Am I wrong in that assumption?


I'm not familiar with this case, but was on the IESG in the past, and am 
not aware of Experimental causing any real issues for layering or 
referencing in other standards.  This is simply called out in the IETF 
Last Call as a downref, recorded in the downref registry, and life goes 
on.  The running code doesn't care how the document is labelled either :)


IMHO, Standards Track carries more weight to say that there are no sharp 
corners, and the IETF is pretty sure this works well. Experimental is 
more cautious saying this looks pretty useful, and you should consider 
trying it out, but it might have some rough edges (e.g. like open 
research questions, identified in several of the AQM drafts).


Just my 2 cents.

___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Wesley Eddy]

Dave, here is a longer answer to your specific questions; I hope this 
helps calibrate where I'm coming from at least:



On 2/4/2016 8:22 PM, Dave Täht wrote:

I realize now that there was a call as to what status it should be
a while.  I figured silence meant there was consensus on informational,
so I was a bit surprised when it was changed. You got my attention!


I think we had discussed focusing on document quality and collecting 
evaluations, test results, etc., and then when ready to publish each 
algorithm, deciding what document track to go on.


For the DOCSIS one, I'm pretty sure there was never any other option 
than Informational, since it was not something IETF was working on. I 
think it is similar in a way to RFC 6057, and others.




so, what criteria apply for experimental vs informational vs standards
track?


For Informational vs Experimental, there is the IESG statement:
https://www.ietf.org/iesg/informational-vs-experimental.html

For the relation with the standards track, I don't know anything that 
describes Experimental vs Standards Track outside of RFC 2026.


AQM's charter allows us to publish on the Standards Track.  I haven't 
seen very much push to do so, personally, and IMHO the standards track 
is not to be taken lightly.  If a lot of people say "yes, definitely, 
algorithm X should be on standards track, and we're comfortable with 
that", that will be convincing ... but the feedback is really light so far.




What blockers apply later, were, for example, another RFC to rely on an
experimental vs informational fq_codel rfc? For example, right now,
fq_codel is the defacto fq+aqm for homenet...

vs standards track?



I don't think there's really any distinction there.  For the IESG, 
Standards Track (or BCP) is definitely better to normatively reference 
from other documents going onto Standards Track, since otherwise the AD 
has to do explain and catalogue the "downref" for IETF Last Call.  
However, this is not uncommon, or very burdensome. There is a running 
list of downrefs, and once an RFC is on it, that can be referenced for 
future IETF LCs:

https://trac.tools.ietf.org/group/iesg/trac/wiki/DownrefRegistry




No field data exists for docsis-pie. Most of the data on pie is from
sims.  Outside of the docsis standard I know of no deployments of pie
and no hardware support for it (please, someone? is there a roadmap from
any manufacturer with pie in it?).


I don't know about this, but it sounds like a good reason why 
Informational and Experimental tracks would be preferable at the moment 
to Standards Track.




and conversely, since the fq-codel draft describes what has already been
done in linux, deployed in systemd, openwrt, the edgerouter products (as
what it is), as part of more than a few other products, (rebranded),
inside several major cos, and with at least one well known deployed ISP...

...

I regarded fq_codel as experimental 4 years ago. it has survived the
test of time with no substantial changes. Certainly I'd like it to be
self tuning below 2.5mbits, but pie does badly there also without tuning.

...


I believe the Experimental vs Standards Track decision on FQ-CoDel is 
independent of anything to do with PIE.




As for declaring a "proposed standard", it seems as though pie's
standardization status itself has not yet been discussed on this list,
either.


I also would like to see more discussion of what people want to do with 
these documents.  I've tried to be clear about my own thoughts.  As a 
chair, my default is to NOT go standards track unless there's explicit 
push from the working group.




My vote is for docis-pie, pie and fq_codel to have the same status,
whatever it winds up being. Informational seemed fine, across the board.
I'm all in favor of more deployment experience.



Understood, though I'm doubtful that our publication track will impact 
deployment.  I believe we need to make the proper choice for each 
algorithm on its own merits, and not tie them together, personally.  
However, if the working group wants to keep everything at an even status 
though, please shout and tell us what that status is!





___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] Experimental vs informational vs standards track

[Wesley Eddy]

On 2/4/2016 8:26 PM, Wesley Eddy wrote:


There is IESG explanation of the distinction here:
https://www.ietf.org/iesg/informational-vs-experimental.html



Quoting from that, I think this is the criteria that makes it most clear 
Informational is appropriate for DOCSIS-PIE:

"""

1. If it's not going to be changed no matter what the result is, it's
   Informational. This is typically the case with vendor protocols; the
   vendor will publish it for the good of the community, but retains
   full change control, and gives no promises about listening to
   community feedback. Case in point: "Microsoft Point-To-Point
   Encryption (MPPE) Protocol" (RFC 3078).

"""

It's simply what was done elsewhere, and not going to be changed.  Greg 
was kind enough to write it up, and it's useful information for the 
community, so the WG had decided to put it forward as Informational.


Does that make sense?



___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm