Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

[Wesley Eddy]

On 1/22/2016 2:17 PM, Fred Baker (fred) wrote:

On Jan 22, 2016, at 10:47 AM, Wesley Eddy  wrote:

I do also (personally) think that if there's a desire to go standards-track 
(rather than just experimental) with AQM algorithms, that having a fairly 
explicit evaluation of the algorithms with regard to the guidelines would be 
very helpful, exactly as Polina asked about.  But I don't think this has really 
happened, and don't think it's necessary at all for experimental RFCs.

As I recall the discussion, we decided up front that since there is no 
interoperability requirement among AQM algorithms (the requirement is that they 
interoperate well with TCP and UDP based applications; the AQM algorithms don' 
actually talk to each other, and the point is to drop or mark at the right rate 
and with the right pattern to encourage transport layer sessions to behave 
well), we didn't need to recommend a single AQM algorithm for all equipment or 
all uses. What we did need to do was identify some AQM algorithms that actually 
worked, and give guidance to the vendors and operators on their use.


I agree with all of this.  The characterization guidelines are aimed at 
helping to identify the AQM algorithms that actually work, or cases 
where they don't work as well (i.e. where some harmful or unintended 
consequence results).



___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

[Klatsky, Carl]

Wes and all,

My comment is in regard to Polina's comment "The WG currently has two AQMs 
(dropping/marking policy) in last call. Did someone evaluate these AQMs 
according to the specified guidelines?".  As I read over 
draft-ietf-aqm-eval-guidelines, I did not think the objective of this memo was 
to arrive at consensus to select one specific AQM.  I thought the objective was 
to publish guidelines for implementers & service providers on how they can 
select an AQM that is best for their environment.  And further that both AQMs 
in last call would complete the process as valid AQMs for implementers & 
service providers as available to use in their environment, with the guidelines 
helping them to decide which is best for them.  Some may chose PIE, some may 
chose FQ_CODEL.  And possibly other future AQMs would go through the IETF 
process for publication, and that would be an additional option for 
implementers & service providers to evaluate according to the guidelines as 
best for their environment.

Is my understand of draft-ietf-aqm-eval-guidelines correct?

Regards,

Carl Klatsky
Comcast

From: aqm [mailto:aqm-boun...@ietf.org] On Behalf Of Wesley Eddy
Sent: Friday, January 22, 2016 9:51 AM
To: aqm@ietf.org; Polina Goltsman
Subject: Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

On 12/7/2015 7:32 AM, Polina Goltsman wrote:


In the abstract, the document says that it describes characterization 
guidelines for an AQM proposal, to decide whether it should be adopted by the 
AQM WG. The WG currently has two AQMs (dropping/marking policy) in last call. 
Did someone evaluate these AQMs according to the specified guidelines?


In my opinion, for "standardization" it would be good to have crisp guidelines. 
 For simply publishing RFCs that are experimental (not standardized), it is 
much less important.




Moreover, it seems to me that the WG is about to conclude. What exactly is the 
purpose of standardizing this document then ?


It's definitely true that the activity level has been low, and so we're trying 
to wrap up the outstanding work items before it flattens completely.  This 
document is not proposed for standards track, only "Informational".  The 
current goal as I see it (with co-chair hat on) is to see if we have rough 
consensus that this is a useful contribution to the community going forward, 
and that any small issues can be addressed.

As I understood your review (please correct if I'm wrong), a main issue you see 
is that there isn't specific guidance about numeric values or ranges to use in 
evaluations?  Nicolas explained why this might be a bit difficult to do in 
general (and specific values published in 2016 may be moot by 2018), so as I 
understand, one of the limitations of this document is that it is only going to 
be able to provide general descriptive guidance in terms of what kinds of tests 
to run, and what types of things to look for, but not prescriptive values and 
conditions to be met.  Do you think that's okay, even though it's probably less 
directly useful than if there were specific values?

___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

[Dave Täht]

The evaluation guide needs to be executable, or rather, turned into
public code and a standardized benchmark suite. Eventually.

Iteratively, flent has many tests that have proven valuable and quite a
few that have not.

The tests in the aqm guide, need to be created, iterated on, and examined.


___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

[Wesley Eddy]

On 1/22/2016 1:32 PM, Klatsky, Carl wrote:


Wes and all,

My comment is in regard to Polina’s comment “The WG currently has two 
AQMs (dropping/marking policy) in last call. Did someone evaluate 
these AQMs according to the specified guidelines?”.  As I read over 
draft-ietf-aqm-eval-guidelines, I did not think the objective of this 
memo was to arrive at consensus to select one specific AQM.  I thought 
the objective was to publish guidelines for implementers & service 
providers on how they can select an AQM that is best for their 
environment. And further that both AQMs in last call would complete 
the process as valid AQMs for implementers & service providers as 
available to use in their environment, with the guidelines helping 
them to decide which is best for them. Some may chose PIE, some may 
chose FQ_CODEL.  And possibly other future AQMs would go through the 
IETF process for publication, and that would be an additional option 
for implementers & service providers to evaluate according to the 
guidelines as best for their environment.


Is my understand of draft-ietf-aqm-eval-guidelines correct?




Yes, you're correct!  My assumption is that someone like a service 
provider would have an idea of some of the ranges of values (rates, 
delays, asymmetries, etc) appropriate for their environment, and would 
be able to use the evaluation guidelines effectively.


I do also (personally) think that if there's a desire to go 
standards-track (rather than just experimental) with AQM algorithms, 
that having a fairly explicit evaluation of the algorithms with regard 
to the guidelines would be very helpful, exactly as Polina asked about.  
But I don't think this has really happened, and don't think it's 
necessary at all for experimental RFCs.




___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

Re: [aqm] I-D Action: draft-ietf-aqm-eval-guidelines-09.txt

[Fred Baker (fred)]

On Jan 22, 2016, at 10:47 AM, Wesley Eddy  wrote:
> I do also (personally) think that if there's a desire to go standards-track 
> (rather than just experimental) with AQM algorithms, that having a fairly 
> explicit evaluation of the algorithms with regard to the guidelines would be 
> very helpful, exactly as Polina asked about.  But I don't think this has 
> really happened, and don't think it's necessary at all for experimental RFCs.

As I recall the discussion, we decided up front that since there is no 
interoperability requirement among AQM algorithms (the requirement is that they 
interoperate well with TCP and UDP based applications; the AQM algorithms don' 
actually talk to each other, and the point is to drop or mark at the right rate 
and with the right pattern to encourage transport layer sessions to behave 
well), we didn't need to recommend a single AQM algorithm for all equipment or 
all uses. What we did need to do was identify some AQM algorithms that actually 
worked, and give guidance to the vendors and operators on their use.




signature.asc
Description: Message signed with OpenPGP using GPGMail
___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm

[aqm] status of PIE drafts WGLC

[Wesley Eddy]

Hello; the working group last call on the PIE drafts generated some 
emails, but I don't think I've seen any response from the editors. 
Specifically, there were a couple of emails with algorithm description 
questoins and technical comments from Rasool Al-Saadi and Ilpo Jarvinen, 
both with specific points that should be addressed.


For the most part, as I understand the comments, these are things that 
can be relatively simply fixed up or the intent clarified, and not 
catastrophic issues that would prevent the PIE docs from being 
publishable.  Please correct me if I misunderstand though.


If the editors can respond and work up a revision that addresses the 
comments to the satisfaction of Rasool and Ilpo, I'd like to keep the 
PIE documents moving forward.




___
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm