Hi Mirja,
Hi Benoit,
I would not expect to see a bis doc here. As I said I would rather
expect a new, separate document to specify and register the metrics
(and providing more details). Given the very different scope and use
case, I would not even see it as a problem if those two documents
don't align fully.
I can go back to the authors and ask if they would be interested in a
directorate review. But I guess that would also mean that they first
have to change the text in section 2.X to the format/structure
guidelines in RFC6390 (sec 5.4.2 only in this case, I'd say), correct?
Not necessarily.
The only question is that important in my mind is:
Al, assuming that someone would like to register this metric in a
registry (RFC6390), are they any grey areas in the performance metric
definitions in the draft?
From what I understand, a point such this one (from Al) is:
Because we are using Goodput, G, I take as given that there
must be a protocol with retransmission capability.
Otherwise, further simplification is possible (with dummy traffic).
But yes, Fs and G need to be reported on payload
at the same layer, so the protocol layer chosen is
an input parameter for this metric.
I still have to say that looking at RFC6390 again I find it not fully
applicable for this case; e.g. Measurement Timing is not a great fit
when you talk about a one time measurement (for one test run) of the
flow completion time. I'd fear that using the template provided in
RFC6390 would make it rather more confusing than clear.
Fine with that.
Regards, Benoit
Mirja
On 10.06.2016 09:36, Benoit Claise wrote:
Thanks Al,
This is exactly the type of feedback requested from the Performance
Metric
Directorate.
Mirja,
I buy into your argument:
I guess as soon as we have a registry, maybe there is someone
interest in IPPM to catch up these metrics again and provide a
RFC6390 definition but I would rather not like this document doing it.
As such, we probably don't need the RFC6390 template in this
document. Fair
enough.
However, what would be a pity is that if/when someone would like to
register
this metric, we end with questions such Al's one below, because the
metric
definitions in the published RFC are not precise. That could result
in a RFCbis.
Therefore, going through the Performance Metric Directorate feedback
now is
important IMO.
Regards, Benoit
Because we are using Goodput, G, I take as given that there
must be a protocol with retransmission capability.
Otherwise, further simplification is possible (with dummy traffic).
But yes, Fs and G need to be reported on payload
at the same layer, so the protocol layer chosen is
an input parameter for this metric.
Al
-----Original Message-----
From: Mirja Kuehlewind (IETF) [mailto:i...@kuehlewind.net]
Sent: Wednesday, June 08, 2016 7:21 AM
To: MORTON, ALFRED C (AL)
Cc:w...@mti-systems.com;aqm-cha...@ietf.org; The IESG; draft-ietf-aqm-
eval-guideli...@ietf.org; Benoit Claise;aqm@ietf.org
Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
guidelines-11: (with DISCUSS and COMMENT)
Actually, it really doesn't matter that much in this case, I’d say. As
we are talking about a lab environment, you might use dummy traffic
that
has some headers or not, that you might take into account of not,
mostly
depending on which information can be more easily accessed. What is
important is that you do the same thing for all schemes that you
compare.
I guess one could add a note that there are different ways to measure
this and that it is important to measure G at the same layer. Does
that
make sense?
Mirja
Am 08.06.2016 um 13:03 schrieb MORTON, ALFRED C (AL)
<acmor...@att.com>:
Here's one area which could use more detail:
...The Flow Completion Time (FCT) is
related to the flow size (Fs) and the goodput for the flow (G) as
follows:
FCT [s] = Fs [Byte] / ( G [Bit/s] / 8 [Bit/Byte] )
What protocol layers are included and excluded from Fs?
Also, G needs to be measured at the same layer, and the
definition in RFC 2647 is a bit vague about layers, too.
It would be good to clarify which bytes to count here.
Al
-----Original Message-----
From: Mirja Kuehlewind (IETF) [mailto:i...@kuehlewind.net]
Sent: Wednesday, June 08, 2016 5:40 AM
To: MORTON, ALFRED C (AL)
Cc: Benoit Claise;w...@mti-systems.com;aqm-cha...@ietf.org;
aqm@ietf.org;draft-ietf-aqm-eval-guideli...@ietf.org; The IESG
Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
guidelines-11: (with DISCUSS and COMMENT)
Hi Al,
what kind of detail are you looking for? Because I thought with the
given equation this one was pretty clear.
Do you have a reference to the benchmarking work?
Mirja
Am 08.06.2016 um 11:18 schrieb MORTON, ALFRED C (AL)
<acmor...@att.com>:
Hi Mirja,
That sounds fairly reasonable to me.
Would it be possible to ask the authors provide a bit more
detail on Flow Completion Time?
Flow Completion Time is close to a definition for a new metric,
and could benefit from more attention, perhaps a few more
details.
RFC6390 will provide some areas for improvement.
I imagine that related benchmarking efforts may wish to measure
this
metric, and there would be independent implementations based on
the description provided here.
regards from Geneve'
Al
-----Original Message-----
From: Mirja Kuehlewind (IETF) [mailto:i...@kuehlewind.net]
Sent: Wednesday, June 08, 2016 4:46 AM
To: MORTON, ALFRED C (AL); Benoit Claise
Cc:w...@mti-systems.com;aqm@ietf.org; The IESG; draft-ietf-aqm-
eval-
guideli...@ietf.org;aqm-cha...@ietf.org
Subject: Re: [aqm] Benoit Claise's Discuss on draft-ietf-aqm-eval-
guidelines-11: (with DISCUSS and COMMENT)
Hi Benoit,
I finally had another look at the document as well as at
RFC6390. I
guess the metrics in question are Flow completion time (sec 2.1.),
Flow
start up time (2.2.) and Packet loss synchronization (2.4.). And
you
are
right that these metric could be see as Application-Specific
Performance
Metric as defined in RFC6390. However, I agree with Al that given
the
scope of this document is providing
"a generic list of scenarios against which an
AQM proposal should be evaluated, considering both potential
performance gain and safety of deployment.“,
I don’t think these metric need to be defined and registered this
way.
I guess as soon as we have a registry, maybe there is someone
interest
in IPPM to catch up these metrics again and provide a RFC6390
definition
but I would rather not like this document doing it.
Is that acceptable for you?
We could add one more sentence in the abstract to make the
scope on
lab
testing during development and before deployment clear. Would that
help?
Mirja
Am 25.05.2016 um 14:22 schrieb Mirja Kuehlewind (IETF)
<i...@kuehlewind.net>:
Hi Al, Benoit, hi all,
thanks for the feedback. Sorry for me delaying this maybe a
little
but
I need t have another look at the document which will be next week
at
this point. In general I agree that this does not need to only
rely
on
registered metrics because is mostly for lab tests; further this
might
probably not the right doc to register new metrics. However, I
would
still like to have another look at the doc and see if we can
improve
anything or figure out if any of the ’new’/non-registed metrics
should/could be taken up by e.g. ippm.
Mirja
Am 20.05.2016 um 14:53 schrieb MORTON, ALFRED C (AL)
<acmor...@att.com>:
All,
a few replies in-line below,
Al
-----Original Message-----
From: Benoit Claise [mailto:bcla...@cisco.com]
Sent: Thursday, May 19, 2016 5:38 AM
To: The IESG
Cc:draft-ietf-aqm-eval-guideli...@ietf.org; wes@mti-
systems.com;
aqm-
cha...@ietf.org;w...@mti-systems.com;aqm@ietf.org; linda
Dunbar;
MORTON, ALFRED C (AL)
Subject: Benoit Claise's Discuss on draft-ietf-aqm-eval-
guidelines-
11:
(with DISCUSS and COMMENT)
Benoit Claise has entered the following ballot position for
draft-ietf-aqm-eval-guidelines-11: Discuss
...
----------------------------------------------------------------
--
--
--
DISCUSS:
----------------------------------------------------------------
--
--
--
Has a RFC6390 performance directorate review done for the 2.X
metrics?
It
should.
[ACM]
I reviewed this draft about 18 months ago.
Mostly, it points to existing RFCs for fundamental metrics,
and discusses others. I read this:
...This document provides characterization guidelines that
can be used to assess the deployability of an AQM, whether it is
candidate for standardization at IETF or not.
as restricted to lab testing.
Seehttp://www.ietf.org/iesg/directorate/performance-
metrics.html
I guess that the metrics will be recorded in the future (See
draft-ietf-ippm-metric-registry-06
), right?
[ACM]
That's up to the authors, they might simply point to
metrics in the registry contributed by others
(when following these guidelines at a future time).
For example, Flow Completion Time and Packet Loss
Synchronization
are
new, I believe.
[ACM]
Flow Completion Time is close to a definition for a new metric,
and could benefit from more attention, perhaps a few more
details.
RFC6390 will provide some areas for improvement.
Packet loss sync full methodology is described in [JAY006],
according to the text.
And some other metrics are already documented in RFC6390
compliant
documents. Pointers should be provided.
[ACM]
Most others are discussion sections and provide references.
See
https://tools.ietf.org/html/draft-ietf-xrblock-independent-
burst-
gap-
discard-01#appendix-A
for an example
----------------------------------------------------------------
--
--
--
COMMENT:
----------------------------------------------------------------
--
--
--
- Random Early Detection (RED), BLUE, and Proportional Integral
controller (PI)
Would you have references?
- BDP is mentioned a few times. Please expand.
- Glossary section = terminology section, right? If we want to
be
consistent across documents
- section 12.2. Why not a MUST below?
In order to understand an AQM's deployment considerations and
performance under a specific environment, AQM proposals SHOULD
describe the parameters that control the macroscopic AQM
behavior,
and identify any parameters that require tuning to operational
conditions.
_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm
.
_______________________________________________
aqm mailing list
aqm@ietf.org
https://www.ietf.org/mailman/listinfo/aqm
