Lars Eggert has entered the following ballot position for
draft-ietf-alto-performance-metrics-19: Discuss
When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)
Please refer to https://www.ietf.org/blog/handling-iesg-ballot-positions/
for more information about how to handle DISCUSS and COMMENT positions.
The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/
--
DISCUSS:
--
This document needs to become much more formal about how it defines the
metrics it wishes to use with ALTO. This could either be done either by
identifying and normatively referencing existing metrics the IETF has defined,
or by defining them here. When normatively referencing existing IETF metrics, it
would need to explain why their use with ALTO makes sense.
At the moment, the document informatively points to a somewhat arbitrary
collection of prior IETF metrics (most of which are from IPPM, residual
bandwidth from IS-IS TE, but then reservable bandwidth from OSPF TE?). But it
only refers to them as "examples", without actually defining how exactly they
are to be used with ALTO, or - if not those - which actual metrics are supposed
to be used.
Defining a mechanism for exposing metric information to clients isn't really
useful unless the content of that information is much more clearly specified.
Section 4.1.3. , paragraph 2, discuss:
>Intended Semantics: To give the throughput of a TCP congestion-
>control conforming flow from the specified source to the specified
>destination; see [RFC3649, Section 5.1 of RFC8312] on how TCP
>throughput is estimated. The spatial aggregation level is specified
>in the query context (e.g., PID to PID, or endpoint to endpoint).
A TCP bandwidth estimate can only be meaningfully be derived for bulk TCP
transfers under a set of pretty strict and simplistic assumptions, making this
metric a meaningless at best and misleading at worst, given that the source of
this information doesn't know what workload, congestion controller and network
conditions the user of this information will use or see.
Also, RFC3649 is an Experimental RFC (from 2003!) and RFC8312 is an
Informational RFC. Since this document normatively refers to them, it needs to
cite them, and this will cause DOWNREFs for PS document. I would argue that
at least RFC3649 is certainly not an appropriate DOWNREF.
Why define this metric at all? The material you point to is the usual
model-based throughput calculation based on RTT and loss rates; a client that
intended to predict TCP performance could simply query ALTO for this and perform
their own computation, which will likely be more accurate, since the client will
hopefully know which congestion controller they will use for the given workload,
and what the characteristics of that workload are.
--
COMMENT:
--
Section 1. , paragraph 6, comment:
>The purpose of this document is to ensure proper usage of the
>performance metrics defined in Table 1; it does not claim novelty of
>the metrics. The "Origin Example" column of Table 1 gives an example
>RFC that has defined each metric.
I don't understand what the purpose of the "origin example" column is. Most of
these point to IPPM metrics, which have a pretty clear and narrowly-defined area
of applicability. Since ALTO isn't performing IPPM-style network testing, it's
not clear why IPPM metrics are referenced here?
Section 2.2. , paragraph 23, comment:
>If a cost metric string does not have the optional statical operator
>string, the statistical operator SHOULD be interpreted as the default
>statical operator in the definition of the base metric. If the
What is a "statical" operator; I am not familiar with the term and it doesn't
seem to appear in other RFCs? (Also occurs elsewhere in this document.)
Section 3.1.4. , paragraph 4, comment:
>link statistics. Another example of a source to estimate the delay
>is the IPPM framework [RFC2330]. It is RECOMMENDED that the
IPPM defines measurement metrics. How would they be a source for estimates?
Section 3.3. , paragraph 1, comment:
> 3.3. Cost Metric: Delay Variation (delay-variation)
Is this supposed to apply to the one-way or bidirectional delay? Also, delay
variation is not independent from path utilization (c.f. bufferbloat), so why is
it being reported independently?
Section 3.5. , paragraph 1, comment:
> 3.5. Cost Metric: Loss Rate (lossrate)
What is this metric supposed to capture? Loss is generally not independent from
network utilization (apart
