Lars Eggert has entered the following ballot position for draft-ietf-alto-performance-metrics-19: Discuss
When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) Please refer to https://www.ietf.org/blog/handling-iesg-ballot-positions/ for more information about how to handle DISCUSS and COMMENT positions. The document, along with other ballot positions, can be found here: https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/ ---------------------------------------------------------------------- DISCUSS: ---------------------------------------------------------------------- This document needs to become much more formal about how it defines the metrics it wishes to use with ALTO. This could either be done either by identifying and normatively referencing existing metrics the IETF has defined, or by defining them here. When normatively referencing existing IETF metrics, it would need to explain why their use with ALTO makes sense. At the moment, the document informatively points to a somewhat arbitrary collection of prior IETF metrics (most of which are from IPPM, residual bandwidth from IS-IS TE, but then reservable bandwidth from OSPF TE?). But it only refers to them as "examples", without actually defining how exactly they are to be used with ALTO, or - if not those - which actual metrics are supposed to be used. Defining a mechanism for exposing metric information to clients isn't really useful unless the content of that information is much more clearly specified. Section 4.1.3. , paragraph 2, discuss: > Intended Semantics: To give the throughput of a TCP congestion- > control conforming flow from the specified source to the specified > destination; see [RFC3649, Section 5.1 of RFC8312] on how TCP > throughput is estimated. The spatial aggregation level is specified > in the query context (e.g., PID to PID, or endpoint to endpoint). A TCP bandwidth estimate can only be meaningfully be derived for bulk TCP transfers under a set of pretty strict and simplistic assumptions, making this metric a meaningless at best and misleading at worst, given that the source of this information doesn't know what workload, congestion controller and network conditions the user of this information will use or see. Also, RFC3649 is an Experimental RFC (from 2003!) and RFC8312 is an Informational RFC. Since this document normatively refers to them, it needs to cite them, and this will cause DOWNREFs for PS document. I would argue that at least RFC3649 is certainly not an appropriate DOWNREF. Why define this metric at all? The material you point to is the usual model-based throughput calculation based on RTT and loss rates; a client that intended to predict TCP performance could simply query ALTO for this and perform their own computation, which will likely be more accurate, since the client will hopefully know which congestion controller they will use for the given workload, and what the characteristics of that workload are. ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- Section 1. , paragraph 6, comment: > The purpose of this document is to ensure proper usage of the > performance metrics defined in Table 1; it does not claim novelty of > the metrics. The "Origin Example" column of Table 1 gives an example > RFC that has defined each metric. I don't understand what the purpose of the "origin example" column is. Most of these point to IPPM metrics, which have a pretty clear and narrowly-defined area of applicability. Since ALTO isn't performing IPPM-style network testing, it's not clear why IPPM metrics are referenced here? Section 2.2. , paragraph 23, comment: > If a cost metric string does not have the optional statical operator > string, the statistical operator SHOULD be interpreted as the default > statical operator in the definition of the base metric. If the What is a "statical" operator; I am not familiar with the term and it doesn't seem to appear in other RFCs? (Also occurs elsewhere in this document.) Section 3.1.4. , paragraph 4, comment: > link statistics. Another example of a source to estimate the delay > is the IPPM framework [RFC2330]. It is RECOMMENDED that the IPPM defines measurement metrics. How would they be a source for estimates? Section 3.3. , paragraph 1, comment: > 3.3. Cost Metric: Delay Variation (delay-variation) Is this supposed to apply to the one-way or bidirectional delay? Also, delay variation is not independent from path utilization (c.f. bufferbloat), so why is it being reported independently? Section 3.5. , paragraph 1, comment: > 3.5. Cost Metric: Loss Rate (lossrate) What is this metric supposed to capture? Loss is generally not independent from network utilization (apart from random corruption loss). So it should be zero for unloaded networks, and depends on utilization otherwise. Also, is this unidirectional or bidirectional loss (wording below is unclear)? Using lowercase "not" together with an uppercase RFC2119 keyword is not acceptable usage. Found: "MUST not" The document has 6 authors, which exceeds the recommended author limit. I assume the sponsoring AD has agreed that this is appropriate? No reference entries found for: [RFC3649] and [RFC8312]. Found terminology that should be reviewed for inclusivity; see https://www.rfc-editor.org/part2/#inclusive_language for background and more guidance: * Term "man"; alternatives might be "individual", "people", "person". ------------------------------------------------------------------------------- All comments below are about very minor potential issues that you may choose to address in some way - or ignore - as you see fit. Some were flagged by automated tools (via https://github.com/larseggert/ietf-reviewtool), so there will likely be some false positives. There is no need to let me know what you did with these suggestions. "Abstract", paragraph 2, nit: - types of cost metric. Since the ALTO base protocol (RFC 7285) + types of cost metrics. Since the ALTO base protocol (RFC 7285) + + Section 1. , paragraph 2, nit: > ] on registering ALTO cost metrics. Hence it specifies the identifier, the in > ^^^^^ A comma may be missing after the conjunctive/linking adverb "Hence". Section 2.2. , paragraph 2, nit: > of the observations. median: the mid point (i.e., p50) of the observations. > ^^^^^^^^^ This word is normally spelled with a hyphen. "IPPM ", paragraph 2, nit: > Also, delay variation is not independent from path utilization (c.f. buffer > ^^^^^^^^^^^^^^^^ The usual collocation for "independent" is "of", not "from". Did you mean "independent of"? Section 3.3.3. , paragraph 7, nit: > apture? Loss is generally not independent from network utilization (apart fr > ^^^^^^^^^^^^^^^^ The usual collocation for "independent" is "of", not "from". Did you mean "independent of"? Section 3.4.3. , paragraph 6, nit: > imation" method. See Section 3.1.4 on on related discussions such as summing > ^^^^^ Possible typo: you repeated a word. Section 3.5.4. , paragraph 3, nit: > [RFC8312]), it helps to specify as much details as possible on the the cong > ^^^^ Use "many" with countable plural nouns like "details". Section 3.5.4. , paragraph 3, nit: > ify as much details as possible on the the congestion control algorithm used > ^^^^^^^ Two determiners in a row. Choose either "the" or "the". These URLs in the document can probably be converted to HTTPS: * http://www.iana.org/assignments/alto-protocol/alto-protocol.xhtml#cost-metrics _______________________________________________ alto mailing list alto@ietf.org https://www.ietf.org/mailman/listinfo/alto
