Re: A question about [Fwd: WG Review: Performance Metrics at Other Layers (pmol)]
Leslie == Leslie Daigle [EMAIL PROTECTED] writes: I doubt I'll use the output in security protocols. Leslie Leslie. Leslie Original Message Subject: WG Review: Leslie Performance Metrics at Other Layers (pmol) Date: Mon, 22 Leslie Oct 2007 14:15:02 -0400 From: IESG Secretary Leslie [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Leslie [EMAIL PROTECTED] Leslie A new IETF working group has been proposed in the Leslie Operations and Management Area. The IESG has not made any Leslie determination as yet. The following draft charter was Leslie submitted, and is provided for informational purposes Leslie only. Please send your comments to the IESG mailing list Leslie ([EMAIL PROTECTED]) by October 29. Leslie +++ Leslie Performance Metrics at Other Layers (pmol) Leslie == Leslie Current Status: Proposed Working Group Leslie WG Chairs: TBD Leslie Operations and Management Area: Dan Romascanu Leslie [EMAIL PROTECTED] Ronald Bonica [EMAIL PROTECTED] Leslie Description: Leslie The successful implementation and operation of IP based Leslie applications often depends on some underlying performance Leslie measurement infrastructure that helps service operators or Leslie network managers to recognize when performance is Leslie unsatisfactory and identify problems affecting service Leslie quality. Standardized performance metrics add the Leslie desirable features of consistent implementation, Leslie interpretation, no comparison. Leslie The IETF has two Working Groups dedicated to the Leslie development of performance metrics however each has strict Leslie limitations in their charters: Leslie - The Benchmarking Methodology WG has addressed a range of Leslie networking technologies and protocols in their long Leslie history (such as IEEE 802.3, ATM, Frame Relay, and Routing Leslie Protocols), but the charter strictly limits their Leslie Performance characterizations to the laboratory Leslie environment. Leslie - The IP Performance Metrics WG has the mandate to develop Leslie metrics applicable to the performance of Internet data Leslie delivery, but it is specifically prohibited from Leslie developing metrics that characterize traffic (such as a Leslie VoIP stream). Leslie The IETF also has current and completed activities related Leslie to the reporting of application performance metrics Leslie (e.g. RAQMON and RTCP XR) and is also actively involved in Leslie the development of reliable transport protocols which Leslie would affect the relationship between IP performance and Leslie application performance. Leslie Thus there is a gap in the currently chartered coverage of Leslie IETF WGs: development of performance metrics for IP-based Leslie protocols and applications that operate over UDP, TCP, Leslie SCTP, DCCP, Forward Error Correction (FEC) and other Leslie robust transport protocols, and that can be used to Leslie characterize traffic on live networks. Leslie The working group will focus on the completion of two Leslie RFCs: Leslie 1. A PMOL framework and guidelines memo that will describe Leslie the necessary elements of performance metrics of protocols Leslie and applications transported over IETF-specified protocols Leslie (such as the formal definition, purpose, and units of Leslie measure) and the various types of metrics that Leslie characterize traffic on live networks (such as metrics Leslie derived from other metrics, possibly on lower layers). The Leslie framework will also address the need to specify the Leslie intended audience and the motivation for the performance Leslie metrics. There will also be guidelines for a performance Leslie metric development process that includes entry criteria Leslie for new proposals (how a proposal might be evaluated for Leslie possible endorsement by a protocol development working Leslie group), and how an successful proposal will be developed Leslie by PMOL WG in cooperation with a protocol development WG. Leslie 2. A proof-of-concept RFC defining performance metrics for Leslie SIP, based on draft-malas-performance-metrics. This memo Leslie would serve as an example of the framework and the PMOL Leslie development process in the IETF. Leslie Discussion of new work proposals is strongly discouraged Leslie under the initial charter of the PMOL WG, except to advise Leslie a protocol development WG when they are evaluating a new Leslie work proposal for related performance metrics. Leslie The PMOL WG will also be guided by a document describing Leslie how memos defining performance metrics are intended to Leslie advance along the IETF Standards track
A question about [Fwd: WG Review: Performance Metrics at Other Layers (pmol)]
At the Application Performance Metrics BoF in Chicago, there was a lot of discussion about whether or how to best capture expertise in the area of performance metrics for application to IETF protocols. There seemed to be 2 separate questions on the table: 1/ how, and 2/ whether to generalize a performance metric framework for IETF protocols. The proposed PMOL WG addresses the first question -- how to do it. On the second point -- the question is really about whether the IETF community as a whole supports/values performance metrics and will invest the effort in using any general framework for existing/new protocols. I believe the folks that joined the PMOL mailing list after the APM BoF are assuming that support is there. I suggest that now is an excellent time for folks _not_ involved in the PMOL effort, but who are working on protocols that could/should make use of its output, to voice some opinions on whether or not this approach is helpful/will be useful to them in future protocols. Leslie. Original Message Subject: WG Review: Performance Metrics at Other Layers (pmol) Date: Mon, 22 Oct 2007 14:15:02 -0400 From: IESG Secretary [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: [EMAIL PROTECTED] A new IETF working group has been proposed in the Operations and Management Area. The IESG has not made any determination as yet. The following draft charter was submitted, and is provided for informational purposes only. Please send your comments to the IESG mailing list ([EMAIL PROTECTED]) by October 29. +++ Performance Metrics at Other Layers (pmol) == Current Status: Proposed Working Group WG Chairs: TBD Operations and Management Area: Dan Romascanu [EMAIL PROTECTED] Ronald Bonica [EMAIL PROTECTED] Description: The successful implementation and operation of IP based applications often depends on some underlying performance measurement infrastructure that helps service operators or network managers to recognize when performance is unsatisfactory and identify problems affecting service quality. Standardized performance metrics add the desirable features of consistent implementation, interpretation, no comparison. The IETF has two Working Groups dedicated to the development of performance metrics however each has strict limitations in their charters: - The Benchmarking Methodology WG has addressed a range of networking technologies and protocols in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing Protocols), but the charter strictly limits their Performance characterizations to the laboratory environment. - The IP Performance Metrics WG has the mandate to develop metrics applicable to the performance of Internet data delivery, but it is specifically prohibited from developing metrics that characterize traffic (such as a VoIP stream). The IETF also has current and completed activities related to the reporting of application performance metrics (e.g. RAQMON and RTCP XR) and is also actively involved in the development of reliable transport protocols which would affect the relationship between IP performance and application performance. Thus there is a gap in the currently chartered coverage of IETF WGs: development of performance metrics for IP-based protocols and applications that operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) and other robust transport protocols, and that can be used to characterize traffic on live networks. The working group will focus on the completion of two RFCs: 1. A PMOL framework and guidelines memo that will describe the necessary elements of performance metrics of protocols and applications transported over IETF-specified protocols (such as the formal definition, purpose, and units of measure) and the various types of metrics that characterize traffic on live networks (such as metrics derived from other metrics, possibly on lower layers). The framework will also address the need to specify the intended audience and the motivation for the performance metrics. There will also be guidelines for a performance metric development process that includes entry criteria for new proposals (how a proposal might be evaluated for possible endorsement by a protocol development working group), and how an successful proposal will be developed by PMOL WG in cooperation with a protocol development WG. 2. A proof-of-concept RFC defining performance metrics for SIP, based on draft-malas-performance-metrics. This memo would serve as an example of the framework and the PMOL development process in the IETF. Discussion of new work proposals is strongly discouraged under the initial charter of the PMOL WG, except to advise a protocol development WG when they are evaluating a new work proposal for related performance metrics. The PMOL WG will also be guided by a document describing how memos defining performance metrics are intended to advance along the IETF
Re: A question about [Fwd: WG Review: Performance Metrics at Other Layers (pmol)]
Since application developers have developed many tricks to smooth out some peculiarities of network-based protocols (e.g. jitter, delay, drops and so on), it is very often hard to related the performance measured/perceived at application layer with those of underlying layers. If this WG provides guidelines (or at least hints) to get over this vagueness, I will be very happy. Regards Leslie Daigle wrote: At the Application Performance Metrics BoF in Chicago, there was a lot of discussion about whether or how to best capture expertise in the area of performance metrics for application to IETF protocols. There seemed to be 2 separate questions on the table: 1/ how, and 2/ whether to generalize a performance metric framework for IETF protocols. The proposed PMOL WG addresses the first question -- how to do it. On the second point -- the question is really about whether the IETF community as a whole supports/values performance metrics and will invest the effort in using any general framework for existing/new protocols. I believe the folks that joined the PMOL mailing list after the APM BoF are assuming that support is there. I suggest that now is an excellent time for folks _not_ involved in the PMOL effort, but who are working on protocols that could/should make use of its output, to voice some opinions on whether or not this approach is helpful/will be useful to them in future protocols. Leslie. Original Message Subject: WG Review: Performance Metrics at Other Layers (pmol) Date: Mon, 22 Oct 2007 14:15:02 -0400 From: IESG Secretary [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: [EMAIL PROTECTED] A new IETF working group has been proposed in the Operations and Management Area. The IESG has not made any determination as yet. The following draft charter was submitted, and is provided for informational purposes only. Please send your comments to the IESG mailing list ([EMAIL PROTECTED]) by October 29. +++ Performance Metrics at Other Layers (pmol) == Current Status: Proposed Working Group WG Chairs: TBD Operations and Management Area: Dan Romascanu [EMAIL PROTECTED] Ronald Bonica [EMAIL PROTECTED] Description: The successful implementation and operation of IP based applications often depends on some underlying performance measurement infrastructure that helps service operators or network managers to recognize when performance is unsatisfactory and identify problems affecting service quality. Standardized performance metrics add the desirable features of consistent implementation, interpretation, no comparison. The IETF has two Working Groups dedicated to the development of performance metrics however each has strict limitations in their charters: - The Benchmarking Methodology WG has addressed a range of networking technologies and protocols in their long history (such as IEEE 802.3, ATM, Frame Relay, and Routing Protocols), but the charter strictly limits their Performance characterizations to the laboratory environment. - The IP Performance Metrics WG has the mandate to develop metrics applicable to the performance of Internet data delivery, but it is specifically prohibited from developing metrics that characterize traffic (such as a VoIP stream). The IETF also has current and completed activities related to the reporting of application performance metrics (e.g. RAQMON and RTCP XR) and is also actively involved in the development of reliable transport protocols which would affect the relationship between IP performance and application performance. Thus there is a gap in the currently chartered coverage of IETF WGs: development of performance metrics for IP-based protocols and applications that operate over UDP, TCP, SCTP, DCCP, Forward Error Correction (FEC) and other robust transport protocols, and that can be used to characterize traffic on live networks. The working group will focus on the completion of two RFCs: 1. A PMOL framework and guidelines memo that will describe the necessary elements of performance metrics of protocols and applications transported over IETF-specified protocols (such as the formal definition, purpose, and units of measure) and the various types of metrics that characterize traffic on live networks (such as metrics derived from other metrics, possibly on lower layers). The framework will also address the need to specify the intended audience and the motivation for the performance metrics. There will also be guidelines for a performance metric development process that includes entry criteria for new proposals (how a proposal might be evaluated for possible endorsement by a protocol development working group), and how an successful proposal will be developed by PMOL WG in cooperation with a protocol development WG. 2. A proof-of-concept RFC defining performance metrics for SIP, based on draft-malas-performance-metrics. This memo would serve as an example of the
