Re: PS Characterization Clarified
On 18 sep. 2013, at 01:54, John C Klensin john-i...@jck.com wrote: Pete, I generally agree with your changes and consider them important -- the IESG should be seen in our procedural documents as evaluating and reflecting the consensus of the IETF, not acting independently of it. Agreed…. Of the various places in the document in which IESG now appears, only one of them should, IMO, even be controversial. It is tied up with what I think is going on in your exchange with Scott: --On Tuesday, September 17, 2013 18:10 -0500 Pete Resnick presn...@qti.qualcomm.com wrote: Section 2: ... the IESG strengthened its review ... The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. I believe that change would be factually incorrect Which part of the above do you think is factually incorrect? The issue here --about which I mostly agree with Scott but still believe your fix is worth making-- is that the impetus for the increased and more intense review, including imposing a number of requirements that go well beyond those of 2026, did not originate in the community but entirely within the IESG. It didn't necessarily originate with explicit decisions. In many cases, it started with an AD taking the position that, unless certain changes were made or things explained to his (or occasionally her) satisfaction, the document would rot in the approval process. Later IESG moves to enable overrides and clarify conditions for discuss positions can be seen as attempts to remedy those abuses but, by then, it was too late for Proposed Standard. And, fwiw, those changes originated within the IESG and were not really subject to a community consensus process either. However, because the document will be read externally, I prefer that it be IETF in all of the places you identify. If we have to hold our noses and claim that the community authorized the IESG actions by failing to appeal or to recall the entire IESG, that would be true if unfortunate. I would not like to see anything in this document that appears to authorize IESG actions or process changes in the future that are not clearly authorized by community consensus regardless of how we interpret what happened in the past. But one of the things that we should try to maintain in making that change is the notion that the IESG does have a almost key-role in doing technical review. You made the point that that is an important distinction between 'us' and formal SDOs. Therefore I propose that that last occurrence reads: cross-area technical review performed by the IETF, exemplified by technical review by the full IESG at last stage of specification development. I think that this language doesn't set precedence and doesn't prescribe how the review is done, only that the IESG does do review. In full context: In fact, the IETF review is more extensive than that done in other SDOs owing to the cross-area technical review performed by the IETF,exemplified by technical review by the full IESG at last stage of specification development. That position is further strengthened by the common presence of interoperable running code and implementation before publication as a Proposed Standard. Does that work? --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
John covered why I said that Pete's assertion is factually incorrect that said, I agree that being accurate here (that the IESG is the final decider and the body that changed the review from what was described in RFC 2026) may be counter productive when the document is reviewed outside of the IETF so changing most of the IESG reverences to read IETF is the right thing to do the one that should not be changed is the one that Olaf calls out at the end of the included message Scott On Sep 18, 2013, at 4:59 AM, Olaf Kolkman o...@nlnetlabs.nl wrote: On 18 sep. 2013, at 01:54, John C Klensin john-i...@jck.com wrote: Pete, I generally agree with your changes and consider them important -- the IESG should be seen in our procedural documents as evaluating and reflecting the consensus of the IETF, not acting independently of it. Agreed…. Of the various places in the document in which IESG now appears, only one of them should, IMO, even be controversial. It is tied up with what I think is going on in your exchange with Scott: --On Tuesday, September 17, 2013 18:10 -0500 Pete Resnick presn...@qti.qualcomm.com wrote: Section 2: ... the IESG strengthened its review ... The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. I believe that change would be factually incorrect Which part of the above do you think is factually incorrect? The issue here --about which I mostly agree with Scott but still believe your fix is worth making-- is that the impetus for the increased and more intense review, including imposing a number of requirements that go well beyond those of 2026, did not originate in the community but entirely within the IESG. It didn't necessarily originate with explicit decisions. In many cases, it started with an AD taking the position that, unless certain changes were made or things explained to his (or occasionally her) satisfaction, the document would rot in the approval process. Later IESG moves to enable overrides and clarify conditions for discuss positions can be seen as attempts to remedy those abuses but, by then, it was too late for Proposed Standard. And, fwiw, those changes originated within the IESG and were not really subject to a community consensus process either. However, because the document will be read externally, I prefer that it be IETF in all of the places you identify. If we have to hold our noses and claim that the community authorized the IESG actions by failing to appeal or to recall the entire IESG, that would be true if unfortunate. I would not like to see anything in this document that appears to authorize IESG actions or process changes in the future that are not clearly authorized by community consensus regardless of how we interpret what happened in the past. But one of the things that we should try to maintain in making that change is the notion that the IESG does have a almost key-role in doing technical review. You made the point that that is an important distinction between 'us' and formal SDOs. Therefore I propose that that last occurrence reads: cross-area technical review performed by the IETF, exemplified by technical review by the full IESG at last stage of specification development. I think that this language doesn't set precedence and doesn't prescribe how the review is done, only that the IESG does do review. In full context: In fact, the IETF review is more extensive than that done in other SDOs owing to the cross-area technical review performed by the IETF,exemplified by technical review by the full IESG at last stage of specification development. That position is further strengthened by the common presence of interoperable running code and implementation before publication as a Proposed Standard. Does that work? --Olaf
Re: PS Characterization Clarified
--On Wednesday, September 18, 2013 10:59 +0200 Olaf Kolkman o...@nlnetlabs.nl wrote: However, because the document will be read externally, I prefer that it be IETF in all of the places you identify. If we have to hold our noses and claim that the community authorized the IESG actions by failing to appeal or to recall the entire IESG, that would be true if unfortunate. I would not like to see anything in this document that appears to authorize IESG actions or process changes in the future that are not clearly authorized by community consensus regardless of how we interpret what happened in the past. ... But one of the things that we should try to maintain in making that change is the notion that the IESG does have a almost key-role in doing technical review. You made the point that that is an important distinction between 'us' and formal SDOs. It doesn't affect the document but can we adjust or vocabulary and thinking to use, e.g., more traditional rather than formal. There is, IMO, too little that we do that is informal any more, but that isn't the point. Therefore I propose that that last occurrence reads: ... I think that this language doesn't set precedence and doesn't prescribe how the review is done, only that the IESG does do review. ... In full context: In fact, the IETF review is more extensive than that done in other SDOs owing to the cross-area technical review performed by the IETF,exemplified by technical review by the full IESG at last stage of specification development. That position is further strengthened by the common presence of interoperable running code and implementation before publication as a Proposed Standard. Does that work? The new sentence does work and is, IMO, excellent. I may be partially responsible for the first sentence but, given other comments, suggest that you at least insert some so that it ends up being ...more extensive than that done in some other SDOs owing That makes it a tad less combative and avoids a potentially-contentious argument about counterexamples. The last sentence is probably ok although, if we were to do an actual count, I'd guess that the fraction of Proposed Standards for which implemented and interoperability-tested conforming running code exists at the time of approval is somewhat less than common. john
Re: PS Characterization Clarified
On 9/18/13 7:19 AM, John C Klensin wrote: In full context: In fact, the IETF review is more extensive than that done in other SDOs owing to the cross-area technical review performed by the IETF,exemplified by technical review by the full IESG at last stage of specification development. That position is further strengthened by the common presence of interoperable running code and implementation before publication as a Proposed Standard. Does that work? The new sentence does work and is, IMO, excellent. I may be partially responsible for the first sentence but, given other comments, suggest that you at least insert some so that it ends up being ...more extensive than that done in some other SDOs owing That makes it a tad less combative and avoids a potentially-contentious argument about counterexamples. All sounds good to me. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: PS Characterization Clarified
Olaf, John, Pete, I know I have more mail to process and that you've already converged. I just wanted to say something about this: draft Proposed rewrite While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly communicated in the document prefereably on the front page of the RFC e.g. in the introduction or a separate statement. I like it much better than earlier versions in the discussion. The review that we perform for documents is not really just the IESG review. There is a plenty of WG and IETF Last Call and directorate and specialist and AD review going on… Secondly, for a long time the preferred IESG approach to saying something about the document's applicability and status has been to put the words in the document itself, abstract, introduction, separate section…. as opposed to an IESG statement. And we'd really like WGs and authors to be the source of those words, rather than the IESG. Of course there may be special situations that demand an IESG statement, but that is another matter. and Pete wrote later: I would like to change IESG to IETF in five places: And given my explanation above, I fully agree with his suggestion. And I see that you seem to have converged on that change, too. Good. Jari
Re: PS Characterization Clarified
Based on the conversation below I converged to: t While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly and prominently communicated in the document e.g. in the abstract, the introduction, or a separate section or statement. /t I will be publishing version 02 shortly. --Olaf [This is a top-post, nothing but context below] On 16 sep. 2013, at 18:04, John C Klensin john-i...@jck.com wrote: --On Monday, September 16, 2013 10:43 -0400 Barry Leiba barryle...@computer.org wrote: ... I agree that we're normally requiring much more of PS documents than we used to, and that it's good that we document that and let external organizations know. At the same time, we are sometimes proposing things that we know not to be fully baked (some of these came out of the sieve, imapext, and morg working groups, for example), but we *do* want to propose them as standards, not make them Experimental. I want to be sure we have a way to continue to do that. The text Olaf proposes is, I think, acceptable for that. In case it wasn't clear, I have no problems with that at all. I was objecting to three things that Olaf's newer text has fixed: (1) It is a very strong assertion to say that the above is exceptional. In particular, exceptional would normally imply a different or supplemental approval process to make the exception. If all that is intended is to say that we don't do it very often, then commonly (Olaf's term), usually, or perhaps even normally are better terms. (2) While it actually may be the common practice, I have difficulty with anything that reinforces the notion that the IESG makes standardization decisions separate from IETF consensus. While it isn't current practice either, I believe that, were the IESG to actually do that in an area of significance, it would call for appeals and/or recalls. Olaf's older text implied that the decision to publish a not-fully-mature or incomplete specification was entirely an IESG one. While the text in 2026, especially taken out of context, is no better (and Olaf just copied the relevant bits), I have a problem with any action that appears to reinforce that view or to grant the IESG authority to act independently of the community. (3) As a matter of policy and RFCs of editorially high quality, I think it is better to have explanations of loose ends and not-fully-baked characteristics of standards integrated into the document rather than using IESG Statements. I don't think Olaf's new front page requirement is correct (although I can live with it) -- I'd rather just say clearly and prominently communicated in the document and leave the is it clear and prominent enough question for Last Call -- but don't want to see it _forced_ into an IESG statement. I do note that front page and Introduction are typically inconsistent requirements (header + abstract + status and copyright boilerplate + TOC usually force the Introduction to the second or third page). More important, if a real explanation of half-baked features (and why they aren't fully baked) may require a section, or more than one, on it own. One would normally like a cross reference to those sections in the Introduction and possibly even mention in the Abstract, but forcing the text into the Introduction (even with preferably given experience with how easily that turns into a nearly-firm requirement) is just a bad idea in a procedures document. We should say clearly, prominently, or both and then leave specifics about what that means to conversations between the authors, the IESG and community, and the RFC Editor. best, john signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
On 16 sep. 2013, at 17:31, John C Klensin john-i...@jck.com wrote: As actionable for this draft I take that I explicitly mention that Section 4.1 2026 is exclusively updated. While I understand your desire to keep this short, the pragmatic reality is that your non-IETF audience is likely to read this document (especially after you hand it to them) and conclude that it is the whole story. Since the natural question that immediately follows why should we accept your standards at all is why can't you hand them off to, e.g., ISO, the way that many national bodies and organizations like IEEE do with many of their documents. Suggestion in the interest of brevity: in addition to mentioning the above, mention explicitly that there are requirements in other sections of 2026 that affect what is standardized and how. Second paragraph of the introduction now reads: This document exclusively updates the characterization of Proposed Standards from RFC2026 Section 4.1.1 and does not speak to or alter the procedures for the maintenance of Standards Track documents from RFC 2026 and xref target=RFC6410RFC 6410/xref. For complete understanding of the requirements for standardization those documents should be read in conjunction with this document. By the way, while I understand all of the reasons why we don't want to actually replace 2026 (and agree with most of them), things are getting to the point that it takes far too much energy to actually figure out what the rules are. Perhaps it is time for someone to create an unofficial redlined version of 2026 that incorporates all of the changes and put it up on the web somewhere. I think we would want a clear introduction and disclaimer that it might be be exactly correct and that only the RFCs are normative, but the accumulation of changes may otherwise be taking us too far into the obscure. If we need a place to put it, it might be a good appendix to the Tao. And constructing it might be a good job for a relative newcomer who is trying to understand the ins and outs of our formal procedures. I guess this is a call for volunteers. --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
On Tue, Sep 17, 2013 at 10:47 AM, Olaf Kolkman o...@nlnetlabs.nl wrote: Based on the conversation below I converged to: t While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly and prominently communicated in the document e.g. in the abstract, the introduction, or a separate section or statement. /t I read John's message as being against the use of the phrase in exceptional cases. I would also like to avoid that; it suggests that some exceptional argument may have to be made, and has the implication that it essentially operates outside the process. I would prefer the less formidable-sounding on occasion, which still implies relative rarity. Dave.
Re: PS Characterization Clarified
On 17/09/2013 11:32, Dave Cridland wrote: On Tue, Sep 17, 2013 at 10:47 AM, Olaf Kolkman o...@nlnetlabs.nl mailto:o...@nlnetlabs.nl wrote: Based on the conversation below I converged to: t While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly and prominently communicated in the document e.g. in the abstract, the introduction, or a separate section or statement. /t I read John's message as being against the use of the phrase in exceptional cases. I would also like to avoid that; it suggests that some exceptional argument may have to be made, and has the implication that it essentially operates outside the process. I would prefer the less formidable-sounding on occasion, which still implies relative rarity. +1.
Re: PS Characterization Clarified
On Sep 17, 2013 6:33 AM, Dave Cridland d...@cridland.net wrote: On Tue, Sep 17, 2013 at 10:47 AM, Olaf Kolkman o...@nlnetlabs.nl wrote: Based on the conversation below I converged to: t While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly and prominently communicated in the document e.g. in the abstract, the introduction, or a separate section or statement. /t I read John's message as being against the use of the phrase in exceptional cases. I would also like to avoid that; it suggests that some exceptional argument may have to be made, and has the implication that it essentially operates outside the process. I would prefer the less formidable-sounding on occasion, which still implies relative rarity. Dave. Exceptions and arguments for and against are part of the process. Having a process with no consideration for exceptions would be exceptional.
Re: PS Characterization Clarified
--On Tuesday, September 17, 2013 11:47 +0200 Olaf Kolkman o...@nlnetlabs.nl wrote: Based on the conversation below I converged to: t While less mature specifications will usually be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly and prominently communicated in the document e.g. in the abstract, the introduction, or a separate section or statement. /t I suggest that communicated in the document e.g. in... now essentially amounts to ... communicated in the document, e.g. in the document. since the examples span the entire set of possibilities. Consequently, for editorial reasons and in the interest of brevity, I recommend just stopping after prominently communicated in the document.. But, since the added words are not harmful, I have no problem with your leaving them if you prefer. john
Re: PS Characterization Clarified
--On Tuesday, September 17, 2013 11:32 +0100 Dave Cridland d...@cridland.net wrote: I read John's message as being against the use of the phrase in exceptional cases. I would also like to avoid that; it suggests that some exceptional argument may have to be made, and has the implication that it essentially operates outside the process. Exactly. I would prefer the less formidable-sounding on occasion, which still implies relative rarity. And on occasion is at least as good or better than my suggestions of usually, commonly, or normally although I think any of the four would be satisfactory. --On Tuesday, September 17, 2013 07:06 -0400 Scott Brim scott.b...@gmail.com wrote: ... Exceptions and arguments for and against are part of the process. Having a process with no consideration for exceptions would be exceptional. Scott, it an IETF technical context, I'd completely agree, although some words like consideration for edge cases would be much more precise if that is actually what you are alluding to. But part of the intent of this revision to 2026 is to make what we are doing more clear to outsiders who are making a good-faith effort to understand us and our standards. In that context, what you say above, when combined with Olaf's text, is likely to be read as: We regularly, and as a matter of course, consider waiving our requirements for Proposed Standard entirely and adopt specifications using entirely different (and undocumented) criteria. That is misleading at best. In the interest of clarity, I don't think we should open the door that sort of interpretation if we can avoid it. I don't think it belongs in this document (it is adequately covered by Olaf's new text about other sections), but it is worth remembering that we do have a procedure for making precisely the type of exceptions my interpretation above implies: the Variance Procedure of Section 9.1 of 2026. I cannot remember that provision being invoked since 2026 was published -- it really is exceptional in that sense. Its existence may be another reason for removing exceptional from the proposed new text because it could be read as implying that we have to use the Section 9.1 procedure for precisely the cases of a well-documented, but slightly incomplete, that most of us consider normal. In particular, it would make the approval of the specs that Barry cited in his examples invalid without invoking the rather complex procedure of Section 9.1. I'd certainly not like to have text in this update that encourages that interpretation and the corresponding appeals -- it would create a different path to the restriction Barry is concerned about. john
Re: PS Characterization Clarified
FYI. I just posted the third version of the draft at: http://tools.ietf.org/html/draft-kolkman-proposed-standards-clarified-02 Diff with the previous document: http://tools.ietf.org/rfcdiff?url2=draft-kolkman-proposed-standards-clarified-02.txt I will let Jari know that I believe we converged and that as far as I am concerned its last call time. --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
On 9/17/13 11:27 AM, Olaf Kolkman wrote: I just posted the third version of the draft at: http://tools.ietf.org/html/draft-kolkman-proposed-standards-clarified-02 I would like to change IESG to IETF in five places: Section 1: the IESG has evolved its review processes Section 2: IESG Reveiew of Proposed Standards the IESG strengthened its review last chance for the IESG to ensure the quality cross-area technical review performed by the IESG The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. Saying the IESG in these places implies precedent setting that I think would be bad. If the IETF capitulated to the IESG changing the rules on its own in the past, so be it, but I think it would be bad to indicate in a BCP that we think it's OK for the IESG to do so unilaterally. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: PS Characterization Clarified
1/ I believe that change would be factually incorrect 2/ I do not see that being factually correct about what happened says anything about the community opinion about any future IESG decision to change processes. Scott On Sep 17, 2013, at 6:48 PM, Pete Resnick presn...@qti.qualcomm.com wrote: On 9/17/13 11:27 AM, Olaf Kolkman wrote: I just posted the third version of the draft at: http://tools.ietf.org/html/draft-kolkman-proposed-standards-clarified-02 I would like to change IESG to IETF in five places: Section 1: the IESG has evolved its review processes Section 2: IESG Reveiew of Proposed Standards the IESG strengthened its review last chance for the IESG to ensure the quality cross-area technical review performed by the IESG The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. Saying the IESG in these places implies precedent setting that I think would be bad. If the IETF capitulated to the IESG changing the rules on its own in the past, so be it, but I think it would be bad to indicate in a BCP that we think it's OK for the IESG to do so unilaterally. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: PS Characterization Clarified
On 9/17/13 5:52 PM, Scott O. Bradner wrote: On Sep 17, 2013, at 6:48 PM, Pete Resnickpresn...@qti.qualcomm.com wrote: I would like to change IESG to IETF in five places: Section 1: the IESG has evolved its review processes Section 2: IESG Reveiew of Proposed Standards the IESG strengthened its review last chance for the IESG to ensure the quality cross-area technical review performed by the IESG The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. I believe that change would be factually incorrect Which part of the above do you think is factually incorrect? pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: PS Characterization Clarified
Pete, I generally agree with your changes and consider them important -- the IESG should be seen in our procedural documents as evaluating and reflecting the consensus of the IETF, not acting independently of it. Of the various places in the document in which IESG now appears, only one of them should, IMO, even be controversial. It is tied up with what I think is going on in your exchange with Scott: --On Tuesday, September 17, 2013 18:10 -0500 Pete Resnick presn...@qti.qualcomm.com wrote: Section 2: ... the IESG strengthened its review ... The IETF as a whole, through directorate reviews, area reviews, doctor reviews, *and* IESG reviews, has evolved, strengthened, ensured, etc., its reviews. I believe that change would be factually incorrect Which part of the above do you think is factually incorrect? The issue here --about which I mostly agree with Scott but still believe your fix is worth making-- is that the impetus for the increased and more intense review, including imposing a number of requirements that go well beyond those of 2026, did not originate in the community but entirely within the IESG. It didn't necessarily originate with explicit decisions. In many cases, it started with an AD taking the position that, unless certain changes were made or things explained to his (or occasionally her) satisfaction, the document would rot in the approval process. Later IESG moves to enable overrides and clarify conditions for discuss positions can be seen as attempts to remedy those abuses but, by then, it was too late for Proposed Standard. And, fwiw, those changes originated within the IESG and were not really subject to a community consensus process either. However, because the document will be read externally, I prefer that it be IETF in all of the places you identify. If we have to hold our noses and claim that the community authorized the IESG actions by failing to appeal or to recall the entire IESG, that would be true if unfortunate. I would not like to see anything in this document that appears to authorize IESG actions or process changes in the future that are not clearly authorized by community consensus regardless of how we interpret what happened in the past. john
Re: PS Characterization Clarified
On 13 sep. 2013, at 21:02, Adrian Farrel adr...@olddog.co.uk wrote: Hey Olaf, Thanks for stubbornly pushing on with this. Comments (sorry I haven't read the thread to see if others have already made these comments)… This is to acknowledge I took the suggestions that I am not quoting. --- Section 2 clarity of the standards document Prefer Standards Track document This is a change to the original 2026 language, but I support it it makes really clear what we are taking about. --- Section 2 over the last decade or more have had extensive review. ...by the IESG? ...by or on behalf of the IESG? That is already explained in the paragraphs above. I propose to just keep the focus on the document having had review and not make an addition to that sentence. On the other hand, elsewhere in the thread John Klensin made the case that we should stress that the IESG does this quality assurance since that is different from other SDOs, this might be a good place to add that extra emphasis. As said, I keep the text as is for now but I will take editorial guidance if this is a serious issue. --- Section 2 Because of this change in review assumptions, IETF Proposed Standards should be considered to be at least as mature as final standards from other standards development organizations. In fact, the IETF review is more extensive than is done in other SDOs due to the cross-area technical review performed by the IESG. I wonder whether you should add ...a position that is further strengthened by the implementation and running code that is often present before publication as a Proposed Standard. Added, but with the word Interoperable added. Could you review if this is still proper English and contact me off-list if it isn't: In fact, the IETF review is more extensive than is done in other SDOs due to the cross-area technical review performed by the IESG, a position that is further strengthened by the regular precense of interoperable implementation and running code before publication as a Proposed Standard. --- Section 3.1 Usually, neither implementation nor operational experience is required for the designation of a specification as a Proposed Standard. However, such experience is highly desirable, and will usually represent a strong argument in favor of a Proposed Standard designation. You could add ... and may be tracked and reported as described in [RFC6982] Hmmm.. I want to be careful with that reference. It is an experimental document and increases the referential complexity for little benefit. Until serious pushback otherwise I propose to not take over this suggestion. --- Section 3.1 Two paragraphs seem to enjoy some duplication in their final sentences. A Proposed Standard specification is stable, has resolved known design choices, is well-understood, has received significant community review, and appears to enjoy enough community interest to be considered valuable. However, as with all technical standards, further experience might result in a change or even retraction of the specification in the future. and A Proposed Standard will have no known technical omissions with respect to the requirements placed upon it. Proposed Standards are of such quality that implementations can be deployed in the Internet. However, as with all technical specifications, Proposed Standards may be revised if problems are found or better solutions are identified, when experiences with deploying implementations of such technologies at scale is gathered. Fixed by removing that final sentence at first occurrence. I'll wait a few days to see discussion converge before posting 02 in the mean time: pre 02 is at: http://www.secret-wg.org/Secret-Media/draft-kolkman-proposed-standards-clarified-pre02-2013091600.txt with a diff at: http://tools.ietf.org/rfcdiff?url1=http://tools.ietf.org/id/draft-kolkman-proposed-standards-clarified-01.txturl2=http://www.secret-wg.org/Secret-Media/draft-kolkman-proposed-standards-clarified-pre02-2013091600.txt Thanks, --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
[Barry added explicitly to the CC as this speaks to 'his' issue] On 13 sep. 2013, at 20:57, John C Klensin klen...@jck.com wrote: [… skip …] * Added the Further Consideration section based on discussion on themailinglist. Unfortunately, IMO, it is misleading to the extent that you are capture existing practice rather than taking us off in new directions. Yeah it is a thin line. But the language was introduced to keep a current practice possible (as argued by Barry I believe). You wrote: While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that does not match the characterizations above as a Proposed Standard. In those cases that fact will be clearly communicated on the front page of the RFC e.g. means of an IESG statement. On the one hand, I can't remember when the IESG has published something as a Proposed Standard with community consensus and with an attached IESG statement that says that they and the community had to hold our collective noses, but decided to approve as PS anyway. Because, at least in theory, a PS represents community consensus, not just IESG consensus (see below), I would expect (or at least hope for) an immediate appeal of an approval containing such as statement unless it (the statement itself, not just the opinion) matched community consensus developed during Last Call. Conversely, the existing rules clearly allow a document to be considered as a Proposed Standard that contains a paragraph describing loose ends and points of fragility, that expresses the hope that the cases won't arise very often and that a future version will clarify how the issues should be handled based on experience. That is no known technical omissions since the issues are identified and therefore known and not omissions. In the current climate, I'd expect such a document to have a very hard time on Last Call as people argued for Experimental or even keeping it as an I-D until all of the loose ends were tied up. But, if there were rough consensus for approving it, I'd expect it to be approved without any prefatory, in-document, IESG notes (snarky or otherwise). The above may or may not be tied up with the generally stable terminology. I could see a spec with explicit this is still uncertain and, if we are wrong, might change language in it on the same basis as the loose end description above. Such language would be consistent with generally stable but, since it suggests a known point of potential instability, it is not consistent with stable. I see where you are going. draft Proposed rewrite While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly communicated in the document prefereably on the front page of the RFC e.g. in the introduction or a separate statement. /draft I hope that removing the example of the IESG statement makes clear that this is normally part of the development process. Additional observations based on mostly-unrelated recent discussions: If you are really trying to clean 2026 up and turn the present document into something that can be circulated to other groups without 2026 itself, then the change control requirement/ assumption of RFC 2026 Section 7.1.3 needs to be incorporated into your new Section 3. It is not only about internal debates, it is our rule against why we can't just endorse a standard developed elsewhere as an IETF standards track specification. Along the same lines but more broadly, both the sections of 2026 you are replacing and your new text, if read in isolation, strongly imply that these are several decisions, including those to approve standardization, that the IESG makes on its own judgment and discretion. I think it is fairly clear from the rest of 2026 (and 2028 and friends and IETF oral tradition) that the IESG is a collector and interpreter of community consensus, not a body that is someone delegated to use its own judgment. I believe that, if an IESG were ever to say something that amounted to the community consensus is X, but they are wrong, so we are selecting or approving not-X, we would either see a revolution of the same character that brought us to 2026 or the end of the IETF's effectiveness as a broadly-based standards body. More important --and related to some of my comments that you deferred to a different discussion-- the IESG as final _technical_ review and interpreter of consensus model is very different from that in some other SDOs in which the final approval step is strictly a procedural and/or legal review that is a consensus review only in the sense of
Re: PS Characterization Clarified
Yeah it is a thin line. But the language was introduced to keep a current practice possible (as argued by Barry I believe). Yes, that was my concern. I see where you are going. draft Proposed rewrite While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly communicated in the document prefereably on the front page of the RFC e.g. in the introduction or a separate statement. /draft I hope that removing the example of the IESG statement makes clear that this is normally part of the development process. I think that version is just fine too, with the typo in preferably corrected. John, to give you the context again: I agree that we're normally requiring much more of PS documents than we used to, and that it's good that we document that and let external organizations know. At the same time, we are sometimes proposing things that we know not to be fully baked (some of these came out of the sieve, imapext, and morg working groups, for example), but we *do* want to propose them as standards, not make them Experimental. I want to be sure we have a way to continue to do that. The text Olaf proposes is, I think, acceptable for that. Barry
Re: PS Characterization Clarified
--On Monday, September 16, 2013 15:58 +0200 Olaf Kolkman o...@nlnetlabs.nl wrote: [Barry added explicitly to the CC as this speaks to 'his' issue] On 13 sep. 2013, at 20:57, John C Klensin klen...@jck.com wrote: [… skip …] * Added the Further Consideration section based on discussion on themailinglist. Unfortunately, IMO, it is misleading to the extent that you are capture existing practice rather than taking us off in new directions. Yeah it is a thin line. But the language was introduced to keep a current practice possible (as argued by Barry I believe). Understood. Barry and I are on the same page wrt not wanting to accidentally restrict established existing practices. You wrote: While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in ... I see where you are going. draft Proposed rewrite While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that still contains areas for improvement or certain uncertainties about whether the best engineering choices are made. In those cases that fact will be clearly communicated in the document prefereably on the front page of the RFC e.g. in the introduction or a separate statement. /draft I hope that removing the example of the IESG statement makes clear that this is normally part of the development process. Yes. Editorial nits: * While commonly less mature specifications will be published... has commonly qualifying less mature. It is amusing to think about what that might mean, but it isn't what you intended. Try While less mature specifications will usually be published Replace usually with commonly or normally if you like, but I think usually is closest to what you are getting at. * prefereably - preferably Additional observations based on mostly-unrelated recent discussions: If you are really trying to clean 2026 up and turn the present document into something that can be circulated to other groups without 2026 itself, then the change control requirement/ ... Along the same lines but more broadly, both the sections of 2026 you are replacing and your new text, if read in isolation, strongly imply that these are several decisions, including those to approve standardization, that the IESG makes on its own judgment and discretion. I think it is ... More important --and related to some of my comments that you deferred to a different discussion-- the IESG as final _technical_ review and interpreter of consensus model is very different from that in some other SDOs in which the final approval step is strictly a procedural and/or legal review that is a consensus review only in the sense of verifying ... So noted. As actionable for this draft I take that I explicitly mention that Section 4.1 2026 is exclusively updated. While I understand your desire to keep this short, the pragmatic reality is that your non-IETF audience is likely to read this document (especially after you hand it to them) and conclude that it is the whole story. Since the natural question that immediately follows why should we accept your standards at all is why can't you hand them off to, e.g., ISO, the way that many national bodies and organizations like IEEE do with many of their documents. Suggestion in the interest of brevity: in addition to mentioning the above, mention explicitly that there are requirements in other sections of 2026 that affect what is standardized and how. By the way, while I understand all of the reasons why we don't want to actually replace 2026 (and agree with most of them), things are getting to the point that it takes far too much energy to actually figure out what the rules are. Perhaps it is time for someone to create an unofficial redlined version of 2026 that incorporates all of the changes and put it up on the web somewhere. I think we would want a clear introduction and disclaimer that it might be be exactly correct and that only the RFCs are normative, but the accumulation of changes may otherwise be taking us too far into the obscure. If we need a place to put it, it might be a good appendix to the Tao. And constructing it might be a good job for a relative newcomer who is trying to understand the ins and outs of our formal procedures. best, john
Re: PS Characterization Clarified
--On Monday, September 16, 2013 10:43 -0400 Barry Leiba barryle...@computer.org wrote: ... I agree that we're normally requiring much more of PS documents than we used to, and that it's good that we document that and let external organizations know. At the same time, we are sometimes proposing things that we know not to be fully baked (some of these came out of the sieve, imapext, and morg working groups, for example), but we *do* want to propose them as standards, not make them Experimental. I want to be sure we have a way to continue to do that. The text Olaf proposes is, I think, acceptable for that. In case it wasn't clear, I have no problems with that at all. I was objecting to three things that Olaf's newer text has fixed: (1) It is a very strong assertion to say that the above is exceptional. In particular, exceptional would normally imply a different or supplemental approval process to make the exception. If all that is intended is to say that we don't do it very often, then commonly (Olaf's term), usually, or perhaps even normally are better terms. (2) While it actually may be the common practice, I have difficulty with anything that reinforces the notion that the IESG makes standardization decisions separate from IETF consensus. While it isn't current practice either, I believe that, were the IESG to actually do that in an area of significance, it would call for appeals and/or recalls. Olaf's older text implied that the decision to publish a not-fully-mature or incomplete specification was entirely an IESG one. While the text in 2026, especially taken out of context, is no better (and Olaf just copied the relevant bits), I have a problem with any action that appears to reinforce that view or to grant the IESG authority to act independently of the community. (3) As a matter of policy and RFCs of editorially high quality, I think it is better to have explanations of loose ends and not-fully-baked characteristics of standards integrated into the document rather than using IESG Statements. I don't think Olaf's new front page requirement is correct (although I can live with it) -- I'd rather just say clearly and prominently communicated in the document and leave the is it clear and prominent enough question for Last Call -- but don't want to see it _forced_ into an IESG statement. I do note that front page and Introduction are typically inconsistent requirements (header + abstract + status and copyright boilerplate + TOC usually force the Introduction to the second or third page). More important, if a real explanation of half-baked features (and why they aren't fully baked) may require a section, or more than one, on it own. One would normally like a cross reference to those sections in the Introduction and possibly even mention in the Abstract, but forcing the text into the Introduction (even with preferably given experience with how easily that turns into a nearly-firm requirement) is just a bad idea in a procedures document. We should say clearly, prominently, or both and then leave specifics about what that means to conversations between the authors, the IESG and community, and the RFC Editor. best, john
Re: PS Characterization Clarified
Colleagues [I have added a number of people who were active in the discussion previously to the CC, my apologies if that is bad etiquette but I wanted to make you explicitly aware of this.] Based on the discussion so far I've made a few modifications to the draft. I am trying to consciously keep this document to the minimum that is needed to achieve 'less is more' and my feeling is that where we are now is close to the sweetspot of consensus. This is the summary of these changes: * Added Updates 2026 and added Sean's initial * Copied the whole characterization pararaph for Internet Standards from 2026, instead of only the line that is the actual characterization itself. * Added the Further Consideration section based on discussion on the mailinglist. See: http://www.ietf.org/id/draft-kolkman-proposed-standards-clarified-01.txt For diff: http://tools.ietf.org/rfcdiff?url2=draft-kolkman-proposed-standards-clarified-01.txt --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
Hi Olaf, At 07:56 13-09-2013, Olaf Kolkman wrote: Based on the discussion so far I've made a few modifications to the draft. I am trying to consciously keep this document to the minimum that is needed to achieve 'less is more' and my feeling is that where we are now is close to the sweetspot of consensus. The intended status would have to be BCP instead of Informational. In Section 3.1: A specific action by the IESG is required to move a specification onto the standards track at the Proposed Standard level. I suggest standards instead of specific action if you (and the other authors) decide that BCP is appropriate. The last paragraph in Section 3.1 is okay. I don't think that Section 4 is necessary. Please note that I do not have a strong opinion about this. I leave it to your discretion. The two references in Section 7 would have to be normative references. I have reason to believe that you mean it when you say we document what we do. draft-kolkman-proposed-standards-clarified-01 proposes that the IETF does that and I think that it is a fine idea. Regards, S. Moonesamy
Re: PS Characterization Clarified
Based on the discussion so far I've made a few modifications to the draft. I am trying to consciously keep this document to the minimum that is needed to achieve 'less is more' and my feeling is that where we are now is close to the sweetspot of consensus. Section 4 makes me happy. I think it's ready to go. Barry
Re: PS Characterization Clarified
On 13 sep. 2013, at 19:17, S Moonesamy sm+i...@elandsys.com wrote: The intended status would have to be BCP instead of Informational. Correct…. fixed on trunk. In Section 3.1: A specific action by the IESG is required to move a specification onto the standards track at the Proposed Standard level. I suggest standards instead of specific action if you (and the other authors) decide that BCP is appropriate. I have used exactly the same term as RFC2026. I have no idea if 'standards action' is defined somewhere. The two references in Section 7 would have to be normative references. Yes. (see PS) Thanks, and best, --Olaf PS. I think this is xml2rfc playing up. The xml contains this: back references title='Normative References' rfc2026; rfc6410; /references section title=Acknowledgements ….. But it seems to not want to translate . If anybody has suggestions, off-list please. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
On Sep 13, 2013, at 2:32 PM, Olaf Kolkman o...@nlnetlabs.nl wrote: On 13 sep. 2013, at 19:17, S Moonesamy sm+i...@elandsys.com wrote: The intended status would have to be BCP instead of Informational. Correct…. fixed on trunk. In Section 3.1: A specific action by the IESG is required to move a specification onto the standards track at the Proposed Standard level. I suggest standards instead of specific action if you (and the other authors) decide that BCP is appropriate. I have used exactly the same term as RFC2026. I have no idea if 'standards action' is defined somewhere. I do not think we should move away from the ted used in RFC 2026 Scott
Re: PS Characterization Clarified
On 13 sep. 2013, at 20:03, Carsten Bormann c...@tzi.org wrote: On Sep 13, 2013, at 16:56, Olaf Kolkman o...@nlnetlabs.nl wrote: * Added the Further Consideration section based on discussion on the mailinglist. I believe the current document is fine for a major part of the IETF standards activities. It is, however, important to keep in mind that the IETF is not a homogeneous organization, not even within each of the quite different areas. Section 4 seems to try to open up the straightjacket created by section 3 a little bit again, but the way it does this is probably the wrong approach. Note this is not trying to change… I is trying to document what we do now. On https://www.iab.org/wp-content/IAB-uploads/2011/04/Bormann.pdf I am trying to see what one gets if one translates the fallacies into positive actions, or answer the question on how do you cope with the fallacy. I notice that your draft observes but doesn't seem to recommend. That is not a value judgement on the text btw but it doesn't give me insight in if 'this is probably wrong' what is the right way? And more important, is there any indication we can get there? I believe we had several tries in doing something different (better, was the intention of all those that took part in that debate) but we never reached consensus. That is why this is not trying to change, but tries to document the realities. Have a nice weekend. --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
On Sep 13, 2013, at 16:56, Olaf Kolkman o...@nlnetlabs.nl wrote: * Added the Further Consideration section based on discussion on the mailinglist. I believe the current document is fine for a major part of the IETF standards activities. It is, however, important to keep in mind that the IETF is not a homogeneous organization, not even within each of the quite different areas. Section 4 seems to try to open up the straightjacket created by section 3 a little bit again, but the way it does this is probably the wrong approach. Instead of writing a long diatribe here, let me just point to section 2.7 of https://www.iab.org/wp-content/IAB-uploads/2011/04/Bormann.pdf (see the references to the other sections there, too). Grüße, Carsten
Re: PS Characterization Clarified
On Sep 13, 2013, at 20:50, Olaf Kolkman o...@nlnetlabs.nl wrote: I am trying to see what one gets if one translates the fallacies into positive actions, or answer the question on how do you cope with the fallacy. I notice that your draft observes but doesn't seem to recommend. Indeed, the 2011 document is heavy on diagnosis, and very light on therapy. The hope was that Selbsterkenntnis ist der erste Schritt zur Besserung (self-awareness is the first step to improvement), so it tries to create some of that awareness. How to go from personal awareness to organizational awareness is indeed the question*). Documenting the status quo *does* change the status quo, by making it more rigid. (Unless one also documents why exactly that wasn't intended.) But my main point is that Proposed Standard is turning into a Procrustes bed, and fixing that will require more work than section 4 seems to want making one believe. Grüße, Carsten *) (Going through the process of turning that 2011 workshop submission into a consensus RFC might be one step towards that, and I apologize for not having attempted that yet.)
Re: PS Characterization Clarified
--On Friday, September 13, 2013 16:56 +0200 Olaf Kolkman o...@nlnetlabs.nl wrote: ... Based on the discussion so far I've made a few modifications to the draft. I am trying to consciously keep this document to the minimum that is needed to achieve 'less is more' and my feeling is that where we are now is close to the sweetspot of consensus. Olaf, I'm afraid I need to keep playing loyal opposition here. * Added the Further Consideration section based on discussion on themailinglist. Unfortunately, IMO, it is misleading to the extent that you are capture existing practice rather than taking us off in new directions. You wrote: While commonly less mature specifications will be published as Informational or Experimental RFCs, the IETF may, in exceptional cases, publish a specification that does not match the characterizations above as a Proposed Standard. In those cases that fact will be clearly communicated on the front page of the RFC e.g. means of an IESG statement. On the one hand, I can't remember when the IESG has published something as a Proposed Standard with community consensus and with an attached IESG statement that says that they and the community had to hold our collective noses, but decided to approve as PS anyway. Because, at least in theory, a PS represents community consensus, not just IESG consensus (see below), I would expect (or at least hope for) an immediate appeal of an approval containing such as statement unless it (the statement itself, not just the opinion) matched community consensus developed during Last Call. Conversely, the existing rules clearly allow a document to be considered as a Proposed Standard that contains a paragraph describing loose ends and points of fragility, that expresses the hope that the cases won't arise very often and that a future version will clarify how the issues should be handled based on experience. That is no known technical omissions since the issues are identified and therefore known and not omissions. In the current climate, I'd expect such a document to have a very hard time on Last Call as people argued for Experimental or even keeping it as an I-D until all of the loose ends were tied up. But, if there were rough consensus for approving it, I'd expect it to be approved without any prefatory, in-document, IESG notes (snarky or otherwise). The above may or may not be tied up with the generally stable terminology. I could see a spec with explicit this is still uncertain and, if we are wrong, might change language in it on the same basis as the loose end description above. Such language would be consistent with generally stable but, since it suggests a known point of potential instability, it is not consistent with stable. Additional observations based on mostly-unrelated recent discussions: If you are really trying to clean 2026 up and turn the present document into something that can be circulated to other groups without 2026 itself, then the change control requirement/ assumption of RFC 2026 Section 7.1.3 needs to be incorporated into your new Section 3. It is not only about internal debates, it is our rule against why we can't just endorse a standard developed elsewhere as an IETF standards track specification. Along the same lines but more broadly, both the sections of 2026 you are replacing and your new text, if read in isolation, strongly imply that these are several decisions, including those to approve standardization, that the IESG makes on its own judgment and discretion. I think it is fairly clear from the rest of 2026 (and 2028 and friends and IETF oral tradition) that the IESG is a collector and interpreter of community consensus, not a body that is someone delegated to use its own judgment. I believe that, if an IESG were ever to say something that amounted to the community consensus is X, but they are wrong, so we are selecting or approving not-X, we would either see a revolution of the same character that brought us to 2026 or the end of the IETF's effectiveness as a broadly-based standards body. More important --and related to some of my comments that you deferred to a different discussion-- the IESG as final _technical_ review and interpreter of consensus model is very different from that in some other SDOs in which the final approval step is strictly a procedural and/or legal review that is a consensus review only in the sense of verifying that the process in earlier stages followed the consensus rules and is not technical review at all. I don't think you need to spend time on that, but you need to avoid things that would make your document misleading to people who start with that model of how standards are made as an initial assumption. best, john
Re: PS Characterization Clarified
The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. Wouldn't such spec come with an applicability statement of sorts? (today, in practice?) That's a good point; probably yes. So if the text here can say something that allows a PS spec to *say* that it's less mature, and that that's being done on purpose, my concern is satisfied. It says that IETF PS specs are at least as mature as final standards from other SDOs. Mostly, that's true. But it doesn't have to be. After this, it would have to be, always, for every PS spec. Are we *sure* that's what we want? This draft is mostly targeted to document what we do, not what we want. Although I can see how you want to keep the door open. As a specific current example, I have the sense that the documents coming out of the repute working group are specifying a protocol that's somewhat less mature than what we usually do -- more comparable to the 2026 version of PS than to this one. Yet I absolutely think they should be PS, *not* Experimental. Barry
Re: PS Characterization Clarified
OK, somebody has to say it. Maybe we should have another state, something like draft standard. [ sob alluded to a private message from me which said ] while i really like the idea of pushing well-tested interoperable documents to full standards, i think tested interop is key here. hence i liked the old interop tests for ds and am not all that happy to see it go (yes, i whined privately to russ). our protocols are rarely based on any testable formalism, computer science, mathematics, ... we live on a pile of crap hacks. interop is about all that keeps us from entropic degeneration. randy
Re: PS Characterization Clarified
On 9/4/2013 11:14 AM, Scott Brim wrote: On Wed, Sep 4, 2013 at 10:34 AM, Barry Leiba barryle...@computer.org wrote: The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. Wouldn't such spec come with an applicability statement of sorts? (today, in practice?) That's a good point; probably yes. So if the text here can say something that allows a PS spec to *say* that it's less mature, and that that's being done on purpose, my concern is satisfied. Not the spec itself but an associated statement about it? There was a proposal to provide an alternative way of publishing applicability statements, in http://tools.ietf.org/html/draft-ietf-newtrk-repurposing-isd-04 (now expired). As a specific current example, I have the sense that the documents coming out of the repute working group are specifying a protocol that's somewhat less mature than what we usually do -- more comparable to the 2026 version of PS than to this one. Yet I absolutely think they should be PS, *not* Experimental. OK, somebody has to say it. Maybe we should have another state, something like draft standard. There were a couple of proposals to provide a way of saying the working group thinks they're finished, but lets hold off on PS until they see how the protocol works. The one I co-authored was at http://tools.ietf.org/html/draft-dawkins-newtrk-wgs-00, Scott Bradner's was at http://tools.ietf.org/html/draft-bradner-ietf-stds-trk-00, in Section 5. Both are now expired. Perhaps those drafts would be helpful background for folks thinking about this? (especially for folks who were too busy doing protocol work to follow NEWTRK? :-) Spencer
Re: PS Characterization Clarified
On Wed, Sep 4, 2013 at 10:34 AM, Barry Leiba barryle...@computer.org wrote: The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. Wouldn't such spec come with an applicability statement of sorts? (today, in practice?) That's a good point; probably yes. So if the text here can say something that allows a PS spec to *say* that it's less mature, and that that's being done on purpose, my concern is satisfied. Not the spec itself but an associated statement about it? As a specific current example, I have the sense that the documents coming out of the repute working group are specifying a protocol that's somewhat less mature than what we usually do -- more comparable to the 2026 version of PS than to this one. Yet I absolutely think they should be PS, *not* Experimental. OK, somebody has to say it. Maybe we should have another state, something like draft standard.
Re: PS Characterization Clarified
On 2 sep. 2013, at 22:14, John C Klensin john-i...@jck.com wrote: --On Monday, 02 September, 2013 14:09 -0400 Scott O Bradner s...@sobco.com wrote: There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more? seems like a quite bad idea (as Randy points out) take extra effort and get some interoperability data More than that. Unless we want to deserve the credibility problems we sometimes accuse others of having, nothing should be a full standard, no matter how popular, unless it reflects good engineering practice. I think there is more flexibility for Proposed Standards, especially if they come with commentary or applicability statements, but I believe that, in general, the community should consider bad design or bad engineering practice to fall into the known defect category of RFC 2026. If RFC 6410 requires, or even allows, that we promote things merely because they are popular, then I suggest there is something seriously wrong with it. John, +1 All, In fact, going back to the language of RFC2026 for Full (now Internet) Standard. It confirms that popularity (significant implementation) is one necessary but not sufficient criterium. 4.1.3 Internet Standard A specification for which significant implementation and successful operational experience has been obtained may be elevated to the Internet Standard level. An Internet Standard (which may simply be referred to as a Standard) is characterized by a high degree of technical maturity and by a generally held belief that the specified protocol or service provides significant benefit to the Internet community. I would hope that any concerns about technical maturity or significant benefit to the Internet community are taken into account when making the decision. If it is the case that members of the community assess that a specification lacks interoperability that should be sufficient grounds to not advance until data proofs otherwise. And for what its worth. One of the concerns most seen are those of IPR. The stamp of Internet Standard is a confirmation of the community that any IPR on the specification can be death with, that is an important piece of information on layer 9. = On a more generic note. The reason I took initiative for this draft is mainly because I believe we need to do what we document and document what we do. As discussed in this thread the practice for the approval of PS has changed, the bar is much higher than 20 years ago. In this case it is good that we document what we do. That shouldn't be a motivation to not do what we document: namely be serious about the maintenance of our standards. And I would hope that somehow we as a community find the energy needed to advance our specification in a way that truly assesses the requirements of RFC2026 sect 4.1.3 * significant implementation * successful operational experience * technical maturity * significant benefit to the Internet community. --Olaf signature.asc Description: Message signed with OpenPGP using GPGMail
Re: PS Characterization Clarified
+1. Well said.
Re: PS Characterization Clarified
Olaf, John, Scott, In fact, going back to the language of RFC2026 for Full (now Internet) Standard. It confirms that popularity (significant implementation) is one necessary but not sufficient criterium. Sorry. I was careless when I wrote about the effort. I didn't mean to suggest that we have an effort to classify standards merely based on popularity. What I meat that we have an effort to move a particular set of specifications to Internet Standard, and will use the usual criteria when deciding whether the documents are ready. While that particular set of specifications happens to be popular, that was just an observation, not a (sole) reason of moving them forward. Hope this clarifies. I would hope that any concerns about technical maturity or significant benefit to the Internet community are taken into account when making the decision. If it is the case that members of the community assess that a specification lacks interoperability that should be sufficient grounds to not advance until data proofs otherwise. Yes, of course. Jari
Re: PS Characterization Clarified
thank you - clarity does help but such an effort will not remove the need for this document imo Scott On Sep 3, 2013, at 9:20 AM, Jari Arkko jari.ar...@piuha.net wrote: Olaf, John, Scott, In fact, going back to the language of RFC2026 for Full (now Internet) Standard. It confirms that popularity (significant implementation) is one necessary but not sufficient criterium. Sorry. I was careless when I wrote about the effort. I didn't mean to suggest that we have an effort to classify standards merely based on popularity. What I meat that we have an effort to move a particular set of specifications to Internet Standard, and will use the usual criteria when deciding whether the documents are ready. While that particular set of specifications happens to be popular, that was just an observation, not a (sole) reason of moving them forward. Hope this clarifies. I would hope that any concerns about technical maturity or significant benefit to the Internet community are taken into account when making the decision. If it is the case that members of the community assess that a specification lacks interoperability that should be sufficient grounds to not advance until data proofs otherwise. Yes, of course. Jari
Re: PS Characterization Clarified
On Sep 3, 2013, at 4:40 PM, Barry Leiba barryle...@computer.org wrote: The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? It seems as if we already do this. It's not unusual to publish things as experimental when we don't think they're baked, and that seems entirely appropriate. The fact is that anything that has PS on it at this point has had _very_ thorough review. Frequently PS have multiple implementations. I think that what you are describing is actually the status quo—it would be very hard to get something really half-baked published as PS at this juncture. That doesn't mean that every PS is high quality, but neither are they half-baked.
Re: PS Characterization Clarified
fwiw - I would love for the IESG to exercise flexibility here but I have not seen that in many years - so I think we are already there without a discernible path back Scott On Sep 3, 2013, at 4:40 PM, Barry Leiba barryle...@computer.org wrote: I mostly agree with this draft, but I have a concern. Let's anchor that concern off of this bit that Jari said: Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. It's certainly clear to all of us that most PS specs are far more mature than what we thought about when we wrote RFC 2026. The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. It says that IETF PS specs are at least as mature as final standards from other SDOs. Mostly, that's true. But it doesn't have to be. After this, it would have to be, always, for every PS spec. Are we *sure* that's what we want? Barry
Re: PS Characterization Clarified
I mostly agree with this draft, but I have a concern. Let's anchor that concern off of this bit that Jari said: Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. It's certainly clear to all of us that most PS specs are far more mature than what we thought about when we wrote RFC 2026. The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. It says that IETF PS specs are at least as mature as final standards from other SDOs. Mostly, that's true. But it doesn't have to be. After this, it would have to be, always, for every PS spec. Are we *sure* that's what we want? Barry
Re: PS Characterization Clarified
Barry, Question, in-line. On Sep 3, 2013, at 10:40 PM, Barry Leiba barryle...@computer.org wrote: I mostly agree with this draft, but I have a concern. Let's anchor that concern off of this bit that Jari said: Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. It's certainly clear to all of us that most PS specs are far more mature than what we thought about when we wrote RFC 2026. The only concern I have is that once we do this -- declare that PS is always more mature than that -- we can't go back. Do we *really* want to say that we will never again approve a PS spec that's partially baked? This is painting us into the room where PS is mature and robust. If we like being in that room, that's fine. But it removes the IESG can put fuzzy stuff out as PS if it thinks that's the right thing to do option. Wouldn't such spec come with an applicability statement of sorts? (today, in practice?) It says that IETF PS specs are at least as mature as final standards from other SDOs. Mostly, that's true. But it doesn't have to be. After this, it would have to be, always, for every PS spec. Are we *sure* that's what we want? This draft is mostly targeted to document what we do, not what we want. Although I can see how you want to keep the door open. --Olaf.
Re: PS Characterization Clarified
Olaf, Scott, Apologies for a late reply on this (I was on vacation after the IETF). But thank you for writing this draft. My general comment is that the draft makes what in my mind is an accurate correction to our documents, aligning the documents to the current reality. I'd be happy to take the document forward. In fact, I think we need to make this change even if we made other, more long term changes. There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more? Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. But enough about my opinions. What do the rest of you think? In terms of specific text, I also wrote a few observations, below. These are purely personal comments. First, I think you assumed this but never made it explicit. While the new characterisation recognises the often final role of PS RFCs, it does not take away the possibility of publishing Internet Standard specifications. Can this be clarified? In the two decades after publication of RFC 2026 [RFC202] the IESG has evolved its review processes of Proposed Standard RFCs and thus RFC 2026 section 4.1.1 no longer accurately describes IETF Proposed Standards. I'd prefer saying the IETF review processes Proposed Standard RFCs have evolved. And leave the details to Section 2. 2. IESG Reveiew of Proposed Standards Review In response, the IESG strengthened its review of Proposed Standards, basically operating as if the Proposed Standard was the last chance for the IESG to ensure the quality of the technology and the clarity of the standards document. That is part of it, but I think the situation is more complicated than that. The world changed around us, and suddenly Internet was big business, global, and we got more careful about impacts to it. The process has evolved, including the number of steps in the ladder. Review practices in general have changed quite a lot, we now have a fairly broad review of RFCs. Progression has also varied, mostly downwards. But as noted, it also seems very much affected by specific initiatives. Here's what I'd say: Initially it was assumed that most IETF technical specifications would progress through a series of maturity stages starting with a relatively early Proposed Standard, then progressing to Draft Standard then, finally, to Internet Standard (see RFC 2026 section 6). Over time, for a number of reasons, this progression became less common. At the same time, the review for Proposed Standard RFCs was strengthened. This strengthening was partially a response by the IESG for the above, and in part a consequence of the growth in the importance of the Internet and broader interest in reviewing new Internet technology. At the time of this writing, the IETF operates as if the Proposed Standard was the last chance for the to ensure the quality of the technology and the clarity of the standards document. The result is that IETF Proposed Standards approved over the last decade or more have had extensive review. Because of this change in review assumptions, IETF Proposed Standards should be considered to be at least as mature as final standards from other standards development organizations. In fact, the IETF review is more extensive than is done in other SDOs due to the cross-area technical review performed in the IETF. Jari
Re: PS Characterization Clarified
On Sep 2, 2013, at 10:23 AM, Jari Arkko jari.ar...@piuha.net wrote: Olaf, Scott, Apologies for a late reply on this (I was on vacation after the IETF). But thank you for writing this draft. My general comment is that the draft makes what in my mind is an accurate correction to our documents, aligning the documents to the current reality. I'd be happy to take the document forward. In fact, I think we need to make this change even if we made other, more long term changes. There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more? seems like a quite bad idea (as Randy points out) take extra effort and get some interoperability data Secondly, the other obvious action we could take is to go back to the original mode of operation, i.e., making PS RFCs truly early and somewhat untested specifications. I am personally opposed to that on the following grounds. First, it would not change the fact that a large part of Internet technology today runs on PS RFCs, and Olaf's problem with getting these RFCs recognised would continue. Second, while I think we need to keep adjusting the level of review performed by the IESG and in IETF Last Call (we sometimes overdo it), I think broad review is actually useful. imo - not a chance in hell of the IESG going back to the original meaning of PS it is not in the IESG genetics, nor has it been for quite a while But enough about my opinions. What do the rest of you think? In terms of specific text, I also wrote a few observations, below. These are purely personal comments. First, I think you assumed this but never made it explicit. While the new characterisation recognises the often final role of PS RFCs, it does not take away the possibility of publishing Internet Standard specifications. Can this be clarified? In the two decades after publication of RFC 2026 [RFC202] the IESG has evolved its review processes of Proposed Standard RFCs and thus RFC 2026 section 4.1.1 no longer accurately describes IETF Proposed Standards. I'd prefer saying the IETF review processes Proposed Standard RFCs have evolved. And leave the details to Section 2. 2. IESG Reveiew of Proposed Standards Review In response, the IESG strengthened its review of Proposed Standards, basically operating as if the Proposed Standard was the last chance for the IESG to ensure the quality of the technology and the clarity of the standards document. That is part of it, but I think the situation is more complicated than that. The world changed around us, and suddenly Internet was big business, global, and we got more careful about impacts to it. The process has evolved, including the number of steps in the ladder. Review practices in general have changed quite a lot, we now have a fairly broad review of RFCs. Progression has also varied, mostly downwards. But as noted, it also seems very much affected by specific initiatives. Here's what I'd say: Initially it was assumed that most IETF technical specifications would progress through a series of maturity stages starting with a relatively early Proposed Standard, then progressing to Draft Standard then, finally, to Internet Standard (see RFC 2026 section 6). Over time, for a number of reasons, this progression became less common. At the same time, the review for Proposed Standard RFCs was strengthened. This strengthening was partially a response by the IESG for the above, and in part a consequence of the growth in the importance of the Internet and broader interest in reviewing new Internet technology. At the time of this writing, the IETF operates as if the Proposed Standard was the last chance for the to ensure the quality of the technology and the clarity of the standards document. The result is that IETF Proposed Standards approved over the last decade or more have had extensive review. Because of this change in review assumptions, IETF Proposed Standards should be considered to be at least as mature as final standards from other standards development organizations. In fact, the IETF review is more extensive than is done in other SDOs due to the cross-area technical review performed in the IETF. wfm Jari
Re: PS Characterization Clarified
Hi Jari, On 03/09/2013 02:23, Jari Arkko wrote: ... At the time of this writing, the IETF operates as if the Proposed Standard was the last chance for the to ensure the quality of the technology and the clarity of the standards document. There's a point that I think should be made here, something like: In practice, interoperable implementations are commonly based on Proposed Standard documents, so whatever design defects those documents have tend to become part of the interoperable network, perhaps in the form of work-arounds. Similarly, in today's Internet, any security defects tend to be exploited at an early stage. Fixing design and security issues in widely deployed code may be difficult or impossible in practice. Therefore, there is now very strong pressure to make the Proposed Standard as mature as possible, rather than being just good enough to meet the RFC 2026 requirements. The result is that IETF Proposed Standards approved over the last decade or more have had extensive review. Exactly. Brian
Re: PS Characterization Clarified
--On Monday, 02 September, 2013 14:09 -0400 Scott O Bradner s...@sobco.com wrote: There is at least one ongoing effort right now that has the potential to reclassify a large set of Proposed Standard RFCs that form the basis of widely used technology. These types of efforts can have a relatively big effect on the standards status of the most commonly used RFCs. Do we want to do more? Can we do more? seems like a quite bad idea (as Randy points out) take extra effort and get some interoperability data More than that. Unless we want to deserve the credibility problems we sometimes accuse others of having, nothing should be a full standard, no matter how popular, unless it reflects good engineering practice. I think there is more flexibility for Proposed Standards, especially if they come with commentary or applicability statements, but I believe that, in general, the community should consider bad design or bad engineering practice to fall into the known defect category of RFC 2026. If RFC 6410 requires, or even allows, that we promote things merely because they are popular, then I suggest there is something seriously wrong with it. john
Re: PS Characterization Clarified
There's a point that I think should be made here, something like: In practice, interoperable implementations are commonly based on Proposed Standard documents, so whatever design defects those documents have tend to become part of the interoperable network, perhaps in the form of work-arounds. Similarly, in today's Internet, any security defects tend to be exploited at an early stage. Fixing design and security issues in widely deployed code may be difficult or impossible in practice. Therefore, there is now very strong pressure to make the Proposed Standard as mature as possible, rather than being just good enough to meet the RFC 2026 requirements. Yes, that is pretty accurate. Jari
