Re: Work effort? (Re: Proposed Standard and Perfection)
On 13 Mar 2004, at 4:21 pm, John C Klensin wrote: snip (1) In your example above, I suggest that, with the amount to scrutiny now going into getting things to Proposed, the types of problems you identify above would be detected there and dealt with. And, if they were not, we have a more serious problem than criteria for draft (in today's world, not an ideal one). (2) We need to look at the system as it exists, not in how things might work if, e.g., everything got the perfect levels of review at the perfect time. If we do that, our situation today is that documents go to Proposed, with whatever quality of (typically pre-implementation) review they get. They get published. Now, consider what might happen next, in an environment in which, for whatever reason (and I think both your observation about implementation reports and mine about document-fussing and general pain are part of the problem), we move almost nothing to Draft... snip To which I reply: Hi John, folks, When I pass my driving test, the Examiner doesn't consider me to be a good driver; merely that I'm unlikely to kill someone so readily. I had thought that this was the analogy for Proposed, but this hasn't been my experience; protocol quality is often considered before we get that far. Given the amount of time we take to raise things to Proposed, *if people are serious about the protocol* then there may well be implementations out there already - a number of protocols have had interoperation tests whilst still I-Ds, simply due to the glacial nature of the process (i.e. people are overloaded) and the requirement for quality (or perfection, depending on who's saying it). Thus the suggestion that implementation reports alone are the key to moves to Draft is interesting; do we merely document the interoperation test results and (skip/fast-track past) Proposed altogether? Unfortunately, describing the rationale and operation of a protocol is not easy, and IMHO one of the main drivers for issuing a new RFC is to clarify things that were understood differently by different implementation teams - for example with ROHC, the first interoperation test exposed that just about everyone interworked, but everyone had got the bit order wrong on the checksum as they interpreted the text that way. Note - this isn't a mistake in the protocol, so putting this into the errata collection is not appropriate. It is however a gotcha, so an Implementer's Guide (i.e. when it says this, they mean that) is useful. It's this documentation step that is needed for some poor sot who has to use this stuff, and it isn't clear to me that it's the main driver for further steps up the standards track. A statement that this feature in section 12.3 and that feature in section 13.4 interworked between implementation X and implementation Y may be true, but not necessarily complete if someone doesn't understand what's in section 12.3 or 13.4 in the same way. Didn't this organisation used to be about defining protocols *and* experience through using them? all the best, Lawrence
Re: Work effort? (Re: Proposed Standard and Perfection)
Maybe I'm confused but, as I understand it, standards track level is already, in principle, completely decoupled from write and publish an RFC in that the standards level is not incorporated in the RFC anywhere but listed separately. In general, I agree with John Klensin as to what are considered the real barriers to advancement, adding that the usual reason a new RFC is required is that some small part/option of a Proposed Standards does not have multiple implementations or if it does they don't interoperabe and people have decided that part/option is of so little use this is not worth fixing. Thus, under our current rules, a new RFC is needed to drop out this more or less failed portion of the Proposed Standard. Thanks, Donald == Donald E. Eastlake 3rd [EMAIL PROTECTED] 155 Beaver Street +1-508-634-2066(h) +1-508-786-7554(w) Milford, MA 01757 USA [EMAIL PROTECTED] On Fri, 12 Mar 2004, John C Klensin wrote: Date: Fri, 12 Mar 2004 12:49:02 -0500 From: John C Klensin [EMAIL PROTECTED] ... (1) We decouple change maturity levels from write and publish an RFC (this means that the listing of maturity levels must be current and authoritative). This also meshes nicely with another proposal that was floated a few years ago, but never really written up (see below) As far a I know all of the above is true. ...
Re: Work effort? (Re: Proposed Standard and Perfection)
--On Friday, 12 March, 2004 20:19 -0500 Keith Moore [EMAIL PROTECTED] wrote: (2) When a document comes up for review for Proposed-Draft, we look for implementations, etc., perhaps following Keith's proposal outline. If the implementations are there, we issue a Last Call for identification of serious technical/definitional flaws in the document as published, where serious is defined as problematic enough to interfere with independent interoperable implementations. If none are found, we advance the maturity level of the document, in place -- no new RFC publication required and hence little or no ... I cannot support the notion that the only technical flaws that should prevent a document from advancing from Proposed to Draft are those that interfere with independent interoperable implementations. A protocol may interoperate quite well with itself, and yet have a peripheral effect on a large number of other protocols. A protocol that does not share channel capacity fairly with TCP can have a disruptive effect on TCP-based protocols. A protocol that defines a new kind of (IPv4 or IPv6) address space that behaves differently from existing address space can break applications even if that protocol does not directly interact with those applications. A protocol which doesn't provide adequate authentication can cause its hosts to be compromised which in turn affects the operation and security of other services. ... Keith, You are quite correct. I oversimplified the situation considerably. Clearly, any significant technical or interoperability difficulty that shows up should be considered and should be a showstopper for something going to Draft. But consider: (1) In your example above, I suggest that, with the amount to scrutiny now going into getting things to Proposed, the types of problems you identify above would be detected there and dealt with. And, if they were not, we have a more serious problem than criteria for draft (in today's world, not an ideal one). (2) We need to look at the system as it exists, not in how things might work if, e.g., everything got the perfect levels of review at the perfect time. If we do that, our situation today is that documents go to Proposed, with whatever quality of (typically pre-implementation) review they get. They get published. Now, consider what might happen next, in an environment in which, for whatever reason (and I think both your observation about implementation reports and mine about document-fussing and general pain are part of the problem), we move almost nothing to Draft... * The Proposed Standard is implemented, causes no problems, turns out to be clear. No one cares about moving it to Draft, so it stays at Proposed. * Flaws are discovered. Our only real mechanism for documenting them is to reissue at Proposed or move the document to Draft (the RFC Editor's Errata are not IETF-authoritative and few people know where to find them). But nothing goes to Draft, so the document sits there at Proposed, warts and all. * There are some other cases, but few result in the documents getting recycled, and fewer result in them going to Draft. Not a good situation. In fact, I think it is a sufficiently bad one that we should be looking at ways that would make things even slightly better in even a subset of cases than figuring what problems those changes would not fix in a perfect world. That, and it's too easy for technical flaws to be missed at Proposed Standard that show up after some implementation and/or deployment experience is gained - and these aren't at all limited to flaws that interfere with interoperation. Sure. Do you have a proposal as to how to get those flaws documented and the documents appropriately reissued? If not, this is an IETF is going downhill, news at 11 sort of observation. I do agree that we should not reopen noncontroversial documents that aren't found to have significant flaws. I have occasionally suggested that rather than re-edit documents advancing in grade, we start out by making lists of things that need to be changed. If that list ends up being less than N pages or X% of the length of the original specification, publish the list with a preface that says RFC , with the following clarifications, changes, and corrections, is approved for Draft Standard. We are in general agreement about this, and it is part of what my turn STDs into real documents idea was intended to address. OTOH, some documents that were controversial at Proposed probably do deserve special scrutiny at Draft. That doesn't mean that we should automatically reopen them, but it might be that given the subsequent experience with the document it no longer meets the criteria for Proposed (e.g. no known technical omissions). Yes. I
Re: Work effort? (Re: Proposed Standard and Perfection)
--On Monday, 08 March, 2004 13:26 -0500 Keith Moore [EMAIL PROTECTED] wrote: It's all well and good to try to retire Proposed Standard documents that don't get implemented. But I think it's even more important to make it easier for documents that do meet the criteria to advance to Draft Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Interesting. I would have described the hardest part as lying in the documentation process itself, in particular... * When a document has been very controversial in getting to Proposed, at least partially because of very loud and continuing objections about specific points from those who disagreed with them, it is often hard for authors, editors, [former?] WG Chairs, etc., to get up the energy to deal with what is likely to be a renewed version of the unpleasantness. * When one or more IESG members have engaged in extensive, time-consuming, and unpleasant nit-picking or bickering about provisions of the original, Proposed, document, authors/ editors/ WG Chairs may have the reasonable expectation that they would do an even better (more extensive) job on a draft coming up for Draft. and * Documents that are, themselves, perfectly ready to advance get stalled indefinitely because they make normative references to documents or protocols that, for one reason or another, less ready. The discussion below does not address the third point; I don't know quite what to do about it. Note that none of these issues is intrinsically related to the protocol quality or deployment of the specification. What the first two share with the implementation report issue is that the documentation issues are independent of the underlying reality. Hence this modest proposal: - For each standards-track document, create a web page that is used to keep track of bug reports, errata, implementation reports, and test reports. ... My not-very-modest proposal (many details would need filling in, but, as an idea...): (1) We decouple change maturity levels from write and publish an RFC (this means that the listing of maturity levels must be current and authoritative). This also meshes nicely with another proposal that was floated a few years ago, but never really written up (see below) (2) When a document comes up for review for Proposed-Draft, we look for implementations, etc., perhaps following Keith's proposal outline. If the implementations are there, we issue a Last Call for identification of serious technical/definitional flaws in the document as published, where serious is defined as problematic enough to interfere with independent interoperable implementations. If none are found, we advance the maturity level of the document, in place -- no new RFC publication required and hence little or no opportunity to reopen old wounds that lack demonstrated interoperability impact. If the document is about to time out in grade, we issue a Last Call, wait a reasonable period for protests and volunteers, and then downgrade it in place. The notion of having to write an RFC, following all of today's complex rules, and get formal consensus in order to declare a Protocol obsolete that isn't implemented and won't operate in today's environment is one of the more astonishing triumphs of bureaucracy over rationality. Similar rules would apply to Draft-Full (or whatever formula newtrk comes up with). (3) If a document is judged defective (which is difficult from judging a protocol defective), we try to find someone to fix it. If a plan (and volunteer(s)) cannot be readily found, we publish a description of the defect(s), presumably as an RFC, but, if that is unworkable, in some other way. Minimize the fuss -- if our customers don't think an updated document is worth the trouble --enough trouble to invest people, dollars for professional editing, or whatever else is required-- then we should stop losing sleep over it. Principle: If we are going to spend as much time and energy as we do getting something to Proposed, then moving it to Draft or Full should usually require only identification of implementations, deployment, and desirability, not extensive and time-consuming document rewriting john -- Footnote -- that other proposal: Some time ago, I noticed that the relationship between the STD definitions and the underlying RFCs had gotten a little too complicated for anyone to usefully understand. By being reserved to full standards, the
Re: Work effort? (Re: Proposed Standard and Perfection)
John, I think the things you describe have very many of the same ideals and targets as draft-loghney-what-standards, currently being discussed in newtrk, which still needs work and significant input to be converted from an idea to a workable process - we may have a rare case of singing in harmony here! Harald
Re: Work effort? (Re: Proposed Standard and Perfection)
(2) When a document comes up for review for Proposed-Draft, we look for implementations, etc., perhaps following Keith's proposal outline. If the implementations are there, we issue a Last Call for identification of serious technical/definitional flaws in the document as published, where serious is defined as problematic enough to interfere with independent interoperable implementations. If none are found, we advance the maturity level of the document, in place -- no new RFC publication required and hence little or no opportunity to reopen old wounds that lack demonstrated interoperability impact. If the document is about to time out in grade, we issue a Last Call, wait a reasonable period for protests and volunteers, and then downgrade it in place. The notion of having to write an RFC, following all of today's complex rules, and get formal consensus in order to declare a Protocol obsolete that isn't implemented and won't operate in today's environment is one of the more astonishing triumphs of bureaucracy over rationality. Similar rules would apply to Draft-Full (or whatever formula newtrk comes up with). John, I cannot support the notion that the only technical flaws that should prevent a document from advancing from Proposed to Draft are those that interfere with independent interoperable implementations. A protocol may interoperate quite well with itself, and yet have a peripheral effect on a large number of other protocols. A protocol that does not share channel capacity fairly with TCP can have a disruptive effect on TCP-based protocols. A protocol that defines a new kind of (IPv4 or IPv6) address space that behaves differently from existing address space can break applications even if that protocol does not directly interact with those applications. A protocol which doesn't provide adequate authentication can cause its hosts to be compromised which in turn affects the operation and security of other services. That, and it's too easy for technical flaws to be missed at Proposed Standard that show up after some implementation and/or deployment experience is gained - and these aren't at all limited to flaws that interfere with interoperation. I do agree that we should not reopen noncontroversial documents that aren't found to have significant flaws. I have occasionally suggested that rather than re-edit documents advancing in grade, we start out by making lists of things that need to be changed. If that list ends up being less than N pages or X% of the length of the original specification, publish the list with a preface that says RFC , with the following clarifications, changes, and corrections, is approved for Draft Standard. OTOH, some documents that were controversial at Proposed probably do deserve special scrutiny at Draft. That doesn't mean that we should automatically reopen them, but it might be that given the subsequent experience with the document it no longer meets the criteria for Proposed (e.g. no known technical omissions). Principle: If we are going to spend as much time and energy as we do getting something to Proposed, then moving it to Draft or Full should usually require only identification of implementations, deployment, and desirability, not extensive and time-consuming document rewriting What if we applied the same principle to software? What if once a program were written there were an expectation that it should never be substantially changed, never be re-evaluated in light of changed conditions or new understanding? Keith
Re: Work effort? (Re: Proposed Standard and Perfection)
--On 9. mars 2004 22:46 -0600 Spencer Dawkins [EMAIL PROTECTED] wrote: I don't KNOW that what I'm thinking is true, but I'm wondering to myself if the target audience for protocol specification maintenance is all in the IETF... not all the audience for protocol specification is in the IETF, so why should we expect the audience for protocol specification maintenance to be..? the IETF works when it has *enough* of the audience, I think. But we do have an answer for those who complain that they are not part of the process: show up!. (note - never said it was an easy answer. but there is one)
Re: Work effort? (Re: Proposed Standard and Perfection)
--On 9. mars 2004 19:54 -0800 Randy Presuhn [EMAIL PROTECTED] wrote: I made the comment that I thought we should apply RFC 2026 and force things to either advance or go historic. Our AD advised us in one case that if our WG wanted one of its RFCs to go historic, we had to write another RFC explaining why. The procedure in RFC 2026 section 6.2 (last paragraph) seems very reasonable, and I like Harald's suggested approach to cleaning up the cruft. backpedalling somewhat I didn't intend to indicate that I had a suggested approach yet - there are lots of details that should be worked out before we start, such as: - who should do it? - what criteria should they use? - which way should they be biased when in doubt? - how much workload is created by doing this? - who does something about the docs that do NOT go to Historic? - how do we document the evaluations - how do we challenge them? The 2026 section 6.2 procedure, with no adjustments, says IESG, judgment about likelyhood of progressing on track, downgrade, don't care about workload, WG if existing, otherwise don't care, IETF-announce list message, appeals procedure. I don't think those are necessarily the answers that can be made to work. that's just a start some thinking about these things can probably go a long way towards reducing the number of un-intended side effects we cause Harald
Re: Work effort? (Re: Proposed Standard and Perfection)
Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Hence this modest proposal: [clip] I rather like the proposal. What's been lacking is any forum for further development of standards outside of mailing lists and IETF meetings -- there's no tracking, and that's a major problem. Rick
Re: Work effort? (Re: Proposed Standard and Perfection)
Hi - From: Spencer Dawkins [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Sunday, March 07, 2004 5:59 AM Subject: Re: Work effort? (Re: Proposed Standard and Perfection) I spent more time trying to capture what people were saying at the plenary than trying to figure out who said what, but I would like to figure out who said [06:43:24] anewton too much time needed to take something out there and take it back to historic. [06:43:44] anewton suggests steps for things to automatically go historic. [06:43:48] anewton harald. [06:43:55] --- AWGY has joined [06:44:20] anewton perhaps have someone else beside IESG do leg work. [06:44:36] anewton ??. on Thursday night - sound familiar to anyone? The last name mentioned in the logs was John Loughney, then Harald replied, and then SOMEONE said too much time needed... I'd love to find who who said this. ... I made the comment that I thought we should apply RFC 2026 and force things to either advance or go historic. Our AD advised us in one case that if our WG wanted one of its RFCs to go historic, we had to write another RFC explaining why. The procedure in RFC 2026 section 6.2 (last paragraph) seems very reasonable, and I like Harald's suggested approach to cleaning up the cruft. Randy
Re: Work effort? (Re: Proposed Standard and Perfection)
--On 8. mars 2004 12:38 -0700 Rick Stewart [EMAIL PROTECTED] wrote: Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Hence this modest proposal: [clip] I rather like the proposal. What's been lacking is any forum for further development of standards outside of mailing lists and IETF meetings -- there's no tracking, and that's a major problem. in draft-iesg-hardie-outline-01, the concept of a maintenance team (called IANA Team in that document) was floated. This didn't get much discussion. Is this something that's worth discussing as an idea? Harald
Re: Work effort? (Re: Proposed Standard and Perfection)
I don't KNOW that what I'm thinking is true, but I'm wondering to myself if the target audience for protocol specification maintenance is all in the IETF... Spencer - Original Message - From: Harald Tveit Alvestrand [EMAIL PROTECTED] To: Rick Stewart [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Tuesday, March 09, 2004 9:48 PM Subject: Re: Work effort? (Re: Proposed Standard and Perfection) --On 8. mars 2004 12:38 -0700 Rick Stewart [EMAIL PROTECTED] wrote: Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Hence this modest proposal: [clip] I rather like the proposal. What's been lacking is any forum for further development of standards outside of mailing lists and IETF meetings -- there's no tracking, and that's a major problem. in draft-iesg-hardie-outline-01, the concept of a maintenance team (called IANA Team in that document) was floated. This didn't get much discussion. Is this something that's worth discussing as an idea? Harald ___ This message was passed through [EMAIL PROTECTED], which is a sublist of [EMAIL PROTECTED] Not all messages are passed. Decisions on what to pass are made solely by IETF_CENSORED ML Administrator ([EMAIL PROTECTED]).
Re: Work effort? (Re: Proposed Standard and Perfection)
Harald, HTA In the steady state (30 docs/month, currently), perhaps 30 man-hours/month 30 documents go to Proposed each month? The steady-state rate of review is the average number of documents that go to Proposed. (well, ok, the average of the number that went to proposed 2 years ago. In any event, we can distinguish documents that newly come up to their 2-year limit, versus dusting out the closet of those that already hit 2 years, before this. Keeping up with the new documents reaching their limit is the critical-path activity. Dusting out the closet can take as long as we want. d/ -- Dave Crocker dcrocker-at-brandenburg-dot-com Brandenburg InternetWorking www.brandenburg.com Sunnyvale, CA USA tel:+1.408.246.8253
Re: Work effort? (Re: Proposed Standard and Perfection)
It's all well and good to try to retire Proposed Standard documents that don't get implemented. But I think it's even more important to make it easier for documents that do meet the criteria to advance to Draft Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Hence this modest proposal: - For each standards-track document, create a web page that is used to keep track of bug reports, errata, implementation reports, and test reports. (yes, I know about the RFC Editor's errata page - this might be a modification of that or it might be something else entirely) - Allow implementors to submit reports via a form on that web page - An implementation report would name an implementation and specify what features it implemented - A test report would, for a given set of implementations, specify which features were tested and whether they interoperated - Allow ADs to designate one or more people to review implementation reports (to eliminate duplicates and cull out bogus reports) - At adoption time + 2 years, every PS document would be Last Called for Draft Standard, for a period of 4 weeks. This would serve as: - a final notice to submit implementation reports to the web site - a final notice to submit bug reports and errata to the web site - At the end of the Last Call period the sheperding AD would review the implementation reports and bug reports and make a recommendation to IESG (similar to the AD writeup) to either: - approve document as-is - submit to author or WG for updates - recommend that the document be reclassified as historic, experimental, or informational Keith p.s. The hardest part of this (and often, the hardest part of interop testing) is defining exactly what tests are needed, especially when features interact or when there are more than two parties participating in a protocol at the same time. Ideally each PS would specify what implementation tests were needed to move the specification to DS, and these would be published along with the specification. But that will have to wait awhile...
Re: Work effort? (Re: Proposed Standard and Perfection)
Over the past couple of years, I've been involved in a W3C effort that might have some useful lessons for this discussion. The working group concerned adopted a working practice of creating test cases for any significant decision that it was required to make. One of the observations that underpinned this approach was that, given text specifying some awkward technical detail, it was usually possible for reasonable implementers to interpret it differently. Not so for test cases. The result was a process analogous to test-led software development, in which the specific test cases (concerning which it was far easier to unambiguously determine WG consensus) were used to drive the adoption of new or changed specification text. A side effect of all this was that we ended up with a suite of test cases that could be used to judge the extent to which the specification was consistently implementable. With these test cases available along with the (purported) complete specification, and compared with the long delay for IETF specs to move from proposed to draft, the process for moving from [equivalent of] Proposed to [equivalent of] Draft is relatively short ... typically a few months, it seems. With this process, a substantial element of the particular hardest part that Keith notes is put together by the working group as the specification is being developed. #g -- At 13:26 08/03/04 -0500, Keith Moore wrote: It's all well and good to try to retire Proposed Standard documents that don't get implemented. But I think it's even more important to make it easier for documents that do meet the criteria to advance to Draft Standard. In my experience the hardest part of getting a document advanced is to collect the implementation report. Hence this modest proposal: - For each standards-track document, create a web page that is used to keep track of bug reports, errata, implementation reports, and test reports. (yes, I know about the RFC Editor's errata page - this might be a modification of that or it might be something else entirely) - Allow implementors to submit reports via a form on that web page - An implementation report would name an implementation and specify what features it implemented - A test report would, for a given set of implementations, specify which features were tested and whether they interoperated - Allow ADs to designate one or more people to review implementation reports (to eliminate duplicates and cull out bogus reports) - At adoption time + 2 years, every PS document would be Last Called for Draft Standard, for a period of 4 weeks. This would serve as: - a final notice to submit implementation reports to the web site - a final notice to submit bug reports and errata to the web site - At the end of the Last Call period the sheperding AD would review the implementation reports and bug reports and make a recommendation to IESG (similar to the AD writeup) to either: - approve document as-is - submit to author or WG for updates - recommend that the document be reclassified as historic, experimental, or informational Keith p.s. The hardest part of this (and often, the hardest part of interop testing) is defining exactly what tests are needed, especially when features interact or when there are more than two parties participating in a protocol at the same time. Ideally each PS would specify what implementation tests were needed to move the specification to DS, and these would be published along with the specification. But that will have to wait awhile... ___ This message was passed through [EMAIL PROTECTED], which is a sublist of [EMAIL PROTECTED] Not all messages are passed. Decisions on what to pass are made solely by IETF_CENSORED ML Administrator ([EMAIL PROTECTED]). Graham Klyne For email: http://www.ninebynine.org/#Contact
Re: Work effort? (Re: Proposed Standard and Perfection)
Harald, In any event, we can distinguish documents that newly come up to their 2-year limit, versus dusting out the closet of those that already hit 2 years, before this. HTA I'd like to tackle both - it seems silly to have all this garbage My comment did not suggest that older Proposed documents be ignored. What I said was that they can be dealt with differently from the steady state handling required to handle new documents, as they come to their time limit. For doing project planning, there is a very big difference between on-going requirements and start-up requirements. Both urgency and resource allocation can be quite different. HTA If we have the consensus of the community that we SHOULD reclassify the HTA documents, then the dusting out is just work. The amount of work required can affect people's willingness to support doing a project. Overestimating can make the task seem more daunting than necessary and thereby reduce support. Underestimating can set unrealistic expectation and thereby cause community frustration later. For that matter, a community in crisis about timeliness, productivity, accountability and funding might wonder how a particular proposal affects any of these urgent problems, especially when the community is severely resource limited. There are many good proposals, but scarce resources mandates setting priorities. d/ -- Dave Crocker dcrocker-at-brandenburg-dot-com Brandenburg InternetWorking www.brandenburg.com Sunnyvale, CA USA tel:+1.408.246.8253
Work effort? (Re: Proposed Standard and Perfection)
it's me again. --On 4. mars 2004 10:59 -0800 Eliot Lear [EMAIL PROTECTED] wrote: We come to different conclusions here. My conclusion is that no standard should remain at proposed for more than 2 years unless it's revised. Either it goes up, it goes away, or it gets revised and goes around again. I spent some time thinking about this comment from the plenary on my way home from Seoul. (An advantage of long flights?) I don't think obsoleting as a regular procedure is a bad idea. But it will take some work to get from here to there. My musings came up with a guesstimate of 1/2 hour of work per document on the standards track that has been abandoned totally (due dilligence in figuring out that *nobody* is using it), and 4 hours of work per document on the standards track that is in use, but has no particularly interesting future and should be moved to informational - and a somewhat larger figure for the documents where there is in fact a reason to revive and revise them. Some calculations using very round numbers. We have approximately 992 Proposed documents, of which I guesstimate that about 800 documents are ready for evaluation under the 2-year rule; if we assume a 12:3:1 distribution of the three categories above, we're looking at around 600x0.5 + 150x4 = 900 man-hours to get there, if we disregard the part about the standards that deserve updating, and disregard the documents that have gotten to Draft and the full standards that deserve honorable retirement. Possibly quite a bit less once the team doing it gets into full swing, or if there are many standards for which it is easy to show that no usage/interest exists. So if we can get 9 people to work at it, and want to be up-to-date in a year, we're looking at an investment of around 100 hours per volunteer - or about 2 hours each week for a year. In the steady state (30 docs/month, currently), perhaps 30 man-hours/month could be enough - lower, because since the docs are newer, people who are asked actually remember. That doesn't look too awful people who want to volunteer for this work can contact me, and we'll see if we can figure out a way to do it Harald
Re: Work effort? (Re: Proposed Standard and Perfection)
I spent more time trying to capture what people were saying at the plenary than trying to figure out who said what, but I would like to figure out who said [06:43:24] anewton too much time needed to take something out there and take it back to historic. [06:43:44] anewton suggests steps for things to automatically go historic. [06:43:48] anewton harald. [06:43:55] --- AWGY has joined [06:44:20] anewton perhaps have someone else beside IESG do leg work. [06:44:36] anewton ??. on Thursday night - sound familiar to anyone? The last name mentioned in the logs was John Loughney, then Harald replied, and then SOMEONE said too much time needed... I'd love to find who who said this. Thanks! Spencer
Re: Work effort? (Re: Proposed Standard and Perfection)
--On 7. mars 2004 17:07 -0800 Dave Crocker [EMAIL PROTECTED] wrote: Harald, HTA In the steady state (30 docs/month, currently), perhaps 30 man-hours/month 30 documents go to Proposed each month? The steady-state rate of review is the average number of documents that go to Proposed. (well, ok, the average of the number that went to proposed 2 years ago. apologies - wrong number - I was counting documents, not standards-track docs. standards-track are slightly more than half that, I think. check the numbers from Allison's presentation - she is a more careful statistician than I am. In any event, we can distinguish documents that newly come up to their 2-year limit, versus dusting out the closet of those that already hit 2 years, before this. Keeping up with the new documents reaching their limit is the critical-path activity. Dusting out the closet can take as long as we want. I'd like to tackle both - it seems silly to have all this garbage cluttering up the history while applying exacting criteria to the ones that had the luck to be passed exactly two years ago. If we have the consensus of the community that we SHOULD reclassify the documents, then the dusting out is just work. Besides, we should learn something from the experience of going through the backlog - there have been a couple of attempts at single-document downgrading already, and at least one met with resistance from people feeling that their document was singled out for harsh treatment (RFC 1628, UPS MIB, 1994), while reclassification of RFC 1314 (Image file format, 1992) as Informational was uncontroversial when Patrik and I tried it back in 1999 or thereabouts. (Both are still listed in the index as Proposed, however) Harald
RE: Work effort? (Re: Proposed Standard and Perfection)
I think that is an excellent idea. Of course, if we keep calling everything a RFC, I do not think this will have any noticeable effect on either moving things forward or out. For example, there would still be little incentive for people to do the hard work of interoperability reports and (worse yet) finding out that they are not fully interoperable. If we don't make things that are obsolete clearly obsolete, the miscreants' marketing department will still be happy to say, our product Z is fully compliant with RFC ; most of the world will not know or care the RFC was aged out. The program works for I-D's, because I-D's disappear after six months. People would have incentive to move things from PS to DS if, for example, RFC becomes something like OBS . -Original Message- From: Harald Tveit Alvestrand [mailto:[EMAIL PROTECTED] Sent: Sunday, March 07, 2004 2:42 AM To: [EMAIL PROTECTED] Subject: Work effort? (Re: Proposed Standard and Perfection) it's me again. --On 4. mars 2004 10:59 -0800 Eliot Lear [EMAIL PROTECTED] wrote: We come to different conclusions here. My conclusion is that no standard should remain at proposed for more than 2 years unless it's revised. Either it goes up, it goes away, or it gets revised and goes around again. I spent some time thinking about this comment from the plenary on my way home from Seoul. (An advantage of long flights?) I don't think obsoleting as a regular procedure is a bad idea. But it will take some work to get from here to there. My musings came up with a guesstimate of 1/2 hour of work per document on the standards track that has been abandoned totally (due dilligence in figuring out that *nobody* is using it), and 4 hours of work per document on the standards track that is in use, but has no particularly interesting future and should be moved to informational - and a somewhat larger figure for the documents where there is in fact a reason to revive and revise them. Some calculations using very round numbers. We have approximately 992 Proposed documents, of which I guesstimate that about 800 documents are ready for evaluation under the 2-year rule; if we assume a 12:3:1 distribution of the three categories above, we're looking at around 600x0.5 + 150x4 = 900 man-hours to get there, if we disregard the part about the standards that deserve updating, and disregard the documents that have gotten to Draft and the full standards that deserve honorable retirement. Possibly quite a bit less once the team doing it gets into full swing, or if there are many standards for which it is easy to show that no usage/interest exists. So if we can get 9 people to work at it, and want to be up-to-date in a year, we're looking at an investment of around 100 hours per volunteer - or about 2 hours each week for a year. In the steady state (30 docs/month, currently), perhaps 30 man-hours/month could be enough - lower, because since the docs are newer, people who are asked actually remember. That doesn't look too awful people who want to volunteer for this work can contact me, and we'll see if we can figure out a way to do it Harald
