On 7/30/11 11:05 AM, Joel M. Halpern wrote: > It seems to me that this does two things, both small but > useful. 1) It makes a minor change in the advancement > procedures so that they are more reasonable. They may still > not be sufficiently reasonable to be used, but it improves > them, and thereby improves the odds.
Actually, there is no evidence that this is an improvement. I'd agree that it is a minor change, but there has been no serious analysis of whether it is an improvement or not. To the extent to which approving this lowers the odds of considering and making changes that might actually have significant effects (e.g., really sorting out what maturity levels mean in a world of increasingly complex standards), it is harmful. See long discussion below. > 2) It is coupled to an > intent to actually behave according to what the document > says. Such an intent is obviously not feasible without some > change. Well, yes and no. My sense of the discussion over the last year is that a significant fraction of the community believes that the critical path change in this area is an adjustment to the threshold for Proposed Standard. That issue is addressed in draft-bradner-restore-proposed, which requires no change to RFC 2026 or other formal procedures at all. It is not addressed in this document. > It is useful to have our behavior and our documented > description of how we work match because the mismatch causes > confusion, at least for new participants, and sometimes even > for experienced participants. I agree with this. But this document doesn't make nearly enough of a contribution in that area to justify approving it. It addresses exactly one provision of our processes in which documentation and practice don't align, a provision about which there is no subtlety or confusion within the community at all (even though new participants may be confused). If the issue is important, then let's make a serious effort to update the places where 2026, 2028, 2031, 2360, 3710, 4071, and probably several other documents have fallen at least somewhat out of line with reality. If the particular issue of the annual review is really more important than any of those other issues, I assume that a stand-along document that changed it would rapidly achieve community consensus (albeit with some complaints about wasted time). Certainly it should not be sufficient to justify approving this document... the change is almost irrelevant to it. > It might be the case that it will improve the advancement > percentage. It might not. I can not imagine that it will > make that even worse. I can because the effects of changes like this are actually very hard to predict. It increases the requirements for the second level, however slightly. Since we have trouble getting things to the second level now, that increased requirement might reduce incentives to bring things off Proposed at a least as much as a theoretical "just one more level and then you are finished" change would increase those incentives. By changing Proposed from "1/3 of the way" to "1/2 way or a bit more", it might remove a small incentive to take the additional step. In addition, the "publication without a new RFC" provision may actually further increase the de facto requirements for Proposed Standards by requiring that those documents be edited to a high standard of clear English and specification precision. While, IMO, we have taken too little advantage of it, the current provisions of RFC 2026 permit publishing a Proposed Standard as soon as the WG understands it, leaving language cleanup (and refined translation from the writing styles of many people in the community whether native speakers of English or not) for Draft Standard versions. More important, for those of us who believe that a maturity system based on rigid named categories that apply to entire documents has outlived its usefulness as our specifications have become more complex, approval of this document is almost certain to cause consideration of such approaches to be postponed by some years as people complain that the changes made in this draft must have time to take effect and be evaluated. --- A more extended analysis of other aspects of the situation with this document follows. Those who don't like extended analyses might want to stop reading at some point. A very long time ago, a then-professor of mine observed that one of the most common sources of failures in engineering, especially computer engineering, was to look at a problem that seemed too large or too intractable, find some easily-changed and tractable part of the problem, do something with it (almost anything), and then claim that significant progress had been made on the original/ real problem because one had started on it. In many cases, the approach actually makes the real problem harder: systems become more complex, applying remedies later turns out to be more complicated, and so on. The narrow "solution" may also use up energy and creates expectations that make it harder than it would have been otherwise to take on the real work when the narrow job fails to solve any significant problems. An even more insidious version of the same approach is to identify a problem and claim it is serious enough that one needs to do _something_. One then proceeds to do something that has nothing at all to do with the problem, just to demonstrate motion (or in the hope that setting a butterfly in motion in one part of the world will cause major change elsewhere). These approaches are very different from taking a large design issue and modularizing it into pieces that can be worked out separately but with clear and well-understood interfaces. Doing that right requires that the overall design issue be understood well enough to define all of the modules and interfaces, not doing one piece, ignoring the poorly-understood rest, and hoping that things will sort themselves out. The IETF periodically falls into this trap. We often see protocols that are designed to handle the easy issues adopted without consideration of the harder ones and edge cases, only to discover that those protocols represent a framework into which solutions for the other issues won't fit. The largest symptoms of this used to show up only with Security issues -- protocols being designed without security provisions with the expectation that security would somehow be patched in later... and despite repeated experiences that such patches are rarely completely satisfactory. Now we see it spreading: I've seen "we knew about those issues but ignored them and now have to clean up the mess somehow" comments in two separate WGs in recent weeks and suspect that there are many more. Many of the comments on this list try to justify approving this draft on the same basis. These "there is a problem so let's do something and hope it will help", "even though we can demonstrate no logic that would predict that the change being proposed will result in the desired effect", and "let's fix something to show that we can" approaches are _always_ justified on the basis of "baby steps" or "incremental efforts". Those justifications would be quite reasonable if we actually had good evidence of the linkages. In the absence of such demonstrated linkages, we are just thrashing around and wasting energy --energy that, under other circumstances, could actually be used to address the problems. So, while I applaud the efforts of the authors of this draft to remove controversial and largely extraneous material, it seems to me that the core issue remains the one that many people have commented on in much earlier versions: At present, we don't get many documents from Proposed Standard to Draft Standard, despite the requirements for Draft Standard being slightly lower than those set forth for Internet Standard in this document. We have a possibly-serious problem with the norms for Proposed Standard being too high in practice --far beyond what RFC 2026 requires. It is noteworthy that draft-housley-two-maturity-levels no longer makes any claim to solve that problem. On the other hand, there has been considerable discussion in the community, previously supported by the strongest advocates for the predecessors of the current draft, that the norms for getting to Proposed Standard _are_ the keystone problem. I don't personally believe that -- I think things are more complicated-- but I do see it as a critical element (see draft-bradner-restore-proposed (expired today but can be reposted if that would be useful)). So this draft now ignores the threshold problem for Proposed Standard and argues that "Changing the Internet Standards Process from three maturity levels to two is intended to create an environment where lessons from implementation and deployment experience are used to improve specifications". But there is no logic to support the belief that intention will be satisfied. The purpose of the three-level system is to create an environment where the same lessons can be learned, applied, and documented. This document does nothing to change or improve that environment unless one believes that, magically, removing the third level will increase motivations for getting things to draft. The problem, which the document still describes, is that documents don't move off the first level, not that somehow the existence of the third one is pressing things down. In progressing by baby steps, actual babies fall down a lot. Usually they learn while falling and keep trying until they get it right. Equally important, for their purposes, the important target is learning how to walk more reliably, not reaching a specific goal or solving a particular problem. The IETF lacks those advantages: First, we are focused on the goal of making a change and have posited that change in terms of a choice between this particular document and doing nothing --all other possibilities have been dismissed without significant consideration. To quote one of the co-authors of this draft from a different thread: > Herein lies the real problem: As with many process and > structure discussions in the IETF, folk often see only a > simplistic, binary choice between whatever they prefer, versus > something akin to chaos. > > The world is more nuanced than that, and the choices more rich. I agree, but this document is being treated as that sort of binary choice. Second, the community is easily exhausted by process debates and the debate about this particular document has now been going on for 13+ months. Even if the suggestion made at the plenary about opening 2026 were serious, does anyone actually believe the community will be able to take up and evaluate more significant process changes in the near future? Or that this document, if approved, will not be used to justify deferring discussion of other changes because we have to see how it works out? So this is not "baby steps". It is one step, a step that makes a change that isn't demonstrably connected to the problem it purports to solve and that leads into a dead end. john > _______________________________________________ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf