Re: [Gen-art] Re: Gen-art review of draft-hartman-mailinglist-experiment-01.txt
Sam, we have some differences of opinion on how these things work, and how they are supposed to work. But I'll try to be constructive. I think that in any experiment that involves giving someone the power to set procedures, there MUST be some words on how those procedures are set (the metaprocedures). What your -01 draft currently says is: The IESG MUST inform the community in a public statement of any procedures for mailing list management approved under this experiment. Such a statement should include the description of the procedure and the description of mailing lists to which it applies or an indication that it applies to all IETF mailing lists. The IESG is encouraged to last call procedures it is considering approving under this experiment. While such last calls are encouraged, they are not required. The reason that the last call is not required is that under RFC 2418, no last call is required; there seems to be no reason to have a procedure more strict than that proposed in RFC 2418. Sanctions made under this memo may be appealed using the procedures outlined in [RFC2026]. I would add to this (suitably wordsmithed): The IESG MUST make the procedures for mailing list management public The IESG MUST make proposed changes public at least 14 days before enacting them. The IESG is NOT required to assert that there is IETF consensus for a change in procedures. (the last also means that I doubt that the term last call is useful here - a request for input, like the one the IAB uses for its documents, is more in line with the procedures you are proposing) ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: [Gen-art] Re: Gen-art review of draft-hartman-mailinglist-experiment-01.txt
Harald == Harald Alvestrand [EMAIL PROTECTED] writes: The primary reason you want to encourage meta-experiments is that a lot of the hypotheses you want to test involve delegation. For example I want to test the hypothesis that the right way to solve the mailing list mess is to delegate it to the IESG. I could delegate it to the IESG as part of an experiment along with a initial procedure. But if I do that I'm testing a different hypothesis: is delegating something to the IESG with the whole IETF designing the initial conditions a way to solve the problem. As you might imagine that's a different hypothesis. Harald I do not believe that is a correct statement. Harald An RFC 3933 experiment by its very nature involves three different Harald sub-experiments: Harald - Whether you can get the IETF to agree to running this experiment Harald - Whether the bodies that administer the experiment can perform the Harald experiment so that we have something to measure Harald - Whether the experiment produces useful results I agree with this characterization. I don't see how this characterization is in conflict with mine. Harald I think what you are proposing will lead to failure of the *first* Harald experiment, which means that you won't get to the other two. Harald If you say give the IESG the power to produce procedures, we're not Harald going to give you even a hint as to what these procedures might be, Harald this is what I would characterize as selling a pig in a poke, and it's Harald perfectly reasonable for the IETF community to ring up a no sale on Harald the consensus register. The IETF certainly may do that. You clearly believe the IETF should do that. I believe the IETF should not do that. Ultimately Brian will have to make a consensus call. I think it is reasonable to delegate comuing up with procedures to the IESG. I think that if we're going to delegate that then the procedures should not be discussed in this draft. If the community wants more oversight, here are some things I think would be reasonable to ask for: * Add criteria the IESG will use for evaluating procedures or for creating procedures. * Requiring proponents of this experiment to submit candidate procedures that demonstrate procedures are possible * Adding explicit review requirements for procedures that the IESG proposes. * Establishing principles the IESG should use If you believe that one of these options would help, I invite you to propose text or direction. You could also propose an experiment to run with specific procedures. I would not object to that experiment but I would also not be interested in contributing to it. Harald The theory that there is a different hypothesis being tested if the Harald IESG sets the initial procedures than if the initial procedures are Harald part of the experiment also seems doubtful to me; in both cases what I Harald think will happen is that the IESG will start off with procedures Harald designed by Sam Hartman, and the big difference is that the community Harald will have seen the initial procedures before deciding to run the Harald experiment. Harald The whole IETF has never designed anything and never will; all design Harald is done by specific people, and (hopefully) reviewed by some Harald large-enough fraction of the IETF. That's how EVERYTHING in the IETF Harald works. Since I'm on the IESG I'm actually in a reasonably good position to negotiate an initial procedure that the IESG will be happy with and that would be similar to a procedure the IESG would come up with on its own. However we want 3933 experiments--even experiments delegating things to the IESG--to be documents that anyone can write. So we should require that authors of 3933 experiments demonstrate stakeholder buy-in for experiments but not require that they take actions as if they were the stakeholders. Harald I do not believe this theory either. So, the main point of the paragraph here is that I don't believe only IESG members should be able to propose experiments that change things for the IESG. I hope we can agree on that. Harald RFC 3933 talks about the IESG and the community. The stakeholder Harald concept is of your introduction, and I believe it serves no benefit. Harald An RFC 3933 proposal author HAS to do three things: Harald - Write up what he's proposing in enough detail that it can be evaluated Harald - Convince the IESG that the experiment has enough merit to warrant a Harald Last Call Harald - Generate enough review and comment in the community that the IESG Harald can confidently say that there's community consensus to run the Harald experiment I proposed the concept of stakeholder buy-in as an attempt to solve the first criteria. I think that
Re: Gen-art review of draft-hartman-mailinglist-experiment-01.txt
Hi. I am going to take my gen-art hat off here because I want to suggest alternatives rather than just assessments. I have no inherent problem with what we are calling meta-experiments, although there are issues regarding whether the community will feel comfortable with just granting the IESG some power to 'do something' without having much idea what it is. Determining what constraints might placed on that power might take as long as actually specifying what should be done. On the other hand if you constrain the powers too tightly you might as well not bother with the meta-experiment. I think that is the problem you have already identified. The main difficulty I have with the draft as it stands is that the last paragraph of s1 says that the experiment will do something definite, but s4 actually just gives the IESG a general power. This is not consistent - it falls between the two stools mentioned above: although s4 suggests what types of delegation are envisaged the IESG can do whatever it likes regarding mailing list control and what actually happens might bear no resemblance to the second half of s4 para 1 and s4 para 2. So I think the draft can go one of two ways: 1 (which I think Sam would prefer): Redraft the memo as purely the meta-experiment and remove the suggestions of what the IETf might do, to avoid any possibility that people might be convinced that this was the only thing that could happen. === Delete the last but one sentence of s1 - the modified document won't actually create any particular level of sanction. Modify the previous sentence: This memo is an RFC 3933[RFC3933] experiment to provide the community with additional mechanisms to manage its mailing lists while the community decides what mailing list guidelines are appropriate. to: This memo is an RFC 3933[RFC3933] experiment which will enable the IESG to provide the community with additional mechanisms to manage its mailing lists while the community decides what mailing list guidelines are appropriate. Remove the 3rd and 4th sentences (from 'Such methods...') of para 1 and the whole of para 2 of s4. Replace the last part of para 4 of s4: While such last calls are encouraged, they are not required. The reason that the last call is not required is that under RFC 2418, no last call is required; there seems to be no reason to have a procedure more strict than that proposed in RFC 2418. Since RFC2418 does not require actions taken to be Last Called, it is explicitly permitted that any procedure put in place under the powers granted by this experiment need not require a Last Call before it is implemented, as there seems to be no reason to have a procedure more strict than is permitted in RFC 2418. If the resulting document is approved as an RFC3933 experiment, then the text removed from paras 1 and 2 can be put to the IESG as a proposal. The main issue with this as an experiment is that there are two sets of evaluations that can be done: - was the meta-experiement a success? - were any procedures adopted under the extra powers succcessful? It is not very obvious how to judge the original meta-experiment especially if the adopted procedures don't work out! = 2. Redraft the document to remove the general power given in s4 - then s1 is accurate, and the experiment would just cover the explicit procedures set up in the second half of para 1 and para 2 of s4. == Thoughts? Regards, Elwyn Sam Hartman wrote: Elwyn == Elwyn Davies [EMAIL PROTECTED] writes: Elwyn I was selected as General Area Review Team reviewer for Elwyn this specification (for background on Gen-ART, please see Elwyn http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html). Hi. I'm sorry it has taken me so long to get back to your comments. I've been busy trying to clear documents before the Dallas IETF. Elwyn Summary: [Note: This is the first time that I have done a Elwyn Process Experiment review and I will have to stretch the Elwyn usual review criteria a bit. Basically I believe I should Elwyn be looking for internal self-consistency, consistency with Elwyn associated processes and likelihood of damage to the Elwyn functioning of the IETF.] That seems like a good approach. I'm also doing this for the first time so your cooperation in helping me is greatly appreciated. Elwyn I think this draft is NOT currently in a suitable state to Elwyn produce a well-defined experiment. The main reason for Elwyn this conclusion is that the experiment consists of enabling Elwyn a meta-mechanism and suggesting something the IESG might Elwyn possibly do with this power rather than explicitly stating Elwyn that this is what the IESG should put in place. I'd like to focus on
Re: Gen-art review of draft-hartman-mailinglist-experiment-01.txt
Hi. Sam Hartman wrote: I am happy to make a change similar to the one you propose in section 1. I'm happy to split the parts of section 4 dealing with what the IESG might do into their own section as an example. That's fine by me.. it should make a self-consistent document. I do not want to remove them completely. Would that be OK As regards gen-art I think it would be fine. Ultimately I am only a small part of the consensus as regards the experiment proposed. Personally, I would prefer that we didn't have to waste inordinate amounts of time on mailing list management. Unfortunately knocking virtual heads together isn't very effective. /Elwyn ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Gen-art review of draft-hartman-mailinglist-experiment-01.txt
Elwyn == Elwyn Davies [EMAIL PROTECTED] writes: Elwyn I was selected as General Area Review Team reviewer for Elwyn this specification (for background on Gen-ART, please see Elwyn http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html). Hi. I'm sorry it has taken me so long to get back to your comments. I've been busy trying to clear documents before the Dallas IETF. Elwyn Summary: [Note: This is the first time that I have done a Elwyn Process Experiment review and I will have to stretch the Elwyn usual review criteria a bit. Basically I believe I should Elwyn be looking for internal self-consistency, consistency with Elwyn associated processes and likelihood of damage to the Elwyn functioning of the IETF.] That seems like a good approach. I'm also doing this for the first time so your cooperation in helping me is greatly appreciated. Elwyn I think this draft is NOT currently in a suitable state to Elwyn produce a well-defined experiment. The main reason for Elwyn this conclusion is that the experiment consists of enabling Elwyn a meta-mechanism and suggesting something the IESG might Elwyn possibly do with this power rather than explicitly stating Elwyn that this is what the IESG should put in place. I'd like to focus on your general comments about the draft not being ready. I think we can come back to specific textual comments after we've reached general agreement. I'd like to first take your issue of the IESG charter and then come back to the meta issue that all this experiment does is enable the IESG to do things. Elwyn My reading Elwyn of s7.2 of the IESG Charter [RFC3710] is that the IESG has Elwyn the power to do this sort of thing already anyway. Note that the IESG charter is an informational document. It cannot give the IESG power that it does not otherwise have. It was not clear to the participants on the IETF list that the IESG has the power to create mechanisms between 3683 and 3934 without a BCP or experiment. If the consensus of the IETF community is that the IESG already has that power and that the IESG's power is already well documented then I will withdraw my draft. My personal opinion is that the power is not well documented and thus is subject to higher probability of successful appeal. Elwyn Were Elwyn the suggested mechanisms eventually adopted, I would have Elwyn some qualms about the possibility of indefinite bans being Elwyn possible without allowing a wider (possibly IETF as opposed Elwyn to IESG) consensus, but that point is currently moot as the Elwyn actual proposals that would be put in place are not Elwyn specified by this document. I don't think the point is moot. If there are specific limits on the IESG's power that should be put in place, here is the place to do it. Alternatively when we evaluate the experiment we could decide additional limits are needed. But let's come back to the question of whether meta-experiments are a good idea. I think that in order for 3933 to be a valuable tool many of the experiments are going to be meta-experiments. So let me first explain why I think that's the case and then discuss how to evaluate a meta experiment. The primary reason you want to encourage meta-experiments is that a lot of the hypotheses you want to test involve delegation. For example I want to test the hypothesis that the right way to solve the mailing list mess is to delegate it to the IESG. I could delegate it to the IESG as part of an experiment along with a initial procedure. But if I do that I'm testing a different hypothesis: is delegating something to the IESG with the whole IETF designing the initial conditions a way to solve the problem. As you might imagine that's a different hypothesis. Since I'm on the IESG I'm actually in a reasonably good position to negotiate an initial procedure that the IESG will be happy with and that would be similar to a procedure the IESG would come up with on its own. However we want 3933 experiments--even experiments delegating things to the IESG--to be documents that anyone can write. So we should require that authors of 3933 experiments demonstrate stakeholder buy-in for experiments but not require that they take actions as if they were the stakeholders. The second reason that you want to allow meta-experiments is that we want to encourage RFC 3933 as the first step in process change. Process change often results in BCPs. You want the 3933 experiment to be reasonably similar to a BCP so that when appropriate you can easily convert a successful experiment into a BCP. You would probably replace any evaluation criteria with results of the evaluation, replace the sunset clause with something else. However you want the operative language to remain the same. A significant result of the mailing list discussion is the concern that our BCPs are too specific and encode operational details. If you
Gen-art review of draft-hartman-mailinglist-experiment-01.txt
I was selected as General Area Review Team reviewer for this specification (for background on Gen-ART, please see http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html). Document: draft-hartman-mailinglist-experiment-01.txt Intended Status: Experimental (RFC3933 Process Experiment) Shepherding AD: Brian Carpenter Review Trigger: IETF Last Call ending 15 March 2006 Summary: [Note: This is the first time that I have done a Process Experiment review and I will have to stretch the usual review criteria a bit. Basically I believe I should be looking for internal self-consistency, consistency with associated processes and likelihood of damage to the functioning of the IETF.] I will disregard all matters of the appropriateness or otherwise of the timing of this experiment vis-a-vis any discussion of ongoing PR actions. This review will only consider the merits of the proposal itself. I think this draft is NOT currently in a suitable state to produce a well-defined experiment. The main reason for this conclusion is that the experiment consists of enabling a meta-mechanism and suggesting something the IESG might possibly do with this power rather than explicitly stating that this is what the IESG should put in place. My reading of s7.2 of the IESG Charter [RFC3710] is that the IESG has the power to do this sort of thing already anyway. Were the suggested mechanisms eventually adopted, I would have some qualms about the possibility of indefinite bans being possible without allowing a wider (possibly IETF as opposed to IESG) consensus, but that point is currently moot as the actual proposals that would be put in place are not specified by this document. [Aside: Posting policies will need to start covering wikis and issue trackers in the immediate future!] Review: s1: I think that this section of the last paragraph of s1 overstates/mis-states what this document actually does: This memo is an RFC 3933[RFC3933] experiment to provide the community with additional mechanisms to manage its mailing lists while the community decides what mailing list guidelines are appropriate. IN particular this experiment creates a level of sanction between RFC 3934 and RFC 3683 for working group lists and creates sanctions other than RFC 3683 for non-working-group lists. In practice all that s4 mandates is: During the experiment period, the IESG MAY approve other methods of mailing list control besides those outlined in RFC 3683 and RFC 3934 to be used on a specified set of IETF mailing lists. i.e., it enables a meta-mechanism. s4 then goes on to suggest some things the IESG *might* adopt (second half of para 1 and all of para 2 of s4) and the procedures that it must go through to get them adopted. The result of this is (in the first instance) merely the provision to allow the IESG to decide to do something. I don't think this constitutes a well-formed experiment. s3: The section ... and any mailing list hosted on any site or system operated by the IASA or otherwise on behalf of the IETF. still leaves considerable scope for discussion of whether some mailing lists associated with IETF stuff would be covered. In particular I am thinking of pre-BOF lists and auxiliary WG mailing lists which are not hosted on IETF controlled systems but are discussing IETF related stuff. s3: The following sentence 'Mailing lists listed at https://datatracker.ietf.org/public/nwg_list.cgi are explicitly included in this definition.' has been specifically crafted to catch some of the lists where there are currently problems even though these lists are not actually hosted on IETF controlled systems. Although I am not aware of any specific problems, there could be potential legal minefields in the IETF attempting to manage (especially retrospectively) posting rights on mailing lists which are not hosted on IETF hardware. Some existing WG mailing lists are also not hosted on IETF hardware. s4: Para 2 talks about 'managers' of lists. This term has not been defined - in particular non-WG lists only have administrators (aka owners in some cases). s4, next to last para: The piece about last calls appears to confuse last calls relating to the specification of procedures and last calls relating to specific PR-actions (as mandated by RFC3683 but not required by RFC2418). The initial part of the para appears to be talking about the definition of the powers and procedures which the IESG takes and decides to follow but this metamorphoses into consideration of individual 'procedures' taken under the resulting procedures.. too many 'procedures'! In my view the Last Call which has provoked this review really ought to be arriving at consensus on the additional procedures currently 'suggested' in the second half of para 1 and in para 2 of s4 of this document, and the next to last para of s4 should just state that any actions taken under these
