Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Hallam-Baker, Phillip wrote: From: Jeffrey Hutzelman [mailto:[EMAIL PROTECTED] I do think that there should be a fixed rule prohibiting members of the IESG being WG chairs. I would also include the IETF chair in this. Most ADs positively want to drop their WG chairships in a hurry, because they don't have time any more. I'm not asserting it's universal, but it was certainly my reaction. One exception is generic Area WGs such as TSVWG, where you will notice that a third chair was appointed, which seems like good practice. I don't. While I agree this should be a rare occurrance, I have seen no evidence of a problem that such a rule would be intended to address. If it's not broken, why spend time trying to fix it? I was unable to appeal to the IESG against what I considered to be a very grave abuse of position by a chair precisely because I was warned that there was no prospect of getting a fair hearing as he was a member of the IESG. That is why we have an appeal chain, i.e. if such proved to be the case you could have appealed further to the IAB. (In fact, you just gave the worst possible reason for not appealing, IMHO.) Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Dave Cridland wrote: It's sometimes difficult to find the drafts you could comment on as they're produced, especially if you're not part of the WG, and it's also tricky to find the background to some of the decisions. It's fustrating, too, to have issues which are brought up continuously in a WG by well-meaning outsiders, when the issues have been discussed to death. But equally, it's virtually impossible to keep an active interest in more than a handful of WGs. Often times what you describe as discussed to death are those very issues where there really isn't community consensus, and so what happens is that people check out of working groups, and sometimes make a stink toward the end of the process. That's when the IESG has a responsibility to make a judgment call as to whether or not a WG has made the right technical decision, rough consensus be damned. Eliot ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Ned Freed wrote: I think that the single change most likely to keep WGs on track is to ensure that they do not have a single dominant participant, eg one who is both chair and author of key I-Ds. The WGs I see most at risk of going round in circles and/or producing output that falls short of what is needed are ones such. Some time ago, I did hear an IESG member talk of this in such a way as to make me think that this was an understood problem, but nothing seems to have changed in the two or so years since then. Perhaps it is an unwritten rule, but I thought this battle was fought and won years ago. Perhaps you should discuss the specific problem with the AD and or the IESG. I can't recall the last time I was involved in a group in which the chair played an active role in authoring. And as someone who did that way many years ago, I strongly advise against it. This is exactly my take as well. I've seen many cases where a chair has refused to become a document author or editor in a group because of the conflict it creates. I've also seen at least one case where a chair stepped down in order to become a document author. Exactly. Of course there are exceptions. The obvious ones are that the conflict is much more limited when there's a non-aothor co-chair, the authorship role is limited to a small subset of the group's documents, or both. For example, I was asked to co-chair the NNTPEXT group in large part because the other co-chair was also the main document author. (I note in passing that in this case we ended up with both roles being done by other people.) Without commenting on the specific case, I think that co-chairs may be the right solution when one of the chairs is an author of a fraction of the documents, but not usually when s/he is *the* main author. In that situation, the author+chair can never make a consensus call, so effectively is recused from the most important part of chairing. In another case, I once tried to weasle out of writing a specification (now RFC 2034) because at the time I was co-chair of the NOTARY group. But the feedback from all concerned was that it one small ancillary document and I was a only the co-chair, and I was, um, persuaded to do the work. Exactly. Brian And, of course, I believe that there is more to good engineering than just engineering eg the right processes. Ding ding! Ditto. Ned ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Brian E Carpenter [mailto:[EMAIL PROTECTED] This is exactly my take as well. I've seen many cases where a chair has refused to become a document author or editor in a group because of the conflict it creates. I've also seen at least one case where a chair stepped down in order to become a document author. Exactly. For the same reason I believe that it should never be the case than an AD is chair of a working group even if it is a different area. Other ADs are not likely to challenge a fellow AD. It means that there is no effective recourse to either the AD supervising the WG or to the full IESG. If someone is abusing their position as WG chair they are quite likely to abuse their position on the IESG as well to retaliate against anyone who agrees with the injured party. I do not think it would be a good thing to make it an inviolate rule that a chair can never be an editor. There are a couple of cases where this may make sense, particularly where the chair decides it will be easier to finish off a draft that has been languishing rather than badger people into finishing it. Also there are some cases where there are a lot of drafts in front of a WG. It should be possible for an editor to hand over that responsibility to become a chair without loosing their authorship credit. I do think that there should be a fixed rule prohibiting members of the IESG being WG chairs. I would also include the IETF chair in this. The position at BOFs is rather different. It is usually helpful to have someone experienced in IETF process to chair a BOF and often the best person to do that is someone who is not directly associated with the actual proposal. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Wednesday, June 28, 2006 09:45:27 AM -0700 Hallam-Baker, Phillip [EMAIL PROTECTED] wrote: I do not think it would be a good thing to make it an inviolate rule that a chair can never be an editor. Nor do I. I do think that there should be a fixed rule prohibiting members of the IESG being WG chairs. I would also include the IETF chair in this. I don't. While I agree this should be a rare occurrance, I have seen no evidence of a problem that such a rule would be intended to address. If it's not broken, why spend time trying to fix it? The position at BOFs is rather different. It is usually helpful to have someone experienced in IETF process to chair a BOF and often the best person to do that is someone who is not directly associated with the actual proposal. I agree. In addition, I would note that while a BOF chair does have the same logistical responsibilities as a WG chair in terms of making the meeting happen, they are not necessarily making a long-term commitment to the proposed work. Of course, many BOF chairs are already active with the proposal, but that's not a requirement and, as you note, may actually not be the best way to run a BOF. -- Jeff ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
- Original Message - From: Keith Moore moore@cs.utk.edu To: Robert Sayre [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; ietf@ietf.org Sent: Tuesday, June 27, 2006 1:38 AM Subject: Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors) I'm much more interested in trying to figure out how to get WGs to stay on track in the first place and to accept useful clue from elsewhere. I maintain that no process will accomplish that. The only way to get a WG to accept a clue is to demonstrate that their output is irrelevant by concrete example. no process can ensure that WGs stay on track, but we can certainly do better than what we have now. I think that the single change most likely to keep WGs on track is to ensure that they do not have a single dominant participant, eg one who is both chair and author of key I-Ds. The WGs I see most at risk of going round in circles and/or producing output that falls short of what is needed are ones such. Some time ago, I did hear an IESG member talk of this in such a way as to make me think that this was an understood problem, but nothing seems to have changed in the two or so years since then. And, of course, I believe that there is more to good engineering than just engineering eg the right processes. Tom Petch. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (w as: moving from hosts to sponsors)
Tom, This would be a bad idea as a general rule - though it is (I believe) one of the things that ADs look at. The problem is that there are good examples of WGs where the chair was a key author as well and it worked just fine. In addition, there are also examples where a chair has had to step in because (as is often the case when dealing with volunteers) nobody else would step up to the task. -- Eric --[SNIP]-- -- -- I think that the single change most likely to keep WGs on track is to ensure -- that they do not have a single dominant participant, eg one who is both chair -- and author of key I-Ds. The WGs I see most at risk of going round in circles -- and/or producing output that falls short of what is needed are ones such. -- --[SNIP]-- ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Hi, I think that the single change most likely to keep WGs on track is to ensure that they do not have a single dominant participant, eg one who is both chair and author of key I-Ds. The WGs I see most at risk of going round in circles and/or producing output that falls short of what is needed are ones such. Some time ago, I did hear an IESG member talk of this in such a way as to make me think that this was an understood problem, but nothing seems to have changed in the two or so years since then. Perhaps it is an unwritten rule, but I thought this battle was fought and won years ago. Perhaps you should discuss the specific problem with the AD and or the IESG. I can't recall the last time I was involved in a group in which the chair played an active role in authoring. And as someone who did that way many years ago, I strongly advise against it. And, of course, I believe that there is more to good engineering than just engineering eg the right processes. Ding ding! Eliot ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
I think that the single change most likely to keep WGs on track is to ensure that they do not have a single dominant participant, eg one who is both chair and author of key I-Ds. The WGs I see most at risk of going round in circles and/or producing output that falls short of what is needed are ones such. Some time ago, I did hear an IESG member talk of this in such a way as to make me think that this was an understood problem, but nothing seems to have changed in the two or so years since then. Perhaps it is an unwritten rule, but I thought this battle was fought and won years ago. Perhaps you should discuss the specific problem with the AD and or the IESG. I can't recall the last time I was involved in a group in which the chair played an active role in authoring. And as someone who did that way many years ago, I strongly advise against it. This is exactly my take as well. I've seen many cases where a chair has refused to become a document author or editor in a group because of the conflict it creates. I've also seen at least one case where a chair stepped down in order to become a document author. Of course there are exceptions. The obvious ones are that the conflict is much more limited when there's a non-aothor co-chair, the authorship role is limited to a small subset of the group's documents, or both. For example, I was asked to co-chair the NNTPEXT group in large part because the other co-chair was also the main document author. (I note in passing that in this case we ended up with both roles being done by other people.) In another case, I once tried to weasle out of writing a specification (now RFC 2034) because at the time I was co-chair of the NOTARY group. But the feedback from all concerned was that it one small ancillary document and I was a only the co-chair, and I was, um, persuaded to do the work. And, of course, I believe that there is more to good engineering than just engineering eg the right processes. Ding ding! Ditto. Ned ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Burger, Eric wrote: Very much agreed. -Original Message- From: Keith Moore [mailto:[EMAIL PROTECTED] I would have little objection to requiring running code as a test of feasibility of a new idea. I would object strongly to an argument that just because someone has running code, means it's a good indication of adequacy of the protocol...DKIM is a great example of a poorly designed protocol that has been justified by running code. I am hoping that you are agreeing with the stated principle, rather than the provided exemplar. As Michael noted, the exemplar asserts the assertion of justification that was never made. (It also asserts a quality assessment that is humorously incorrect, but gosh I suspect we ought not to slide down that slope.) Anyhow, I am probably missing something basic about this thread, since it seems to be re-discovering an item already in the formal IETF standards process: 4.1.1 Proposed Standard ... Usually, neither implementation nor operational experience is required for the designation of a specification as a Proposed Standard. However, such experience is highly desirable, and will usually represent a strong argument in favor of a Proposed Standard designation. The IESG may require implementation and/or operational experience prior to granting Proposed Standard status to a specification that materially affects the core Internet protocols or that specifies behavior that may have significant operational impact on the Internet. If folks are suggesting changes to this text, then what are the changes? d/ ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On 25-jun-2006, at 6:18, Keith Moore wrote: All too often, we (and I include myself here) shoot down ideas before they've had a fair hearing. We take an idea that is embryonic and - because we can _imagine_ that idea going somewhere that _might_ have a bad result - we do our best to kill it before it can breed. In this way we discard many good ideas not because they have no merit, but because they bring with them the potential to change things. The result is stagnation. Are you the same Keith Moore as the one I was talking to ealier?? Trouble is, in our current process, there's rarely any formal request for feedback, and little external visibility of a WG's output, until Last Call. That's what charters are for, aren't they? But I see your point. When I first attended an IETF meeting I was surprised to see how inward looking wgs are. There is very little, if any, effort to present what the working group is doing to people outside of the wg. So when I'm saying that working groups need multiple stages of formal, external review, what I'm really saying is that we need a structure for working groups in which we can have confidence that sufficient feedback will be obtained early enough to put good ideas on the right track and to see that truly bad ideas get weeded out in due time, most of the time. Hm, I think trying to kill bad ideas is largely a waste of time. (Saying this both as someone who came up with some ideas that others think are bad and someone who has tried to convince others that their ideas are bad.) Often, the fatal flaws will show up as the idea is developed, so a lot of them go away without doing anything anyway. The trouble is, that if someone develops a presumably bad idea in a draft, that draft is going to be deleted after six months. So the only way to keep that work around is to put it on a private website where it's probably going to be lost between the billions of pages that make up the web, or push for publication as an RFC. And like it or not, this lends a lot of credibility to an idea as most people don't understand the informational/experimental/standards track classification. It might make sense to create a third class of published documents that sits somewhere between draft and RFC. This would avoid the ridiculous situation where a draft is implemented and the email announcing its existance is archived, but the draft itself is deleted after some time. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Mon, 26 Jun 2006, Iljitsch van Beijnum wrote: On 25-jun-2006, at 6:18, Keith Moore wrote: Trouble is, in our current process, there's rarely any formal request for feedback, and little external visibility of a WG's output, until Last Call. But I see your point. When I first attended an IETF meeting I was surprised to see how inward looking wgs are. There is very little, if any, effort to present what the working group is doing to people outside of the wg. I guess that if people are interested in the WG then they will participate, or at least read the drafts as they come up in the i-d announcement list. You can't force volunteers to take an interest. Perhaps requirements documents should be more like rationale documents: we chose to solve this problem in this way because of this... Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ SOUTHEAST ICELAND: SOUTH VEERING SOUTHWEST 4 OR 5, OCCASIONALLY 6 IN NORTH. RAIN AT TIMES. MODERATE OR GOOD. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Keith Moore [mailto:[EMAIL PROTECTED] there is one important class of bad ideas that doesn't go away in IETF -- the class of bad ideas that is obviously bad from a wider perspective but which looks good to a set of people who are focused on a narrow problem. and in IETF what we often do with those ideas is to protect them and encourage development of them in isolation by giving them a working group. we sometimes even write those groups' charters in such a way as to discourage clue donation or discussion of other ways of solving the problem. That is a somewhat cynical way to describe IPSEC isn't it? Care to mention any other groups that fit that description? The IESG and the IETF in general has hardly demonstrated an infalible understanding of what is and is not a bad idea, nor for that matter has anyone else. This is a research area and there are plenty of areas where the great and the good get it wrong. Take Gopher for example, I remember the days when the assumption was that the Web would merge into gopher rather than the other way round. After all the Gopher people knew so much more about networking. Only they did not understand the UI issue and it turned out thsat Tim had a much more powerful idea despite not being an IETF longtimer. My theory is that Vint and Jon set up the whole IETF infrastructure as a Gordian knot test. Keep the systems safe from over tampering until someone comes along who is decisive enough and addressing a need that is so urgent that either the layers of obfustication will yield or they will snap. No Keith, you are not Vint Cerf, or Tim Berners-Lee and neither is anyone else here including me. I know that folk focused on narrow problems have tended to come up with narrow solutions. That is hardly suprising, the rules of engagement here prohibit the discussion of the general. Take DKIM for example we are about to discuss a one off policy language to serve a single protocol, not because there is only a single protocol that requires policy but because there are people in the establishment who tried policy fifteen years ago, failled to solve the problem and have declared it 'insoluble'. There is also the problem of the other group who need s to be part of the policy discussion which has repeatedly demonstrated itself to be unwilling to listen to any outside view. Try to explain a problem to them and its 'la la la I'm not listening'. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Keith Moore [mailto:[EMAIL PROTECTED] there is one important class of bad ideas that doesn't go away in IETF -- the class of bad ideas that is obviously bad from a wider perspective but which looks good to a set of people who are focused on a narrow problem. and in IETF what we often do with those ideas is to protect them and encourage development of them in isolation by giving them a working group. we sometimes even write those groups' charters in such a way as to discourage clue donation or discussion of other ways of solving the problem. That is a somewhat cynical way to describe IPSEC isn't it? Care to mention any other groups that fit that description? DKIM comes to mind, as does zeroconf. But I've seen so many examples of this over the years (including IPsec) that I've lost track. The IESG and the IETF in general has hardly demonstrated an infalible understanding of what is and is not a bad idea, nor for that matter has anyone else. true. but the fact that we're not infallable doesn't mean we shouldn't try to improve things. No Keith, you are not Vint Cerf, or Tim Berners-Lee and neither are the real Vint or Tim, respectively. (both smart guys whom I respect, but there's a difference between any real person and his reputation. and this isn't an discussion about personalities, it's a discussion about how to do protocol enginering) I know that folk focused on narrow problems have tended to come up with narrow solutions. That is hardly suprising, the rules of engagement here prohibit the discussion of the general. which is my point - we need to change the rules of engagement. Take DKIM for example we are about to discuss a one off policy language to serve a single protocol, not because there is only a single protocol that requires policy but because there are people in the establishment who tried policy fifteen years ago, failled to solve the problem and have declared it 'insoluble'. There is also the problem of the other group who need s to be part of the policy discussion which has repeatedly demonstrated itself to be unwilling to listen to any outside view. Try to explain a problem to them and its 'la la la I'm not listening'. DKIM as currently described in the I-Ds is a lot more broken than that, but they're not listening either. But it's a lot bigger problem than any single working group. I find DKIM a convenient example in this discussion because it's current, and because of my long history of working with email I have a keener interest in that WG than most. But it's not hard to find other examples. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Trouble is, in our current process, there's rarely any formal request for feedback, and little external visibility of a WG's output, until Last Call. But I see your point. When I first attended an IETF meeting I was surprised to see how inward looking wgs are. There is very little, if any, effort to present what the working group is doing to people outside of the wg. I guess that if people are interested in the WG then they will participate, or at least read the drafts as they come up in the i-d announcement list. You can't force volunteers to take an interest. the problem is the assumption that people are either interested enough to participate in a WG or they're not interested at all in the outcome. for many WGs there are a significant number of people who will potentially be adversely affected by the outcome but who only have a peripheral interest in the design. we need a better way for WGs to get clues from those with peripheral interests than to expect those people to read every new I-D that comes out and try to evaluate it without benefit of mailing list context. Perhaps requirements documents should be more like rationale documents: we chose to solve this problem in this way because of this... In general we need to discourage the meme requirements documents. We need problem definition documents and design goal documents from the early phases of the process, design rationale documents to explain particular decisions made at later phases. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Sat, 24 Jun 2006, Tim Bray wrote: When standards orgs go out to invent stuff in unexplored territory you get disasters like OSI networking, CORBA, and in the current landscape, WS-*. I suppose there are exceptions but I don't know of any. Speaking of CORBA: http://www.acmqueue.com/modules.php?name=Contentpa=showpagepid=396 Tony. -- f.a.n.finch [EMAIL PROTECTED] http://dotat.at/ PORTPATRICK: ANOTHER AREA OF STRONG SOUTHERLY WINDS WILL SPREAD TO MOST WESTERN AREAS, AND FASTNET, ON THURSDAY AND FRIDAY, WITH GALES POSSIBLE IN ROCKALL AND BAILEY. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Perhaps one option would be if each WG maintained a problem-statement and rationale draft. This could reference the documents that provided the solutions to the problems found, and act as an overview to outsiders of the WG's previously discarded arguments, etc, as well as how they see the specifications used in practise. This would act in part as Keith's requirements document, and also act as a lure for external (to the WG, at least) reviewers. please note: I am _not_ a proponent of requirements documents. I think WGs need to define their problems and refine their scopes beyond that stated in their charters, write down their design goals, and perhaps to try to characterize design tradeoffs.A few of those design goals thus identified might qualify as hard-and-fast requirements. But asking WGs to state things in terms of requirements is tantamount to asking them to make those design tradeoffs before they are understood. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Monday, June 26, 2006 11:24:55 AM -0400 Keith Moore moore@cs.utk.edu wrote: Perhaps requirements documents should be more like rationale documents: we chose to solve this problem in this way because of this... In general we need to discourage the meme requirements documents. No, we don't. We just need to discourage the practice of waiting to write such a document until after protocol design is complete, rather than doing it first. need problem definition documents and design goal documents from the early phases of the process Yes, absolutely. That's what a requirements document is supposed to be. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Jeffrey Hutzelman [mailto:[EMAIL PROTECTED] No, we don't. We just need to discourage the practice of waiting to write such a document until after protocol design is complete, rather than doing it first. That depends. Take the recently renamed WAE. The original idea of this group was to go after phishing. I work on phishing every day, I think they would do far better and produce something with much more immediate impact by instead applying their technology to the problem of Internet pedophiles. Changing bank authentication infrastructure is hard and the liability issues are savage. Bottom up work is what this particular community does best. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On 6/24/06, Keith Moore moore@cs.utk.edu wrote: In other words, we don't want to distract WGs with useful input ... better that they should keep their heads in the sand for the entire 2-3 years of their existence and then produce irrelevant or even harmful output. And that way, maybe a few influential people within the WG can coerce the WG into producing something that favors their employers' short-term interest even if it harms other interests or glosses over important limitations. If the errors are sufficiently grave, it is easy to fork the WG documents and have them replaced or completely rewritten. it's not easy at all - because even if you replace the WG you'll have most of the same individuals active in the new one as in the old one - only they'll be angrier than the first time around, and there's a good chance that any people you lose in the transition will include those who had more clue. I'm much more interested in trying to figure out how to get WGs to stay on track in the first place and to accept useful clue from elsewhere. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On 6/26/06, Keith Moore moore@cs.utk.edu wrote: If the errors are sufficiently grave, it is easy to fork the WG documents and have them replaced or completely rewritten. it's not easy at all - because even if you replace the WG you'll have most of the same individuals active in the new one as in the old one - only they'll be angrier than the first time around, I meant replace the documents, rather than the WG. But you're right, they do get angrier, at first. For example, in the somewhat-successful Atompub group, this happened at least twice. The protocol document still contains a lot of cruft that's only in there so that a few vendors can call whatever it is they're building Atom. It might not be fatal stuff, so forking it won't work. The resulting document will be easy to improve on, if someone wants to, but it sure is a heck of a lot more practical than WS-*. I'm much more interested in trying to figure out how to get WGs to stay on track in the first place and to accept useful clue from elsewhere. I maintain that no process will accomplish that. The only way to get a WG to accept a clue is to demonstrate that their output is irrelevant by concrete example. -- Robert Sayre ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF?(was: moving from hosts to sponsors)
From: Robert Sayre [mailto:[EMAIL PROTECTED] The advice WGs need most often is probably most of your goals are false ones, and this design is 10x too complicated. In the apps area, stop using W3C XML Schema is probably up there, too. That is not good advice, the SGML DTD mechanism is even more grossly defective. It should be studdied as a prime example of how not to architect standards. The only way to understand the idiotic and arbitrary limits imposed in the DTD scheme is that there is no coherent design, merely a description of a program and ad-hoc hacks performed to the program to support a random set of poorly articulated requirements. The solution in this case is to propose an alternative schema mechanism. People speak highly of relax. My own solution which can be seen in SAML and XKMS (or rather could when I was editor) was to develop my own schema language which is considerably simpler and less flexible. I then generated the schema for the specification from my own schema language using some simple tools. So for example XML schema allows an element name to have multiple definitions within a document which is idiotic. There is no excuse for context sensitive grammars in a protocol spec. XML schema allows for a whole range of silly stuff, like Qnames (which I have made the mistake of using myself at some point). The only thing you can't fix that way is the basic idiocy of the way schema inclusion works. Schema inclusion requires the introduction of a new prefix namespace which results in the ludicrous complexity of XML documents, as well as the verbosity. There is no conflict between the schemas of XKMS, XML Signature, XML Encryption. It should be possible for the XKMS schema to include the other two schemas without declaring additional prefixes. It is an issue but not a major one, certainly XML Schema is broken and should be fixed but the solution is not to bar its use, that is predjudice masquerading as architecture. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF?(was: moving from hosts to sponsors)
On 6/26/06, Hallam-Baker, Phillip [EMAIL PROTECTED] wrote: From: Robert Sayre [mailto:[EMAIL PROTECTED] The advice WGs need most often is probably most of your goals are false ones, and this design is 10x too complicated. In the apps area, stop using W3C XML Schema is probably up there, too. That is not good advice, the SGML DTD mechanism is even more grossly defective. No comment, but I didn't say use DTDs. The solution in this case is to propose an alternative schema mechanism. People speak highly of relax. Use Relax NG. It is an issue but not a major one, certainly XML Schema is broken and should be fixed Can't be fixed, from what I know. I do think it is a major issue. but the solution is not to bar its use, that is predjudice masquerading as architecture. The banning suggestion was ha-ha-only-serious, but it would not be prejudice. It would be discrimination, in the best sense of the word. -- Robert Sayre ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF?(was: moving from hosts to sponsors)
From: Robert Sayre [mailto:[EMAIL PROTECTED] The solution in this case is to propose an alternative schema mechanism. People speak highly of relax. Use Relax NG. It is an issue but not a major one, certainly XML Schema is broken and should be fixed Can't be fixed, from what I know. I do think it is a major issue. but the solution is not to bar its use, that is predjudice masquerading as architecture. The banning suggestion was ha-ha-only-serious, but it would not be prejudice. It would be discrimination, in the best sense of the word. The solution is not to ban XML Schema, rather it is to insist on Relax NG or to make it clear that Relax is the prefered route. When I was writing XML specs it was far too soon to be making categorical judgements such as avoid XML Schema. But what I do protest is the insertion of ill judged opinion as authority. For example there is good reason to make sure that an email security scheme plays nicely with both S/MIME and OpenPGP. There is absolutely no point in insisting on support for PEM, MOSS or any other aborted start. There is good reason to require Web Services to support layering on top of the SOAP stack. It makes absolutely no sense to require support for BEEP as if it was still an equally viable alternative. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
I'm much more interested in trying to figure out how to get WGs to stay on track in the first place and to accept useful clue from elsewhere. I maintain that no process will accomplish that. The only way to get a WG to accept a clue is to demonstrate that their output is irrelevant by concrete example. no process can ensure that WGs stay on track, but we can certainly do better than what we have now. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Such as, a requirement for formal cross-area review of the design goals document and of preliminary specifications as a prerequisite before producing a reference implementation. The IETF standards process is already so slow and uncertain that people throw up their hands in exasperation and go around it. Changing the process to make it even slower and add yet more hoops to jump through will solve the problem by ensuring that nobody bothers any more. R's, John ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On 24 Jun 2006 10:58:31 - John Levine [EMAIL PROTECTED] wrote: Such as, a requirement for formal cross-area review of the design goals document and of preliminary specifications as a prerequisite before producing a reference implementation. The IETF standards process is already so slow and uncertain that people throw up their hands in exasperation and go around it. True. Which is why it's necessary to handle the reviews in a pipelined rather than a stop-and-wait fashion. But part of the reason IETF's process is so slow is that the only meaningful checks we place are at the end - so a working group typically labors to the point of exhaustion without having received any external feedback, so when the feedback does arrive the working group is so dysfunctional that it's nearly incapable of fixing anything (and it is often in denial about what is wrong). Providing more early feedback will speed up the process rather than slow it down. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Maybe I'm unduly pessimistic, but I would have trouble working up a lot of enthusiasm for doing work on a project if I knew that some gatekeeper working in parallel might later tell me that I was wasting my time. I have trouble working up a lot of enthusiasm for contributing to work that is going off in the weeds. What I have in mind is not a gatekeeper but external review. The external review would not be expected to tell a WG that it were wasting its time (though that is a possibility), but rather, how to make better use of its time by producing a specification that was more relevant. Apparently you think that an artist is the best judge of the relevance of his own work. Although I completely disagree with your prediction that DKIM will be useless, there are worse problems than standards that turn out not to be used, so long as they are designed so they don't interfere with other more useful work. We seem to be managing that last issue reasonably well. DKIM is a good counterexample of that also. The existence of DKIM distracts attention from more useful work that could be done - partly because it's consuming energy from those who would work on more useful goals if they were chartered, partly because of the need for damage control, and partly because of the widespread assumption that since IETF has chartered DKIM that DKIM is the solution that will be promoted by IETF. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
but rather, how to make better use of its time by producing a specification that was more relevant. ... Apparently you think that an artist is the best judge of the relevance of his own work. I find the assumption that external reviewers are better able to score a project's relevance, whatever that is, rather than the people who are doing it and plan to use it, truly breathtaking. - partly because it's consuming energy from those who would work on more useful goals if they were chartered, partly because of the need for damage control, This must be a different group of people from the ones who I find on the DKIM list. If we wanted to work on something else, we would be doing so, and although I may overrate our collective wisdom, I don't think we're all working on DKIM purely because we are too dim to imagine anything else. and partly because of the widespread assumption that since IETF has chartered DKIM that DKIM is the solution that will be promoted by IETF. Seems to me that if it weren't so difficult to charter and complete WGs, there would be more of them, and people would be less likely to overestimate the importance of one or another of them. But I wouldn't generalize too much about DKIM, or MARID, or ASRG, because people have been looking for a magic spam bullet for 10 years and nothing we tell them is going to stop that. Regards, John Levine, [EMAIL PROTECTED], Primary Perpetrator of The Internet for Dummies, Information Superhighwayman wanna-be, http://www.johnlevine.com, Mayor I shook hands with Senators Dole and Inouye, said Tom, disarmingly. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Keith Moore wrote: True. Which is why it's necessary to handle the reviews in a pipelined rather than a stop-and-wait fashion. But part of the reason IETF's process is so slow is that the only meaningful checks we place are at the end - so a working group typically labors to the point of exhaustion without having received any external feedback, so when the feedback does arrive the working group is so dysfunctional that it's nearly incapable of fixing anything (and it is often in denial about what is wrong). Providing more early feedback will speed up the process rather than slow it down. There's already a means for external reviewers to do so: read the drafts, make comments, add issues to the issue tracker. It's really not rocket science. Having some new sclerotic pipeline involved with the life blood of a working group sounds like a recipe for working group infarction to me. Mike ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
but rather, how to make better use of its time by producing a specification that was more relevant. ... Apparently you think that an artist is the best judge of the relevance of his own work. I find the assumption that external reviewers are better able to score a project's relevance, whatever that is, rather than the people who are doing it and plan to use it, truly breathtaking. As I said, apparently you think that an artist is the best judge of the relevance of his own work. I find this preposterous. and partly because of the widespread assumption that since IETF has chartered DKIM that DKIM is the solution that will be promoted by IETF. Seems to me that if it weren't so difficult to charter and complete WGs, there would be more of them, and people would be less likely to overestimate the importance of one or another of them. One reason that it's difficult to charter and complete WGs might be that WGs have demonstrated a huge potential to do more harm than good. But I wouldn't generalize too much about DKIM, or MARID, or ASRG, because people have been looking for a magic spam bullet for 10 years and nothing we tell them is going to stop that. True, the spam area is more difficult than most. But that ought to compel us to rely even more heavily on engineering disciplines, rather than to invest a huge amount of effort in a dubious direction out of a naive belief that doing something, anything at all, is better than doing nothing. Keth ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
There's already a means for external reviewers to do so: read the drafts, make comments, add issues to the issue tracker. It's really not rocket science. That's not quite sufficient, because most WGs aren't proceeding according to good engineering discipline (e.g. they're doing things in the wrong order, like trying to define the protocol before the problem space is understood), there's little external visibility of what the WG is doing (so it's difficult to make timely input without actually following the entirety of the mailing list discussion), and often, nobody with any authority over the WG is checking to see whether the WG is actually responding appropriately to such comments. A more formal process is necessary. Having some new sclerotic pipeline involved with the life blood of a working group sounds like a recipe for working group infarction to me. In other words, we don't want to distract WGs with useful input ... better that they should keep their heads in the sand for the entire 2-3 years of their existence and then produce irrelevant or even harmful output. And that way, maybe a few influential people within the WG can coerce the WG into producing something that favors their employers' short-term interest even if it harms other interests or glosses over important limitations. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Keith Moore wrote: There's already a means for external reviewers to do so: read the drafts, make comments, add issues to the issue tracker. It's really not rocket science. That's not quite sufficient, because most WGs aren't proceeding according to good engineering discipline (e.g. they're doing things in the wrong order, like trying to define the protocol before the problem space is understood), there's little external visibility of what the WG is doing (so it's difficult to make timely input without actually following the entirety of the mailing list discussion), and often, nobody with any authority over the WG is checking to see whether the WG is actually responding appropriately to such comments. A more formal process is necessary. By whose standard? If you think that's going on, I'd think it would be appropriate to take it up with the relevent WG chairs and AD's. Last I heard, they're the stakeholders. Having some new sclerotic pipeline involved with the life blood of a working group sounds like a recipe for working group infarction to me. In other words, we don't want to distract WGs with useful input ... better that they should keep their heads in the sand for the entire 2-3 years of their existence and then produce irrelevant or even harmful output. And that way, maybe a few influential people within the WG can coerce the WG into producing something that favors their employers' short-term interest even if it harms other interests or glosses over important limitations. I repeat: There's already a means for external reviewers to do so: read the drafts, make comments, add issues to the issue tracker. It's really not rocket science. What you seem to want is some sort of cabal of dilettants who don't actually have to pay very much attention, but by their rank alone get to exercise veto power. Yuck. I don't suppose you have anybody in mind for such an IETF version of landed gentry? Mike ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Michael, Merely calling something or someone a name like sclerotic or dillitante is unconvincing. You haven't seen a proposal yet, you have no idea of how this would work (or apparently, how a review of an engineering effort should work), and yet you dismiss the very idea out of hand. That's very similar the kind of failure that causes many working groups to produce poor or irrelevant output. You haven't refuted anything, you've merely provided a case example of why working groups need adult supervision. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Merely calling something or someone a name like sclerotic or dillitante is unconvincing. You haven't seen a proposal yet, you have no idea of how this would work Good point. The sooner you send around a proposal, the sooner we can figure out how it might improve the process. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Jun 24, 2006, at 8:55 AM, Keith Moore wrote: That's not quite sufficient, because most WGs aren't proceeding according to good engineering discipline (e.g. they're doing things in the wrong order, like trying to define the protocol before the problem space is understood) I'd generalize that. I have never seen *any* standards org do a good job of inventing new technology. I've been working with the soon-to- wind-down Atompub group for a couple of years and we got a pretty good result I think (if you can judge by implementations deployments). There were a few things in our favor - a high level of interest and energy, lots of experience on the WG, decent editors - but the key thing was there was a ton of hands-on experience in the space (syndication technology). A whole lot of the key arguments could be resolved by appeal to example and experience. When standards orgs go out to invent stuff in unexplored territory you get disasters like OSI networking, CORBA, and in the current landscape, WS-*. I suppose there are exceptions but I don't know of any. -Tim ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Sat, 24 Jun 2006 17:34:09 -0700 Tim Bray [EMAIL PROTECTED] wrote: That's not quite sufficient, because most WGs aren't proceeding according to good engineering discipline (e.g. they're doing things in the wrong order, like trying to define the protocol before the problem space is understood) I'd generalize that. I have never seen *any* standards org do a good job of inventing new technology. good point. anytime a significant number of people participating in a WG are lacking some important set of expertise or experience, there's going to be a lot of time spent educating everyone and adopting a common language. and in IETF, because we try to engineer things for the Internet as a whole, it's entirely normal for a significant plurality of WG participants to lack some fundamental kind of expertise that is required to make the protocol work well at Internet scale. so we have our work cut out for us. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Merely calling something or someone a name like sclerotic or dillitante is unconvincing. You haven't seen a proposal yet, you have no idea of how this would work Good point. The sooner you send around a proposal, the sooner we can figure out how it might improve the process. Fair enough. But I think Michael's note illustrates a somewhat different, but related, and very common phenomenon in IETF that is relevant to this problem. All too often, we (and I include myself here) shoot down ideas before they've had a fair hearing. We take an idea that is embryonic and - because we can _imagine_ that idea going somewhere that _might_ have a bad result - we do our best to kill it before it can breed. In this way we discard many good ideas not because they have no merit, but because they bring with them the potential to change things. The result is stagnation. I think we do this because we lack confidence in the IETF working group process. We all know that once something gets to the working group stage, it's very hard to kill. No matter how badly it turns out, the IESG is likely to end up holding its nose - maybe demand a trivial text change or two or add a nasty disclaimer - but basically approve it. By the time the WG has spent 2-3 years or more, exhausted itself and declared itself to be done, it takes a lot of Last Call feedback and IESG gumption to push back on it. Simple fixes can of course be made, but significant design flaws can rarely be corrected. At the same time, experienced WG participants know that even small, apparently trivial objections made late in a WG's lifetime have the potential to delay the WG's output for months or sometimes years. So once there is a significant investment in a particular direction within a WG, there tends to be tremendous resistance on the part of core participants to feedback that would call that direction into question. Trouble is, in our current process, there's rarely any formal request for feedback, and little external visibility of a WG's output, until Last Call. Which basically means that working groups get most of the external feedback about their fundamental design choices long after the designs are frozen and the working group is too exhausted to fix anything. Ideally our process should encourage these differences to be resolved early - before there is a significant investment in the WG's output that would encourage denial on the part of its proponents, and before the WG is exhausted to the point that it cannot respond quickly and flexibility to external input. And if we had a process that made it likely that significant problems were discovered early, the latter stages of review should be less time consuming and less risky. So when I'm saying that working groups need multiple stages of formal, external review, what I'm really saying is that we need a structure for working groups in which we can have confidence that sufficient feedback will be obtained early enough to put good ideas on the right track and to see that truly bad ideas get weeded out in due time, most of the time.You (or Michael) and I might have somewhat different ideas about how to gain that confidence, or about the details for how to make it work without bogging down working groups, but the concept working groups need earlier external feedback is sound. Keith p.s. as for additional review stages bogging things down, any good protocol engineer ought to understand that sliding windows produce better throughput than stop-and-wait, that a fin-wait state is unavoidable if you want a clean close, but the way to reduce time in fin-wait is to minimize rtt and packet loss... ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
I would offer that in *some* groups the running code bar is reasonable. For example, in SIPPING, the problem space is pretty well-defined, and there are third-party specifications and requirements out there. There have been way too many half-baked ideas floated for consideration, and that has sucked the life blood out of the work group. As I have mentioned on the SIPPING list, this is NOT a prescription for all proposals or work groups in the IETF. However, the idea is reasonable for *some* work groups in the IETF. I think raising the bar isn't going to hurt. The they'll take the idea to another standards body doesn't hold water if there is no code / prototype / product to promote. -Original Message- From: Keith Moore [mailto:[EMAIL PROTECTED] Sent: Friday, March 24, 2006 2:50 PM To: Dave Cridland Cc: [EMAIL PROTECTED]; ietf@ietf.org Subject: Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors) [snip] It may be that we place too much emphasis on running code in IETF today. In ARPAnet days, when the user community was small and homogeneous but platforms were diverse (different word/character sizes, different character sets, different limitations of operating systems and networking hardware), and goals for protocols were modest, merely being able to implement a protocol across different platforms was one of the biggest barriers to adoption. In that environment, being able to demonstrate running code on multiple platforms was nearly sufficient to demonstrate the viability of a protocol. Besides, since the net was small, it wasn't terribly hard to make changes should they be found to be necessary. These days running code serves as proof-of-concept and also as a way to validate the specification. It doesn't say anything about the quality of the design - not efficiency, nor usability, nor scalability, nor security. etc. [snip] ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri, 23 Jun 2006 16:18:40 -0400 Burger, Eric [EMAIL PROTECTED] wrote: I would offer that in *some* groups the running code bar is reasonable. I would have little objection to requiring running code as a test of feasibility of a new idea. I would object strongly to an argument that just because someone has running code, means it's a good indication of adequacy of the protocol...DKIM is a great example of a poorly designed protocol that has been justified by running code. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Friday, June 23, 2006 05:24:11 PM -0400 Keith Moore moore@cs.utk.edu wrote: On Fri, 23 Jun 2006 16:18:40 -0400 Burger, Eric [EMAIL PROTECTED] wrote: I would offer that in *some* groups the running code bar is reasonable. I would have little objection to requiring running code as a test of feasibility of a new idea. I would object strongly to an argument that just because someone has running code, means it's a good indication of adequacy of the protocol. Specific examples aside, I agree. Running code should be a necessary condition for something to progress, but not a sufficient one. -- Jeffrey T. Hutzelman (N3NHS) [EMAIL PROTECTED] Sr. Research Systems Programmer School of Computer Science - Research Computing Facility Carnegie Mellon University - Pittsburgh, PA ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
I would have little objection to requiring running code as a test of feasibility of a new idea. I would object strongly to an argument that just because someone has running code, means it's a good indication of adequacy of the protocol. Specific examples aside, I agree. Running code should be a necessary condition for something to progress, but not a sufficient one. I think we would do well to require a reference implementation as a condition for Proposed Standards from new working groups or individual submitters...but there are other conditions that we should impose that are far more important. Such as, a requirement for formal cross-area review of the design goals document and of preliminary specifications as a prerequisite before producing a reference implementation. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Title: Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors) Very much agreed. -- Sent from my Palm Treo - Sorry if Terse -Original Message- From: Keith Moore [mailto:moore@cs.utk.edu] Sent: Friday, June 23, 2006 05:25 PM Eastern Standard Time To: Burger, Eric Cc: ietf@ietf.org Subject: Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors) On Fri, 23 Jun 2006 16:18:40 -0400 Burger, Eric [EMAIL PROTECTED] wrote: I would offer that in *some* groups the running code bar is reasonable. I would have little objection to requiring running code as a test of feasibility of a new idea. I would object strongly to an argument that just because someone has running code, means it's a good indication of adequacy of the protocol...DKIM is a great example of a poorly designed protocol that has been justified by running code. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Tue, Mar 28, 2006 at 06:47:46AM -0800, Dave Crocker wrote: [EMAIL PROTECTED] wrote: ah yes, the IETF as a FormulaOne race car. I'll approach CocaCola Visa for branding rights if that would help (esp for those folks denied a 770) ah yes, the ad absurdem form of argumentation. testing the boundaries of an assertion is occasionally wise. The reality in having a host is that we already experience quite a bit of marketing from that host. we do. My suggestion was cast in terms of permitting that level to continue, not in permiting every attendee eye-blink to experience a new injection of promotional material. i was more concerned wiht Jordi's suggestion that we aquire sustaining sponsors. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Thu, Mar 23, 2006 at 07:34:00PM -0800, Andy Bierman wrote: Dave Crocker wrote: Michael StJohns wrote: What I think Jordi is saying is that he wants the US sponsors to subsidize the cost of the overseas meetings. At least that's what it works out to be This view can be mapped to a classic model that would have significant benefits for the IETF: A host gets all sorts of marketing leverage out of the role in producing an IETF. There is nothing that requires that the event site management effort be coupled with a particular host's venue. If we moved to a model of having companies provide sponsorship funds, in return for which they get appropriate marketing presence, then we could have meeting venue management move to the sort of predictable and timely basis -- ie, far enough ahead of time -- that has been a concern for many years. Amen! And maybe the meeting fees could actually go down with enough sponsors. An additional room like the terminal room (not out in the open) could be used. Also, the IETF could maintain control of the network if there were multiple sponsors instead of a single host. They would not be allowed to ignore the advice of the NOC team, and let the wireless meltdown right off the bat. d/ Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ah yes, the IETF as a FormulaOne race car. I'll approach CocaCola Visa for branding rights if that would help (esp for those folks denied a 770) --bill ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
[EMAIL PROTECTED] wrote: ah yes, the IETF as a FormulaOne race car. I'll approach CocaCola Visa for branding rights if that would help (esp for those folks denied a 770) ah yes, the ad absurdem form of argumentation. The reality in having a host is that we already experience quite a bit of marketing from that host. My suggestion was cast in terms of permitting that level to continue, not in permiting every attendee eye-blink to experience a new injection of promotional material. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Meeting format (Re: Moving from hosts to sponsors)
Andy, As I hope everybody knows, about 60% of the IASA budget comes from meeting fees, and we must make enough surplus on the meetings to fund the secretariat. So, if we did decide to change the nature of any of our meetings, we'd really have to understand the budget implications. That being said, we should certainly explore alternatives to the present way of doing things. My own experience as a WG chair is that interim meetings are highly effective when starting out on a new effort and consensus has to be reached on fundamentals. I'm less convinced that longer meetings are useful when refining complex documents - that is when editorial teams and issue trackers seem to be effective. As I said the other day, cross-fertilization remains important. It's hard to imagine how we could run multiple in-depth meetings in parallel without losing cross-fertilization. Maybe we could experiment with a meeting with one day (out of 4.5) dedicated to 7 in-depth WG meetings, and the other 3.5 days traditional? Brian Andy Bierman wrote: Harald Alvestrand wrote: Andy Bierman wrote: Ray Pelletier wrote: ... A more workable model would be to treat the current type of meeting as an Annual Plenary, full of Power-Point laden 2 hour BOFs, and status meetings of almost no value in the production of standards-track protocols. The other 2 IETFs would be Working Group Meetings. Essentially, this as a collection of WG Interim Meetings. WGs meet for 1-3 days and mash through documents and get them done fast. Decisions validated on the WG mailing list within 2 weeks of IETF Friday. are you seeing these as meetings of *all* WGs, or do you envision multiple meetings, each with some subset of WGs? If (guesstimate) 50 of 120 WGs decide to meet in this format, the number of parallell tracks at one monster meeting will run between 15 and 30. There would need to be lots of overlap. Lots of preparation for joint-meetings would be needed. Not everybody can work on everything at once. (More accurately, most people won't be able to read email while ignoring the meeting in as many WGs. ;-) Meeting slots would be divided into 3 categories, times, allocations, and proportions decided by the IESG. - WG - intra-area - inter-area This might even lead to more shared work, more cross area review, more consistency. You know -- proper Engineering. (Maybe we have way too many WGs. That problem is out of scope here.) It should take 1 year to get a standards-track RFC out the door. Not 6+ years. The solution is obvious. Quit messing around and get some work done. This turtle pace has got to end. Interim meetings should not be a almost-never way to get work done. Any IETF veteran knows it's the only way anything gets done around here. I'm intrigued by the idea of replacing one or more IETF meetings with large interims, but would like to work through exactly what's being proposed. (Of course, scheduling 2 years out DOES interfere with quick experiments) The meeting slot allocation (General Agenda) is set in stone 30 days in advance of IETF Sunday. BOF agendas are due 30 days in advance of IETF Sunday. WG agendas are due 15 days in advance of IETF Sunday. No exceptions. and cancellation on those WGs who do not provide such agendas? Just checking Remote audio and jabber for all meetings of course, and better remote meeting participation tools over time. If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. The meeting fee represented 41% of the total cost of my attendance to IETF 65. IMO, sponsors at the Plenary meeting would be appropriate, and could help the IETF fund a cheaper, more stable, IETF-controlled, conference. My cost for this meeting (VERY rough, off the top of my head): - Hotel: 800 dollars - Food: 400 dollars - Airfare: 1200 dollars - Meeting fee: 550 dollars - Misc: 50 dollars Total: 3000 dollars. Meeting fee percentage: 18%. But we've been around the how much does the meeting fee matter bush before. Next time I go to Europe instead of US, our expense ratios will be reversed. These 2 data points aren't meaningful. Cheaper is better, and how much it matters varies widely between people. Harald Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian, this is not universally true. With cheaper air fares and not staying in the overpriced Hilton hotel rooms, my IETF65 meeting fee was almost exactly the same as my combined hotel and air fare costs. For those of us not on corporate expense accounts, it's the total amount that matters. Understood, but you are fortunate to find cheap airfares from where you happen to live. Also note that the meeting rooms have to be paid for, either by filling our room block at the main hotel, or with hard cash. Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
... I guess I was just wishing out loud when I said maybe the meeting fee could trend down instead of up. I would be happy if the IETF had more control over the meetings We have complete control since December 15, 2005. so the fees were stable, The fees have to cover our costs. It would be wonderful if the costs were stable too, but they're not. the network was stable (use sponsor money to buy more gear and let our ace NOC team control it), Firstly our NOC team is a group of volunteers. Secondly, they are always significantly helped by the local host. Thirdly, we do have some equipment but it's unclear that it's practical to own 100% of what we need and to ship it to international locations. the venues were set far enough in advance to give me the maximum travel options. Agreed, we are trying to get ahead of the game on fixing locations. The old single sponsor system isn't working anymore. I'm concerned the IETF will fix the problem by raising the meeting fee $50 every 6 months. This year is the first time we have actually had full visibility of the budget, now 100% controlled by IASA. So while I fully agree with this concern, it's only when we plan the 2007 budget that we'll have a complete handle on these questions. Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Brian E Carpenter wrote: Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. this is not universally true. With cheaper air fares and not staying ... Understood, but you are fortunate to find cheap airfares from where you happen to live. Also note that the meeting rooms have to be paid for, either by filling our room block at the main hotel, or with hard cash. Brian, He is already staying at some other hotel. Are we having a problem filling the room block. More generally, how does the question of lower attendance fees and broader participation have anything to do with the room block? d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Dave Crocker wrote: Brian E Carpenter wrote: Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. this is not universally true. With cheaper air fares and not staying ... Understood, but you are fortunate to find cheap airfares from where you happen to live. Also note that the meeting rooms have to be paid for, either by filling our room block at the main hotel, or with hard cash. Brian, He is already staying at some other hotel. Are we having a problem filling the room block. I don't have the numbers for Dallas yet, but it's a concern for every meeting where the meeting rooms are free unless we don't fill the block. More generally, how does the question of lower attendance fees and broader participation have anything to do with the room block? Because money is fungible. More people paying a lower fee who all stay at a budget hotel might make the overall budget better or worse; it just depends how the numbers work out. Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. this is not universally true. With cheaper air fares and not staying ... Understood, but you are fortunate to find cheap airfares from where you happen to live. Also note that the meeting rooms have to be paid for, either by filling our room block at the main hotel, or with hard cash. Oh. This is the 'what if everyone does it' argument. Given that they everyone doesn't go to the cheaper place now, what makes you think that reducing the meeting fees will cause those (types of attendees) currently staying in the conference hotel will not continue to do so? There are multiple factors that come into the choice of hotel. For some, price dominates. For others convenience to the event and comfort in their room are more important. That won't change. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Meeting format (Re: Moving from hosts to sponsors)
Brian E Carpenter wrote: Andy, As I hope everybody knows, about 60% of the IASA budget comes from meeting fees, and we must make enough surplus on the meetings to fund the secretariat. So, if we did decide to change the nature of any of our meetings, we'd really have to understand the budget implications. That being said, we should certainly explore alternatives to the present way of doing things. My own experience as a WG chair is that interim meetings are highly effective when starting out on a new effort and consensus has to be reached on fundamentals. I'm less convinced that longer meetings are useful when refining complex documents - that is when editorial teams and issue trackers seem to be effective. As I said the other day, cross-fertilization remains important. It's hard to imagine how we could run multiple in-depth meetings in parallel without losing cross-fertilization. Maybe we could experiment with a meeting with one day (out of 4.5) dedicated to 7 in-depth WG meetings, and the other 3.5 days traditional? This would be great. If we could get the sponsors who who have paid for the entire interim meeting costs somewhere else chip in, the the IETF could extend room and network services until 8 PM Friday, and we could have Interim Friday. This would have the least impact on regular IETF activities. Since only dedicated people show up to interims anyway, holding them on Friday won't impact the masses in the slightest. Of course, you would need volunteer WGs who even want to have a 1 day interim instead of a 2 hour slot in Montreal. (NETCONF WG volunteers right now ;-) (IMO, BOFs should be early in the week, not on Friday. Cross-area review of new ideas is just as important as anything else.) Brian Andy Andy Bierman wrote: Harald Alvestrand wrote: Andy Bierman wrote: Ray Pelletier wrote: ... A more workable model would be to treat the current type of meeting as an Annual Plenary, full of Power-Point laden 2 hour BOFs, and status meetings of almost no value in the production of standards-track protocols. The other 2 IETFs would be Working Group Meetings. Essentially, this as a collection of WG Interim Meetings. WGs meet for 1-3 days and mash through documents and get them done fast. Decisions validated on the WG mailing list within 2 weeks of IETF Friday. are you seeing these as meetings of *all* WGs, or do you envision multiple meetings, each with some subset of WGs? If (guesstimate) 50 of 120 WGs decide to meet in this format, the number of parallell tracks at one monster meeting will run between 15 and 30. There would need to be lots of overlap. Lots of preparation for joint-meetings would be needed. Not everybody can work on everything at once. (More accurately, most people won't be able to read email while ignoring the meeting in as many WGs. ;-) Meeting slots would be divided into 3 categories, times, allocations, and proportions decided by the IESG. - WG - intra-area - inter-area This might even lead to more shared work, more cross area review, more consistency. You know -- proper Engineering. (Maybe we have way too many WGs. That problem is out of scope here.) It should take 1 year to get a standards-track RFC out the door. Not 6+ years. The solution is obvious. Quit messing around and get some work done. This turtle pace has got to end. Interim meetings should not be a almost-never way to get work done. Any IETF veteran knows it's the only way anything gets done around here. I'm intrigued by the idea of replacing one or more IETF meetings with large interims, but would like to work through exactly what's being proposed. (Of course, scheduling 2 years out DOES interfere with quick experiments) The meeting slot allocation (General Agenda) is set in stone 30 days in advance of IETF Sunday. BOF agendas are due 30 days in advance of IETF Sunday. WG agendas are due 15 days in advance of IETF Sunday. No exceptions. and cancellation on those WGs who do not provide such agendas? Just checking Remote audio and jabber for all meetings of course, and better remote meeting participation tools over time. If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. The meeting fee represented 41% of the total cost of my attendance to IETF 65. IMO, sponsors at the Plenary meeting would be appropriate, and could help the IETF fund a cheaper, more stable, IETF-controlled, conference. My cost for this meeting (VERY rough, off the top of my head): - Hotel: 800 dollars - Food: 400 dollars - Airfare: 1200 dollars - Meeting fee: 550 dollars - Misc: 50 dollars Total: 3000 dollars. Meeting fee percentage: 18%. But we've been around the how much does the meeting fee matter bush before. Next time I go to Europe instead of US, our expense ratios will be reversed. These 2 data points aren't meaningful. Cheaper is better, and how
Re: Meeting format (Re: Moving from hosts to sponsors)
(IMO, BOFs should be early in the week, not on Friday. Cross-area review of new ideas is just as important as anything else.) This is an interesting suggestion. Any meeting that is early in the week gets the benefit of follow-on hallways discussions, during the rest of the week. However BOFs are a specific class of meetings that tend to need this more than most. BOFs are trying to gain interest and even a good BOF will tend to engender some fuzziness, with resulting efforts at clarification. This often benefits most from the combinations of smaller discussions that are so common during IETF meetings. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Meeting format (Re: Moving from hosts to sponsors)
For what it's worth, this approach seemed to work reasonably well for the SIP P2P BOF + ad-hoc (or interim) meeting. The former was on Tuesday, the latter on Friday afternoon. Dave Crocker wrote: (IMO, BOFs should be early in the week, not on Friday. Cross-area review of new ideas is just as important as anything else.) This is an interesting suggestion. Any meeting that is early in the week gets the benefit of follow-on hallways discussions, during the rest of the week. However BOFs are a specific class of meetings that tend to need this more than most. BOFs are trying to gain interest and even a good BOF will tend to engender some fuzziness, with resulting efforts at clarification. This often benefits most from the combinations of smaller discussions that are so common during IETF meetings. d/ ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Sat, Mar 25, 2006 at 12:43:57PM -0500, Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian, this is not universally true. With cheaper air fares and not staying in the overpriced Hilton hotel rooms, my IETF65 meeting fee was almost exactly the same as my combined hotel and air fare costs. For those of us not on corporate expense accounts, it's the total amount that matters. Being trapped away from the IETF hotel for one night by the flood, the quality of a nearish (5-10 min drive, probably 1h walk) motel was a little of an eye opener, very similar quality for $69+taxes, and a bigger bathroom. Why the Hilton creates such enormous rooms with such small en-suites is a mystery to me :) -- Tim/::1 ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
technical tutorials (was: RE: Moving from hosts to sponsors)
-Original Message- From: Hallam-Baker, Phillip [mailto:[EMAIL PROTECTED] I don't think that the current meetings are power-point laden summaries, but that would actually be useful. I often end up going to sessions at conferences to find out what a WG is intended to achieve. This only happens at IETF in the BOFs. I am not too worried about ending up with a trade show. The real danger as I see it is adding a speaker track or having open access to the trade show. A secondary risk is that people who want to go to attend the IETF would get seconded to man the booth. I believe that I made this proposal in the past, in a plenary session a while ago, when numbers in the IETF particpation were the issue. Discussions hold then led to the edu track, which is however focused on IETF process and not on technical or tutorial content. I do not see why should not the IETF offer a full Sunday track of tutorials with technical content. Why should one go to a industry conference or trade show to hear what is going on in an IETF WG, when the principal contributors (WG chairs, editors) who usually give these talks are all attending the IETF meetings? Having a full Sunday track of tutorials would not only attract new people to come to the IETF and help them justify to their employers and to themselves the cost of the travel, but also improve the level of understanding of the technical material in the WGs, increasing the chances that new attendees would become active participants in a shorter time. We can even play with different fees structure (conference only, tutorial only, conference + tutorial) to help people optimize their costs. The extra money resulting from the tutorial fees and increased participation would lower sponsoring costs, and hopefully the meeting fees for the technical contributors. Dan ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Sun, 26 Mar 2006, Tim Chown wrote: On Sat, Mar 25, 2006 at 12:43:57PM -0500, Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian, this is not universally true. With cheaper air fares and not staying in the overpriced Hilton hotel rooms, my IETF65 meeting fee was almost exactly the same as my combined hotel and air fare costs. For those of us not on corporate expense accounts, it's the total amount that matters. Being trapped away from the IETF hotel for one night by the flood, the quality of a nearish (5-10 min drive, probably 1h walk) motel was a little of an eye opener, very similar quality for $69+taxes, and a bigger bathroom. Why the Hilton creates such enormous rooms with such small en-suites is a mystery to me :) The hotel was built by Trammell Crow and has a been a loews, wyndham and now a hilton (but only for 3 months). So, while hilton hotels inc gets some blame for certain things (rooms had better soap when it was wyndham) it hardly assumes blame for all of them. -- -- Joel Jaeggli Unix Consulting [EMAIL PROTECTED] GPG Key Fingerprint: 5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2 ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: technical tutorials (was: RE: Moving from hosts to sponsors)
--On Sunday, 26 March, 2006 14:50 +0200 Romascanu, Dan \\(Dan\\) [EMAIL PROTECTED] wrote: I believe that I made this proposal in the past, in a plenary session a while ago, when numbers in the IETF particpation were the issue. Discussions hold then led to the edu track, which is however focused on IETF process and not on technical or tutorial content. I do not see why should not the IETF offer a full Sunday track of tutorials with technical content. Why should one go to a industry conference or trade show to hear what is going on in an IETF WG, when the principal contributors (WG chairs, editors) who usually give these talks are all attending the IETF meetings? Having a full Sunday track of tutorials would not only attract new people to come to the IETF and help them justify to their employers and to themselves the cost of the travel, but also improve the level of understanding of the technical material in the WGs, increasing the chances that new attendees would become active participants in a shorter time. We can even play with different fees structure (conference only, tutorial only, conference + tutorial) to help people optimize their costs. The extra money resulting from the tutorial fees and increased participation would lower sponsoring costs, and hopefully the meeting fees for the technical contributors. Dan, I see one major problem with this. I tried to raise it with the EDU team before Dallas but, other than one set of offline comments from an individual, have gotten no response. Despite all of the noise in the IPR WG, the biggest risks to a standards body involve claims that the review and approval process have been captured or manipulated by particular interests, causing the documents that are produced to reflect those manipulations rather than open and balanced community consensus. A tutorial whose subject matter is how to get things done in the IETF -- how we are structured, how we do business, the tools we use, and even what one needs to know technically and structurally to write an I-D or RFC -- are not problematic. But, as soon as we start giving technical tutorials that related to areas that are under standardization, there is a risk of someone later claiming that the tutorial content was biased in one way or another that impacted the standardization choices we made. That would be extremely bad news... possibly of the variety that could have the EDU team or the IESG neck-deep in lawyers. So, if there are to be technical tutorials, I suggest that you start working on an organizational structure that would keep the decisions about which sessions to hold and their content at arms-length or further from anyone with decision-making leadership in the IETF. Even then, there are risks. But a decision made by an EDU team that operates under even general IESG supervision, or with a lecturer who is involved in the standards process and who is taking positions there (or is associated with a company that is doing so), are really poor ideas if we want to preserve both the fact and appearance of fairness in the standards process. john ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. My head hurts. If more people can participate how come we would need *smaller* venues? And by what miracle does lowering the fee allow us to reduce the cost per participant? In my case, the meeting fees are small compared to travel and hotel costs. Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
one thing, the first thing we would have to do in the IETF - if we adopted a model like this - is to establish a marketing over-sight function to ensure fair and equitable disposition of sponsorship funds. Eric, I am not sure why this would be required. In so far as it's required, it's clearly the IAOC's job. Brian ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian, this is not universally true. With cheaper air fares and not staying in the overpriced Hilton hotel rooms, my IETF65 meeting fee was almost exactly the same as my combined hotel and air fare costs. For those of us not on corporate expense accounts, it's the total amount that matters. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Henning Schulzrinne wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. Brian, this is not universally true. With cheaper air fares and not staying in the overpriced Hilton hotel rooms, my IETF65 meeting fee was almost exactly the same as my combined hotel and air fare costs. For those of us not on corporate expense accounts, it's the total amount that matters. Sometimes, people even use frequent flier miles, double up in rooms, and don't eat every meal in an over-priced restaurant, just to attend an IETF. (Not me, but some people ;-) For people paying their own way, the meeting fee is the only fixed cost in the trip. It's expensive already, and trending upwards (not expensive if you stay all 5 days, but some of us don't). I guess I was just wishing out loud when I said maybe the meeting fee could trend down instead of up. I would be happy if the IETF had more control over the meetings so the fees were stable, the network was stable (use sponsor money to buy more gear and let our ace NOC team control it), and the venues were set far enough in advance to give me the maximum travel options. The old single sponsor system isn't working anymore. I'm concerned the IETF will fix the problem by raising the meeting fee $50 every 6 months. Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Brian E Carpenter wrote: Indeed. Not only is it small, it isn't where corporate bean counters put their attention, which is air fare, hotel, and per diem. 1. There are more bean counters to consider, that only those in commercial corporations, if the IETF still considers it important to be inclusive beyond the interests of corporations. 2. Even corporations pay attention to the total cost for going to an event. -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
I think that the IETF neglects (or, rather, has neglected in the past) many possible opportunities for sponsorship. That implies that increasing the income from sponsorship should be possible. People who are concerned with this issue should talk (or email) our IAD, Ray Pelletier, who has a number of ideas in this area (and who reads this list). If people feel that some sorts of sponsorship are not appropriate, I am sure that Ray would like that input too. In the new IASA / NeuStar system, there is no choice but to be realistic with cost figures. Note that the registration fee and the attendance both went up with this meeting, which of course means that revenue increased. I actually think that, with revenue and sponsorship both increasing, the IETF should be able to improve the meeting support and experience even more in the future. Regards Marshall On Mar 24, 2006, at 12:52 AM, JORDI PALET MARTINEZ wrote: I don't think the meeting fees could actually go down, may be more in the other way around if we are realistic with the cost figures. Actually the cost is already high for a sponsor, and I believe trying to get more from the industry (or other kind of sponsors) for each meeting will be really difficult. Regards, Jordi De: Andy Bierman [EMAIL PROTECTED] Responder a: [EMAIL PROTECTED] Fecha: Thu, 23 Mar 2006 19:34:00 -0800 Para: [EMAIL PROTECTED] CC: Keith Moore moore@cs.utk.edu, ietf@ietf.org ietf@ietf.org, [EMAIL PROTECTED], Michael StJohns [EMAIL PROTECTED] Asunto: Re: Moving from hosts to sponsors Dave Crocker wrote: Michael StJohns wrote: What I think Jordi is saying is that he wants the US sponsors to subsidize the cost of the overseas meetings. At least that's what it works out to be This view can be mapped to a classic model that would have significant benefits for the IETF: A host gets all sorts of marketing leverage out of the role in producing an IETF. There is nothing that requires that the event site management effort be coupled with a particular host's venue. If we moved to a model of having companies provide sponsorship funds, in return for which they get appropriate marketing presence, then we could have meeting venue management move to the sort of predictable and timely basis -- ie, far enough ahead of time -- that has been a concern for many years. Amen! And maybe the meeting fees could actually go down with enough sponsors. An additional room like the terminal room (not out in the open) could be used. Also, the IETF could maintain control of the network if there were multiple sponsors instead of a single host. They would not be allowed to ignore the advice of the NOC team, and let the wireless meltdown right off the bat. d/ Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ** The IPv6 Portal: http://www.ipv6tf.org Barcelona 2005 Global IPv6 Summit Slides available at: http://www.ipv6-es.com This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Marshall Eubanks wrote: I think that the IETF neglects (or, rather, has neglected in the past) many possible opportunities for sponsorship. That implies that increasing the income from sponsorship should be possible. People who are concerned with this issue should talk (or email) our IAD, Ray Pelletier, who has a number of ideas in this area (and who reads this list). If people feel that some sorts of sponsorship are not appropriate, I am sure that Ray would like that input too. In the new IASA / NeuStar system, there is no choice but to be realistic with cost figures. Note that the registration fee and the attendance both went up with this meeting, which of course means that revenue increased. I actually think that, with revenue and sponsorship both increasing, the IETF should be able to improve the meeting support and experience even more in the future. I know you've heard this all before, but it's been getting increasingly difficult for us WG Chairs to get all the key people working on a protocol to fly across the planet for a 2 hour meeting. These are busy people who can't afford to block out an entire week because they don't know when or where the 2 hour meeting is going to be. (This even applies to some WG Chairs ;-) I would support any plan that will make meetings cheaper and easier to attend for everybody. I'd like quick action, not a 2 year study to think about it. Regards Marshall Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
JORDI PALET MARTINEZ wrote: I don't think the meeting fees could actually go down, may be more in the other way around if we are realistic with the cost figures. Actually the cost is already high for a sponsor, and I believe trying to get more from the industry (or other kind of sponsors) for each meeting will be really difficult. IMO, the current trend and situation is not sustainable. It may be fine for professional standards attenders, but I'm trying to get some people to show up in my WG who actually write code and run networks for a living. They don't want to come here anymore. Regards, Jordi Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Mar 24, 2006, at 7:37 AM, Andy Bierman wrote: Marshall Eubanks wrote: I think that the IETF neglects (or, rather, has neglected in the past) many possible opportunities for sponsorship. That implies that increasing the income from sponsorship should be possible. People who are concerned with this issue should talk (or email) our IAD, Ray Pelletier, who has a number of ideas in this area (and who reads this list). If people feel that some sorts of sponsorship are not appropriate, I am sure that Ray would like that input too. In the new IASA / NeuStar system, there is no choice but to be realistic with cost figures. Note that the registration fee and the attendance both went up with this meeting, which of course means that revenue increased. I actually think that, with revenue and sponsorship both increasing, the IETF should be able to improve the meeting support and experience even more in the future. I know you've heard this all before, but it's been getting increasingly difficult for us WG Chairs to get all the key people working on a protocol to fly across the planet for a 2 hour meeting. These are busy people who can't afford to block out an entire week because they don't know when or where the 2 hour meeting is going to be. (This even applies to some WG Chairs ;-) I would support any plan that will make meetings cheaper and easier to attend for everybody. I'd like quick action, not a 2 year study to think about it. Now THAT is a different matter. I think that there should be a block in period, starting maybe 1 week in advance, after which the schedule should only change because of force majeure. (Yes, that includes floods.) If people are flying across the planet to a meeting, the time of the meeting needs to be predictable something more than one RTT by airplane in advance. Any policy to the contrary is just broken and wrong. Regards Marshall Regards Marshall Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Meeting format (Re: Moving from hosts to sponsors)
Andy Bierman wrote: Ray Pelletier wrote: Andy Bierman wrote: JORDI PALET MARTINEZ wrote: I don't think the meeting fees could actually go down, may be more in the other way around if we are realistic with the cost figures. Actually the cost is already high for a sponsor, and I believe trying to get more from the industry (or other kind of sponsors) for each meeting will be really difficult. IMO, the current trend and situation is not sustainable. It may be fine for professional standards attenders, but I'm trying to get some people to show up in my WG who actually write code and run networks for a living. They don't want to come here anymore. What changes would have to be made so that they would want to attend? my $0.02: Nothing -- not in the current meeting format. A more workable model would be to treat the current type of meeting as an Annual Plenary, full of Power-Point laden 2 hour BOFs, and status meetings of almost no value in the production of standards-track protocols. The other 2 IETFs would be Working Group Meetings. Essentially, this as a collection of WG Interim Meetings. WGs meet for 1-3 days and mash through documents and get them done fast. Decisions validated on the WG mailing list within 2 weeks of IETF Friday. are you seeing these as meetings of *all* WGs, or do you envision multiple meetings, each with some subset of WGs? If (guesstimate) 50 of 120 WGs decide to meet in this format, the number of parallell tracks at one monster meeting will run between 15 and 30. I'm intrigued by the idea of replacing one or more IETF meetings with large interims, but would like to work through exactly what's being proposed. (Of course, scheduling 2 years out DOES interfere with quick experiments) The meeting slot allocation (General Agenda) is set in stone 30 days in advance of IETF Sunday. BOF agendas are due 30 days in advance of IETF Sunday. WG agendas are due 15 days in advance of IETF Sunday. No exceptions. and cancellation on those WGs who do not provide such agendas? Just checking Remote audio and jabber for all meetings of course, and better remote meeting participation tools over time. If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. The meeting fee represented 41% of the total cost of my attendance to IETF 65. IMO, sponsors at the Plenary meeting would be appropriate, and could help the IETF fund a cheaper, more stable, IETF-controlled, conference. My cost for this meeting (VERY rough, off the top of my head): - Hotel: 800 dollars - Food: 400 dollars - Airfare: 1200 dollars - Meeting fee: 550 dollars - Misc: 50 dollars Total: 3000 dollars. Meeting fee percentage: 18%. But we've been around the how much does the meeting fee matter bush before. Cheaper is better, and how much it matters varies widely between people. Harald ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Andy Bierman [EMAIL PROTECTED] writes: my $0.02: Nothing -- not in the current meeting format. A more workable model would be to treat the current type of meeting as an Annual Plenary, full of Power-Point laden 2 hour BOFs, and status meetings of almost no value in the production of standards-track protocols. The other 2 IETFs would be Working Group Meetings. Essentially, this as a collection of WG Interim Meetings. WGs meet for 1-3 days and mash through documents and get them done fast. Decisions validated on the WG mailing list within 2 weeks of IETF Friday. ... Remote audio and jabber for all meetings of course, and better remote meeting participation tools over time. If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. I think there are some good ideas here. I find that WG meetings are too short to get anything useful done, and all the issues that would benefit of longer face-to-face discussions are taken to the mailing list before any concrete proposal are fleshed out. I agree that interim WG meetings would be useful, but here is a further proposal: Have open virtual interim-meetings for specific WG work items. A working group could declare to have a two-hour meeting on jabber (and possibly audio) on a specific date to sort out all technical problems with a specific document. The audience will then be more inclined to have actually read the document. If there is an agenda and list of open issues, going through the open issues until there is one (or more) fully fleshed out proposed solution is hopefully not too un-realistic. The proposal can then be written up and taken back to the WG for mailing list discussions. /Simon ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
I agree that interim WG meetings would be useful, but here is a further proposal: There are quite a few really good ideas for improvements to IETF productivity. The problem with taking a particular suggestion and then adding others to it will be that nothing gets considered in detail and nothing gets done. The original suggestion was quite specific: Take the kinds of funds spent by meeting hosts and, instead, have them become meeting sponsors, with meeting venue logistics handled by the IETF itself, separately. In return for meeting sponsorship, give the sponsor various marketing opportunities as the meeting, similar to what hosts currently enjoy. In other words, I am suggesting a single, conceptually small change to the current model. Its purpose is to permit vastly better meeting planning than we currently can achieve, due to the delays inherent in having meeting hosts. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. In my case, the meeting fees are small compared to travel and hotel costs. I think there are some good ideas here. I find that WG meetings are too short to get anything useful done, and all the issues that would benefit of longer face-to-face discussions are taken to the mailing list before any concrete proposal are fleshed out. But is the WG the place to have the discussion? In most of the WG's that I attended this week, technical discussions were typically between 3 to 5 experts in the field who know everything about the topic, the rest of the room either couldn't follow the discussion or had nothing to contribute. That means that there are 50 or so people sitting there doing nothing. While I agree that face-2-face discussions are useful, I much rather see the discussion take place in the hallway, then have one person report on the outcome. Henk -- Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The NetherlandsThe NetherlandsMobile: +31.6.55861746 -- 1160438400. Watch this space... ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: Moving from hosts to sponsors
Dave, Certainly there are organizations that do this. Those organizations are significantly different from the IETF. For one thing, the first thing we would have to do in the IETF - if we adopted a model like this - is to establish a marketing over-sight function to ensure fair and equitable disposition of sponsorship funds. -- Eric -- -Original Message- -- From: Dave Crocker [mailto:[EMAIL PROTECTED] -- Sent: Thursday, March 23, 2006 7:45 PM -- To: Michael StJohns -- Cc: Keith Moore; ietf@ietf.org; [EMAIL PROTECTED] -- Subject: Moving from hosts to sponsors -- -- Michael StJohns wrote: -- What I think Jordi is saying is that he wants the US sponsors to -- subsidize the cost of the overseas meetings. At least -- that's what it -- works out to be -- -- This view can be mapped to a classic model that would have -- significant benefits -- for the IETF: -- -- -- A host gets all sorts of marketing leverage out of the -- role in producing an -- IETF. -- -- There is nothing that requires that the event site -- management effort be coupled -- with a particular host's venue. -- -- If we moved to a model of having companies provide -- sponsorship funds, in return -- for which they get appropriate marketing presence, then we -- could have meeting -- venue management move to the sort of predictable and timely -- basis -- ie, far -- enough ahead of time -- that has been a concern for many years. -- -- -- d/ -- -- -- -- -- Dave Crocker -- Brandenburg InternetWorking -- http://bbiw.net -- -- ___ -- Ietf mailing list -- Ietf@ietf.org -- https://www1.ietf.org/mailman/listinfo/ietf -- ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
On Fri Mar 24 16:20:26 2006, Simon Josefsson wrote: Henk Uijterwaal [EMAIL PROTECTED] writes: That means that there are 50 or so people sitting there doing nothing. While I agree that face-2-face discussions are useful, I much rather see the discussion take place in the hallway, then have one person report on the outcome. I think virtual interim meetings may have a similar end result. Virtual interims are far easier to organize, and therefore could be organized around a specific issue or small set of them, rather than trying to concern themselves with the entire output of a WG (or in the case of some WGs, not only their entire output but the output of two or more other WGs as well). One would hope that organizing several specific virtual mini-interims would yield just the handful of interested parties making the time to attend, and not yield 50 people staring blankly at laptops. (Or if they do, at least their cost and travel ought to be minimal). Dave. -- You see things; and you say Why? But I dream things that never were; and I say Why not? - George Bernard Shaw ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
one thing, the first thing we would have to do in the IETF - if we adopted a model like this - is to establish a marketing over-sight function to ensure fair and equitable disposition of sponsorship funds. Eric, I am not sure why this would be required. The IETF already takes in money for events and spends money for events. The change would be that a host also does this and we would, instead, have the host give the money to the IETF/ISOC funds pot for events. I would think that management of the larger pot of IETF event funds would not require any additional mechanisms. d/ -- Dave Crocker Brandenburg InternetWorking http://bbiw.net ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Dave Crocker wrote: I agree that interim WG meetings would be useful, but here is a further proposal: There are quite a few really good ideas for improvements to IETF productivity. The problem with taking a particular suggestion and then adding others to it will be that nothing gets considered in detail and nothing gets done. The original suggestion was quite specific: Take the kinds of funds spent by meeting hosts and, instead, have them become meeting sponsors, with meeting venue logistics handled by the IETF itself, separately. In return for meeting sponsorship, give the sponsor various marketing opportunities as the meeting, similar to what hosts currently enjoy. In other words, I am suggesting a single, conceptually small change to the current model. Its purpose is to permit vastly better meeting planning than we currently can achieve, due to the delays inherent in having meeting hosts. This is much closer to the model that most IEEE and ACM conferences use, and it works well. It does, however, require a persistent organization with legal and financial oversight, and who has sufficient pockets to bear deficits occasionally. Joe ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
There are quite a few really good ideas for improvements to IETF productivity. The problem with taking a particular suggestion and then adding others to it will be that nothing gets considered in detail and nothing gets done. you say that like it's a bad thing. not to pick on you personally, but since you brought this up - it's hard to escape the impression that you'd rather have one half-baked idea adopted without much discussion, than to have serious discussion about what the real problems are and how to solve them. come to think of it, that resembles a lot of what passes for 'engineering' of IETF protocols. Keith Let me follow up to my own message here because I think I responded a bit too quickly, and because I think my original reply didn't really address the point I was trying to make. In particular I didn't really want to single out Dave - he just happened to be the person who sent the message that triggered this response. I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. When people express that frustration, they often seem to think that the solution to this frustration is to do something rather than just talk about it. In other words, they prefer experimentation to analysis. I share the frustration, but have some doubts about the solution. There are circumstances where experimentation is appropriate. Offhand, these seem to include: (a) the cost of failure of the experiment is small relative to the potential gain, (b) the problem being addressed is so poorly understood or so complex that analysis isn't an option, and (c) the results of the experiment can be evaluated with a reasonable degree of objectivity to inform a decision about whether to do things that way in the future. Even when those circumstances are met, the experiment is usually more valuable if it is carefully designed based on such undertstanding and analysis as is available. What really bothers me is the apparent popularity of a mindset, in a group of people that claims to be doing engineering, that we should just try something without really thinking about it, and without a good way to evaluate the experiment objectively. The fundamental assumption of engineering is that you can make better (more effective, reliable, and cost-effective) solutions to problems if you (a) first understand what problem you are trying to solve, and (b) analyze your proposed solutions (and choose and/or refine them based on analysis) before building them. More and more in IETF we seem to be insisting that we (a) artifically and prematurely limit the scope of an engineering effort to a narrow solution space, (b) decide on that solution without doing any analysis, and (c) let the marketplace or the Internet community pay to conduct very expensive and poorly designed experiments without any good way to evaulate the results or much of an ability to change direction based on what is learned. Examples that pop into my head of this include DKIM, IMA, ZEROCONF. I'm not claiming that this is a complete or representative sample, these are just three things that popped into my head in three seconds. (and hopefully a diverse enough sample that nobody thinks I'm picking on him personally) Of course our management and process issues are different than our protocols, and it would make some sense for us to attack those problems differently. It would be understandable if, out of familiarity, we tried to apply engineering techniques to solving our management and process problems...and it would be understandable if we found that it didn't work well because we didn't know how to do the right kinds of analysis. But what seems to be the case instead is that we hardly use engineering discipline in protocol design. We guess about what will make a good protocol design, and guess about what will improve our management and process also. If it seems like we make good progress on the former and poor progress on the latter, perhaps it's because for management and process issues that affect all of us, we have no good way to artifically narrow the scope of the discussion (and marginalize the nay-sayers) in order to get the apperance of agreement. I'm all for experimentation about how we run meetings as long as we take reasonable care in designing the experiments, identify effective ways to evaluate the results of the experiments, and leave ourselves room to adopt a different course if we don't like the results. It helps if we have controls also. For instance, if we wanted to experiment with allowing vendors to pay for greater exposure, we could do this at one meeting per year for a couple of years and see how we like it in comparison with other meetings. I'm much more concerned about how we design our protocols. Our longstanding habit of marginalizing dissenting voices in order to get agreement within a working group - our
Re: Meeting format (Re: Moving from hosts to sponsors)
Harald Alvestrand wrote: Andy Bierman wrote: Ray Pelletier wrote: ... A more workable model would be to treat the current type of meeting as an Annual Plenary, full of Power-Point laden 2 hour BOFs, and status meetings of almost no value in the production of standards-track protocols. The other 2 IETFs would be Working Group Meetings. Essentially, this as a collection of WG Interim Meetings. WGs meet for 1-3 days and mash through documents and get them done fast. Decisions validated on the WG mailing list within 2 weeks of IETF Friday. are you seeing these as meetings of *all* WGs, or do you envision multiple meetings, each with some subset of WGs? If (guesstimate) 50 of 120 WGs decide to meet in this format, the number of parallell tracks at one monster meeting will run between 15 and 30. There would need to be lots of overlap. Lots of preparation for joint-meetings would be needed. Not everybody can work on everything at once. (More accurately, most people won't be able to read email while ignoring the meeting in as many WGs. ;-) Meeting slots would be divided into 3 categories, times, allocations, and proportions decided by the IESG. - WG - intra-area - inter-area This might even lead to more shared work, more cross area review, more consistency. You know -- proper Engineering. (Maybe we have way too many WGs. That problem is out of scope here.) It should take 1 year to get a standards-track RFC out the door. Not 6+ years. The solution is obvious. Quit messing around and get some work done. This turtle pace has got to end. Interim meetings should not be a almost-never way to get work done. Any IETF veteran knows it's the only way anything gets done around here. I'm intrigued by the idea of replacing one or more IETF meetings with large interims, but would like to work through exactly what's being proposed. (Of course, scheduling 2 years out DOES interfere with quick experiments) The meeting slot allocation (General Agenda) is set in stone 30 days in advance of IETF Sunday. BOF agendas are due 30 days in advance of IETF Sunday. WG agendas are due 15 days in advance of IETF Sunday. No exceptions. and cancellation on those WGs who do not provide such agendas? Just checking Remote audio and jabber for all meetings of course, and better remote meeting participation tools over time. If the meeting fees could be lowered over time because smaller venues are needed 2 out of 3 IETFs, then more people will be able to participate. The meeting fee represented 41% of the total cost of my attendance to IETF 65. IMO, sponsors at the Plenary meeting would be appropriate, and could help the IETF fund a cheaper, more stable, IETF-controlled, conference. My cost for this meeting (VERY rough, off the top of my head): - Hotel: 800 dollars - Food: 400 dollars - Airfare: 1200 dollars - Meeting fee: 550 dollars - Misc: 50 dollars Total: 3000 dollars. Meeting fee percentage: 18%. But we've been around the how much does the meeting fee matter bush before. Next time I go to Europe instead of US, our expense ratios will be reversed. These 2 data points aren't meaningful. Cheaper is better, and how much it matters varies widely between people. Harald Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
Hi - From: Keith Moore moore@cs.utk.edu To: ietf@ietf.org Cc: [EMAIL PROTECTED] Sent: Friday, March 24, 2006 9:47 AM Subject: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors) ... My question is - do others see this as a problem, and (without trying to propose a concrete solution that will be seen as a threat) is there a shared sense that this is a problem and general willingness to try new ways of conducting our discussions? ... I agree with much of your analysis, but I think the problem goes beyond just the ways in which we conduct our discussions. The process of conducting BOFs and developing WG charters has a way of framing discussion, that, although it serves to keep things focused, may also marginalize attempts to look at the problem from a broader architectural perspective. If we could count on perfect architectural foresight in the formulation of WG charters and deliverables, this would not be a serious problem. In some cases, however, I think working groups have carefully engineered a solution to a problem very different from the one which originally served to motivate the work, and may even have completely missed the mark, all while satisfying their charters to the letter. That said, I think there is much to learn about what does and does not work, both from experience in other organizations as well as from what has been tried by various WGs. I expect, however, that we'll find that there is no one size fits all solution. I hope we end up with a toolkit of techniques appropriate to different sizes and shapes of WGs for doing new work, revisions, maintenance, integration and retrofitting. Randy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri Mar 24 17:47:04 2006, Keith Moore wrote: I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. When people express that frustration, they often seem to think that the solution to this frustration is to do something rather than just talk about it. In other words, they prefer experimentation to analysis. I share the frustration, but have some doubts about the solution. What you're saying, I've also heard people complain about in reverse. In other words, there are working groups where a substantial number of people involved in the discussion are not only not going to be implementing the proposals, but don't actually do any kind of implementation within the sphere - we're talking about people discussing the precise semantics of some HTTP extension who aren't involved in doing any webserver related programming, or some people discussing an email issue who limit their interaction with email to having an email address. Or, if you prefer, people are talking and not doing the running code bit. What really bothers me is the apparent popularity of a mindset, in a group of people that claims to be doing engineering, that we should just try something without really thinking about it, and without a good way to evaluate the experiment objectively. Now, wait - I agree up to a point. Yes, we need to carefully analyze what we're doing, because experimentation won't easily show if a proposed solution will actually scale to the level we need, is secure enough, and is flexible enough to cope with future demands that we've not thought of. This much is, hopefully, not up for debate. But there's a really simple experiment that's easy to do, and results in a useful, concrete result. The hypothesis to test is does it actually work, the experiment is suck it and see, and the result is, one hopes, yeah, I did this, with an optional but this bit was tricky that we can feed back into the design process. Unless that experiment is done, we aren't engineers, we're philosophers. The fundamental assumption of engineering is that you can make better (more effective, reliable, and cost-effective) solutions to problems if you (a) first understand what problem you are trying to solve, and (b) analyze your proposed solutions (and choose and/or refine them based on analysis) before building them. We're lucky, because we work in computers, so we can actually make a distinction between building and deploying. Exchanging the word building in this portion of your message for deploying makes me happier with what it says. Changing analyze for building, and I'm in agreement. Dave. -- You see things; and you say Why? But I dream things that never were; and I say Why not? - George Bernard Shaw ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri Mar 24 17:47:04 2006, Keith Moore wrote: I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. When people express that frustration, they often seem to think that the solution to this frustration is to do something rather than just talk about it. In other words, they prefer experimentation to analysis. I share the frustration, but have some doubts about the solution. What you're saying, I've also heard people complain about in reverse. In other words, there are working groups where a substantial number of people involved in the discussion are not only not going to be implementing the proposals, but don't actually do any kind of implementation within the sphere - we're talking about people discussing the precise semantics of some HTTP extension who aren't involved in doing any webserver related programming, or some people discussing an email issue who limit their interaction with email to having an email address. I don' t have a problem with that. IMHO we tend to design with too little regard for the needs of end users, and we need more input from knowledgable users, rather than less. Or, if you prefer, people are talking and not doing the running code bit. It may be that we place too much emphasis on running code in IETF today. In ARPAnet days, when the user community was small and homogeneous but platforms were diverse (different word/character sizes, different character sets, different limitations of operating systems and networking hardware), and goals for protocols were modest, merely being able to implement a protocol across different platforms was one of the biggest barriers to adoption. In that environment, being able to demonstrate running code on multiple platforms was nearly sufficient to demonstrate the viability of a protocol. Besides, since the net was small, it wasn't terribly hard to make changes should they be found to be necessary. These days running code serves as proof-of-concept and also as a way to validate the specification. It doesn't say anything about the quality of the design - not efficiency, nor usability, nor scalability, nor security. etc. What really bothers me is the apparent popularity of a mindset, in a group of people that claims to be doing engineering, that we should just try something without really thinking about it, and without a good way to evaluate the experiment objectively. Now, wait - I agree up to a point. Yes, we need to carefully analyze what we're doing, because experimentation won't easily show if a proposed solution will actually scale to the level we need, is secure enough, and is flexible enough to cope with future demands that we've not thought of. This much is, hopefully, not up for debate. But there's a really simple experiment that's easy to do, and results in a useful, concrete result. The hypothesis to test is does it actually work, the experiment is suck it and see, and the result is, one hopes, yeah, I did this, with an optional but this bit was tricky that we can feed back into the design process. Unless that experiment is done, we aren't engineers, we're philosophers. I agree that those kinds of experiments can be quite valuable, though I'm having a hard time remembering when such an experiment was indicated in an IETF WG that I've been involved in. I have seen several kinds of experiments of the form let's see what happens if we do this nonstandard thing with SMTP - will existing servers handle it? and I've generally regarded those experiments as invalid because they tend to lack any analysis of the sample space or any attempt to get a representative sample. They can prove that something doesn't work, but rarely can they demonstrate that something does work reliably in the wild. (OTOH if you know reliably that there are only a few implementations of a protocol, such experiments might be more valuable.) The fundamental assumption of engineering is that you can make better (more effective, reliable, and cost-effective) solutions to problems if you (a) first understand what problem you are trying to solve, and (b) analyze your proposed solutions (and choose and/or refine them based on analysis) before building them. We're lucky, because we work in computers, so we can actually make a distinction between building and deploying. Exchanging the word building in this portion of your message for deploying makes me happier with what it says. Changing analyze for building, and I'm in agreement. I should have said analyze...before deploying. I also believe in building prototypes and reference implementations, but that's not a substitute for analysis. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Keith Moore [mailto:[EMAIL PROTECTED] I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. Quite so, which is why most of us feel that there should be a strong bias in favor of action and experimentation rather than inertia and analysis. I would like to see much more interchange across the standards forums. One of the benefits of doing that is that more people would have experience of working different ways and with different groups of people. I would like to be in a position of choosing the venue acording to the type of work and the benefits of the particular forum rather than having to take the efficiency of the venues into account. We have a competitive market in standards forums. This is a good thing not least because there is no possibility that the IETF could ever have managed the workload of W3C and OASIS in addition to its own interests. smime.p7s Description: S/MIME cryptographic signature ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
RE: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
From: Keith Moore [mailto:[EMAIL PROTECTED] From: Keith Moore [mailto:[EMAIL PROTECTED] I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. Quite so, which is why most of us feel that there should be a strong bias in favor of action and experimentation rather than inertia and analysis. I hope you're wrong about that, because this is supposed to be an engineering organization. It is infeasible to determine by experimentation how well a protocol will work at Internet scale. We are talking about experimentation in ways of doing business at the scale of an organization with approximately 2,500 or so active members. The individual working groups have at most 100 active members. That is four orders of magnitude less than Internet scale. The comparison is utterly ludicrous and overblown. The IETF has singularly failed to scale to Internet size. It has not even made a serious effort. It is extremely unlikely that any organization would work at that scale. smime.p7s Description: S/MIME cryptographic signature ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri Mar 24 19:50:15 2006, Keith Moore wrote: In other words, there are working groups where a substantial number of people involved in the discussion are not only not going to be implementing the proposals, but don't actually do any kind of implementation within the sphere - we're talking about people discussing the precise semantics of some HTTP extension who aren't involved in doing any webserver related programming, or some people discussing an email issue who limit their interaction with email to having an email address. I don' t have a problem with that. IMHO we tend to design with too little regard for the needs of end users, and we need more input from knowledgable users, rather than less. That input needs to be present in defining the problem, not the solution. Or, if you prefer, people are talking and not doing the running code bit. It may be that we place too much emphasis on running code in IETF today. I'd say we place too little. In ARPAnet days, when the user community was small and homogeneous but platforms were diverse (different word/character sizes, different character sets, different limitations of operating systems and networking hardware), and goals for protocols were modest, merely being able to implement a protocol across different platforms was one of the biggest barriers to adoption. In that environment, being able to demonstrate running code on multiple platforms was nearly sufficient to demonstrate the viability of a protocol. Besides, since the net was small, it wasn't terribly hard to make changes should they be found to be necessary. We have fewer platforms, and they're all running with the same 8-bit byte, (or as close as makes no difference), and they all do UTF-8 easily, let alone ASCII, so yes, that kind of problem has largely gone away. However, if you're extending IMAP, say, there's a large number of IMAP servers out there which are, internally, massively different beasts, so the in my day argument merely highlights that problems move, they don't go away. These days running code serves as proof-of-concept and also as a way to validate the specification. It doesn't say anything about the quality of the design - not efficiency, nor usability, nor scalability, nor security. etc. No. It doesn't say much about the efficiency, usability, scalability, or security, but it does say a little, and it gives me, for one, a lot better an idea about where the problems in all those areas lie. Maybe I'm a drooling idiot, and this is the equivalent of having to read aloud, in which case I'm sorry. What really bothers me is the apparent popularity of a mindset, in a group of people that claims to be doing engineering, that we should just try something without really thinking about it, and without a good way to evaluate the experiment objectively. Now, wait - I agree up to a point. Yes, we need to carefully analyze what we're doing, because experimentation won't easily show if a proposed solution will actually scale to the level we need, is secure enough, and is flexible enough to cope with future demands that we've not thought of. This much is, hopefully, not up for debate. But there's a really simple experiment that's easy to do, and results in a useful, concrete result. The hypothesis to test is does it actually work, the experiment is suck it and see, and the result is, one hopes, yeah, I did this, with an optional but this bit was tricky that we can feed back into the design process. Unless that experiment is done, we aren't engineers, we're philosophers. I agree that those kinds of experiments can be quite valuable, though I'm having a hard time remembering when such an experiment was indicated in an IETF WG that I've been involved in. It's weird, because I thought that pretty well everyone implemented stuff to this level. For a long time - years - it never occured to me that PoC and probably even deployed implementations didn't exist for some specifications, let alone those going onto the standards track. I have seen several kinds of experiments of the form let's see what happens if we do this nonstandard thing with SMTP - will existing servers handle it? and I've generally regarded those experiments as invalid because they tend to lack any analysis of the sample space or any attempt to get a representative sample. They can prove that I've seen discussions of a similar nature, not formal experiments - perhaps we're saying the same thing. I've also seen discussions concerning are we sure that feature X works in the wild, with the result as we have done X for some time and have seen no failures. something doesn't work, but rarely can they demonstrate that something does work reliably in the wild. (OTOH if you know reliably that there are only a few implementations of a protocol, such experiments might be more valuable.) I'm not sure that
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
I think that Dave's message reflects a common frustration in IETF that we talk a lot about particular problems and never seem to do anything about them. Quite so, which is why most of us feel that there should be a strong bias in favor of action and experimentation rather than inertia and analysis. I hope you're wrong about that, because this is supposed to be an engineering organization. It is infeasible to determine by experimentation how well a protocol will work at Internet scale. We are talking about experimentation in ways of doing business at the scale of an organization with approximately 2,500 or so active members. The individual working groups have at most 100 active members. That is four orders of magnitude less than Internet scale. Ah, the message you replied to was talking about both. But as I said in that message: I'm all for experimentation about how we run meetings as long as we take reasonable care in designing the experiments, identify effective ways to evaluate the results of the experiments, and leave ourselves room to adopt a different course if we don't like the results. and you could generalize this a bit from how we run meetings to how we conduct business. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri Mar 24 19:50:15 2006, Keith Moore wrote: In other words, there are working groups where a substantial number of people involved in the discussion are not only not going to be implementing the proposals, but don't actually do any kind of implementation within the sphere - we're talking about people discussing the precise semantics of some HTTP extension who aren't involved in doing any webserver related programming, or some people discussing an email issue who limit their interaction with email to having an email address. I don' t have a problem with that. IMHO we tend to design with too little regard for the needs of end users, and we need more input from knowledgable users, rather than less. That input needs to be present in defining the problem, not the solution. Such input would also be useful in evaluating a potential solution. As for developing the solution, good solutions tend to be developed by small design teams of clueful people. Often, it takes more than developer clue to make a good design team - protocol design clue, operational clue, security clue, human factors clue, etc. would all be useful. But I agree that it's the rare user who would make significant contributions to a protocol design team. Or, if you prefer, people are talking and not doing the running code bit. It may be that we place too much emphasis on running code in IETF today. I'd say we place too little. [...] We have fewer platforms, and they're all running with the same 8-bit byte, (or as close as makes no difference), and they all do UTF-8 easily, let alone ASCII, so yes, that kind of problem has largely gone away. However, if you're extending IMAP, say, there's a large number of IMAP servers out there which are, internally, massively different beasts, so the in my day argument merely highlights that problems move, they don't go away. I guess I would say that testing of new features on a wide variety of platforms, while sometimes useful or even necessary, usually isn't sufficient to validate the design of a protocol or extension.By all means let's do the testing when it's appropriate to do so. But let's not take the results of those tests by themselves to mean the protocol is good. I've seen lots of arguments of the form X is implemented, therefore it's good and that's often a totally bogus argument. (e.g. for X == DKIM) It's weird, because I thought that pretty well everyone implemented stuff to this level. For a long time - years - it never occured to me that PoC and probably even deployed implementations didn't exist for some specifications, let alone those going onto the standards track. Scary. Though of course, our process doesn't require that at Proposed. I think the assumption was that we'd get early feedback on implementations before deployment, and that things wouldn't get widely deployed before Draft. Of course, it hasn't worked out that way. Keith ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On Fri Mar 24 19:50:15 2006, Keith Moore wrote: In other words, there are working groups where a substantial number of people involved in the discussion are not only not going to be implementing the proposals, but don't actually do any kind of implementation within the sphere - we're talking about people discussing the precise semantics of some HTTP extension who aren't involved in doing any webserver related programming, or some people discussing an email issue who limit their interaction with email to having an email address. I don' t have a problem with that. IMHO we tend to design with too little regard for the needs of end users, and we need more input from knowledgable users, rather than less. That input needs to be present in defining the problem, not the solution. Exactly. Or, if you prefer, people are talking and not doing the running code bit. It may be that we place too much emphasis on running code in IETF today. I'd say we place too little. And I would agree. I still try to implement everything I write specifications for. The few times I've deviated from this practice I've regretted it. (Can you say RFC 2231?) In ARPAnet days, when the user community was small and homogeneous but platforms were diverse (different word/character sizes, different character sets, different limitations of operating systems and networking hardware), and goals for protocols were modest, merely being able to implement a protocol across different platforms was one of the biggest barriers to adoption. In that environment, being able to demonstrate running code on multiple platforms was nearly sufficient to demonstrate the viability of a protocol. Besides, since the net was small, it wasn't terribly hard to make changes should they be found to be necessary. We have fewer platforms, and they're all running with the same 8-bit byte, (or as close as makes no difference), and they all do UTF-8 easily, let alone ASCII, so yes, that kind of problem has largely gone away. Agree on the 8 bit byte part. I wish I could agree on the UTF-8 part, but I have too much (recent) experience to the contrary. It is getting better though, and at least we do have a workablle reasonably universal solution in this space, which is a heck of a lot better than where we were when I first started working in the IETF back in 1991. However, if you're extending IMAP, say, there's a large number of IMAP servers out there which are, internally, massively different beasts, so the in my day argument merely highlights that problems move, they don't go away. Exactly. And arguing about whehere they have or haven't gone away really misses the point. There's always a new impedance mismatch lurking just around the corner. These days running code serves as proof-of-concept and also as a way to validate the specification. It doesn't say anything about the quality of the design - not efficiency, nor usability, nor scalability, nor security. etc. No. It doesn't say much about the efficiency, usability, scalability, or security, but it does say a little, and it gives me, for one, a lot better an idea about where the problems in all those areas lie. Maybe I'm a drooling idiot, and this is the equivalent of having to read aloud, in which case I'm sorry. To put it another way, the existance of such code says very little, but the experience of writing that code says a lot as long as you're being honest with yourself. Our ability to selld-delude is often part of the problem, but it is much easier to go off the rails with no implementation experience. Ned ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: are we willing to do change how we do discussions in IETF? (was: moving from hosts to sponsors)
On 18:47 24/03/2006, Keith Moore said: My question is - do others see this as a problem, and (without trying to propose a concrete solution that will be seen as a threat) is there a shared sense that this is a problem and general willingness to try new ways of conducting our discussions? I do. I would add to the list of examples from my own experience IDNA, Langtags, ethics. I proposed a very simple solution which is the position links: every active participant maintains a position statement page. The consensus us when all the pages are white or equivalent. Considerably reduces volume. Everyone can evaluate rough consensus easily. In case of appeal the full file is here. Every outsider poping into the debate (for example to advise on an area of his expertise) can brief himself in minutes, and leave/update a comment of reference. This was worked at at the WG-Review of the DNSO. I implemented it for some arduous debates. It works well. It would however be great if someone worked out a plink tool (as there are for blogs). Chair could use it to list the points he currently sees as needing to be solved. Rule here as always: KISS. jfc ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Dave Crocker wrote: Michael StJohns wrote: What I think Jordi is saying is that he wants the US sponsors to subsidize the cost of the overseas meetings. At least that's what it works out to be This view can be mapped to a classic model that would have significant benefits for the IETF: A host gets all sorts of marketing leverage out of the role in producing an IETF. There is nothing that requires that the event site management effort be coupled with a particular host's venue. If we moved to a model of having companies provide sponsorship funds, in return for which they get appropriate marketing presence, then we could have meeting venue management move to the sort of predictable and timely basis -- ie, far enough ahead of time -- that has been a concern for many years. Amen! And maybe the meeting fees could actually go down with enough sponsors. An additional room like the terminal room (not out in the open) could be used. Also, the IETF could maintain control of the network if there were multiple sponsors instead of a single host. They would not be allowed to ignore the advice of the NOC team, and let the wireless meltdown right off the bat. d/ Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
I don't think the meeting fees could actually go down, may be more in the other way around if we are realistic with the cost figures. Actually the cost is already high for a sponsor, and I believe trying to get more from the industry (or other kind of sponsors) for each meeting will be really difficult. Regards, Jordi De: Andy Bierman [EMAIL PROTECTED] Responder a: [EMAIL PROTECTED] Fecha: Thu, 23 Mar 2006 19:34:00 -0800 Para: [EMAIL PROTECTED] CC: Keith Moore moore@cs.utk.edu, ietf@ietf.org ietf@ietf.org, [EMAIL PROTECTED], Michael StJohns [EMAIL PROTECTED] Asunto: Re: Moving from hosts to sponsors Dave Crocker wrote: Michael StJohns wrote: What I think Jordi is saying is that he wants the US sponsors to subsidize the cost of the overseas meetings. At least that's what it works out to be This view can be mapped to a classic model that would have significant benefits for the IETF: A host gets all sorts of marketing leverage out of the role in producing an IETF. There is nothing that requires that the event site management effort be coupled with a particular host's venue. If we moved to a model of having companies provide sponsorship funds, in return for which they get appropriate marketing presence, then we could have meeting venue management move to the sort of predictable and timely basis -- ie, far enough ahead of time -- that has been a concern for many years. Amen! And maybe the meeting fees could actually go down with enough sponsors. An additional room like the terminal room (not out in the open) could be used. Also, the IETF could maintain control of the network if there were multiple sponsors instead of a single host. They would not be allowed to ignore the advice of the NOC team, and let the wireless meltdown right off the bat. d/ Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf ** The IPv6 Portal: http://www.ipv6tf.org Barcelona 2005 Global IPv6 Summit Slides available at: http://www.ipv6-es.com This electronic message contains information which may be privileged or confidential. The information is intended to be for the use of the individual(s) named above. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, including attached files, is prohibited. ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
Re: Moving from hosts to sponsors
Andy, I have been involved as local host now for two times (although I wasn't very local this time ;-)). I agree that it doesn't make sense to build a network each and every time completely from scratch. It is an enormous effort to beg potential sponsors for accesspoints (or spend a lot of money to buy them), to figure out how to build a terminal room and how to equip it, to buy servers and install monitoring software that gets wiped out right after the meeting to mention just a few examples. Luckily, we and the very experienced group of volunteers that helped us did have some memories (nightmares?) from previous meetings but it would have been way more efficient if a lot of the building blocks were simply already in place before a host even volunteers to be the host (and I think a host would more easily take on this role if the job was a bit more manageable). I personally believe that we would be better off if the same experienced (paid for) group would build the network each and every time with the same equipment owned by IETF, while the sponsor does what they are best at, and that is providing funding for the actual meeting. David Kessens PS it will also be easier to deal with complaints: no cookies at the break ? well, maybe you or employer should have sponsored the break then. --- On Thu, Mar 23, 2006 at 07:34:00PM -0800, Andy Bierman wrote: Dave Crocker wrote: Michael StJohns wrote: What I think Jordi is saying is that he wants the US sponsors to subsidize the cost of the overseas meetings. At least that's what it works out to be This view can be mapped to a classic model that would have significant benefits for the IETF: A host gets all sorts of marketing leverage out of the role in producing an IETF. There is nothing that requires that the event site management effort be coupled with a particular host's venue. If we moved to a model of having companies provide sponsorship funds, in return for which they get appropriate marketing presence, then we could have meeting venue management move to the sort of predictable and timely basis -- ie, far enough ahead of time -- that has been a concern for many years. Amen! And maybe the meeting fees could actually go down with enough sponsors. An additional room like the terminal room (not out in the open) could be used. Also, the IETF could maintain control of the network if there were multiple sponsors instead of a single host. They would not be allowed to ignore the advice of the NOC team, and let the wireless meltdown right off the bat. Andy ___ Ietf mailing list Ietf@ietf.org https://www1.ietf.org/mailman/listinfo/ietf
