Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Andy Bierman]

On Fri, May 17, 2013 at 7:29 PM, Keith Moore wrote:

> On 05/17/2013 10:21 PM, Andy Bierman wrote:
>
>>
>> I notice that nowhere on this list is any mention of the charter
>> milestones
>> or dates.  Is the Foo Proto draft due in 14 months or is it 14 months
>> behind
>> schedule?  If the latter, why isn't the Foo WG meeting at the IETF?
>>
>>
> I don't think milestones will be useful unless and until:
>
> (a) they're defined in terms of not only concrete but also meaningful
> goals (e.g. "complete problem definition", "identify affected parties and
> groups representing their interests", "complete outline of initial design",
> but NOT "revise document X");
> (b) we start automatically suspending the activities of groups that fail
> to meet them (no meetings, no new I-Ds accepted, mailing list traffic
> blocked), until such groups are formally rechartered; and
> (c) IESG is reluctant to recharter groups that have repeatedly failed to
> meet milestones, especially if those groups haven't produced evidence of
> significant progress.
>
>
I think we can find some middle ground between "ignore charter milestones
completely"
and "autobot to terminate WGs behind schedule". :-)

Keith
>
>
Andy

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Andy Bierman]

On Fri, May 17, 2013 at 6:29 PM, Keith Moore wrote:

> On 05/17/2013 04:36 PM, Yoav Nir wrote:
>
>> On May 17, 2013, at 6:37 PM, Dave Crocker  wrote:
>>
>>  On 5/17/2013 7:01 AM, Keith Moore wrote:
>>>
 But WGs should be able to periodically summarize what they're doing -
 what problem they're trying to solve, what approach they're taking, what
 technologies they're using, what major decisions they've made, what the
 current sticking points seem to be, what problems are as yet unresolved,
 what potential for cross-group and cross-area effects have been
 identified, and what efforts have been made to get the affected parties
 in the loop.   For most groups that summary should be maybe 2-3 pages.
 The ADs should be able to verify that those summaries are accurate and
 reasonably complete, or appoint a trusted WG observer other than the
 chair to review each summary. ADs and other members of the community
 should be able to view those summaries and comment on their accuracy.

>>>
>>> The idea that working groups should be required to issue periodic
>>> project progress reports seems strikingly reasonable and useful.
>>>
>>> This makes the folks who are the most knowledgeable responsible for
>>> assessing their work, and should facilitate public review. Recording the
>>> sequence of reports into the wg datatracker could nicely allow evaluating
>>> progress over time.
>>>
>>> It also, of course, nicely distributes the work.
>>>
>>> d/
>>>
>> "
>> From: WG Chair
>> To: ietf@ietf.org
>> Sunbject: Progress Report - Foo WG
>>
>> There has been zero activity on the FOO list in the last three months
>> (except for that "Fake Conference" message everybody got last month). I've
>> tried emailing the WG document authors twice, but they're not answering my
>> emails.
>>
>> So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and
>> draft-ietf-foo-proto-01.
>>
>> The use case document is just about done, but we haven't really started
>> discussing the proto document. We haven't met in Orlando, and are unlikely
>> to meet in Berlin
>>
>> That's it for this report.
>>
>> Cheers
>>
>> WGC
>>
>> "
>>
>
> Instead of a WG progress report, what I had in mind was a separate report
> for each work item.   The report should briefly describe
>
> 1. The purpose of the work being undertaken
> 2. A description of the work being undertaken (including mention of major
> technologies on which the work is based)
> 3. All known parties and interests likely to be affected by the work
> 4. The current state of the work (what's been done, what hasn't been done)
> 5. Any major issues that have been identified but not resolved
> 6. Pointers to the WG's charter, the datatracker pages for each of the
> current draft document(s) associated with that work item, and any other
> useful material (e.g. open issues list, summaries of design decisions
> already taken and the rationale for each)
>
> A person who is knowledgeable about current Internet protocols should be
> able to read that single document and understand what the WG is doing with
> this particular work item, what state it's in, whether or not it will
> affect that person's are of interest, and where to look for detailed
> information.
>
>
This seems like a good idea, although maybe a bit formal.
IMO, big surprises at the tail end that cause lots of work to
be thrown out are evidence of a management failure.
The IESG and WG chairs should be more proactive wrt/ avoiding
late surprises from ever happening.

I notice that nowhere on this list is any mention of the charter milestones
or dates.  Is the Foo Proto draft due in 14 months or is it 14 months behind
schedule?  If the latter, why isn't the Foo WG meeting at the IETF?



Keith
>
>
Andy

RE: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[l.wood]

> I already requested before that all WGs SHOULD
> discuss their milestones and update it in each
> meeting or on the list.

No-one cares what you requested.

Didn't you get banned from the MANET list for lack of useful content?

Lloyd Wood
http://sat-net.com/L.Wood

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Abdussalam Baryun]

> Instead of a WG progress report, what I had in mind was a separate report for 
> each work item. The report should briefly describe

I agree with you totally, that work-item-report SHOULD be copied to AD
and WG. That report is needed mostly when the work does not target its
milestone, requesting new milestone with plans/reason.

AB

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Abdussalam Baryun]

The problem is that WG participants SHOULD follow/update their
milestones and take responsibility to progress work to thoes goals
direction. The Chair SHOULD follow the WG requests, or the Chair
SHOULD encourage discussing the milestones. I already requested before
that all WGs SHOULD discuss their milestones and update it in each
meeting or on the list. If no one cares then the result is WG failing
some-goals which no one realise until long, but some people outside
the IETF are watching such performance.

AB

On Fri, May 17, 2013 at 7:29 PM, Keith Moore  network-heretics.com> wrote:
>
>
> I don't think milestones will be useful unless and until:
>
> (a) they're defined in terms of not only concrete but also meaningful
> goals (e.g. "complete problem definition", "identify affected parties
> and groups representing their interests", "complete outline of initial
> design", but NOT "revise document X");
> (b) we start automatically suspending the activities of groups that
> fail to meet them (no meetings, no new I-Ds accepted, mailing list
> traffic blocked), until such groups are formally rechartered; and
> (c) IESG is reluctant to recharter groups that have repeatedly failed
> to meet milestones, especially if those groups haven't produced
> evidence of significant progress.
>

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/17/2013 10:37 PM, Andy Bierman wrote:


On Fri, May 17, 2013 at 7:29 PM, Keith Moore 
mailto:mo...@network-heretics.com>> wrote:



I don't think milestones will be useful unless and until:

(a) they're defined in terms of not only concrete but also
meaningful goals (e.g. "complete problem definition", "identify
affected parties and groups representing their interests",
"complete outline of initial design", but NOT "revise document X");
(b) we start automatically suspending the activities of groups
that fail to meet them (no meetings, no new I-Ds accepted, mailing
list traffic blocked), until such groups are formally rechartered; and
(c) IESG is reluctant to recharter groups that have repeatedly
failed to meet milestones, especially if those groups haven't
produced evidence of significant progress.


I think we can find some middle ground between "ignore charter 
milestones completely"

and "autobot to terminate WGs behind schedule". :-)

Actually I think it might require an autobot.   Because someone 
(probably the responsible AD) has to evaluate a WG's progress, and ADs 
don't want to take the heat for shutting WGs down.   Better to put the 
responsibility on the chairs for completing the milestones and reporting 
to the AD before the shutdown deadline.


(of course, there could be a generous grace period between the milestone 
deadline and the actual shutdown, with warning messages sent to the WG 
chairs and ADs, etc.)


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/17/2013 10:21 PM, Andy Bierman wrote:


I notice that nowhere on this list is any mention of the charter 
milestones
or dates.  Is the Foo Proto draft due in 14 months or is it 14 months 
behind

schedule?  If the latter, why isn't the Foo WG meeting at the IETF?



I don't think milestones will be useful unless and until:

(a) they're defined in terms of not only concrete but also meaningful 
goals (e.g. "complete problem definition", "identify affected parties 
and groups representing their interests", "complete outline of initial 
design", but NOT "revise document X");
(b) we start automatically suspending the activities of groups that fail 
to meet them (no meetings, no new I-Ds accepted, mailing list traffic 
blocked), until such groups are formally rechartered; and
(c) IESG is reluctant to recharter groups that have repeatedly failed to 
meet milestones, especially if those groups haven't produced evidence of 
significant progress.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/17/2013 04:36 PM, Yoav Nir wrote:

On May 17, 2013, at 6:37 PM, Dave Crocker  wrote:


On 5/17/2013 7:01 AM, Keith Moore wrote:

But WGs should be able to periodically summarize what they're doing -
what problem they're trying to solve, what approach they're taking, what
technologies they're using, what major decisions they've made, what the
current sticking points seem to be, what problems are as yet unresolved,
what potential for cross-group and cross-area effects have been
identified, and what efforts have been made to get the affected parties
in the loop.   For most groups that summary should be maybe 2-3 pages.
The ADs should be able to verify that those summaries are accurate and
reasonably complete, or appoint a trusted WG observer other than the
chair to review each summary. ADs and other members of the community
should be able to view those summaries and comment on their accuracy.


The idea that working groups should be required to issue periodic project 
progress reports seems strikingly reasonable and useful.

This makes the folks who are the most knowledgeable responsible for assessing 
their work, and should facilitate public review. Recording the sequence of 
reports into the wg datatracker could nicely allow evaluating progress over 
time.

It also, of course, nicely distributes the work.

d/

"
From: WG Chair
To: ietf@ietf.org
Sunbject: Progress Report - Foo WG

There has been zero activity on the FOO list in the last three months (except for that 
"Fake Conference" message everybody got last month). I've tried emailing the WG 
document authors twice, but they're not answering my emails.

So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and 
draft-ietf-foo-proto-01.

The use case document is just about done, but we haven't really started 
discussing the proto document. We haven't met in Orlando, and are unlikely to 
meet in Berlin

That's it for this report.

Cheers

WGC

"


Instead of a WG progress report, what I had in mind was a separate 
report for each work item.   The report should briefly describe


1. The purpose of the work being undertaken
2. A description of the work being undertaken (including mention of 
major technologies on which the work is based)

3. All known parties and interests likely to be affected by the work
4. The current state of the work (what's been done, what hasn't been done)
5. Any major issues that have been identified but not resolved
6. Pointers to the WG's charter, the datatracker pages for each of the 
current draft document(s) associated with that work item, and any other 
useful material (e.g. open issues list, summaries of design decisions 
already taken and the rationale for each)


A person who is knowledgeable about current Internet protocols should be 
able to read that single document and understand what the WG is doing 
with this particular work item, what state it's in, whether or not it 
will affect that person's are of interest, and where to look for 
detailed information.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Yoav Nir]

On May 17, 2013, at 6:37 PM, Dave Crocker  wrote:

> On 5/17/2013 7:01 AM, Keith Moore wrote:
>> But WGs should be able to periodically summarize what they're doing -
>> what problem they're trying to solve, what approach they're taking, what
>> technologies they're using, what major decisions they've made, what the
>> current sticking points seem to be, what problems are as yet unresolved,
>> what potential for cross-group and cross-area effects have been
>> identified, and what efforts have been made to get the affected parties
>> in the loop.   For most groups that summary should be maybe 2-3 pages.
>> The ADs should be able to verify that those summaries are accurate and
>> reasonably complete, or appoint a trusted WG observer other than the
>> chair to review each summary. ADs and other members of the community
>> should be able to view those summaries and comment on their accuracy.
> 
> 
> The idea that working groups should be required to issue periodic project 
> progress reports seems strikingly reasonable and useful.
> 
> This makes the folks who are the most knowledgeable responsible for assessing 
> their work, and should facilitate public review. Recording the sequence of 
> reports into the wg datatracker could nicely allow evaluating progress over 
> time.
> 
> It also, of course, nicely distributes the work.
> 
> d/

"
From: WG Chair
To: ietf@ietf.org
Sunbject: Progress Report - Foo WG

There has been zero activity on the FOO list in the last three months (except 
for that "Fake Conference" message everybody got last month). I've tried 
emailing the WG document authors twice, but they're not answering my emails.

So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and 
draft-ietf-foo-proto-01.

The use case document is just about done, but we haven't really started 
discussing the proto document. We haven't met in Orlando, and are unlikely to 
meet in Berlin

That's it for this report.

Cheers

WGC

"

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Brian Haberman]

Dave,

On 5/17/13 11:37 AM, Dave Crocker wrote:

On 5/17/2013 7:01 AM, Keith Moore wrote:

But WGs should be able to periodically summarize what they're doing -
what problem they're trying to solve, what approach they're taking, what
technologies they're using, what major decisions they've made, what the
current sticking points seem to be, what problems are as yet unresolved,
what potential for cross-group and cross-area effects have been
identified, and what efforts have been made to get the affected parties
in the loop.   For most groups that summary should be maybe 2-3 pages.
The ADs should be able to verify that those summaries are accurate and
reasonably complete, or appoint a trusted WG observer other than the
chair to review each summary. ADs and other members of the community
should be able to view those summaries and comment on their accuracy.



The idea that working groups should be required to issue periodic
project progress reports seems strikingly reasonable and useful.

This makes the folks who are the most knowledgeable responsible for
assessing their work, and should facilitate public review. Recording the
sequence of reports into the wg datatracker could nicely allow
evaluating progress over time.

It also, of course, nicely distributes the work.

  d/



We (the INT ADs) have been trying to do that for the past year.  See 
http://trac.tools.ietf.org/area/int/trac/wiki/IETF86 as an example.  The 
goal is to provide a place for INT area WGs to summarize what they have 
been doing so that others within INT can get an idea of what is going 
on.  It is not to the level of a "progress report", but it does provide 
a useful summation (at least based on the feedback that I have received 
to date).


Regards,
Brian

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Dave Crocker]

On 5/17/2013 7:01 AM, Keith Moore wrote:

But WGs should be able to periodically summarize what they're doing -
what problem they're trying to solve, what approach they're taking, what
technologies they're using, what major decisions they've made, what the
current sticking points seem to be, what problems are as yet unresolved,
what potential for cross-group and cross-area effects have been
identified, and what efforts have been made to get the affected parties
in the loop.   For most groups that summary should be maybe 2-3 pages.
The ADs should be able to verify that those summaries are accurate and
reasonably complete, or appoint a trusted WG observer other than the
chair to review each summary. ADs and other members of the community
should be able to view those summaries and comment on their accuracy.



The idea that working groups should be required to issue periodic 
project progress reports seems strikingly reasonable and useful.


This makes the folks who are the most knowledgeable responsible for 
assessing their work, and should facilitate public review. Recording the 
sequence of reports into the wg datatracker could nicely allow 
evaluating progress over time.


It also, of course, nicely distributes the work.

 d/

--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/17/2013 05:32 AM, Yoav Nir wrote:

On May 17, 2013, at 1:38 AM, Fred Baker (fred)  wrote:


On May 16, 2013, at 1:46 PM, Yoav Nir  wrote:


There is a problem, though, that this will increase the load on ADs. Other 
concerns raised during IETF LC may lead to revised I-Ds, which the ADs will 
need to re-read (or at least look at the diff). I don't know how significant 
this extra work would be, but it would come at a time that we're thinking of 
ways to reduce AD workload. It might also require prolonging the LC time, 
because there would be actual discussion in it.

If they raise the issue later in a "discuss", will they not have to do this 
anyway? How does this relate to the timing of the comment or the vehicle by which it is 
conveyed?

If you review early, you later might feel like you need to review again, 
because the document has changed some. Hence - more work.
Yes, but you only need to review what has changed. It _can_ be more 
work, particularly for documents that have changed a lot, or if it's 
been too long since the last review.But I really wouldn't suggest 
that ADs should do several reviews of each document.I suspect that 
the trick is to review a document at the time the WG thinks that it's 
maybe 70-80% done (which is to say the WG is probably closer to 40-50% 
done), but when some issues still seem unresolved.   That's when the 
ADs' input, and external input in general, can be the most valuable.
That way, ADs should be able to say "yes, but you're totally ignoring 
this very important issue here, and you really have to deal with it" at 
a time when the WG isn't yet so exhausted or off in the weeds that it 
can't focus on it.   Then the ADs just need to track the changes from 
that point forward, to make sure that the issues identified were dealt 
with satisfactorily.


Of course new issues can still be identified, and WGs can address 
previously identified issues in unsatisfactory ways.   But at some point 
(well prior to WGLC) it might be appropriate to raise the bar for new 
issues.   At some point it should take a serious defect to significantly 
delay publication of a document, and at that point it might make more 
sense to consider other remedies for identified less-serious issues, 
especially if the existing protocol isn't seen to be actually harmful 
and it appears that the protocol can be extended to address the issues 
in a manner that is compatible with existing implementations.   In those 
cases, it might make sense to go ahead and publish, but charter the WG 
to extend the protocol to address those issues.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/17/2013 05:31 AM, Yoav Nir wrote:

On May 17, 2013, at 12:58 AM, Keith Moore  wrote:


On 05/16/2013 04:46 PM, Yoav Nir wrote:

The time for asking whether the group has considered making this field fixed 
length instead of variable, or whether RFC 2119 language is used in an 
appropriate way, or whether the protocol is extensible enough is at IETF last 
call.

Actually the time for asking these questions is long before IETF-wide Last 
Call.  We need widespread review of proposals for standards-track documents 
long before a WG thinks it's finished with those documents.   It's a gaping 
hole in our process.

Sure. But we have opinionated ADs who read every draft that comes to the IESG. 
There is no way they have time to participate in all of the working groups. I, 
as a participant, can read drafts as they are discussed in working groups, 
because I'm free to ignore all the drafts that are not interesting to me. ADs 
don't have that luxury.


Unless things have changed a great deal since I was on IESG, ADs do have 
the luxury of not reading drafts.   When I was an AD I tried to read 
every draft that was in my area (Applications), and every draft that 
seemed to have the ability to affect applications developers.The 
lower in the protocol stack, the less likely that I'd feel like I'd have 
anything useful to say about a draft.   Even when I "read" a draft 
outside of my area, in many cases it was just skimming the draft looking 
for red flags.   I developed a pretty good sense of whether a group had 
done due diligence or whether there were serious technical omissions 
that they were trying to ignore.   Only in the latter (rare) cases did I 
feel like such drafts needed more of my attention.


I certainly agree that ADs don't have time to participate in all working 
groups, or even probably 10% of our working groups.   But WGs should be 
able to periodically summarize what they're doing - what problem they're 
trying to solve, what approach they're taking, what technologies they're 
using, what major decisions they've made, what the current sticking 
points seem to be, what problems are as yet unresolved, what potential 
for cross-group and cross-area effects have been identified, and what 
efforts have been made to get the affected parties in the loop.   For 
most groups that summary should be maybe 2-3 pages.   The ADs should be 
able to verify that those summaries are accurate and reasonably 
complete, or appoint a trusted WG observer other than the chair to 
review each summary. ADs and other members of the community should be 
able to view those summaries and comment on their accuracy.   And I 
think it would be reasonable for everyone on IESG to read through those 
summaries once in awhile - at least for groups that seemed likely to 
affect their areas of concern.   I think that such summaries could 
actually lessen IESG's workload.

Fix that problem, and most of the conflicts between IESG and WGs that surround 
DISCUSS votes will go away.

Good reviewers are a scarce resource, and there are 500(*) working group drafts 
competing for their attention. That's a hard problem to fix.


That's a related but IMO somewhat different problem.   Working groups 
are producing far too many documents.   That's one reason (far from the 
only one) why WGs shouldn't undertake documents that aren't specifically 
authorized in their charters.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Jari Arkko]

Dave, Ralph,

>> Jari has expressed the goal of having AD concerns be raised more publicly.  
>> Moving AD review and comment to the IETF Last Call venue nicely accomplishes 
>> this, too.
> 
> I just posted elsewhere a suggestion to move this review even earlier, to WG 
> last call.  Accomplishes most of the same ends, while putting the discussion 
> in front of the IETF participants who are, presumably, most invested in the 
> resulting document.

We've also said that we'd like to move the directorate reviews earlier, to WGLC 
time. (Subject to us finding a way to do that without increasing directorate 
workload too much.) See the thread "ways forward with the tail-heavy aspects of 
the IETF process".

But overall, we have to be careful that we don't move the same processes to a 
process step that has a different name but in reality will not impact how 
things are done. Part of the idea for moving directorate reviews earlier is 
that more of the responsibility for dealing with these could reside with the 
WGs rather than ADs. And that is a change that affects substance and may 
improve scalability. Similar reasons are needed for other changes.

I'm not opposed to moving AD reviews to IETF LC time or even earlier, but we'd 
have to find a way to accommodate both technical comments as well as those 
relating to taking LC feedback into account. ("draft-foo LC feedback from Joe 
was was not appropriately addressed" and similar end-of-process checks.)

Jari

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Stephen Farrell]

On 05/17/2013 10:18 AM, Yoav Nir wrote:
> 
> On May 16, 2013, at 11:55 PM, Stephen Farrell  
> wrote:
> 
>>
>> I think Dave's idea is worth looking at, but have one comment:
>>
>> On 05/16/2013 09:46 PM, Yoav Nir wrote:
>>> There is a problem, though, that this will increase the load on ADs.
>>
>> There is that. But don't forget that ADs mostly read everything
>> in IESG review and often comment. Even leaving aside DISCUSSes,
>> if every IETF LC draft got a bunch of AD review comments, then
>> the IETF LC would be much busier than now. And its in the nature
>> of IETFers to chime in. So this would I suspect increase everyone's
>> load, if it works, and not just ADs.
> 
> More community participation at IETF last call is a good thing. 
> 
> The problem with AD's time is that they will need to review at IETF last 
> call, and then may need to review again after some changes have been made 
> (including changes not related to their own issue). That's where the 
> increased AD load comes from.

I'm not so sure that's a problem. The additional time I think
(for me) would be in reading the 100's of ietf-discuss mails
that'd get triggered by my or some other AD's comments. Not
all comments would trigger that of course, but if you look at
the minutes for any IESG telechat [1] and imagine that all
those comments had been sent to ietf-discuss during IETF LC
then you might worry about what'd happen on this list.

I'm not against considering this idea at all, but we should
bear in mind that any random thread on this list has a non-zero
probability of turning into major amounts of traffic. Most of
that traffic would doubtless be excellent IETF LC stuff, but
we maybe need to be careful in what we wish for too.

Cheers,
S.

[1] https://www.ietf.org/iesg/minutes/2013/index.html

> Yoav
> 
> 
> 

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Yoav Nir]

On May 17, 2013, at 1:38 AM, Fred Baker (fred)  wrote:

> 
> On May 16, 2013, at 1:46 PM, Yoav Nir  wrote:
> 
>> There is a problem, though, that this will increase the load on ADs. Other 
>> concerns raised during IETF LC may lead to revised I-Ds, which the ADs will 
>> need to re-read (or at least look at the diff). I don't know how significant 
>> this extra work would be, but it would come at a time that we're thinking of 
>> ways to reduce AD workload. It might also require prolonging the LC time, 
>> because there would be actual discussion in it.
> 
> If they raise the issue later in a "discuss", will they not have to do this 
> anyway? How does this relate to the timing of the comment or the vehicle by 
> which it is conveyed?

If you review early, you later might feel like you need to review again, 
because the document has changed some. Hence - more work.

Yoav

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Yoav Nir]

On May 17, 2013, at 12:58 AM, Keith Moore  wrote:

> On 05/16/2013 04:46 PM, Yoav Nir wrote:
>> The time for asking whether the group has considered making this field fixed 
>> length instead of variable, or whether RFC 2119 language is used in an 
>> appropriate way, or whether the protocol is extensible enough is at IETF 
>> last call. 
> 
> Actually the time for asking these questions is long before IETF-wide Last 
> Call.  We need widespread review of proposals for standards-track documents 
> long before a WG thinks it's finished with those documents.   It's a gaping 
> hole in our process.

Sure. But we have opinionated ADs who read every draft that comes to the IESG. 
There is no way they have time to participate in all of the working groups. I, 
as a participant, can read drafts as they are discussed in working groups, 
because I'm free to ignore all the drafts that are not interesting to me. ADs 
don't have that luxury.

> Fix that problem, and most of the conflicts between IESG and WGs that 
> surround DISCUSS votes will go away.

Good reviewers are a scarce resource, and there are 500(*) working group drafts 
competing for their attention. That's a hard problem to fix.

Yoav

(*) Went to datatracker, and searched for active drafts that start with  
"draft-ietf-". I probably hit some things that have already progressed, but 
OTOH the 500 number is too round, and may be a limitation of datatracker.

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Yoav Nir]

On May 16, 2013, at 11:55 PM, Stephen Farrell  wrote:

> 
> I think Dave's idea is worth looking at, but have one comment:
> 
> On 05/16/2013 09:46 PM, Yoav Nir wrote:
>> There is a problem, though, that this will increase the load on ADs.
> 
> There is that. But don't forget that ADs mostly read everything
> in IESG review and often comment. Even leaving aside DISCUSSes,
> if every IETF LC draft got a bunch of AD review comments, then
> the IETF LC would be much busier than now. And its in the nature
> of IETFers to chime in. So this would I suspect increase everyone's
> load, if it works, and not just ADs.

More community participation at IETF last call is a good thing. 

The problem with AD's time is that they will need to review at IETF last call, 
and then may need to review again after some changes have been made (including 
changes not related to their own issue). That's where the increased AD load 
comes from.

Yoav

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/16/2013 06:09 PM, joel jaeggli wrote:


Fix that problem, and most of the conflicts between IESG and WGs that 
surround DISCUSS votes will go away.
Maybe but I wouldn't take that as an article of faith. You're going to 
get pressure for more changes when fresh eyes review something.


Yeah, every new set of eyes that looks at a document is going to have 
some new ideas for what the document should be.   The trick is to get 
those eyes to look at the document earlier in the process, when it's 
easier to fix problems. Maybe if we did the early review well 
enough, the scope of Last Call comments could be limited in some way.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Fred Baker (fred)]

On May 16, 2013, at 1:46 PM, Yoav Nir  wrote:

> There is a problem, though, that this will increase the load on ADs. Other 
> concerns raised during IETF LC may lead to revised I-Ds, which the ADs will 
> need to re-read (or at least look at the diff). I don't know how significant 
> this extra work would be, but it would come at a time that we're thinking of 
> ways to reduce AD workload. It might also require prolonging the LC time, 
> because there would be actual discussion in it.

If they raise the issue later in a "discuss", will they not have to do this 
anyway? How does this relate to the timing of the comment or the vehicle by 
which it is conveyed?


The ignorance of how to use new knowledge stockpiles exponentially. 
   - Marshall McLuhan

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Peter Saint-Andre]

On 5/16/13 4:07 PM, Ralph Droms wrote:
> 
> On May 16, 2013, at 5:58 PM 5/16/13, Keith Moore  
> wrote:
> 
>> On 05/16/2013 04:46 PM, Yoav Nir wrote:
>>> The time for asking whether the group has considered making this field 
>>> fixed length instead of variable, or whether RFC 2119 language is used in 
>>> an appropriate way, or whether the protocol is extensible enough is at IETF 
>>> last call. 
>>
>> Actually the time for asking these questions is long before IETF-wide Last 
>> Call.  We need widespread review of proposals for standards-track documents 
>> long before a WG thinks it's finished with those documents.   It's a gaping 
>> hole in our process.
> 
> Hear, hear.

One suggestion I've heard several times is a kind of cross-area buddy
system (e.g., ask or assign someone with clue in Area X to help WG Y in
Area Z early and often with regard to specific issues that are often
addressed in Area X). Unfortunately, this suffers from the same problem
that too many WGs suffer from on their own: participant burnout. But I
still think it's a good idea...

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[joel jaeggli]

On 5/16/13 2:58 PM, Keith Moore wrote:

On 05/16/2013 04:46 PM, Yoav Nir wrote:
The time for asking whether the group has considered making this 
field fixed length instead of variable, or whether RFC 2119 language 
is used in an appropriate way, or whether the protocol is extensible 
enough is at IETF last call. 


Actually the time for asking these questions is long before IETF-wide 
Last Call.  We need widespread review of proposals for standards-track 
documents long before a WG thinks it's finished with those 
documents.   It's a gaping hole in our process.


As a Chair and as an AD I have asked for external and cross area reviews 
of some documents before they were considered for WG acceptance. this 
doesn't apply to all work we processed but it does apply to those where 
we were clear that such input was going to be useful.  One case you can 
see for that today is with capwap extensions that are potentially in 
opsawg.
Fix that problem, and most of the conflicts between IESG and WGs that 
surround DISCUSS votes will go away.
Maybe but I wouldn't take that as an article of faith. You're going to 
get pressure for more changes when fresh eyes review something.

Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Ralph Droms]

On May 16, 2013, at 5:58 PM 5/16/13, Keith Moore  
wrote:

> On 05/16/2013 04:46 PM, Yoav Nir wrote:
>> The time for asking whether the group has considered making this field fixed 
>> length instead of variable, or whether RFC 2119 language is used in an 
>> appropriate way, or whether the protocol is extensible enough is at IETF 
>> last call. 
> 
> Actually the time for asking these questions is long before IETF-wide Last 
> Call.  We need widespread review of proposals for standards-track documents 
> long before a WG thinks it's finished with those documents.   It's a gaping 
> hole in our process.

Hear, hear.

> 
> Fix that problem, and most of the conflicts between IESG and WGs that 
> surround DISCUSS votes will go away.

Well, you might be a little optimistic here, but I agree in theory.

> 
> Keith
> 

- Ralph

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Ralph Droms]

Dave - I hope you'll indulge my selective quoting as I have a couple of 
specific points to address.  My apologies if I end up quoting you out of 
context...

On May 16, 2013, at 12:23 PM 5/16/13, Dave Crocker  wrote:

> [...]
> 
> So here's a simple proposal that pays attention to AD workload and includes a 
> simple efficiency hack:
> 
> When the IETF Last Call is issued, wait a few days, to see whether any 
> serious issues are raised by the community.  The really serious ones usually 
> are raised quickly.  If there are none, it's pretty certain the document will 
> advance to an IESG vote.  That leaves 7-10 days of IETF Last Call for ADs to 
> get educated and ask questions, just like everyone else.
> 
> Jari has expressed the goal of having AD concerns be raised more publicly.  
> Moving AD review and comment to the IETF Last Call venue nicely accomplishes 
> this, too.

I just posted elsewhere a suggestion to move this review even earlier, to WG 
last call.  Accomplishes most of the same ends, while putting the discussion in 
front of the IETF participants who are, presumably, most invested in the 
resulting document.

> 
> 
> [...]
> In terms of quality assurance, the idea that we have a process that relies on 
> the sudden insight of a single AD, at the end of a many-month process, is 
> broken.  It's fine if that person sees something that everyone else has 
> missed until then, but that is quite different from designing a process that 
> is claimed to rely on it.

As you and I have discussed in person, I am 100% in agreement with this 
comment.  As much as I liked to think of myself, when I was an AD, as a 
rock-god Network Expert with complete and in-depth insight into every document 
I reviewed, I know the reality was that any problems I might have found were 
related to the old observation that "even a blind squirrel finds an acorn 
occasionally".
> 
> And of course, the reality is that we allow bad specs out the door all the 
> time; we just allow fewer of them than many/most other standards bodies...

You're such an optimist.

- Ralph

> 
> d/
> 
> 
> -- 
> Dave Crocker
> Brandenburg InternetWorking
> bbiw.net

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/16/2013 04:46 PM, Yoav Nir wrote:
The time for asking whether the group has considered making this field 
fixed length instead of variable, or whether RFC 2119 language is used 
in an appropriate way, or whether the protocol is extensible enough is 
at IETF last call. 


Actually the time for asking these questions is long before IETF-wide 
Last Call.  We need widespread review of proposals for standards-track 
documents long before a WG thinks it's finished with those documents.   
It's a gaping hole in our process.


Fix that problem, and most of the conflicts between IESG and WGs that 
surround DISCUSS votes will go away.


Keith

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Ralph Droms]

On May 16, 2013, at 5:00 PM 5/16/13, "Fred Baker (fred)"  wrote:

> 
> On May 16, 2013, at 9:40 AM, Adrian Farrel  wrote:
> 
>> On the whole, I am told that if an AD weighs in with her comments during 
>> working
>> group last call, her fearsome personality may overwhelm some of the WG
>> participants and she may dominate the WG consensus.
> 
> There may be places where that happens, but I would be surprised if it 
> happened in my working group. I think it is fair to say that the AD (or an 
> IAB member, or someone who has recognized expertise on the topic) is likely 
> to be listened to more carefully than some others might. Heck, I'm careful 
> when I make a technical comment on a document in my working group, flagging 
> it with "" to ensure that it is seen as intended - a comment by a 
> competent practitioner of the art, not a process remark or an attempt to 
> trump some other view. Speaking personally, I would prefer to see those 
> comments in the WGLC, not IETF Last Call, if we can make that happen. For 
> example, I'd like to get directorate reviews done (gen-art, security 
> directorate, etc) in the timeframe of WGLC.

I think Fred is returning to an earlier theme here, when he asks for earlier 
review.

Perhaps, as has been already suggested in this thread, we should think about 
SIRSbis? 

First, from draft-carpenter-icar-sirs-01:

 The procedure described in this document is intended to
 solve, or palliate, a number of related problems that
 have been observed in the IETF process [PROBLEM]:

  *submission of documents to the IESG that
   still have significant problems (leading
   to delay)

  *failure to detect fundamental problems
   and Internet- wide issues at an early
   stage

 Particularly because of the second point, it is
 impossible to resolve these problems simply by
 giving additional responsibility to working groups
 themselves. An additional procedure is needed.

In my opinion, it's important to assign responsibility (and accountability) to 
all WGs for producing publication-ready documents.  I agree that some 
additional work is needed before WGs send documents to the IESG.  Perhaps we 
can accomplish these goals through reorganizing the work we are 

I suggest we might want to combine the need for more responsibility with the 
discussion of a new "really close to being ready" document state.  However, 
rather than a new document state, suppose we codify the expectation that a 
document that has passed WG last call is essentially ready-to-publish?  
Correspondingly, any significant problems found in a document after WG last 
call would be considered a serious defect.

   Discussion:  I realize that, elsewhere in this thread, it has been
   asserted (or at least implied), that WGs already have this responsibility
   and DISCUSSes on document are usually unnecessary.  In practice, while
   there may still be unnecessary DISCUSSes, my experience as AD was that
   most DISCUSSes were appropriate and each one referred to a problem that
   the WG had missed.

Let's get all the expert review possible - directorate, AD, cross-area - in the 
WG last call review.  What pops out *should* be ready for publication.  Any 
issues raised by these reviews in WG last call will be exposed to and can be 
discussed by the WG at large, rather than being buried in the noise of IETF 
last call discussions or being fixed in more focused discussions among the IESG 
and the document authors.  This procedure diverges some from 
draft-carpenter-icar-sirs-01, in that it doesn't add a new form of review 
process.  Instead, it reschedules reviews that were going to take place anyway 
earlier in the process, so there is little or no new work added to the document 
publication process.

Perhaps the WG chairs would want to assign document shepherds earlier in the 
process, as well, investing the document shepherds with the responsibility of 
getting the right reviews and advising the WG chairs as to the readiness of the 
document for advancement.

Any WGs willing to volunteer as experimental subjects?  There is really no new 
process to invent ... it's mostly a matter of realigning expectations and 
responsibilities out to 

- Ralph

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Brian E Carpenter]

Dave,

On 17/05/2013 04:23, Dave Crocker wrote:
...
> The problem here is that basic reviewing is being done by the ADs too
> late in the process.  

You are making a lot of assumptions in that sentence. At least these:

1. "Basic" reviewing means 

2. At some stage before approval, ADs should perform basic reviewing.

3. Doing basic reviewing after the document has completed IETF LC
   is too late.

Now, if "basic" reviewing means (A) "looking for fundamental flaws"
that is one thing. If it means (B) "getting a general understanding
of the topic" it's another. I'm really not sure which you mean.

> We are making the mistake of having ADs be exempt
> from IETF Last Call, and allowing them to be unprepared for the IESG
> vote.  So we are combining "education" with "voting".  That's a paradigm
> error.
> 
> By the time the IESG schedules the vote, ADs need to already have
> educated themselves about the document.

Ah, maybe you mean basic (B). Well, I think this is a totally
unrealistic expectation. There are too many drafts on too diverse
a range of subjects for ADs to educate themselves at the time
of IETF LC, which may be weeks or months before a draft hits
the agenda. (I'm not talking about the sponsoring AD or her
co-AD, of course: they are presumed to be aware of what's going
on in their area.) Busy people are deadline driven, and the
IESG agenda is an imminent deadline for ADs in a way that an
IETF LC in another Area isn't.

> Of course, the IESG discussion during the voting process well might
> uncover an actual, serious issue with the document; 

And that's basic (B). I think we all agree that it's unfortunate
if a basic flaw shows up during the IESG ballot phase, but
please don't blame the IESG for that. Blame the WG and the
IETF as a whole for failing to pick it up during WG LC and
IETF LC. Thank the IESG for finding it.

> this should be
> exceptional and it should be an issue that the /IESG/ agrees needs to be
> resolved and it means that voting should not take place until it is. But
> that is quite different from the usual "let's talk about it" kind of
> Discuss imposed by individual ADs.

I'm not quite sure what you're saying here, but I think you're
saying that the majority of clarity and cross-area issues should
be picked up earlier. I couldn't agree more, but don't expect the
over-worked IESG to fix that. The rest of us need to fix that.

> 
> So here's a simple proposal that pays attention to AD workload and
> includes a simple efficiency hack:
> 
>  When the IETF Last Call is issued, wait a few days, to see whether
> any serious issues are raised by the community.  The really serious ones
> usually are raised quickly.  If there are none, it's pretty certain the
> document will advance to an IESG vote.  That leaves 7-10 days of IETF
> Last Call for ADs to get educated and ask questions, just like everyone
> else.

Which, as I said above, is a totally unrealistic expectation.

> Jari has expressed the goal of having AD concerns be raised more
> publicly.  Moving AD review and comment to the IETF Last Call venue
> nicely accomplishes this, too.

Thanks but no thanks. My mail is full enough with discussion of drafts
I will never read as it is. AD reviews should certainly go to the
WG list unless they are only nits, but isn't that what everybody does
anyway?

> 
> 
> 
> On 5/15/2013 8:55 AM, Thomas Narten wrote:
>> 1) Discuss criteria should be principles, not rigid rules. The details
>> of the issue at hand always matter, and it will sometimes come down to
>> judgement calls where reasonable individuals just might disagree.
> 
> We've been doing protocol specification for 40 years.  If we can't
> codify the major concerns that warrant blocking advancement of a
> protocol, we're just being lazy.  Really.  That a given situation might
> cause the IESG to decide to enhance the list does not mean that the list
> shouldn't seek to be complete and precise and that ADs be held to it.

I agree with Thomas. We need to be able to apply common sense. Rules
and targets are the enemy of common sense.

> At every turn, the approach we've taken to the IESG review and approval
> process is to limit actual accountability to the community.  Everyone is
> well-intentioned, but really this makes the process a matter of
> personalities and not the rigor of serious professionalism.
> 
> The IESG's process is far, far better than it was 10-15 years ago, but
> it still lacks meaningful predictability and serious accountability; it

I find that the tracker in its present state has substantially improved
both visibility and predictability.

> is not reasonable for the process to require that authors and chairs
> take the initiative of complaining when an AD is being problematic.

Huh? Who else will complain?

> In terms of quality assurance, the idea that we have a process that
> relies on the sudden insight of a single AD, at the end of a many-month
> process, is broken.  It's fine if that person sees somethi

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Fred Baker (fred)]

On May 16, 2013, at 9:40 AM, Adrian Farrel  wrote:

> On the whole, I am told that if an AD weighs in with her comments during 
> working
> group last call, her fearsome personality may overwhelm some of the WG
> participants and she may dominate the WG consensus.

There may be places where that happens, but I would be surprised if it happened 
in my working group. I think it is fair to say that the AD (or an IAB member, 
or someone who has recognized expertise on the topic) is likely to be listened 
to more carefully than some others might. Heck, I'm careful when I make a 
technical comment on a document in my working group, flagging it with 
"" to ensure that it is seen as intended - a comment by a competent 
practitioner of the art, not a process remark or an attempt to trump some other 
view. Speaking personally, I would prefer to see those comments in the WGLC, 
not IETF Last Call, if we can make that happen. For example, I'd like to get 
directorate reviews done (gen-art, security directorate, etc) in the timeframe 
of WGLC.

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Stephen Farrell]

I think Dave's idea is worth looking at, but have one comment:

On 05/16/2013 09:46 PM, Yoav Nir wrote:
> There is a problem, though, that this will increase the load on ADs.

There is that. But don't forget that ADs mostly read everything
in IESG review and often comment. Even leaving aside DISCUSSes,
if every IETF LC draft got a bunch of AD review comments, then
the IETF LC would be much busier than now. And its in the nature
of IETFers to chime in. So this would I suspect increase everyone's
load, if it works, and not just ADs.

Still worth a look though.

S.

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Yoav Nir]

On May 16, 2013, at 9:08 PM, Scott Brim 
mailto:scott.b...@gmail.com>> wrote:

On Thursday, May 16, 2013, Dave Crocker wrote:
By the time the IESG schedules the vote, ADs need to already have educated 
themselves about the document.

Oh, so you're suggesting adding another phase to the process: IESG education.  
OK.

I don't speak for Dave, but it makes sense that when a document reaches the 
IESG telechat, the questions to be asked are (1) Did this get sufficient review 
by sufficiently capable people, and (2) Has the process been followed, 
including have all issues been addressed.

The time for asking whether the group has considered making this field fixed 
length instead of variable, or whether RFC 2119 language is used in an 
appropriate way, or whether the protocol is extensible enough is at IETF last 
call. This is true of any participants, and ADs can do this too, regardless of 
whether their presence might intimidate the crowd. It could even be that if ADs 
voice their concerns at that time, others might join in, making the LC time 
more useful and less about just waiting two or four weeks.

There is a problem, though, that this will increase the load on ADs. Other 
concerns raised during IETF LC may lead to revised I-Ds, which the ADs will 
need to re-read (or at least look at the diff). I don't know how significant 
this extra work would be, but it would come at a time that we're thinking of 
ways to reduce AD workload. It might also require prolonging the LC time, 
because there would be actual discussion in it.

Yoav

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[joel jaeggli]

On 5/16/13 10:01 AM, Dave Crocker wrote:

On 5/16/2013 9:40 AM, Adrian Farrel wrote:

That's a good question Dave.

The community might like to comment.

On the whole, I am told that if an AD weighs in with her comments 
during working

group last call, her fearsome personality may overwhelm some of the WG
participants and she may dominate the WG consensus.


Only women ADs? [1]

But seriously...


If Tiresias prophet of Thebes is in your wg, I'd listen.

Is it possible that the same would happen in IETF last call?


I think that the public sway of ADs on the IETF list is less of a risk 
than on wg mailing lists.  And note that my suggestion has ADs waiting 
to weigh in until community-generated active disagreement with the 
spec, or its passive agreement, has been established.


Early participation AD's weighing in on draft's generally takes the form 
of just another participant unless you're asking for an opinion on a 
process point. wearing different hats on the course of the lifecycle is 
normal imho. opinion at the mic or on the list is not iesg review. The 
sponsoring AD (who is presumably the most involved) is not likely the 
one with the discuss later in the process since they had to do the 
initial review, put it through the ietf last call and then ballot it, so 
fundamentally they should have a document they can live with when it 
gets to that stage.
In any event, the current reality of having an AD weigh in with a a 
process-blocking Discuss is not exactly /under-/whelming...


Simply put:  We are starting with the reality that an AD is going to 
be expressing their opinions.  The question is when and how. It's not 
going to be /less/ intimidating to have it done as a Discuss.


Contrary to some of the mythology that has been expressed about this, 
the frequent reality is that the typical wg goal is to clear the 
Discuss, not really to engage in debate with an AD who is blocking 
document progress.  (I've watched this reality many times over the 
years.)  That is far less healthy than having the AD's concern become 
part of the /public/ review process.




I agree that this would bring the tail forward a little (not a lot).
I don't believe it would reduce AD work-load (which is another issue 
entirely).


Yes, the timing change is small.  But the context change is enormous.


Lastly, I think I disagree with you about "really serious" IETF last 
call
comments coming in the first few days. It seems to me that we also 
get an number


Some, sure.  But I said it was an efficiency hack.  The goal isn't for 
it to be perfect, just helpful.


d/


[1] The question of proper referential language is a continuing 
surprise to me, given that the currently-popular conventions create 
more problems than they solve -- and it's even a question on a PSAT 
preparatory example that I saw yesterday.  There's a pretty 
straightforward alternative that works nearly all the time:


   http://dcrocker.net/#gender

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Scott Brim]

On Thursday, May 16, 2013, Dave Crocker wrote:

> By the time the IESG schedules the vote, ADs need to already have educated
> themselves about the document.
>

Oh, so you're suggesting adding another phase to the process: IESG
education.  OK.


>
> So here's a simple proposal that pays attention to AD workload and
> includes a simple efficiency hack:
>
>  When the IETF Last Call is issued, wait a few days, to see whether
> any serious issues are raised by the community.  The really serious ones
> usually are raised quickly.  If there are none, it's pretty certain the
> document will advance to an IESG vote.  That leaves 7-10 days of IETF Last
> Call for ADs to get educated and ask questions, just like everyone else.
>

Placing a time limit on some phase of the process doesn't produce quality.
 The process should take as long as it must in order to produce a good
outcome.  The question shouldn't be how long the process takes (that's just
a cheap shortcut), it should be how to make it more efficient in doing
well.  I have an idea: let's place a time limit on working groups: they
need to finish drafts in six weeks or there will be penalties.  Why not?

Scott

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Martin Rex]

Dave Crocker wrote:
> 
> And of course, the reality is that we allow bad specs out the door all 
> the time; we just allow fewer of them than many/most other standards 
> bodies...

But different to (at least some) other standards bodies, we lack an
official means to publish defect reports (aka errata) to document defects
in a _timely_(!!) fashion.  (Timely = can be found where the RFC says
that it can be found, and within at most a few weeks after the
defect/omission has been found by an implementor).

In theory, we have the errata process, and recent RFC even include
a direct URL pointer to the RFC-Editors errata page on the title page:

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   http://www.rfc-editor.org/info/rfc.

("" containing the real number of the RFC on which this appears).

However, the Errata process is currently working poorly, primarily
because a number of folks (including some IETF leadership) currently
thinks that posting something a trivial as a missing vital one-line
clarification to a published RFC as a "substantial change" that can
only be performed by publishing a whole new RFC.


-Martin

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Ted Lemon]

On May 16, 2013, at 1:01 PM, Dave Crocker  wrote:
>   http://dcrocker.net/#gender

That's what I do.  It gets a bit awkward with verb agreement and constructs 
like "themself," which elicits the dreaded red snake underline of doom.   But I 
find it more comfortable than just subverting the sexist paradigm with a 
differently sexist paradigm.

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Dave Crocker]

On 5/16/2013 9:40 AM, Adrian Farrel wrote:

That's a good question Dave.

The community might like to comment.

On the whole, I am told that if an AD weighs in with her comments during working
group last call, her fearsome personality may overwhelm some of the WG
participants and she may dominate the WG consensus.


Only women ADs? [1]

But seriously...



Is it possible that the same would happen in IETF last call?


I think that the public sway of ADs on the IETF list is less of a risk 
than on wg mailing lists.  And note that my suggestion has ADs waiting 
to weigh in until community-generated active disagreement with the spec, 
or its passive agreement, has been established.


In any event, the current reality of having an AD weigh in with a a 
process-blocking Discuss is not exactly /under-/whelming...


Simply put:  We are starting with the reality that an AD is going to be 
expressing their opinions.  The question is when and how.  It's not 
going to be /less/ intimidating to have it done as a Discuss.


Contrary to some of the mythology that has been expressed about this, 
the frequent reality is that the typical wg goal is to clear the 
Discuss, not really to engage in debate with an AD who is blocking 
document progress.  (I've watched this reality many times over the 
years.)  That is far less healthy than having the AD's concern become 
part of the /public/ review process.




I agree that this would bring the tail forward a little (not a lot).
I don't believe it would reduce AD work-load (which is another issue entirely).


Yes, the timing change is small.  But the context change is enormous.



Lastly, I think I disagree with you about "really serious" IETF last call
comments coming in the first few days. It seems to me that we also get an number


Some, sure.  But I said it was an efficiency hack.  The goal isn't for 
it to be perfect, just helpful.


d/


[1] The question of proper referential language is a continuing surprise 
to me, given that the currently-popular conventions create more problems 
than they solve -- and it's even a question on a PSAT preparatory 
example that I saw yesterday.  There's a pretty straightforward 
alternative that works nearly all the time:


   http://dcrocker.net/#gender



--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

RE: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Adrian Farrel]

That's a good question Dave.

The community might like to comment. 

On the whole, I am told that if an AD weighs in with her comments during working
group last call, her fearsome personality may overwhelm some of the WG
participants and she may dominate the WG consensus.

Is it possible that the same would happen in IETF last call? I know, of course,
that a number of you have a robust ability to defend your opinions, but how
would you (the community, not Dave) feel if ADs (speaking as ADs, not speaking
as individual contributors) made their review comments during IETF last call?

I agree that this would bring the tail forward a little (not a lot).
I don't believe it would reduce AD work-load (which is another issue entirely).
I have mixed feelings about whether it would change the processing time for a
draft. That might depend on how the review is handled in IETF last call.

Lastly, I think I disagree with you about "really serious" IETF last call
comments coming in the first few days. It seems to me that we also get an number
of late comments of substance resulting from people who are not familiar with
the document reviewing it (such as directorate reviews and people who didn't
notice the I-D sneaking through).

Still thinking,
Adrian

> -Original Message-
> From: Dave Crocker [mailto:d...@dcrocker.net]
> Sent: 16 May 2013 17:23
> To: adr...@olddog.co.uk
> Cc: ietf@ietf.org
> Subject: Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF
process]
> 
> On 5/15/2013 1:30 PM, Adrian Farrel wrote:
> > Suppose the AD raised her concern by writing a Comment or sending an email
> and
> > balloting "No Objection." That would mean that the I-D would be approved for
> > publication.
> >
> > At this point either:
> > - the discussion goes on, but the document becomes an RFC anyway
> > or
> > - the responsible AD holds the document pending satisfactory completion of
> the
> > discussion.
> >
> > I suggest that the former is a bad result.
> 
> 
> > Personally (but this may reflect my Discusses) I find that an active
engagement
> > by the authors and the Discussing AD on the issue and the potential
resolution,
> > always leads to speedy progression of the document either with the AD
feeling
> > stupid, or the document improved. Both are adequate results.
> 
> 
> Adrian,
> 
> I suggest we all take a look at the original text of the Subject field.
> 
> The problem here is that basic reviewing is being done by the ADs too
> late in the process.  We are making the mistake of having ADs be exempt
> from IETF Last Call, and allowing them to be unprepared for the IESG
> vote.  So we are combining "education" with "voting".  That's a paradigm
> error.
> 
> By the time the IESG schedules the vote, ADs need to already have
> educated themselves about the document.
> 
> Of course, the IESG discussion during the voting process well might
> uncover an actual, serious issue with the document; this should be
> exceptional and it should be an issue that the /IESG/ agrees needs to be
> resolved and it means that voting should not take place until it is.
> But that is quite different from the usual "let's talk about it" kind of
> Discuss imposed by individual ADs.
> 
> So here's a simple proposal that pays attention to AD workload and
> includes a simple efficiency hack:
> 
>   When the IETF Last Call is issued, wait a few days, to see whether
> any serious issues are raised by the community.  The really serious ones
> usually are raised quickly.  If there are none, it's pretty certain the
> document will advance to an IESG vote.  That leaves 7-10 days of IETF
> Last Call for ADs to get educated and ask questions, just like everyone
> else.
> 
> Jari has expressed the goal of having AD concerns be raised more
> publicly.  Moving AD review and comment to the IETF Last Call venue
> nicely accomplishes this, too.
> 
> 
> 
> On 5/15/2013 8:55 AM, Thomas Narten wrote:
>  > 1) Discuss criteria should be principles, not rigid rules. The details
>  > of the issue at hand always matter, and it will sometimes come down to
>  > judgement calls where reasonable individuals just might disagree.
> 
> We've been doing protocol specification for 40 years.  If we can't
> codify the major concerns that warrant blocking advancement of a
> protocol, we're just being lazy.  Really.  That a given situation might
> cause the IESG to decide to enhance the list does not mean that the list
> shouldn't seek to be complete and precise and that ADs be held to it.
> 
> At every turn, the approach we've taken to the IESG review and approval
> proc

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Dave Crocker]

On 5/15/2013 1:30 PM, Adrian Farrel wrote:

Suppose the AD raised her concern by writing a Comment or sending an email and
balloting "No Objection." That would mean that the I-D would be approved for
publication.

At this point either:
- the discussion goes on, but the document becomes an RFC anyway
or
- the responsible AD holds the document pending satisfactory completion of the
discussion.

I suggest that the former is a bad result.




Personally (but this may reflect my Discusses) I find that an active engagement
by the authors and the Discussing AD on the issue and the potential resolution,
always leads to speedy progression of the document either with the AD feeling
stupid, or the document improved. Both are adequate results.



Adrian,

I suggest we all take a look at the original text of the Subject field.

The problem here is that basic reviewing is being done by the ADs too 
late in the process.  We are making the mistake of having ADs be exempt 
from IETF Last Call, and allowing them to be unprepared for the IESG 
vote.  So we are combining "education" with "voting".  That's a paradigm 
error.


By the time the IESG schedules the vote, ADs need to already have 
educated themselves about the document.


Of course, the IESG discussion during the voting process well might 
uncover an actual, serious issue with the document; this should be 
exceptional and it should be an issue that the /IESG/ agrees needs to be 
resolved and it means that voting should not take place until it is. 
But that is quite different from the usual "let's talk about it" kind of 
Discuss imposed by individual ADs.


So here's a simple proposal that pays attention to AD workload and 
includes a simple efficiency hack:


 When the IETF Last Call is issued, wait a few days, to see whether 
any serious issues are raised by the community.  The really serious ones 
usually are raised quickly.  If there are none, it's pretty certain the 
document will advance to an IESG vote.  That leaves 7-10 days of IETF 
Last Call for ADs to get educated and ask questions, just like everyone 
else.


Jari has expressed the goal of having AD concerns be raised more 
publicly.  Moving AD review and comment to the IETF Last Call venue 
nicely accomplishes this, too.




On 5/15/2013 8:55 AM, Thomas Narten wrote:
> 1) Discuss criteria should be principles, not rigid rules. The details
> of the issue at hand always matter, and it will sometimes come down to
> judgement calls where reasonable individuals just might disagree.

We've been doing protocol specification for 40 years.  If we can't 
codify the major concerns that warrant blocking advancement of a 
protocol, we're just being lazy.  Really.  That a given situation might 
cause the IESG to decide to enhance the list does not mean that the list 
shouldn't seek to be complete and precise and that ADs be held to it.


At every turn, the approach we've taken to the IESG review and approval 
process is to limit actual accountability to the community.  Everyone is 
well-intentioned, but really this makes the process a matter of 
personalities and not the rigor of serious professionalism.


The IESG's process is far, far better than it was 10-15 years ago, but 
it still lacks meaningful predictability and serious accountability; it 
is not reasonable for the process to require that authors and chairs 
take the initiative of complaining when an AD is being problematic.


In terms of quality assurance, the idea that we have a process that 
relies on the sudden insight of a single AD, at the end of a many-month 
process, is broken.  It's fine if that person sees something that 
everyone else has missed until then, but that is quite different from 
designing a process that is claimed to rely on it.


And of course, the reality is that we allow bad specs out the door all 
the time; we just allow fewer of them than many/most other standards 
bodies...


d/


--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Dale R. Worley]

> From: Thomas Narten 
> 
> What I do think the IETF should do is *require* that participants
> identify themselves. That means knowing who they are (a name and email
> contact) and an affiliation. For 80% of the participants, this info is
> not very hard to figure out (see below). But we also have participants
> that use obscure email handles that don't correlate to anything
> obvious, whether a real person or to a name in the list of registered
> attendees, etc. I suspect these folk are *not* intenending to be
> anonymous participants, but in practice they are.
> 
> And yes, knowing who someone is, their background and who they work
> for is important to me in figuring out how to guage their input. E.g.,
> I would likely pay more attention to an operator's comments on a
> proposed use case than from someone else.

I believe that this is a less good idea than first appears.

One objection is that the concept of "affiliation" is less simple than
it appears.  People normally expect it to mean "Who is your employer?"

As John mentions, it can mean "What is the organization whose
interests you are promoting?"  But in regard to your observation about
use cases, it can also mean "What area of technology do you have
experience in?"

In many cases, "affiliation" is related to "Who is paying for your
attendance?"  But there are situations where your employer may be
willing to pay for your attendance but doesn't want you to be seen as
officially representing them.

And (at least in common-law countries) one can simply invent an
(unincorporated!) company name and claim it as your affiliation.

In practice, the way we deal with these problems is that we consider
everyone to be individuals, rather than as "the representative of
company XYZ", and it takes time to build a reputation.  One is thus
unwilling to sacrifice that expensive reputation to advocate a poor
solution because one's employer favors it.

3) Google names, look at authorship info in RFCs, linked in,
etc. Works in a lot of cases, but is sometimes more work than
seems appropriate.

If you can't find any information about them, they have no reputation.

Dale

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Barry Leiba]

>> Putting arbitrary time limits on will hurry things up but it will not
>> produce higher quality results.
>
> Ok, so do you agree, that if who holds the work, at least should tell
> us HOW long he is holding or what is the time PLAN. Do you think
> working without plan is efficient and gives good quality.

We generally rely on judgment in this regard, not on time limits and
deadlines.  We do sometimes set deadlines (milestones in charters,
timeouts on last calls, fixed telechat dates, alerts when things take
too long), but we also give leeway (we know how solid charter
milestones aren't, and no one will ignore important, useful input just
because it came in after last call).

Someone is always responsible for determining when a discussion is no
longer productive (shepherds, chairs, responsible ADs), and someone is
responsible for moving things forward.  Reminders to those who are
responsible, telling them that things seem to be dragging, are fine.
Strict time limits are generally not how we prefer to work.

Barry

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Abdussalam Baryun]

On 5/16/13, Scott Brim  wrote:
> Please distinguish between (1) making the system efficient and (2) making
> individual documents go through it quickly.  If you put time limits on
> parts of the process, you're not allowing them to function correctly.
>  Putting arbitrary time limits on will hurry things up but it will not
> produce higher quality results.
>

Ok, so do you agree, that if who holds the work, at least should tell
us HOW long he is holding or what is the time PLAN. Do you think
working without plan is efficient and gives good quality.

AB

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Abdussalam Baryun]

Hi Loa,

I agree with you discussions are our friend. I was focusing on
processing time, not document quality. No dought if you stay longer
time you will get better quality, but what about progress. So I mean
call for discussions is for a time limit, as if no discussion happends
then the call matures and the holding of the work stops as well. If
DISCUSS is continue I never like to close it as long as it is
continuous, but not delayed. In practice I never seen that DISCUSS of
ADs with the WGs are having much continuous communication, it is delay
with no time schedule.

> 99,9% of the DISCUSSES improve our documents or the understanding
> of them.

I know that DISCUSSES with WGs improves our documents (don't forget
that WGs are following milestones and WGLC periods), but DISCUSSES
that have no time limit makes more delays. I hope we see similar times
of WG into the IESG, so the communication can improve the processing
time.

AB

On 5/16/13, Loa Andersson  wrote:
>
>
> On 2013-05-16 14:38, Abdussalam Baryun wrote:
>
>> Discussions should have a time limit (can be one week),
>
> I totally disagree, DISCUSSES are our friends, they need to be
> discussed until we have rough consensus; it seems to be a
> manifestly bad idea to draw a deadline after seven days, if
> someone come up with a satisfactory solution on the eighth.
>
> 99,9% of the DISCUSSES improve our documents or the understanding
> of them.
>
> /Loa
>
>
> like we have
>> in meetings (2hours), if there is time we can know when things are
>> needed to respond to, I usually ignore when there is no milestones or
>> planing-time. Does IESG have milestones for documents
>> processing/discussions?
>>
>
>
> --
>
>
> Loa Anderssonemail: l...@mail01.huawei.com
> Senior MPLS Expert  l...@pi.nu
> Huawei Technologies (consultant) phone: +46 739 81 21 64
>

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Scott Brim]

Please distinguish between (1) making the system efficient and (2) making
individual documents go through it quickly.  If you put time limits on
parts of the process, you're not allowing them to function correctly.
 Putting arbitrary time limits on will hurry things up but it will not
produce higher quality results.

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Loa Andersson]

On 2013-05-16 14:38, Abdussalam Baryun wrote:


Discussions should have a time limit (can be one week),


I totally disagree, DISCUSSES are our friends, they need to be
discussed until we have rough consensus; it seems to be a
manifestly bad idea to draw a deadline after seven days, if
someone come up with a satisfactory solution on the eighth.

99,9% of the DISCUSSES improve our documents or the understanding
of them.

/Loa


like we have

in meetings (2hours), if there is time we can know when things are
needed to respond to, I usually ignore when there is no milestones or
planing-time. Does IESG have milestones for documents
processing/discussions?




--


Loa Anderssonemail: l...@mail01.huawei.com
Senior MPLS Expert  l...@pi.nu
Huawei Technologies (consultant) phone: +46 739 81 21 64

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Abdussalam Baryun]

On May 15, 2013, at 4:30 PM, Adrian Farrel  wrote:
> The claim (or one of the claims) is that some ADs may place Discusses that
> are
> intended to raise a discussion with the authors/WG that could equally have
> been
> raised with a Comment or through direct email. This, it is claimed, may
> unnecessarily delay the document from completing the publication process.

Discussions should have a time limit (can be one week), like we have
in meetings (2hours), if there is time we can know when things are
needed to respond to, I usually ignore when there is no milestones or
planing-time. Does IESG have milestones for documents
processing/discussions?

>
> Now the dangerous bit,
>
> Suppose the AD raised her concern by writing a Comment or sending an email
> and
> balloting "No Objection." That would mean that the I-D would be approved
> for
> publication.
>
> At this point either:
> - the discussion goes on, but the document becomes an RFC anyway
> or
> - the responsible AD holds the document pending satisfactory completion of
> the
> discussion.

That AD SHOULD not hold for unlimited time, also should discuss the
issue with the WG related in limited time.
>
> I suggest that the former is a bad result. Not that the authors/WG will
> ignore
> the discussion, but if they disagree on something the AD considers very
> important, the authors/WG have no incentive to participate in the
> discussion.

Only community rough concensus will decide the final result,
> Of
> course, all participants in this thread so far would never behave like that,
> but
> there is a possibility that this will happen for some authors.

Yes only if there is no time limits for work that should be done,

>
> I also suggest that the latter introduces exactly the same amount of delay
> as
> the Discuss.

There is always possibility of large delay in systems that have no
time limits for processing or responds. Our time/work used is
important for IETF, IMO, no one should hold work/time only if able to
decide/notify/plan when/how to leave it go for all reaction
possibilities.

AB

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

I must say that I have enjoyed reading the discussion between the three of you, 
and think it is immensely valuable in explaining what the IESG ought to be 
doing.   You three should write it up.

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Keith Moore]

On 05/15/2013 12:25 PM, Thomas Narten wrote:

I don't think the IETF needs to be in the profile/resume
business. There are plenty of other places that do a fine job already.

What I do think the IETF should do is *require* that participants
identify themselves. That means knowing who they are (a name and email
contact) and an affiliation.


I have mixed feelings about this.   On one hand I would like to know who 
(if anyone) is paying someone to do IETF work, because there's always 
some potential for inappropriate bias even among people with a high 
degree of integrity - and of course not everyone has such integrity.


On the other hand I don't think that a contributor's affiliation should 
mean anything at all when evaluating that contributor's input to IETF.   
If people treat contributors from major companies as having more weight 
than other contributors, it makes a joke out of the whole notion of 
consensus-based decision making.   (And we all know that people do this 
sometimes.)


I also don't think that anyone should automatically presume that a 
contributor's input is representing his employer's interests.


On balance, I do sometimes find it helpful when IETF contributors 
disclose their employers.   But I don't think it should be required.   
And if everybody stopped disclosing their employers in the context of 
IETF conversations, it might be a good thing.


Keith

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/16/2013 01:44 AM, John C Klensin wrote:


--On Thursday, May 16, 2013 00:55 -0400 Keith Moore
 wrote:


Which is to say, if there is only a single AD "blocking" a
document,  that block is essentially a 2 week affair if you
are willing to push  the point. No need for negotiating; if
the WG decides that the AD is  totally off base, tell your
sponsoring AD that you're waiting the two  weeks. People
(unfortunately IMO) don't push the point nearly enough.

I think it's very unfortunate that IESG has adopted rules that
work this way.   Part of IESG's job is to provide independent
review of WG output.   It that review can be circumvented
merely by waiting two weeks, that's a bug in the process.
And if an AD raises a DISCUSS about a matter of technical or
document quality (or for that matter, about a process
violation), and the WG isn't even willing to discuss the
point, but instead relies on the two week timeout, I think
that's grounds for appeal to the IAB.

Keith,

I generally agree with Pete although I share your low opinion of
a lot of current IETF work, but your comment above seems to call
for comment from someone, like myself, who has often been
critical of the IESG.  I don't think the current rules are
ideal, but the effect of the ones that you and Pete cite isn't
really that a WG can circumvent a review by waiting two weeks.
First, the dissenting AD has those same two weeks to convince
others on the IESG that his or her position is reasonable or at
least needs more extensive consideration.  If that is possible,
then there is no longer a single AD objecting and the two week
rule does not apply.  If it is not possible, then there is
either something wrong with the objection or the AD making it
and probably it is reasonable for the process to move forward.


I don't think I agree.   I do agree that a single AD shouldn't be able 
to indefinitely delay a WG's output, and that you want IESG to be able 
resolve such issues internally in the vast majority of cases (because 
appeals to IAB are very time- and resource-consuming). But I don't think 
two weeks is long enough in general to get other ADs to thoroughly 
review a document.  Two months would be much better.


Now it might be the case that IESG members will help each other out, and 
collaborate to log multiple DISCUSS votes to give everybody more time to 
review a document.   People of good will acting in good faith can 
usually find ways to work around buggy processes to make the right thing 
happen, at least in the most important cases.   But it's not ideal, and 
it relies on ADs "getting along" with each other - which creates 
incentives for ADs to not do an adequate job of reviewing some 
documents, so that they'll be able to call in favors from other ADs when 
they need them.   So overall I think the balloting process that IESG 
currently follows is seriously flawed.



Conversely, a WG that decides to avoid actually engaging on an
issue in the hope of letting the two-week clock run out is
putting itself at considerable risk.  The IESG still have the
ability to fire WG leadership and even to close WGs.


In theory, yes.   But that hardly seems like a good tool for resolving 
issues in a single document, especially when the WG has other ongoing 
work that might turn out to be useful.Also, at least when I was on 
IESG all of us seemed to be aware that though we did have the authority 
and responsibility to push back on poor work, we also had to be mindful 
of the potential for the community to mutiny.




Under any
reasonable circumstances, I assume that the IESG would respond
very strongly if an AD pointed out the a WG had ignored an
effort to discuss a substantive issue even if the rest of the
IESG disagreed with that particular AD about that issue.


I'd hope so.  But I also think that the process should work even for an 
objection raised by an AD who was unpopular with the rest of IESG.  Of 
course I hope that ADs get along well with one another and work together 
to make a better Internet.  But I also know from experience that some 
ADs tend to have more clout than others, and that the ones with the most 
clout aren't always the ones with the best technical judgment.


Keith

Re: call for ideas: tail-heavy IETF process

[John C Klensin]

--On Thursday, May 16, 2013 00:55 -0400 Keith Moore
 wrote:

>> Which is to say, if there is only a single AD "blocking" a
>> document,  that block is essentially a 2 week affair if you
>> are willing to push  the point. No need for negotiating; if
>> the WG decides that the AD is  totally off base, tell your
>> sponsoring AD that you're waiting the two  weeks. People
>> (unfortunately IMO) don't push the point nearly enough.
> 
> I think it's very unfortunate that IESG has adopted rules that
> work this way.   Part of IESG's job is to provide independent
> review of WG output.   It that review can be circumvented
> merely by waiting two weeks, that's a bug in the process.
> And if an AD raises a DISCUSS about a matter of technical or
> document quality (or for that matter, about a process
> violation), and the WG isn't even willing to discuss the
> point, but instead relies on the two week timeout, I think
> that's grounds for appeal to the IAB.

Keith,

I generally agree with Pete although I share your low opinion of
a lot of current IETF work, but your comment above seems to call
for comment from someone, like myself, who has often been
critical of the IESG.  I don't think the current rules are
ideal, but the effect of the ones that you and Pete cite isn't
really that a WG can circumvent a review by waiting two weeks.
First, the dissenting AD has those same two weeks to convince
others on the IESG that his or her position is reasonable or at
least needs more extensive consideration.  If that is possible,
then there is no longer a single AD objecting and the two week
rule does not apply.  If it is not possible, then there is
either something wrong with the objection or the AD making it
and probably it is reasonable for the process to move forward.

Conversely, a WG that decides to avoid actually engaging on an
issue in the hope of letting the two-week clock run out is
putting itself at considerable risk.  The IESG still have the
ability to fire WG leadership and even to close WGs.  Under any
reasonable circumstances, I assume that the IESG would respond
very strongly if an AD pointed out the a WG had ignored an
effort to discuss a substantive issue even if the rest of the
IESG disagreed with that particular AD about that issue.  I
can't remember any instances of a WG Chair being fired, or a WG
being shut down entirely, while documents were post IETF Last
Call and under IESG discussion, but I presume that is because
the IESG has never been provoked quite enough rather than
because there is any rule that would prevent it.  Firing a Chair
or closing a WG on the grounds that they refused to engage on a
substantive issue would, I assume, be sustained on any appeal.

Similarly...
 
> I agree that we should reject kings.   But we should also
> reject stubborn working groups and stubborn authors.   We
> shouldn't presume that a WG knows better than the IESG.  The
> two have different points-of-view, but neither is inherently
> superior to the other.

I don't think we should presume that a WG knows better than the
IESG either.  I do think it is reasonable to assume that a WG
knows better than a single AD who cannot convince others on the
IESG of the merit of his or her position or even that the issue
is important enough that they should read the document carefully
and form their own informed opinions.

best,
  john

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[John C Klensin]

--On Wednesday, May 15, 2013 14:28 -0700 Doug Ewell
 wrote:

>...
> I did this because
> the WG at the time included a malicious contributor who had
> already contacted the HR department of another contributor's
> employer, asking them to professionally discipline the
> employee, because he had supported an RFC 3683 PR-action
> against the first contributor. Full disclosure can be a
> dangerous thing.

I know that sort of "contact the employer and ask them to chew
out the employee" stuff goes on because I've had it tried on me
at least twice although both involved technical issues and
choices, not, e.g., a PR-action.  In the first instance, the
employer said something about academic freedom and essentially
told the person complaining to kiss off.  In the second, the
employer laughed.  

I may have been luckier in my choices of employers (and clients)
and I know bad stuff happens sometimes, but the sort of case you
outline doesn't seem to me to be a good argument against
disclosure of affiliations.  It seems to me that, very rare edge
cases aside, either:

(i) Your employer knows what you are doing and saying in
the IETF, at least to the extent that they care, and
will back you should such complaints arrive.

(ii) You and your employer have an agreement that you
participate in the IETF as an independent activity that
they don't try to control.  Should a complaint arise,
they presumably tell the complaining party that unless
your IETF behavior is an embarrassment to the company,
in which case (iii) applies.

(iii) Your IETF behavior is, as far as your employer is
concerned, that of a loose cannon.  You regularly work
against company positions or the company's best
interests and haven't laid the internal foundation for
that to be acceptable.  A complaint associated with
something you have disclosed could get you into big
trouble, but complaints are equally likely from people
who know where you are working, disclosure or no, such
as fellow employees of the same organization.

So I suggest that, if your behavior is proper and above board,
disclosure will rarely create a problem.  If your behavior
isn't, then disclosure may be the least of your difficulties.  

In addition, the IETF may be in need of a mechanism for
documenting and disclosing the identities of anyone who thinks
that complaining to someone's employer about his or her
reasonable behavior at the IETF.  I imagine the community could
figure out appropriate and completely informal ways to
discourage that particular style of trying to influence IETF
decision-making.

best,
john

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/15/2013 09:07 PM, Pete Resnick wrote:
I initially replied to address Keith's comment. But a few things on 
Joe's:


On 5/15/13 7:41 AM, Keith Moore wrote:

On 05/15/2013 10:39 AM, Joe Touch wrote:

On 5/14/2013 9:54 PM, Keith Moore wrote:

Publishing broken or unclear documents is not progress.


Broken, agreed.

Unclear, nope - please review the NON-DISCUSS criteria, notably:

The motivation for a particular feature of a protocol is not clear 
enough. At the IESG review stage, protocols should not be blocked 
because they provide capabilities beyond what seems necessary to 
acquit their responsibilities.


The DISCUSS isn't there to make documents "better" - that's for 
COMMENTs.


Exactly right. Sometimes we forget; it's a good thing to remind us.

A DISCUSS there to catch a set of problems and to *block* the 
document's progress until that problem is resolved.


Mostly correct. However:

- If there is only one AD who wishes to DISCUSS and no other AD agrees 
with the DISCUSS holder, at the next telechat the document is 
unblocked. (See .)
- Even if others agree with the DISCUSS, the chair can be asked to use 
the alternate procedure, which requires 2/3 of non-recused ADs to 
agree to publication.


Which is to say, if there is only a single AD "blocking" a document, 
that block is essentially a 2 week affair if you are willing to push 
the point. No need for negotiating; if the WG decides that the AD is 
totally off base, tell your sponsoring AD that you're waiting the two 
weeks. People (unfortunately IMO) don't push the point nearly enough.


I think it's very unfortunate that IESG has adopted rules that work this 
way.   Part of IESG's job is to provide independent review of WG 
output.   It that review can be circumvented merely by waiting two 
weeks, that's a bug in the process.   And if an AD raises a DISCUSS 
about a matter of technical or document quality (or for that matter, 
about a process violation), and the WG isn't even willing to discuss the 
point, but instead relies on the two week timeout, I think that's 
grounds for appeal to the IAB.


(For the record, the IESG has *never* used the latter of the two 
procedures.)


That said, I am also of the view that there is a third way, but I have 
never seen a WG attempt it:


DISCUSS should in fact require discussing. 


We agree on that much.
Assuming there was some good faith effort on the part of the WG to 
figure out what the AD was on about and they really assess that the AD 
has it wrong, a WG could say, "Sorry, we got this right. You are 
confused (or wrong). We are not changing the document. We are done 
discussing." At that point, I am of the opinion that the AD cannot 
hold the DISCUSS any longer. The AD must move to ABSTAIN. 


I disagree with this in the strongest possible terms.  I believe that 
for IESG to have rules that insist on or encourage voting ABSTAIN when 
the AD really means "this document is not acceptable", is both 
irresponsible and dishonest on the IESG's part.  I also believe that it 
tends to mean that ADs who didn't actually review the document get more 
say than ADs who did.   The proper thing to do in that case is for 
either side to appeal to the IAB.


The discussion is, for all intents and purposes, over and to continue 
to DISCUSS is (IMO) an appealable offense. Then it is a matter of the 
IESG deciding whether there are enough ADs supporting the document 
(YES or NO OBJECTION) to count as consensus of the IESG. We ostensibly 
use 2/3 of non-recused ADs to mean "consensus", since (I think the 
theory goes) if you can't get 2/3 of ADs to agree that it's OK to 
publish a document, that is a sign of a lack of rough consensus in the 
IETF (and probably a serious problem in WG operation). Indeed, if the 
ADs are so off of their rockers that more than 1/3 of them are against 
a perfectly reasonable document, it's time for the appeals and recall 
procedures to be used.


I don't think IESG voting should be thought of as a consensus process, 
except perhaps in the deadlock breaking procedure.   The typical number 
of reviewers on the IESG is too small for a consensus process to be 
meaningful.   With so few thorough reviewers outside of the WG, you need 
a procedure that at least initially assumes that all review comments 
have to be taken seriously.



Now, as to Keith's comments:

I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS 
isn't just for blocking documents.   And document quality is as 
important (in the sense that poor document quality can lead to as 
many interoperability or other problems) as technical correctness.


Why are people trying to sabotage IESG?


I'm sorry Keith, but your last line is rubbish. To claim that what 
people in this thread are talking about amounts to an attempt to 
sabotage the IESG is the height of hubris. 


"sabotage" is probably not the best word I could have chosen.   But I do 
have the strong impress

Re: call for ideas: tail-heavy IETF process

[Pete Resnick]

I initially replied to address Keith's comment. But a few things on Joe's:

On 5/15/13 7:41 AM, Keith Moore wrote:

On 05/15/2013 10:39 AM, Joe Touch wrote:

On 5/14/2013 9:54 PM, Keith Moore wrote:

Publishing broken or unclear documents is not progress.


Broken, agreed.

Unclear, nope - please review the NON-DISCUSS criteria, notably:

The motivation for a particular feature of a protocol is not clear 
enough. At the IESG review stage, protocols should not be blocked 
because they provide capabilities beyond what seems necessary to 
acquit their responsibilities.


The DISCUSS isn't there to make documents "better" - that's for COMMENTs.


Exactly right. Sometimes we forget; it's a good thing to remind us.

A DISCUSS there to catch a set of problems and to *block* the 
document's progress until that problem is resolved.


Mostly correct. However:

- If there is only one AD who wishes to DISCUSS and no other AD agrees 
with the DISCUSS holder, at the next telechat the document is unblocked. 
(See .)
- Even if others agree with the DISCUSS, the chair can be asked to use 
the alternate procedure, which requires 2/3 of non-recused ADs to agree 
to publication.


Which is to say, if there is only a single AD "blocking" a document, 
that block is essentially a 2 week affair if you are willing to push the 
point. No need for negotiating; if the WG decides that the AD is totally 
off base, tell your sponsoring AD that you're waiting the two weeks. 
People (unfortunately IMO) don't push the point nearly enough.


(For the record, the IESG has *never* used the latter of the two 
procedures.)


That said, I am also of the view that there is a third way, but I have 
never seen a WG attempt it:


DISCUSS should in fact require discussing. Assuming there was some good 
faith effort on the part of the WG to figure out what the AD was on 
about and they really assess that the AD has it wrong, a WG could say, 
"Sorry, we got this right. You are confused (or wrong). We are not 
changing the document. We are done discussing." At that point, I am of 
the opinion that the AD cannot hold the DISCUSS any longer. The AD must 
move to ABSTAIN. The discussion is, for all intents and purposes, over 
and to continue to DISCUSS is (IMO) an appealable offense. Then it is a 
matter of the IESG deciding whether there are enough ADs supporting the 
document (YES or NO OBJECTION) to count as consensus of the IESG. We 
ostensibly use 2/3 of non-recused ADs to mean "consensus", since (I 
think the theory goes) if you can't get 2/3 of ADs to agree that it's OK 
to publish a document, that is a sign of a lack of rough consensus in 
the IETF (and probably a serious problem in WG operation). Indeed, if 
the ADs are so off of their rockers that more than 1/3 of them are 
against a perfectly reasonable document, it's time for the appeals and 
recall procedures to be used.


(This is, BTW, where I disagree with Dave Crocker: Yes, a DISCUSS can be 
used as an exercise in authority, but only if it is allowed to be an 
"assertion that changes are being required". If the rest of the IETF 
started saying, "Sorry, no change is going to be made" instead of making 
changes simply to satisfy the AD, we'd actually be better off and the 
DISCUSS would stop being seen -- and used -- as authoritative.)


Finally, do keep in mind that, although there have been times where the 
numbers have been much different, currently there are 16 documents with 
outstanding DISCUSSes (and I think the total number has been pretty 
stable for a while), and 7 of those are on tomorrow's telechat and 
therefore should hopefully be cleared within a few days of when the 
document could most quickly have been approved anyway. So I'm not sure 
(and we should look at the statistics) how much of a problem this is for 
most documents we are producing.


Now, as to Keith's comments:

I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS 
isn't just for blocking documents.   And document quality is as 
important (in the sense that poor document quality can lead to as many 
interoperability or other problems) as technical correctness.


Why are people trying to sabotage IESG?


I'm sorry Keith, but your last line is rubbish. To claim that what 
people in this thread are talking about amounts to an attempt to 
sabotage the IESG is the height of hubris. In my opinion, the IESG has 2 
jobs as far as document review goes: (1) Check for serious technical 
problems that the WG might not have considered and make sure they get 
considered; and (2) Make sure that our processes have been followed to 
assure that the WG products are truly the product of (IETF-wide) 
consensus and fit within the policies of the IETF. I am *positive* that 
I have failed to limit my DISCUSSes on documents to one of those two 
reasons; I expect I will err and do so in the future. I should be 
corrected. We reject kings, and should.


As for the rest: The I

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Edwin A. Opare]

In all earnestness I don't think a resume will be necessary at all :).

--
Regards,

Edwin A. Opare



On Wed, May 15, 2013 at 11:33 PM, Stephen Farrell  wrote:

>
>
> On 05/15/2013 07:38 PM, John C Klensin wrote:
> > So, what would you have me (and others like me) put on
> > registration forms so that I'm not part of that undifferentiated
> > "180 names"?
>
> How about 7 densely worded paragraphs?
>
> Sorry, couldn't resist:-)
>
> S.
>
>

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Stephen Farrell]

On 05/15/2013 07:38 PM, John C Klensin wrote:
> So, what would you have me (and others like me) put on
> registration forms so that I'm not part of that undifferentiated
> "180 names"? 

How about 7 densely worded paragraphs?

Sorry, couldn't resist:-)

S.

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Thomas Narten]

Hi John.

I agree there are a number of specific cases where there is no
simple/obvious way to handle. I'd be OK with something fairly simple
as "unaffiliated" or "consultant" or something more nuanced. But I'd
like to think those are edge cases (but could of course be wrong in
how common they are). I was specifically thinking of much more obvious
cases where I'm pretty sure that their main employers is sending them.

One thing I've done at times is look at the attendance figures at
meetings to try to identify some of the entities that send the most
attendees to meetings (e.g.,, to track trends). As a result of that,
I've noticed that the number of "no affiliation given" seems high to
me, and I've seen obvious cases where I know the person and where they
were working in the past, and then when I check later, it turns out
they are still working at the same place (e.g. a Cisco, Google or
Huawei). Maybe its just the case that the registration form makes it
to easy to leave it blank.

Just loooking at the list
https://www.ietf.org/registration/ietf86/attendance.py?sortkey=3&login=
I easily counted 20 names of folk I know that don't list an
affiliation, where if any of them have changed employers, that number
would likely be very small.

Thomas

RE: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[l.wood]

You want resumes? You've got linkedin for that.

The sort of thing Doug describes is actually quite common.

For example, I once had a group chair threaten to have me disciplined by the 
company
I worked for, for pointing out the technical failings of his pet protocol.

The IETF isn't a lovey-dovey bunch of hippies working in harmony for humanity.
The IETF is hardball.

Lloyd Wood
http://sat-net.com/L.Wood/



From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Doug Ewell 
[d...@ewellic.org]
Sent: 15 May 2013 22:28
To: ietf@ietf.org
Subject: Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF   
process]

John C Klensin  wrote:

> I think it is all very well to ask for affiliations in principle
> and, also in principle, I agree that it is a good idea. But, in
> practice, I think there are a lot of clarifications and other
> changes that would be required and that might or might not be
> practical.

I used the term "Consultant" in RFCs 4645 and 5645, instead of revealing
the name of my company (which was different each time, and neither of
which contributed any time or money to my WG effort). I did this because
the WG at the time included a malicious contributor who had already
contacted the HR department of another contributor's employer, asking
them to professionally discipline the employee, because he had supported
an RFC 3683 PR-action against the first contributor. Full disclosure can
be a dangerous thing.

--
Doug Ewell | Thornton, CO, USA
http://ewellic.org | @DougEwell ­

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Doug Ewell]

John C Klensin  wrote:

> I think it is all very well to ask for affiliations in principle
> and, also in principle, I agree that it is a good idea. But, in
> practice, I think there are a lot of clarifications and other
> changes that would be required and that might or might not be
> practical. 

I used the term "Consultant" in RFCs 4645 and 5645, instead of revealing
the name of my company (which was different each time, and neither of
which contributed any time or money to my WG effort). I did this because
the WG at the time included a malicious contributor who had already
contacted the HR department of another contributor's employer, asking
them to professionally discipline the employee, because he had supported
an RFC 3683 PR-action against the first contributor. Full disclosure can
be a dangerous thing.

--
Doug Ewell | Thornton, CO, USA
http://ewellic.org | @DougEwell ­

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/15/2013 7:49 AM, Ralph Droms wrote:

>The DISCUSS isn't there to make documents "better" - that's for COMMENTs. A 
DISCUSS there to catch a set of problems and to*block*  the document's progress until that 
problem is resolved.



I'll agree with you *if*  you consider an unclear description of a
feature of a protocol, severe enough that reader of the specification
are not able to build interoperable implementations, as a problem for
which a DISCUSS can be posted.

In my opinion, there are many ways in which document can be unclear
beyond the "motivation for a particular feature of a protocol is not
clear enough."

- Ralph


Absolutely. Note that issues interfering with interoperability or 
correctness are among the explicit DISCUSS criteria.


Joe

Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Ted Lemon]

On May 15, 2013, at 4:30 PM, Adrian Farrel  wrote:
> I suggest that the former is a bad result. Not that the authors/WG will ignore
> the discussion, but if they disagree on something the AD considers very
> important, the authors/WG have no incentive to participate in the discussion. 
> Of
> course, all participants in this thread so far would never behave like that, 
> but
> there is a possibility that this will happen for some authors.

This can also happen purely by accident—it need not involve malice on anyone's 
part.

Is this an elephant? [Was: call for ideas: tail-heavy IETF process]

[Adrian Farrel]

The claim (or one of the claims) is that some ADs may place Discusses that are
intended to raise a discussion with the authors/WG that could equally have been
raised with a Comment or through direct email. This, it is claimed, may
unnecessarily delay the document from completing the publication process.

Now the dangerous bit,

Suppose the AD raised her concern by writing a Comment or sending an email and
balloting "No Objection." That would mean that the I-D would be approved for
publication.

At this point either:
- the discussion goes on, but the document becomes an RFC anyway
or
- the responsible AD holds the document pending satisfactory completion of the
discussion.

I suggest that the former is a bad result. Not that the authors/WG will ignore
the discussion, but if they disagree on something the AD considers very
important, the authors/WG have no incentive to participate in the discussion. Of
course, all participants in this thread so far would never behave like that, but
there is a possibility that this will happen for some authors.

I also suggest that the latter introduces exactly the same amount of delay as
the Discuss.

Personally (but this may reflect my Discusses) I find that an active engagement
by the authors and the Discussing AD on the issue and the potential resolution,
always leads to speedy progression of the document either with the AD feeling
stupid, or the document improved. Both are adequate results.

Adrian

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/15/2013 11:33 AM, Yoav Nir wrote:

On May 15, 2013, at 6:06 PM, Keith Moore 
  wrote:


IMO, IESG should have grounds to reject any document that isn't specifically 
authorized in a WG's charter.

- Keith


Why? There's definitely a process failure there, and it should be blamed on the 
WG chairs and/or the AD, who should have either moved the work out of the 
working group or worked on updating the charter.


ADs shouldn't have to micro-manage the WGs and keep track of whether 
every single document that a WG is working on is authorized by its 
charter.  Fundamentally, it's the WG chair's job to stay within the charter.


What I was addressing with my above statement is that there seems to be 
a presumption on the part of some people that a WG can produce anything 
it wants, and that the IESG is under an obligation to approve such work 
unless it can object on some very specific grounds.I can understand 
something resembling such a presumption for work that the WG is 
specifically chartered to do.   We don't want WGs investing their 
members' time and energy to produce something that will never see the 
light of day, and WG members need some assurance that their efforts are 
likely to bear fruit at least as far as publication is concerned.   But 
I see no reason that a WG should be able to presume that the IESG should 
ultimately accept something that they weren't specifically chartered to 
do in the first place.


Though probably a better remedy than to reject the document outright, 
would be for IESG to treat such documents as individual submissions.


Keith

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/15/2013 02:48 PM, Joe Touch wrote:



On 5/15/2013 11:08 AM, Ted Lemon wrote:

I don't think this is a topic that the IETF as a whole is likely to
find very interesting. However, if anyone is curious, they are welcome
to read the DISCUSS here and see if they agree with your
characterization of my question:
http://datatracker.ietf.org/doc/draft-ietf-tcpm-experimental-options/ballot/ 



For those who may be interested, the last sentence of the first
paragraph is the motivation for this being a DISCUSS position (as
opposed to a comment).


Which is "I think that using a 4-byte ExID runs a real risk of
overflowing the available space in the TCP header in real-world 
circumstances."


Except that the document already describes the ExID as either 16-bit 
or 32-bit:


   >> All ExIDs MUST be either 16-bits or 32-bits long.

Motivation for the additional two bytes is already explained in the 
document in several places, notably:


   The second two bytes serve as a "magic number".
...
   Using the additional magic number bytes helps the option contents
   have the same byte alignment in the TCP header as they would have if
   (or when) a conventional (non-experiment) TCP option codepoint is
   assigned. Use of the same alignment reduces the potential for
   implementation errors, especially in using the same word-alignment
   padding, if the same software is later modified to use a
   conventional codepoint. Use of the longer, 32-bit ExID further
   decreases the probability of such a false positive compared to those
   using shorter, 16-bit ExIDs.
...
   Use of the longer, 32-bit ExID consumes more
   space, but provides more protection against false positives.

Which is why I feel this motivation isn't sufficient for a DISCUSS.


It certainly seems like a valid topic for an AD to want to discuss with 
a WG.And that's all that DISCUSS inherently means.


Keith

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/15/2013 11:08 AM, Ted Lemon wrote:

On May 15, 2013, at 1:23 PM, Joe Touch  wrote:

You don't agree that the motivation for the difference between using 16-bit vs. 
32-bit ExIDs is sufficient, even though that is already discussed in the 
document.


I don't think this is a topic that the IETF as a whole is likely to
find very interesting. However, if anyone is curious, they are welcome
to read the DISCUSS here and see if they agree with your
characterization of my question:
http://datatracker.ietf.org/doc/draft-ietf-tcpm-experimental-options/ballot/

For those who may be interested, the last sentence of the first
paragraph is the motivation for this being a DISCUSS position (as
opposed to a comment).


Which is "I think that using a 4-byte ExID runs a real risk of
overflowing the available space in the TCP header in real-world 
circumstances."


Except that the document already describes the ExID as either 16-bit or 
32-bit:


   >> All ExIDs MUST be either 16-bits or 32-bits long.

Motivation for the additional two bytes is already explained in the 
document in several places, notably:


   The second two bytes serve as a "magic number".
...
   Using the additional magic number bytes helps the option contents
   have the same byte alignment in the TCP header as they would have if
   (or when) a conventional (non-experiment) TCP option codepoint is
   assigned. Use of the same alignment reduces the potential for
   implementation errors, especially in using the same word-alignment
   padding, if the same software is later modified to use a
   conventional codepoint. Use of the longer, 32-bit ExID further
   decreases the probability of such a false positive compared to those
   using shorter, 16-bit ExIDs.
...
   Use of the longer, 32-bit ExID consumes more
   space, but provides more protection against false positives.

Which is why I feel this motivation isn't sufficient for a DISCUSS.

I'd be glad to hear others' view of this as well.

Joe

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[John C Klensin]

--On Wednesday, May 15, 2013 18:25 +0200 Thomas Narten
 wrote:

> I don't think the IETF needs to be in the profile/resume
> business. There are plenty of other places that do a fine job
> already.
> 
> What I do think the IETF should do is *require* that
> participants identify themselves. That means knowing who they
> are (a name and email contact) and an affiliation. For 80% of
> the participants, this info is not very hard to figure out
> (see below). But we also have participants that use obscure
> email handles that don't correlate to anything obvious,
> whether a real person or to a name in the list of registered
> attendees, etc. I suspect these folk are *not* intenending to
> be anonymous participants, but in practice they are.

Thomas, 

I completely agree, but..

> Also, for some reason, some people who register don't bother
> giving an affiliation. In some cases this is intentional, but
> there are others where it doesn't make sense (e.g., someone
> who has worked for the same employer for 10+ years and is
> still working for that employer).
> 
> E.g., if you look at the registration list for Orland0, fully
> 180 names don't list affiliations -- and there are a number
> pretty obvious surprises in that list...

Ok.  I'm probably one of the 180, although I would be surprised
if I were one of the "obvious surprises".  I had a corporate
affiliation until somewhat over a decade ago and showed that
affiliation when asked for it on registration, IAB and IESG
mini-biographies, etc.  Since then, I have been an independent
consultant.  I don't consider the way my business is organized,
whether it has other employees or not, etc., to be any of the
IETF's business.  In order to respond to an implied part of your
question, in that last decade or so, I have never been paid by a
client to attend IETF or to represent that client's interests.
That has been largely by choice and driven by a desire to stay
absolutely independent.   I have accepted travel money and
waivers of registration fees on a few (very few) occasions, but
never, in my post-corporate life, compensation for my time spent
of IETF.

So, what would you have me (and others like me) put on
registration forms so that I'm not part of that undifferentiated
"180 names"?  I objected several years ago to the Secretariat
listing me as "Independent" because I know of multiple
organizations/ enterprises with "Independent" in their names
(including, apparently, six separate ones with second-level
domain names in six popular gTLDs I checked).  You tell me how
to fill in that form a way that doesn't misrepresent my
situation or disclose information that is irrelevant to the IETF
and I will happily do it.  Up through the last IETF meeting, I
could most closely approximate that condition by leaving the
information box blank.

To pursue this a tad further, for the purposes for which you
want affiliations, someone who is legally a consultant or
independent contractor rather than an employee but who is being
paid by a firm to attend IETF may have no less obligation to
consider the interests of that firm in their IETF actions than
you have to IBM.  They might even have more of an obligation.
If you want affiliation information in the interest of openness
--and, again, I think that would be a really good idea-- than
such a person is not "independent" or representing only
themselves.   If we are trying to determinate affiliation to
evaluate positions, he or she is definitely affiliated with that
firm and should be listing it (ideally as "consultant to..." or
"contractor to...").

On of the other side of that coin, in prior professional lives,
I've been in situations in which an employer or research sponsor
has essentially said "you are welcome to attend IETF as long as
you can show (in a way that will satisfy auditors) that you did
so entirely on your own time, with no portion of your salary
during that time or your expenses attributable to the company.
In most cases, that person is as independent of company control
over positions taken as I am -- far more so than a consultant
who is being paid to represent a company at IETF or to come to
IETF in order to advise the company on IETF strategy.  She may
even be constrained to not mention the company's name in
conjunction with IETF efforts for fear that mention will be
taken as endorsement.   What would you like to see on the
registration form?

Finally, do any of the answers change if the consultant or
contractor either is employed directly by a consulting firm that
pays a salary or is an individual but more organized as a
business than I have chosen to be? Is the affiliation you want
the name of the nominal employer of the organizational identity
of whomever is really paying the bills and/or calling the tune?

Relative to overall IETF participation, all of the above may be
edge cases.  For reasons similar to those of the "diversity"
debates, I have no way to guess how many there actually are in
the registrant or partic

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 15, 2013, at 1:23 PM, Joe Touch  wrote:
> You don't agree that the motivation for the difference between using 16-bit 
> vs. 32-bit ExIDs is sufficient, even though that is already discussed in the 
> document.

I don't think this is a topic that the IETF as a whole is likely to find very 
interesting.   However, if anyone is curious, they are welcome to read the 
DISCUSS here and see if they agree with your characterization of my question: 
http://datatracker.ietf.org/doc/draft-ietf-tcpm-experimental-options/ballot/

For those who may be interested, the last sentence of the first paragraph is 
the motivation for this being a DISCUSS position (as opposed to a comment).

Re: call for ideas: tail-heavy IETF process

[SM]

At 08:06 15-05-2013, Keith Moore wrote:
IMO, IESG should have grounds to reject any document that isn't 
specifically authorized in a WG's charter.


I read a few charters:

core:
  Dec 2099 - HOLD (date TBD) Constrained security bootstrapping specification
  submitted to IESG
ancp:
  Sep 2010 - Access Node Control Protocol (ANCP) Last Call
  Dec 2010 - ANCP MIB Last Call
  Dec 2010 - ANCP Multicast Extensions last call
  Jan 2011 - ANCP applicability to PON last call
  Mar 2011 - Re-charter or conclude Working Group

6man:
 Jan 2008 - Submit PPP Compression Negotiation specification to IESG as a
Proposed Standard
 Mar 2008 - Determine way forward for ULA-C specification

l2tpext:
  Mar 2008 - Submit Internet-Draft of PPP over L2TPv3 to IESG
  Done - WG Last Call on TDM over L2TPv3
  Jun 2008 - WG Last Call on IP over L2TPv3

drinks:
  Apr 2012 - Request publication of SPPP over SOAP Document
  Apr 2012 - Request publication of Framework Document

straw:
  Nov 2011 - Specification for a SIP overload control mechanism based on
 implicit/explicit feedback to IESG for publication as 
Proposed Standard
  Feb 2012 - Specification for a SIP load filtering mechaism to IESG 
for publication

 as Proposed Standard
idr:
  Mar 2010 - Solicit work items for scalability improvements
  Aug 2010 - Submit AS-wide Unique BGP Identifier for BGP-4 to IESG 
as a Proposed Standard
  Aug 2010 - Submit Dynamic Capability for BGP-4 to IESG as a 
Proposed Standard

  Aug 2010 - Submit ASpath ORF draft to IESG as a Proposed Standard
  Aug 2010 - Submit MIB v2 for BGP-4 to IESG as a Proposed Standard
  Nov 2010 - Submit BGP Support for Four-octet AS Number Space 
(revised version) to

 IESG as a Proposed Standard
  Nov 2010 - Submit Revisions to the BGP 'Minimum Route 
Advertisement Interval' to

 IESG as a Proposed Standard
  Nov 2010 - Submit Advertisement of Multiple Paths in BGP to IESG 
as a Proposed Standard
  Nov 2010 - Submit BGP Link Bandwidth Extended Community to IESG as 
a Proposed Standard

  Nov 2010 - Submit Advertisement of the best external route in BGP to IESG as
  a Proposed Standard
  Dec 2010 - Submit Multisession BGP to IESG as a Proposed Standard
  Dec 2010 - Submit Error Handling for Optional Transitive BGP Attributes to
 IESG as a Proposed Standard
  Dec 2010 - Submit ASpath ORF to IESG as a Proposed Standard
  Dec 2010 - Revise WG charter

pim:
  Aug 2011 - First WG version of udated RFC 4601
  Aug 2011 - Submit a more reliable PIM solution (refresh reduction) 
to the IESG

  Nov 2011 - Submit a population count extension to PIM to the IESG
  Dec 2011 - Submit update of RFC 4601 to IESG for advancement to 
Draft Standard


pkix:
  Sep 2007 - Progression of CRMF, CMP, and CMP Transport to DRAFT Standard
  Sep 2007 - Progression of Qualified Certificates Profile RFC to 
DRAFT Standard

  Sep 2007 - Progression of Certificate & CRL Profile RFC to DRAFT Standard
  Sep 2007 - Progression of Time Stamp Protocols RFC to DRAFT Standard
  Sep 2007 - Progression of Logotype RFC to DRAFT Standard
  Nov 2007 - Progression of Proxy Certificate RFC to DRAFT Standard
  Nov 2007 - Progression of Attribute Certificate Profile RFC to 
DRAFT standard

  Feb 2008 - Update to CMC approved as PROPOSED Standard
  Mar 2008 - ECC Algorithms approved as PROPOSED Standard RFC
  Mar 2008 - Progression of CMC RFCs to DRAFT Standard
  Mar 2008 - SCVP approved as PROPOSED Standard RFC

ippm:
  Nov 2010 - Final version of process draft
  Nov 2010 - Implementation report based on process draft
  Mar 2011 - Revise charter

ppsp:
  Dec 2010 - Submit problem statement to IESG as Informational
  Apr 2011 - Submit architectural survey to IESG as Informational
  Apr 2011 - Submit requirements document to IESG as Informational
  Aug 2011 - Submit PPSP peer protocol to IESG as Proposed Standard
  Aug 2011 - Submit PPSP tracker protocol to IESG as Proposed Standard
  Dec 2011 - Submit usage guide to IESG to IESG as Informational

I did not verify whether the drafts mentioned about left the working 
group or not.  The IESG would be rejecting a lot of documents if it 
looked into what was authorized by the charter.


At 08:33 15-05-2013, Yoav Nir wrote:
Why? There's definitely a process failure there, and it should be 
blamed on the WG chairs and/or the AD, who should have either moved 
the work out of the working group or worked on updating the charter.


There would be a lot of WG chairs and/or ADs to blame as there are 
dates up to five years in the past in the charter extracts mentioned above.


At 07:25 15-05-2013, Joe Touch wrote:
Well, there are IESG members who stand their ground even when it's 
incorrect, such as:


- claiming that determining WG consensus is up to the AD,
then repeatedly demanding evidence of that consensus


If there was WG consensus it shouldn't be much of a problem to 
provide evidence.  I read a write-up 

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/15/2013 10:15 AM, Ted Lemon wrote:

On May 15, 2013, at 12:36 PM, Joe Touch  wrote:

I'm impressed that you have such a specific interpretation that this
criteria refers to the entire document, even when it talks about the
"feature of a protocol".


"The motivation for a feature of a protocol is not clear enough."
What's ambiguous or subject to interpretation about that? The commentary
exactly echoes what I said. This does not mean that all lacks of clarity
are not DISCUSS criteria: only that a lack of clarity with respect to
motivation is not.


And yet that is the precise issue of your pending DISCUSS on my current 
document.


You don't agree that the motivation for the difference between using 
16-bit vs. 32-bit ExIDs is sufficient, even though that is already 
discussed in the document.


Joe

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 15, 2013, at 12:36 PM, Joe Touch  wrote:
> I'm impressed that you have such a specific interpretation that this
> criteria refers to the entire document, even when it talks about the
> "feature of a protocol".

"The motivation for a feature of a protocol is not clear enough."   What's 
ambiguous or subject to interpretation about that?   The commentary exactly 
echoes what I said.   This does not mean that all lacks of clarity are not 
DISCUSS criteria: only that a lack of clarity with respect to motivation is not.

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/15/2013 7:55 AM, Ted Lemon wrote:

On May 15, 2013, at 10:41 AM, Keith Moore  wrote:

The motivation for a particular feature of a protocol is not
clearenough. At the IESG review stage, protocols should not be blocked
because they provide capabilities beyond what seems necessary to acquit
their responsibilities.


I strongly disagree with what the NON-DISCUSS criteria say.
DISCUSS isn't just for blocking documents. And document quality is
as important (in the sense that poor document quality can lead to
as many interoperability or other problems) as technical
correctness.


The interpretation of this particular NON-DISCUSS criterion that Joe
has given is simply wrong. The key word to pay attention to to see the
error is "motivation." The point of this criterion is to eliminate a
very specific sort of stall that has been known to happen in the past:
the stall where the AD doesn't understand why the document is being put
forward at all, and therefore blocks the document until the authors
explain the motivation behind the document to the satisfaction of the AD
who is holding the DISCUSS.


I'm impressed that you have such a specific interpretation that this
criteria refers to the entire document, even when it talks about the
"feature of a protocol".

So now we're supposed to accept your interpretation of this rather than 
the explicit text?



This is a real issue that has created real problems in the past, and
that is why it is in the NON-DISCUSS criteria.   But this criterion
_does not_ mean that a criticism that the document itself is unclear
is not a valid reason to hold a DISCUSS on it.


Agreed - this does not refer to the document. It refers to a specific 
feature of a protocol.



In fact, it's an
excellent reason to hold a DISCUSS on it.   A lack of clarity in a
document can result in it being implemented incorrectly, or in the
case of a BCP, interpreted incorrectly.


I agree that THESE are good reasons for a DISCUSS, but simply not being 
clear is NOT.



Or in extreme cases, not
read at all.


There's nothing you can do about that; RFCs are not read all the time.

Joe

Re: Gather Profiles/Resumes [was Re: call for ideas: tail-heavy IETF process]

[Thomas Narten]

I don't think the IETF needs to be in the profile/resume
business. There are plenty of other places that do a fine job already.

What I do think the IETF should do is *require* that participants
identify themselves. That means knowing who they are (a name and email
contact) and an affiliation. For 80% of the participants, this info is
not very hard to figure out (see below). But we also have participants
that use obscure email handles that don't correlate to anything
obvious, whether a real person or to a name in the list of registered
attendees, etc. I suspect these folk are *not* intenending to be
anonymous participants, but in practice they are.

And yes, knowing who someone is, their background and who they work
for is important to me in figuring out how to guage their input. E.g.,
I would likely pay more attention to an operator's comments on a
proposed use case than from someone else.

How to figure out who someone is:

1) look at the list of registered attendees. (but that doesn't include
email addresses, so no clean way to map attendee name into email
addresses being used).

Also, for some reason, some people who register don't bother giving an
affiliation. In some cases this is intentional, but there are others
where it doesn't make sense (e.g., someone who has worked for the same
employer for 10+ years and is still working for that employer).

E.g., if you look at the registration list for Orland0, fully 180 names
don't list affiliations -- and there are a number pretty obvious
surprises in that list...

2) look at email addresses. But nowadays they are often generic (e.g.,
gmail) and don't correlate back to an obvious sponsor.

3) Google names, look at authorship info in RFCs, linked in,
etc. Works in a lot of cases, but is sometimes more work than seems
appropriate. And for those with less history in the IETF, knowing
where to look for this stuff is trickier.

But even doing the above, there are people participating (e.g.,
posting on the IETF list) who I don't know who they are, even after
spending some time trying to figure who who they are and what their
background is. For an open standards organization, that somehow
doesn't seem quite right.

Thomas

Re: call for ideas: tail-heavy IETF process

[Thomas Narten]

+1 to what Jari says below.

>From my perspective, the important things to keep in mind:

1) Discuss criteria should be principles, not rigid rules. The details
of the issue at hand always matter, and it will sometimes come down to
judgement calls where reasonable individuals just might disagree. And
if we try to replace judgement with rigid, overly specified rules, we
surely won't like the results. i.e, the process requires we do X, when
X is really a bad outcome overall.

2) There will always be judgement calls, where an AD and an author (or
WG) don't necessarily agree. Fine tuning the rules won't fix that. By
default, when there is disagreement, the presumption needs to be the
AD could be right and needs to be convinced. That is the essence of
what a DISCUSS is intended to be. If the dialog between author/WG/AD
is not progressing, bring in someone else. I.e., the way to overrule a
"stubborn" AD is to convince other ADs (and/or WG members) to weigh in
based on the merits. If an author/WG can't do that, that says
something...

Thomas

Jari Arkko  writes:

> I feel that the discussion is stuck on the different perceptions on
> whether an AD's actions are either blocking reasonable progress, or
> an essential correction to a mistake that went undetected.

> I'd like to make a couple of observations. First of all, we at the
> IESG process 10-25 documents every second week, and several points
> (comment or discuss) are raised for each one. Given that both
> working groups and the IESG consist of humans, I'm guessing that
> none of us are making perfect decisions all the time. While the
> Discuss Criteria document is very useful, I'd warn against trying to
> codify the proper behaviour too much. It would be very difficult,
> and ultimately it comes down to making a call on whether an issue is
> really crucial for the Internet or not.

> I feel that the discussion and pressure from other ADs and authors
> and the rest of the working group is a more useful avenue to ensure
> that we really are fixing the necessary bugs and only those. And I
> know you are doing that - lets make sure we keep doing it. "If you
> see something, say something". It is everyone's responsibility to
> try to do the right thing, and if you feel it is not happening, say
> so. I think the current IESG has a very good mood for receiving
> feedback and acting on it. (Speaking as the longest serving member
> in the current IESG, who frequently gets shown to be wrong by other
> ADs or WG folk.)

> In addition, as discussed separately, moving more of the issue
> resolution to the open and to the WG is a good thing.

> Jari

Re: call for ideas: tail-heavy IETF process

[Yoav Nir]

On May 15, 2013, at 6:06 PM, Keith Moore 
 wrote:

> IMO, IESG should have grounds to reject any document that isn't specifically 
> authorized in a WG's charter.
> 
> - Keith
> 

Why? There's definitely a process failure there, and it should be blamed on the 
WG chairs and/or the AD, who should have either moved the work out of the 
working group or worked on updating the charter.

But the IESG handles documents from both working groups and from individuals, 
and individuals don't have charters.  Why should an otherwise good document be 
rejected after it has gone through IETF last call and has been brought to the 
IESG by one of their own?

Yoav

Re: call for ideas: tail-heavy IETF process

[Andy Bierman]

Hi,

The evidence seems to show that the IESG is getting faster
at their job and WGs are getting slower at theirs.  I don't
see any need for "DISCUSS Rules".  All the AD reviews I've
experienced have improved the document, sometimes a lot.
All DISCUSS issues got cleared with reasonable (even good)
solutions.

IMO, there is no question that applying the right expertise at the
appropriate time in the WG draft process would improve the
entire system and avoid surprises during I* Last Call.
The question is the best way to do that.

Andy


On Tue, May 14, 2013 at 4:57 PM, Joel M. Halpern wrote:

> And your bottom line is exactly what te rules say, what I said, what Ted
> said, and what Joe agreed is reasonable.  It also matchesthe practice I
> have seen.  Even the discuss that I had a lot of arguments with did include
> proposals for paths forward.  Sometimes they were ard to understand.
>  That's probably inevitable with all these opinionated humans doing this.
>
> Yours,
> Joel
>
> On 5/14/2013 7:15 PM, Dave Crocker wrote:
>
>> On 5/14/2013 3:46 PM, Andrew Sullivan wrote:
>>
>>  To be fair, for what it's worth as a WG chair I've had the latter
>>> experience at least as often as the former in the use of DISCUSS, and
>>> I've observed some DISCUSSes cleared without any change at all to the
>>> document in question.
>>>
>>
>> We suffer a continuing logic error in the IETF.  We use "sometimes it
>> happens the other way" as if that negates the existence and problem
>> cause by what is being criticized.
>>
>> So, yeah, of course a Discuss /sometimes/ causes a small delay with no
>> changes.  /Sometimes/ ADs use the sledgehammer of the Discuss to ask for
>> a bit of conversation.  That's all irrelevant.
>>
>> What's relevant is the nature of the mechanisms, its capability, and the
>> cost it can and does impose on authors and the working group.
>>
>> When a serious defect is identified, it's entirely worth the cost.
>>
>> When it isn't, it isn't.
>>
>> In all cases, the person imposing the cost has an obligation to
>> facilitate closing it, including making clear the criteria for closing
>> it.  It is unreasonable to have those who must do the work to clear it
>> play a guessing game.
>>
>>

Re: call for ideas: tail-heavy IETF process

[Jari Arkko]

Joe,

> Broken, agreed.

Yep.

> Unclear, nope - please review the NON-DISCUSS criteria, notably:
> 
> The motivation for a particular feature of a protocol is not clear enough. At 
> the IESG review stage, protocols should not be blocked because they provide 
> capabilities beyond what seems necessary to acquit their responsibilities.
> 
> The DISCUSS isn't there to make documents "better" - that's for COMMENTs. A 
> DISCUSS there to catch a set of problems and to *block* the document's 
> progress until that problem is resolved.


Yes, but note that there are multiple aspects of "unclear". You cite above the 
motivation aspect. There's also a DISCUSS criteria for other forms of unclear, 
e.g., if I can't figure out what I should do in the implementation, it would be 
an issue. The criteria document confirms:

"The specification is impossible to implement due to technical or clarity 
issues."


> Sure, but note that there is a specific NON-DISCUSS criteria on this point:
> 
> Disagreement with informed WG decisions that do not exhibit problems outlined 
> in Section 3.1 (DISCUSS Criteria). In other words, disagreement in 
> preferences among technically sound approaches.
> 
> Finding technical mistakes is good, but imposing the IESG's preferred 
> technical solution over the WG's preference is inappropriate, but happens.

If you are hit with a Discusss that is about preferring another technical 
solution, you should push back.

Jari

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

IMO, IESG should have grounds to reject any document that isn't specifically 
authorized in a WG's charter.

- Keith

On May 15, 2013, at 10:55 AM, Ted Lemon  wrote:

> On May 15, 2013, at 10:41 AM, Keith Moore  wrote:
>>> The motivation for a particular feature of a protocol is not clear enough. 
>>> At the IESG review stage, protocols should not be blocked because they 
>>> provide capabilities beyond what seems necessary to acquit their 
>>> responsibilities.
>> 
>> I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS isn't 
>> just for blocking documents.   And document quality is as important (in the 
>> sense that poor document quality can lead to as many interoperability or 
>> other problems) as technical correctness.
> 
> The interpretation of this particular NON-DISCUSS criterion that Joe has 
> given is simply wrong.   The key word to pay attention to to see the error is 
> "motivation."   The point of this criterion is to eliminate a very specific 
> sort of stall that has been known to happen in the past: the stall where the 
> AD doesn't understand why the document is being put forward at all, and 
> therefore blocks the document until the authors explain the motivation behind 
> the document to the satisfaction of the AD who is holding the DISCUSS.
> 
> This is a real issue that has created real problems in the past, and that is 
> why it is in the NON-DISCUSS criteria.   But this criterion _does not_ mean 
> that a criticism that the document itself is unclear is not a valid reason to 
> hold a DISCUSS on it.   In fact, it's an excellent reason to hold a DISCUSS 
> on it.   A lack of clarity in a document can result in it being implemented 
> incorrectly, or in the case of a BCP, interpreted incorrectly.   Or in 
> extreme cases, not read at all.   This is a bad outcome, worth spending time 
> on, even if the authors would rather be quit of it.
> 

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 15, 2013, at 10:41 AM, Keith Moore  wrote:
>> The motivation for a particular feature of a protocol is not clear enough. 
>> At the IESG review stage, protocols should not be blocked because they 
>> provide capabilities beyond what seems necessary to acquit their 
>> responsibilities.
> 
> I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS isn't 
> just for blocking documents.   And document quality is as important (in the 
> sense that poor document quality can lead to as many interoperability or 
> other problems) as technical correctness.

The interpretation of this particular NON-DISCUSS criterion that Joe has given 
is simply wrong.   The key word to pay attention to to see the error is 
"motivation."   The point of this criterion is to eliminate a very specific 
sort of stall that has been known to happen in the past: the stall where the AD 
doesn't understand why the document is being put forward at all, and therefore 
blocks the document until the authors explain the motivation behind the 
document to the satisfaction of the AD who is holding the DISCUSS.

This is a real issue that has created real problems in the past, and that is 
why it is in the NON-DISCUSS criteria.   But this criterion _does not_ mean 
that a criticism that the document itself is unclear is not a valid reason to 
hold a DISCUSS on it.   In fact, it's an excellent reason to hold a DISCUSS on 
it.   A lack of clarity in a document can result in it being implemented 
incorrectly, or in the case of a BCP, interpreted incorrectly.   Or in extreme 
cases, not read at all.   This is a bad outcome, worth spending time on, even 
if the authors would rather be quit of it.

Re: call for ideas: tail-heavy IETF process

[Ralph Droms]

On May 15, 2013, at 10:39 AM 5/15/13, Joe Touch  wrote:

> 
> 
> On 5/14/2013 9:54 PM, Keith Moore wrote:
>> Publishing broken or unclear documents is not progress.
>> 
>> Keith
> 
> Broken, agreed.
> 
> Unclear, nope - please review the NON-DISCUSS criteria, notably:
> 
> The motivation for a particular feature of a protocol is not clear enough. At 
> the IESG review stage, protocols should not be blocked because they provide 
> capabilities beyond what seems necessary to acquit their responsibilities.
> 
> The DISCUSS isn't there to make documents "better" - that's for COMMENTs. A 
> DISCUSS there to catch a set of problems and to *block* the document's 
> progress until that problem is resolved.

I'll agree with you *if* you consider an unclear description of a feature of a 
protocol, severe enough that reader of the specification are not able to build 
interoperable implementations, as a problem for which a DISCUSS can be posted.

In my opinion, there are many ways in which document can be unclear beyond the 
"motivation for a particular feature of a protocol is not clear enough."  

- Ralph

> 
> Joe

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 10:05 PM, Keith Moore wrote:
...

For that matter, working groups are too often echo chambers where a set
of people manage to isolate themselves from input from those whose work
they might otherwise effect.Some people seem to think that working
group output should be sacrosanct.   There's absolutely no reason to
believe that.  WGs often make technical mistakes that are uncovered by
external review.   Even when no such mistakes are encountered, WG output
rarely represents rough consensus of all interested parties, and WGs
often fail to do due diligence in considering the interests of the broad
spectrum of those potentially affected by their work.

Of course IESG isn't infallible either, and shouldn't behave as if it
is.  But review by experts from outside of the WG generally seems to
improve the IETF's output.


Sure, but note that there is a specific NON-DISCUSS criteria on this point:

Disagreement with informed WG decisions that do not exhibit problems 
outlined in Section 3.1 (DISCUSS Criteria). In other words, disagreement 
in preferences among technically sound approaches.


Finding technical mistakes is good, but imposing the IESG's preferred 
technical solution over the WG's preference is inappropriate, but happens.



As important as the DISCUSS criteria are, there are NON-DISCUSS
criteria that ought to be more carefully followed - including the
point that disagreements with the WG or clarifications are not
justification for DISCUSS.


Strongly disagree.  First, DISCUSS only means that there's something the
AD wants to discuss.


As noted in another post, it means "hold this doc until the DISCUSS is 
resolved". Discussions can happen as a result of COMMENTs in any IESG 
position.



 In particular, disagreements with the WG about
technical quality are always appropriate for IESG to raise.


Yes.


same is
true of requests for clarification.


Sometimes - there's a specific NON-CRITERIA about clarification, however:

The motivation for a particular feature of a protocol is not clear 
enough. At the IESG review stage, protocols should not be blocked 
because they provide capabilities beyond what seems necessary to acquit 
their responsibilities.



Ted pointed out that DISCUSS doesn't mean the IESG doesn't like a
document - agreed, but it *does* hold up a document until the IESG
member clears it.


But there are also procedures within IESG to ignore a single DISCUSS
vote.   So ultimately it takes multiple DISCUSS votes to hold up a
document indefinitely.


Those procedures take time, so even a single DISCUSS holds things up.


DISCUSS is a heavyweight mechanism that holds up document resolution;
it should be used only where absolutely appropriate. If the IESG wants
to have a "discussion" with the authors, they are welcome to
participate in the WGs or IETF LC, or to contact them out of band.


DISCUSS is not supposed to be a heavyweight mechanism, and actually it's
hard to imagine a lighter weight mechanism that gives IESG review any
weight.


Oh, I'm not suggesting removing the DISCUSS mechanism.

First, it ought to have a new name - one that makes clear that this is 
heavyweight. Perhaps "HOLD FOR REVISION", which is the net effect it 
already has.


The key bug is that the IESG can't easily issue a question without 
casting a ballot position, but that may also be a feature. It might be 
useful to be able to issue a QUESTION withouth changing a ballot 
position, though.


> Informal communication doesn't generally work well in practice

because it lacks transparency, and it can cause additional delay without
resolving the problem.   Voting DISCUSS puts pressure on BOTH the AD and
the WG to resolve the issue.


Which is appropriate only when the IESG should have the right to force 
that resolution, and when the path to that resolution is sufficiently 
clear. If there is no path, then vote NO. If forcing resolution and 
putting pressure isn't warranted, issue a COMMENT on any other ballot 
position.


Joe

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/15/2013 10:39 AM, Joe Touch wrote:



On 5/14/2013 9:54 PM, Keith Moore wrote:

Publishing broken or unclear documents is not progress.

Keith


Broken, agreed.

Unclear, nope - please review the NON-DISCUSS criteria, notably:

The motivation for a particular feature of a protocol is not clear 
enough. At the IESG review stage, protocols should not be blocked 
because they provide capabilities beyond what seems necessary to 
acquit their responsibilities.


The DISCUSS isn't there to make documents "better" - that's for 
COMMENTs. A DISCUSS there to catch a set of problems and to *block* 
the document's progress until that problem is resolved.


I strongly disagree with what the NON-DISCUSS criteria say. DISCUSS 
isn't just for blocking documents.   And document quality is as 
important (in the sense that poor document quality can lead to as many 
interoperability or other problems) as technical correctness.


Why are people trying to sabotage IESG?

Keith

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 9:54 PM, Keith Moore wrote:

Publishing broken or unclear documents is not progress.

Keith


Broken, agreed.

Unclear, nope - please review the NON-DISCUSS criteria, notably:

The motivation for a particular feature of a protocol is not clear 
enough. At the IESG review stage, protocols should not be blocked 
because they provide capabilities beyond what seems necessary to acquit 
their responsibilities.


The DISCUSS isn't there to make documents "better" - that's for 
COMMENTs. A DISCUSS there to catch a set of problems and to *block* the 
document's progress until that problem is resolved.


Joe

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 4:03 PM, Ted Lemon wrote:

The whole point of a DISCUSS is to have a discussion.


The *whole* point of a DISCUSS is to hold a document in IETF review 
until a discussion is *resolved*.


There are thus three parts:
- having a discussion
- pausing the document
- waiting for resolution

Nothing limits the IESG to having a "discussion" by issuing a DISCUSS; 
they can issue COMMENTs in any position and simply ask for a dialogue.


Because DISCUSS includes these other properties, it influences authors 
to make changes simply to make a DISCUSS go away, due to pressure by the 
IESG or their own ADs.


That's why they need to be used only where that pressure is appropriate 
and not inappropriate; that's the reason there are both DISCUSS and 
NON-DISCUSS criteria, and why it's very important for those in IESG 
review to hold the IESG to that distinction.


Joe

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 4:57 PM, Joel M. Halpern wrote:

And your bottom line is exactly what te rules say, what I said, what Ted
said, and what Joe agreed is reasonable.


Unfortunately, it's not what happens/is happening right now.

Joe

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 5:53 PM, Ted Lemon wrote:

On May 14, 2013, at 8:27 PM, Joe Touch  wrote:

That is what happens exactly because the DISCUSS holds up the
document, and most ADs don't want to burn time stalling their documents
if there's a way around that delay.


It can only happen if an author values getting their document
through  the process more than getting it right, in which case one has to wonder
why they tried to publish the document in the first place. (I assume you
meant "authors," not "ADs" above).


No; there are times when the document authors are pressured by ADs to do 
anything to resolve pending DISCUSSes, rather than stand up to the fact 
that the issue is either incorrect or the DISCUSS is invalid.



The IESG processes documents quite
quickly; I don't think it's valid to say that there is some terrifying
stall in the document process as a result of the IESG, such that an
author needs to chew off their leg to finally get the thing through.


Well, there are IESG members who stand their ground even when it's 
incorrect, such as:


- claiming that determining WG consensus is up to the AD,
then repeatedly demanding evidence of that consensus

- failing to drop a DISCUSS even when it meets their own
criteria

Those hold up a document too, as you should know (these are examples 
from your review of my doc). As does demanding a document revision while 
there remain open ballot positions, as was done today - on this 
document, to address your pending DISCUSS.


Joe

Re: call for ideas: tail-heavy IETF process

[Jari Arkko]

I feel that the discussion is stuck on the different perceptions on whether an 
AD's actions are either blocking reasonable progress, or an essential 
correction to a mistake that went undetected.

I'd like to make a couple of observations. First of all, we at the IESG process 
10-25 documents every second week, and several points (comment or discuss) are 
raised for each one. Given that both working groups and the IESG consist of 
humans, I'm guessing that none of us are making perfect decisions all the time. 
While the Discuss Criteria document is very useful, I'd warn against trying to 
codify the proper behaviour too much. It would be very difficult, and 
ultimately it comes down to making a call on whether an issue is really crucial 
for the Internet or not.

I feel that the discussion and pressure from other ADs and authors and the rest 
of the working group is a more useful avenue to ensure that we really are 
fixing the necessary bugs and only those. And I know you are doing that - lets 
make sure we keep doing it. "If you see something, say something". It is 
everyone's responsibility to try to do the right thing, and if you feel it is 
not happening, say so. I think the current IESG has a very good mood for 
receiving feedback and acting on it. (Speaking as the longest serving member in 
the current IESG, who frequently gets shown to be wrong by other ADs or WG 
folk.) 

In addition, as discussed separately, moving more of the issue resolution to 
the open and to the WG is a good thing.

Jari

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/14/2013 04:45 PM, Joe Touch wrote:

Brian, et al.,

On 5/14/2013 1:10 PM, Brian E Carpenter wrote:

I think this exchange between Cullen and Ted says it all, except
for one tweak: the IESG is allowed, even encouraged, to apply common
sense when considering the DISCUSS criteria. They are guidance,
not rules.

Also, everybody needs to take the word "discuss" literally. An
entirely possible outcome is that after discussion, the AD says
"Oh. You're correct. Pretend I never spoke!". Or the author says
"Oh. You're correct. We'll change it." Either outcome is good.


The trouble with this assumption is the IESG review process.

COMMENTS are appropriate for feedback, but the IESG review process is 
too often considered an *opportunity* for the IESG to "make the 
document better", or (in some case) have an opportunity for their input.


For that matter, working groups are too often echo chambers where a set 
of people manage to isolate themselves from input from those whose work 
they might otherwise effect.Some people seem to think that working 
group output should be sacrosanct.   There's absolutely no reason to 
believe that.  WGs often make technical mistakes that are uncovered by 
external review.   Even when no such mistakes are encountered, WG output 
rarely represents rough consensus of all interested parties, and WGs 
often fail to do due diligence in considering the interests of the broad 
spectrum of those potentially affected by their work.


Of course IESG isn't infallible either, and shouldn't behave as if it 
is.  But review by experts from outside of the WG generally seems to 
improve the IETF's output.


As important as the DISCUSS criteria are, there are NON-DISCUSS 
criteria that ought to be more carefully followed - including the 
point that disagreements with the WG or clarifications are not 
justification for DISCUSS.


Strongly disagree.  First, DISCUSS only means that there's something the 
AD wants to discuss.In particular, disagreements with the WG about 
technical quality are always appropriate for IESG to raise. The same is 
true of requests for clarification.


Ted pointed out that DISCUSS doesn't mean the IESG doesn't like a 
document - agreed, but it *does* hold up a document until the IESG 
member clears it.


But there are also procedures within IESG to ignore a single DISCUSS 
vote.   So ultimately it takes multiple DISCUSS votes to hold up a 
document indefinitely.




DISCUSS is a heavyweight mechanism that holds up document resolution; 
it should be used only where absolutely appropriate. If the IESG wants 
to have a "discussion" with the authors, they are welcome to 
participate in the WGs or IETF LC, or to contact them out of band.


DISCUSS is not supposed to be a heavyweight mechanism, and actually it's 
hard to imagine a lighter weight mechanism that gives IESG review any 
weight.   Informal communication doesn't generally work well in practice 
because it lacks transparency, and it can cause additional delay without 
resolving the problem.   Voting DISCUSS puts pressure on BOTH the AD and 
the WG to resolve the issue.


Keith

Re: call for ideas: tail-heavy IETF process

[Keith Moore]

On 05/14/2013 06:30 PM, Dave Crocker wrote:

On 5/14/2013 3:12 PM, Ted Lemon wrote:

On May 14, 2013, at 6:00 PM, "Joel M. Halpern" 
wrote:

At the same time, discussions do have to be resolvable.  If there
is no way to address it, then it is not a discuss.  But "required
to clar" is the wrong picture as far as I can tell.


Exactly right.   It would actually be pretty presumptuous for an AD
to say what is required to clear the DISCUSS.   That would tend to
imply that the DISCUSS is a directive, not an invitation to, well,
discuss.


It isn't an 'invitation'.  It's an exercise in political authority by
blocking progress of the document.


It's a mistake to inherently define "progress" in terms of advancing the 
document.   Many documents should not be published in the form in which 
they first reach IESG; a few documents that reach IESG should probably 
never be published.



We came up with the term "Discuss" when I was an AD because, at the
time, the IESG had little authority and wanted to encourage a
constructive tone; we didn't want to sound like we were saying 'no'.

And of course, that's still everyone's preference.  But the reality is
that the imposition of the Discuss is an assertion that changes are
being required.


That's neither what Discuss means, or what it should mean.   Though I've 
seen many cases where WGs or authors demanded that ADs tell them what to 
fix.   It's understandable that they should want clarity from the AD, 
and yet fixing the document is not the AD's job.


For reference, that milder uses of Discuss, which is something akin to
"I'd like something clarified" does not require a Discuss.  It requires
a query to the working group and some dialogue.

Disagree.   I've seen a number of cases where informal conversations 
caused more problems and delay than voting DISCUSS.   By the time a 
document has reached final IESG review, it's generally much better to 
have the level of formality and transparency that comes with voting 
DISCUSS.



Of course we have to _try_ to say what we think would

clear the discuss, but I don't think we can go beyond that; it's
virtually impossible for us to have complete information.


That makes no sense.  The AD is the one choosing to block progress.  It
will be the AD who decides to clear the discuss.


I disagree in the strongest possible terms.

If an AD believes that there is something wrong with a document, even 
something that needs clarification, the proper thing for the AD is to 
vote DISCUSS.   This is NOT "choosing to block progress". Publishing 
broken or unclear documents is not progress.


Keith

Re: call for ideas: tail-heavy IETF process

[Sam Hartman]

I'll say that about a year and a half ago I found myself pushing back on 
discusses
that in my opinion clearly were not within the discuss criteria
significantly more than I ever had to do as an AD. My role was as WG
chair/editor.

Interestingly it's been less of an issue in my experience lately.

Definitely not statistically significant data set for any of this.

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 14, 2013, at 8:27 PM, Joe Touch  wrote:
> That is what happens exactly because the DISCUSS holds up the document, and 
> most ADs don't want to burn time stalling their documents if there's a way 
> around that delay.

It can only happen if an author values getting their document through the 
process more than getting it right, in which case one has to wonder why they 
tried to publish the document in the first place.   (I assume you meant 
"authors," not "ADs" above).   The IESG processes documents quite quickly; I 
don't think it's valid to say that there is some terrifying stall in the 
document process as a result of the IESG, such that an author needs to chew off 
their leg to finally get the thing through.

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 4:03 PM, Ted Lemon wrote:

If the authors think that the goal is to "please the AD," something's
wrong.  This would suggest that they will just do what the AD says
without debate, which is exactly the wrong thing.  The whole point of
a DISCUSS is to have a discussion.   Frankly, it's pretty
disrespectful both to the AD and to the working group if the authors
make changes to "please the AD."


That is what happens exactly because the DISCUSS holds up the document, 
and most ADs don't want to burn time stalling their documents if there's 
a way around that delay.


If the suggestion is a suggestion, then change your DISCUSS to some 
other position, and issue the suggestion as a COMMENT. Yes, that's both 
general advice and specific to the case we have pending, Ted.


Joe

Re: call for ideas: tail-heavy IETF process

[Joel M. Halpern]

And your bottom line is exactly what te rules say, what I said, what Ted 
said, and what Joe agreed is reasonable.  It also matchesthe practice I 
have seen.  Even the discuss that I had a lot of arguments with did 
include proposals for paths forward.  Sometimes they were ard to 
understand.  That's probably inevitable with all these opinionated 
humans doing this.


Yours,
Joel

On 5/14/2013 7:15 PM, Dave Crocker wrote:

On 5/14/2013 3:46 PM, Andrew Sullivan wrote:


To be fair, for what it's worth as a WG chair I've had the latter
experience at least as often as the former in the use of DISCUSS, and
I've observed some DISCUSSes cleared without any change at all to the
document in question.


We suffer a continuing logic error in the IETF.  We use "sometimes it
happens the other way" as if that negates the existence and problem
cause by what is being criticized.

So, yeah, of course a Discuss /sometimes/ causes a small delay with no
changes.  /Sometimes/ ADs use the sledgehammer of the Discuss to ask for
a bit of conversation.  That's all irrelevant.

What's relevant is the nature of the mechanisms, its capability, and the
cost it can and does impose on authors and the working group.

When a serious defect is identified, it's entirely worth the cost.

When it isn't, it isn't.

In all cases, the person imposing the cost has an obligation to
facilitate closing it, including making clear the criteria for closing
it.  It is unreasonable to have those who must do the work to clear it
play a guessing game.

Re: call for ideas: tail-heavy IETF process

[Dave Crocker]

On 5/14/2013 3:46 PM, Andrew Sullivan wrote:


To be fair, for what it's worth as a WG chair I've had the latter
experience at least as often as the former in the use of DISCUSS, and
I've observed some DISCUSSes cleared without any change at all to the
document in question.


We suffer a continuing logic error in the IETF.  We use "sometimes it
happens the other way" as if that negates the existence and problem
cause by what is being criticized.

So, yeah, of course a Discuss /sometimes/ causes a small delay with no
changes.  /Sometimes/ ADs use the sledgehammer of the Discuss to ask for
a bit of conversation.  That's all irrelevant.

What's relevant is the nature of the mechanisms, its capability, and the
cost it can and does impose on authors and the working group.

When a serious defect is identified, it's entirely worth the cost.

When it isn't, it isn't.

In all cases, the person imposing the cost has an obligation to
facilitate closing it, including making clear the criteria for closing
it.  It is unreasonable to have those who must do the work to clear it
play a guessing game.




On 5/14/2013 4:03 PM, Ted Lemon wrote:> On May 14, 2013, at 6:30 PM,
Dave Crocker  wrote:

And of course, that's still everyone's preference.  But the
reality is that the imposition of the Discuss is an assertion
that changes are being required.


No, it absolutely is not.   That may have been the theory when you
were AD, but I can tell you from personal experience dealing with
DISCUSSes on drafts for which I am the responsible AD that that is
not the theory now.


It's the practice now.  It's also often/typically the intent now.

 "'DISCUSS' is a blocking position; the document cannot proceed
until any issues are resolved to the satisfaction of the Area Director
who issued the DISCUSS."

This, of course, sounds very conversational (except for the blocking
part.)  And sometimes that's how it is used.  Sledgehammer to demand
polite conversation.  Talk with me, or else.

But look over the Discuss Criteria (Section 3.1) and at least half of
the bullets pertain to core technical or procedural failings.  These are
not matters of misunderstanding, resolved with some polite
conversations.  They require change.


d/
--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 14, 2013, at 6:30 PM, Dave Crocker  wrote:
> And of course, that's still everyone's preference.  But the reality is
> that the imposition of the Discuss is an assertion that changes are
> being required.

No, it absolutely is not.   That may have been the theory when you were AD, but 
I can tell you from personal experience dealing with DISCUSSes on drafts for 
which I am the responsible AD that that is not the theory now.

> For reference, that milder uses of Discuss, which is something akin to
> "I'd like something clarified" does not require a Discuss.  It requires
> a query to the working group and some dialogue.

One of the DISCUSS criteria requires that you not keep heaping on DISCUSSes, so 
it seems to me that escalating a No Objection to a DISCUSS would be worse, but 
perhaps you disagree?

> That makes no sense.  The AD is the one choosing to block progress.  It
> will be the AD who decides to clear the discuss.

I don't agree with your characterization here, but certainly it's true that the 
AD has to clear the DISCUSS.

> How can it be reasonable for the AD to provide no basis for knowing what
> it will take to get the AD to do this?

It's not.   But that's not what Joe said.   What Joe said was that the AD 
should say _precisely_ what will clear the DISCUSS, and that is a very 
difficult thing to do with incomplete information.

> I suspect you are confusing 'what' with 'how'.  The issue is not one of
> imposing the details of the engineering or documentation changes that
> are being demanded by the AD, but of the criteria that will be applied
> in evaluated the adequacy of the changes.  Othrwise, the authors have to
> play a guessing game, trying to figure out what will please the AD well
> enough to clear the Discuss.

If the authors think that the goal is to "please the AD," something's wrong.  
This would suggest that they will just do what the AD says without debate, 
which is exactly the wrong thing.  The whole point of a DISCUSS is to have a 
discussion.   Frankly, it's pretty disrespectful both to the AD and to the 
working group if the authors make changes to "please the AD."

Re: call for ideas: tail-heavy IETF process

[Andrew Sullivan]

On Tue, May 14, 2013 at 03:30:52PM -0700, Dave Crocker wrote:
> 
> And of course, that's still everyone's preference.  But the reality is
> that the imposition of the Discuss is an assertion that changes are
> being required.
> 
> For reference, that milder uses of Discuss, which is something akin to
> "I'd like something clarified" does not require a Discuss.  It requires
> a query to the working group and some dialogue.

To be fair, for what it's worth as a WG chair I've had the latter
experience at least as often as the former in the use of DISCUSS, and
I've observed some DISCUSSes cleared without any change at all to the
document in question.

A

-- 
Andrew Sullivan
a...@anvilwalrusden.com

Re: call for ideas: tail-heavy IETF process

[Dave Crocker]

On 5/14/2013 3:12 PM, Ted Lemon wrote:

On May 14, 2013, at 6:00 PM, "Joel M. Halpern" 
wrote:

At the same time, discussions do have to be resolvable.  If there
is no way to address it, then it is not a discuss.  But "required
to clar" is the wrong picture as far as I can tell.


Exactly right.   It would actually be pretty presumptuous for an AD
to say what is required to clear the DISCUSS.   That would tend to
imply that the DISCUSS is a directive, not an invitation to, well,
discuss.


It isn't an 'invitation'.  It's an exercise in political authority by
blocking progress of the document.

We came up with the term "Discuss" when I was an AD because, at the
time, the IESG had little authority and wanted to encourage a
constructive tone; we didn't want to sound like we were saying 'no'.

And of course, that's still everyone's preference.  But the reality is
that the imposition of the Discuss is an assertion that changes are
being required.

For reference, that milder uses of Discuss, which is something akin to
"I'd like something clarified" does not require a Discuss.  It requires
a query to the working group and some dialogue.


  Of course we have to _try_ to say what we think would

clear the discuss, but I don't think we can go beyond that; it's
virtually impossible for us to have complete information.


That makes no sense.  The AD is the one choosing to block progress.  It
will be the AD who decides to clear the discuss.

How can it be reasonable for the AD to provide no basis for knowing what
it will take to get the AD to do this?

I suspect you are confusing 'what' with 'how'.  The issue is not one of
imposing the details of the engineering or documentation changes that
are being demanded by the AD, but of the criteria that will be applied
in evaluated the adequacy of the changes.  Othrwise, the authors have to
play a guessing game, trying to figure out what will please the AD well
enough to clear the Discuss.

d/

--
Dave Crocker
Brandenburg InternetWorking
bbiw.net

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 14, 2013, at 6:00 PM, "Joel M. Halpern"  wrote:
> At the same time, discussions do have to be resolvable.  If there is no way 
> to address it, then it is not a discuss.  But "required to clar" is the wrong 
> picture as far as I can tell.

Exactly right.   It would actually be pretty presumptuous for an AD to say what 
is required to clear the DISCUSS.   That would tend to imply that the DISCUSS 
is a directive, not an invitation to, well, discuss.   Of course we have to 
_try_ to say what we think would clear the discuss, but I don't think we can go 
beyond that; it's virtually impossible for us to have complete information.

Re: call for ideas: tail-heavy IETF process

[Joel M. Halpern]

Below:

On 5/14/2013 6:04 PM, Joe Touch wrote:



On 5/14/2013 3:00 PM, Joel M. Halpern wrote:

It seems to me that if it is really a discussion, then there may be many
possible things which could resolve it, and the AD raising the question
may not know exactly what is feasible to clear it.  Otherwise it is a
demand, not a discussions.  And in my experience while ADs can be pushy
(like the rest of us), they are generally prepared to have discussion.

Thus, I find your second item below to be inappropriate.

At the same time, discussions do have to be resolvable.  If there is no
way to address it, then it is not a discuss.  But "required to clar" is
the wrong picture as far as I can tell.


Point taken - at least some indication on what is expected to be changed
toward a path of resolution.

As you note, otherwise it's not a DISCUSS.

Joe
Thanks Joe.  I am glad we are on the same apage about the kind of 
clarity of description needed in discusses.  We may have different 
experiences, but the vast majority of the discusses I have dealt with 
have been sufficiently clear on the question of how to approach 
resolving them.  Sometimes an actual conversation with the AD has been 
necessary to understand the issue and reach a practical point for 
resolving it, but that is a communication style issue, not a discuss 
substance issue.


Yours,
Joel





Yours,
Joel

On 5/14/2013 5:12 PM, Joe Touch wrote:

I am *not* suggesting getting rid of it.

I *am* suggesting that it needs to be used only where necessary, and
that 'necessary' ought to be clearly proved by:

 - citing the specific DISCUSS criteria involved

 - providing explicit information on what would
 be required to clear the DISCUSS

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 3:00 PM, Joel M. Halpern wrote:

It seems to me that if it is really a discussion, then there may be many
possible things which could resolve it, and the AD raising the question
may not know exactly what is feasible to clear it.  Otherwise it is a
demand, not a discussions.  And in my experience while ADs can be pushy
(like the rest of us), they are generally prepared to have discussion.

Thus, I find your second item below to be inappropriate.

At the same time, discussions do have to be resolvable.  If there is no
way to address it, then it is not a discuss.  But "required to clar" is
the wrong picture as far as I can tell.


Point taken - at least some indication on what is expected to be changed 
toward a path of resolution.


As you note, otherwise it's not a DISCUSS.

Joe



Yours,
Joel

On 5/14/2013 5:12 PM, Joe Touch wrote:

I am *not* suggesting getting rid of it.

I *am* suggesting that it needs to be used only where necessary, and
that 'necessary' ought to be clearly proved by:

 - citing the specific DISCUSS criteria involved

 - providing explicit information on what would
 be required to clear the DISCUSS

Re: call for ideas: tail-heavy IETF process

[Joel M. Halpern]

It seems to me that if it is really a discussion, then there may be many 
possible things which could resolve it, and the AD raising the question 
may not know exactly what is feasible to clear it.  Otherwise it is a 
demand, not a discussions.  And in my experience while ADs can be pushy 
(like the rest of us), they are generally prepared to have discussion.


Thus, I find your second item below to be inappropriate.

At the same time, discussions do have to be resolvable.  If there is no 
way to address it, then it is not a discuss.  But "required to clar" is 
the wrong picture as far as I can tell.


Yours,
Joel

On 5/14/2013 5:12 PM, Joe Touch wrote:

I am *not* suggesting getting rid of it.

I *am* suggesting that it needs to be used only where necessary, and
that 'necessary' ought to be clearly proved by:

 - citing the specific DISCUSS criteria involved

 - providing explicit information on what would
 be required to clear the DISCUSS

Re: call for ideas: tail-heavy IETF process

[Ted Lemon]

On May 14, 2013, at 1:41 PM, Stephen Farrell  wrote:
> I've not found that a real problem. When its happened that we
> did turn up something bigger than we thought after the telechat
> (and updating your discuss points before or during the telechat
> is considered fair game) then I think the authors/chairs have
> always also agreed that stuff needed to be done.

Yup, I'm not convinced that it's a problem in practice.   I just happened to 
notice that it could have become an issue the other day when discussing a 
DISCUSS.

Re: call for ideas: tail-heavy IETF process

[Joe Touch]

On 5/14/2013 1:59 PM, Stephen Farrell wrote:


Joe,

On 05/14/2013 09:45 PM, Joe Touch wrote:

As important as the DISCUSS criteria are, there are NON-DISCUSS criteria
that ought to be more carefully followed - including the point that
disagreements with the WG or clarifications are not justification for
DISCUSS.


I had assumed that the term discuss-criteria meant [1] which includes
both. Not sure if that's also what you meant but worth adding the URL
here just in case some folks aren't familiar with it.

[1] https://www.ietf.org/iesg/statement/discuss-criteria.html


DISCUSS is a heavyweight mechanism that holds up document resolution;


Agreed. But its a keystone in the current process. So getting
rid of it would be fairly revolutionary. (Not that I'm against
a bit of revolving now and then:-)


I am *not* suggesting getting rid of it.

I *am* suggesting that it needs to be used only where necessary, and 
that 'necessary' ought to be clearly proved by:


- citing the specific DISCUSS criteria involved

- providing explicit information on what would
be required to clear the DISCUSS


it
should be used only where absolutely appropriate.


s/absolutely appropriate/appropriate/ would be better.


If the IESG wants to
have a "discussion" with the authors, they are welcome to participate in
the WGs or IETF LC, or to contact them out of band.


With our current tail-heavy process, and ~100 WGs that's impossible
in almost all cases.


The NON-DISCUSS criteria imply that DISCUSS is not an opportunity for 
late input. I appreciate that early input isn't practical, but that does 
NOT provide a rationale for violating the NON-DISCUSS criteria 
(technical disagreement with the WG, etc. - see the list).


The IESG can offer its advice in COMMENTS in other positions. It can 
make suggestions, and let the authors and ADs decide what to do.


But it should NEVER hold a document hostage with a DISCUSS without 
specific merit.


Since some of you asked, here's a *current* example:

--

Ted Lemon issued a DISCUSS on a doc I have pending - here's the link:

http://datatracker.ietf.org/doc/draft-ietf-tcpm-experimental-options/ballot/

Note that his DISCUSS raises the point that 32-bit ExIDs "don't add 
value" and are "pointless" (despite having several paragraphs of 
justification in the doc). That's second-guessing the WG, which is 
listed among the explicit NON-DISCUSS criteria.


There's no clear no collateral damage to the Internet, just damage to 
those who might use a longer TCP option, and there are implementations 
that already use this (as well as other TCP options that are much less 
frugal in their use of option space). It affects only shared use of 
experimental option codepoints, a situation that's intended to be 
transient if an option becomes widely used. The choice of 32-bit ExIDs 
has pros and cons listed in the doc, and both 16-bit and 32-bit ExIDs 
are allowed.


Right now, he's waiting for the doc to be changed to recommend 16-bit 
ExIDs. I don't disagree with that suggestion, but is it really 
appropriate to hold a document hostage using a DISCUSS this way?


I don't think so.

I questioned this process, and he suggested that debating the validity 
of a DISCUSS was out of scope during IESG review. That's not acceptable 
to me, and I hope to others.


Joe

Re: call for ideas: tail-heavy IETF process

[Stephen Farrell]

Joe,

On 05/14/2013 09:45 PM, Joe Touch wrote:
> As important as the DISCUSS criteria are, there are NON-DISCUSS criteria
> that ought to be more carefully followed - including the point that
> disagreements with the WG or clarifications are not justification for
> DISCUSS.

I had assumed that the term discuss-criteria meant [1] which includes
both. Not sure if that's also what you meant but worth adding the URL
here just in case some folks aren't familiar with it.

   [1] https://www.ietf.org/iesg/statement/discuss-criteria.html

> DISCUSS is a heavyweight mechanism that holds up document resolution; 

Agreed. But its a keystone in the current process. So getting
rid of it would be fairly revolutionary. (Not that I'm against
a bit of revolving now and then:-)

> it
> should be used only where absolutely appropriate. 

s/absolutely appropriate/appropriate/ would be better.

> If the IESG wants to
> have a "discussion" with the authors, they are welcome to participate in
> the WGs or IETF LC, or to contact them out of band.

With our current tail-heavy process, and ~100 WGs that's impossible
in almost all cases.

On 05/14/2013 09:46 PM, Joe Touch wrote:>
> On 5/14/2013 10:18 AM, Dave Crocker wrote:
>> And a Discuss should be required to assert which criteria apply and how.
>
> +1

That'd be -1 from me fwiw. There aren't enough disputes about that
to make it worthwhile and as I said the IESG is, believe it or not,
pretty good at policing itself in this respect.

S.