Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Jean-Baptiste Onofré]

Great summary.

And good idea for a meeting (even if only mailing list counts ;)).

Regards
JB



On Aug 10, 2016, 06:09, at 06:09, Frances Perry  wrote:
>So to summarize where I think this thread is at -- we'd like to more
>clearly lay out the expectations for larger proposals.
>- Explain what the design doc / proposal should include (like is done
>in
>https://cwiki.apache.org/confluence/display/KAFKA/
>Kafka+Improvement+Proposals)
>- Clearly track the open proposals (potentially in JIRA with a known
>label
>and incrementing proposal IDs).
>- Set expectations around the timelines for proposals -- both to ensure
>enough feedback is gathered and perhaps inactive proposals are
>archived.
>
>Another suggestion: How about if we try resurrecting the (virtual)
>community meetings? Anything that's a deep model change or potentially
>contentious can be presented there. Often a 15 minute overview of these
>topics can be helpful context when reading the detailed proposal.
>
>On Tue, Aug 9, 2016 at 10:18 AM, Kenneth Knowles
>
>wrote:
>
>> I didn't have a specific rubric, but here are some factors:
>>
>>  - Impact on users
>>  - Impact on other devs (while we are incubating, this is possibly a
>big
>> deal)
>>  - Backwards compatibility (not that important until stable release
>if it
>> is minor)
>>  - Amount of detail needed to understand the proposal
>>  - Whether the proposal needs multiple re-readings to understand
>thoroughly
>>  - Whether the proposal will take a while to implement, or is
>basically a
>> one-PR thing
>>
>> I think any of these is enough to consider a BIP. I'm sure others
>will
>> think of other considerations.
>>
>> All my "no" answers are pretty mild on all categories IMO. Most of
>the
>> "yes" answers are heavy in more than one.
>>
>> So actually I didn't specifically consider whether it was a model
>change,
>> but only the impact on users and backwards compatibility. For your
>example
>> of PipelineResult#waitToFinish, if we had a stable release then I
>would
>> have said "yes" for these reasons.
>>
>> The "maybe" answers were all testing infrastructure, because they
>take a
>> while to complete and have high impact on development processes. But
>now
>> that I write these criteria down, I would change the "maybe" answers
>to
>> "no".
>>
>> Thoughts?
>>
>> Kenn
>>
>> On Tue, Aug 9, 2016 at 1:15 AM, Ismaël Mejía 
>wrote:
>>
>> > Kenn, just to start the discussion, what was your criteria to
>decide what
>> > proposals are worth been a BIP ?
>> >
>> > I can clearly spot the most common case to create a BIP:  Changes
>to the
>> > model / SDK (this covers most of the 'yes' in your list, with the
>> exception
>> > of Pipeline#waitToFinish).
>> >
>> > Do you guys have ideas for other criteria ? (e.g. are new runners
>and
>> DSLs
>> > worth a BIP ?, or do Infrastructure issues deserve a BIP ?).
>> >
>> > Ismael
>> >
>> >
>> > On Mon, Aug 8, 2016 at 10:05 PM, Kenneth Knowles
>> >
>> > wrote:
>> >
>> > > +1 to the overall idea, though I would limit it to large and/or
>> long-term
>> > > proposals.
>> > >
>> > > I like:
>> > >
>> > >  - JIRA for tracking: that's what it does best.
>> > >  - Google Docs for detailed commenting and revision - basically a
>wiki
>> > with
>> > > easier commenting
>> > >  - Beam site page for process description and list of current
>"BIPs",
>> > just
>> > > a one liner and a link to JIRA. A proposal to dev@beam could
>include a
>> > > link
>> > > to a PR against the asf-site to add the BIP. However, I would
>agree
>> with
>> > > the counter-argument that this could just be a JIRA component or
>tag.
>> > > Either one works for me. Or a page with the process that links to
>a
>> JIRA
>> > > saved search. The more formal list mostly just makes it even more
>> > visible,
>> > > right?
>> > >
>> > > I think that the number can be small. Here are examples scraped
>from
>> the
>> > > mailing list archives (in random order) and whether I would use a
>> "BIP":
>> > >
>> > >  - Runner API: yes
>> > >  - Serialization tech: no
>> > >  - Dynamic parameters: yes
>> > >  - Splittable DoFn: yes
>> > >  - Scio: yes
>> > >  - Pipeline#waitToFinish(), etc: no
>> > >  - DoFn setup / teardown: yes
>> > >  - State & Timers: yes
>> > >  - Pipeline job naming changes: no
>> > >  - CoGBK as primitive: yes
>> > >  - New website design: no
>> > >  - new DoFn: yes
>> > >  - Cluster infrastructure for tests: maybe
>> > >  - Beam recipes: no
>> > >  - Two spark runners: no
>> > >  - Nightly builds by Jenkins: maybe
>> > >
>> > > When I write them all down it really is a lot :-)
>> > >
>> > > Of course, the first thing that could be discussed in a
>[PROPOSAL]
>> thread
>> > > would be whether to file a "BIP".
>> > >
>> > > Kenn
>> > >
>> > > On Mon, Aug 8, 2016 at 8:29 AM, Lukasz Cwik
>
>> > > wrote:
>> > >
>> > > > +1 for the cwiki approach that Aljoshca and Ismael gave
>examples of.

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Kenneth Knowles]

I didn't have a specific rubric, but here are some factors:

 - Impact on users
 - Impact on other devs (while we are incubating, this is possibly a big
deal)
 - Backwards compatibility (not that important until stable release if it
is minor)
 - Amount of detail needed to understand the proposal
 - Whether the proposal needs multiple re-readings to understand thoroughly
 - Whether the proposal will take a while to implement, or is basically a
one-PR thing

I think any of these is enough to consider a BIP. I'm sure others will
think of other considerations.

All my "no" answers are pretty mild on all categories IMO. Most of the
"yes" answers are heavy in more than one.

So actually I didn't specifically consider whether it was a model change,
but only the impact on users and backwards compatibility. For your example
of PipelineResult#waitToFinish, if we had a stable release then I would
have said "yes" for these reasons.

The "maybe" answers were all testing infrastructure, because they take a
while to complete and have high impact on development processes. But now
that I write these criteria down, I would change the "maybe" answers to
"no".

Thoughts?

Kenn

On Tue, Aug 9, 2016 at 1:15 AM, Ismaël Mejía  wrote:

> Kenn, just to start the discussion, what was your criteria to decide what
> proposals are worth been a BIP ?
>
> I can clearly spot the most common case to create a BIP:  Changes to the
> model / SDK (this covers most of the 'yes' in your list, with the exception
> of Pipeline#waitToFinish).
>
> Do you guys have ideas for other criteria ? (e.g. are new runners and DSLs
> worth a BIP ?, or do Infrastructure issues deserve a BIP ?).
>
> Ismael
>
>
> On Mon, Aug 8, 2016 at 10:05 PM, Kenneth Knowles 
> wrote:
>
> > +1 to the overall idea, though I would limit it to large and/or long-term
> > proposals.
> >
> > I like:
> >
> >  - JIRA for tracking: that's what it does best.
> >  - Google Docs for detailed commenting and revision - basically a wiki
> with
> > easier commenting
> >  - Beam site page for process description and list of current "BIPs",
> just
> > a one liner and a link to JIRA. A proposal to dev@beam could include a
> > link
> > to a PR against the asf-site to add the BIP. However, I would agree with
> > the counter-argument that this could just be a JIRA component or tag.
> > Either one works for me. Or a page with the process that links to a JIRA
> > saved search. The more formal list mostly just makes it even more
> visible,
> > right?
> >
> > I think that the number can be small. Here are examples scraped from the
> > mailing list archives (in random order) and whether I would use a "BIP":
> >
> >  - Runner API: yes
> >  - Serialization tech: no
> >  - Dynamic parameters: yes
> >  - Splittable DoFn: yes
> >  - Scio: yes
> >  - Pipeline#waitToFinish(), etc: no
> >  - DoFn setup / teardown: yes
> >  - State & Timers: yes
> >  - Pipeline job naming changes: no
> >  - CoGBK as primitive: yes
> >  - New website design: no
> >  - new DoFn: yes
> >  - Cluster infrastructure for tests: maybe
> >  - Beam recipes: no
> >  - Two spark runners: no
> >  - Nightly builds by Jenkins: maybe
> >
> > When I write them all down it really is a lot :-)
> >
> > Of course, the first thing that could be discussed in a [PROPOSAL] thread
> > would be whether to file a "BIP".
> >
> > Kenn
> >
> > On Mon, Aug 8, 2016 at 8:29 AM, Lukasz Cwik 
> > wrote:
> >
> > > +1 for the cwiki approach that Aljoshca and Ismael gave examples of.
> > >
> > > On Mon, Aug 8, 2016 at 2:57 AM, Ismaël Mejía 
> wrote:
> > >
> > > > +1 for a more formal "Improvement Proposals" with ids we can refer
> to:
> > > >
> > > > like Flink does too:
> > > > https://cwiki.apache.org/confluence/display/FLINK/
> > > > Flink+Improvement+Proposals
> > > >
> > > >
> > > > On Mon, Aug 8, 2016 at 10:14 AM, Jean-Baptiste Onofré <
> j...@nanthrax.net
> > >
> > > > wrote:
> > > >
> > > > > Same think at Karaf: https://cwiki.apache.org/
> > > confluence/display/KARAF/
> > > > >
> > > > > Combine with Jira.
> > > > >
> > > > > Regards
> > > > > JB
> > > > >
> > > > >
> > > > > On 08/08/2016 10:03 AM, Aljoscha Krettek wrote:
> > > > >
> > > > >> Please have a look at this:
> > > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+
> > > > >> Improvement+Proposals
> > > > >>
> > > > >> We recently started using this process in Flink and so far are
> quite
> > > > happy
> > > > >> with it.
> > > > >>
> > > > >> On Mon, 8 Aug 2016 at 06:52 Jean-Baptiste Onofré  >
> > > > wrote:
> > > > >>
> > > > >> Good point Ben.
> > > > >>>
> > > > >>> I would say a "discussion" Jira can "evolve" to a implementation
> > > "Jira"
> > > > >>> (just changing the component).
> > > > >>>
> > > > >>> WDYT ?
> > > > >>>
> > > > >>> Regards
> > > > >>> JB
> > > > >>>
> > > > >>> On 08/08/2016 06:50 AM, Ben Chambers wrote:
> > > > >>>
> > > >  Would we use the 

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Ismaël Mejía]

+1 for a more formal "Improvement Proposals" with ids we can refer to:

like Flink does too:
https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals


On Mon, Aug 8, 2016 at 10:14 AM, Jean-Baptiste Onofré 
wrote:

> Same think at Karaf: https://cwiki.apache.org/confluence/display/KARAF/
>
> Combine with Jira.
>
> Regards
> JB
>
>
> On 08/08/2016 10:03 AM, Aljoscha Krettek wrote:
>
>> Please have a look at this:
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+
>> Improvement+Proposals
>>
>> We recently started using this process in Flink and so far are quite happy
>> with it.
>>
>> On Mon, 8 Aug 2016 at 06:52 Jean-Baptiste Onofré  wrote:
>>
>> Good point Ben.
>>>
>>> I would say a "discussion" Jira can "evolve" to a implementation "Jira"
>>> (just changing the component).
>>>
>>> WDYT ?
>>>
>>> Regards
>>> JB
>>>
>>> On 08/08/2016 06:50 AM, Ben Chambers wrote:
>>>
 Would we use the same Jira to track the series of PRs implementing the
 proposal (if accepted) or would it be discussion only (possibly linked
 to
 the implementation tasks)?

 On Sun, Aug 7, 2016, 9:48 PM Frances Perry 

>>> wrote:
>>>

 I'm a huge fan of keeping all the details related to a topic in a
>
 relevant
>>>
 jira issue.
>
> On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
> wrote:
>
> Hi guys,
>>
>> we have now several technical discussions, sent on the mailing list
>>
> with
>>>
 link to document for details.
>>
>> I think it's not easy for people to follow the different discussions,
>>
> and
>>>
 to look for the e-mail containing the document links.
>>
>> Of course, it's required to have the discussion on the mailing list
>>
> (per
>>>
 Apache rules). However, maybe it could be helpful to have a place to
>>
> find
>>>
 open discussions, with the link to the mailing list discussion thread,
>>
> and
>
>> to the detailed document.
>> It could be on the website (but maybe not easy to maintain and
>>
> publish),
>>>
 or on Jira (one Jira per discussion), or a wiki.
>>
>> WDYT ?
>>
>> Regards
>> JB
>> --
>> Jean-Baptiste Onofré
>> jbono...@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>>
>

>>> --
>>> Jean-Baptiste Onofré
>>> jbono...@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>>
>>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Jean-Baptiste Onofré]

Same think at Karaf: https://cwiki.apache.org/confluence/display/KARAF/

Combine with Jira.

Regards
JB

On 08/08/2016 10:03 AM, Aljoscha Krettek wrote:

Please have a look at this:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals

We recently started using this process in Flink and so far are quite happy
with it.

On Mon, 8 Aug 2016 at 06:52 Jean-Baptiste Onofré  wrote:


Good point Ben.

I would say a "discussion" Jira can "evolve" to a implementation "Jira"
(just changing the component).

WDYT ?

Regards
JB

On 08/08/2016 06:50 AM, Ben Chambers wrote:

Would we use the same Jira to track the series of PRs implementing the
proposal (if accepted) or would it be discussion only (possibly linked to
the implementation tasks)?

On Sun, Aug 7, 2016, 9:48 PM Frances Perry 

wrote:



I'm a huge fan of keeping all the details related to a topic in a

relevant

jira issue.

On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
wrote:


Hi guys,

we have now several technical discussions, sent on the mailing list

with

link to document for details.

I think it's not easy for people to follow the different discussions,

and

to look for the e-mail containing the document links.

Of course, it's required to have the discussion on the mailing list

(per

Apache rules). However, maybe it could be helpful to have a place to

find

open discussions, with the link to the mailing list discussion thread,

and

to the detailed document.
It could be on the website (but maybe not easy to maintain and

publish),

or on Jira (one Jira per discussion), or a wiki.

WDYT ?

Regards
JB
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com







--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com





--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Aljoscha Krettek]

Please have a look at this:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals

We recently started using this process in Flink and so far are quite happy
with it.

On Mon, 8 Aug 2016 at 06:52 Jean-Baptiste Onofré  wrote:

> Good point Ben.
>
> I would say a "discussion" Jira can "evolve" to a implementation "Jira"
> (just changing the component).
>
> WDYT ?
>
> Regards
> JB
>
> On 08/08/2016 06:50 AM, Ben Chambers wrote:
> > Would we use the same Jira to track the series of PRs implementing the
> > proposal (if accepted) or would it be discussion only (possibly linked to
> > the implementation tasks)?
> >
> > On Sun, Aug 7, 2016, 9:48 PM Frances Perry 
> wrote:
> >
> >> I'm a huge fan of keeping all the details related to a topic in a
> relevant
> >> jira issue.
> >>
> >> On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
> >> wrote:
> >>
> >>> Hi guys,
> >>>
> >>> we have now several technical discussions, sent on the mailing list
> with
> >>> link to document for details.
> >>>
> >>> I think it's not easy for people to follow the different discussions,
> and
> >>> to look for the e-mail containing the document links.
> >>>
> >>> Of course, it's required to have the discussion on the mailing list
> (per
> >>> Apache rules). However, maybe it could be helpful to have a place to
> find
> >>> open discussions, with the link to the mailing list discussion thread,
> >> and
> >>> to the detailed document.
> >>> It could be on the website (but maybe not easy to maintain and
> publish),
> >>> or on Jira (one Jira per discussion), or a wiki.
> >>>
> >>> WDYT ?
> >>>
> >>> Regards
> >>> JB
> >>> --
> >>> Jean-Baptiste Onofré
> >>> jbono...@apache.org
> >>> http://blog.nanthrax.net
> >>> Talend - http://www.talend.com
> >>>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Jean-Baptiste Onofré]

Good point Ben.

I would say a "discussion" Jira can "evolve" to a implementation "Jira" 
(just changing the component).


WDYT ?

Regards
JB

On 08/08/2016 06:50 AM, Ben Chambers wrote:

Would we use the same Jira to track the series of PRs implementing the
proposal (if accepted) or would it be discussion only (possibly linked to
the implementation tasks)?

On Sun, Aug 7, 2016, 9:48 PM Frances Perry  wrote:


I'm a huge fan of keeping all the details related to a topic in a relevant
jira issue.

On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
wrote:


Hi guys,

we have now several technical discussions, sent on the mailing list with
link to document for details.

I think it's not easy for people to follow the different discussions, and
to look for the e-mail containing the document links.

Of course, it's required to have the discussion on the mailing list (per
Apache rules). However, maybe it could be helpful to have a place to find
open discussions, with the link to the mailing list discussion thread,

and

to the detailed document.
It could be on the website (but maybe not easy to maintain and publish),
or on Jira (one Jira per discussion), or a wiki.

WDYT ?

Regards
JB
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com







--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Jean-Baptiste Onofré]

Me too ;)

What I'm doing in other Apache projects (like Karaf) is a Jira component 
named "discussion" where I describe the discussion and attach the 
related documents.


So, with a quick query, we can find all pending discussion, etc.

My $0.01

Regards
JB

On 08/08/2016 06:47 AM, Frances Perry wrote:

I'm a huge fan of keeping all the details related to a topic in a relevant
jira issue.

On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
wrote:


Hi guys,

we have now several technical discussions, sent on the mailing list with
link to document for details.

I think it's not easy for people to follow the different discussions, and
to look for the e-mail containing the document links.

Of course, it's required to have the discussion on the mailing list (per
Apache rules). However, maybe it could be helpful to have a place to find
open discussions, with the link to the mailing list discussion thread, and
to the detailed document.
It could be on the website (but maybe not easy to maintain and publish),
or on Jira (one Jira per discussion), or a wiki.

WDYT ?

Regards
JB
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com





--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [PROPOSAL] Website page or Jira to host all current proposal discussion and docs

[Ben Chambers]

Would we use the same Jira to track the series of PRs implementing the
proposal (if accepted) or would it be discussion only (possibly linked to
the implementation tasks)?

On Sun, Aug 7, 2016, 9:48 PM Frances Perry  wrote:

> I'm a huge fan of keeping all the details related to a topic in a relevant
> jira issue.
>
> On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré 
> wrote:
>
> > Hi guys,
> >
> > we have now several technical discussions, sent on the mailing list with
> > link to document for details.
> >
> > I think it's not easy for people to follow the different discussions, and
> > to look for the e-mail containing the document links.
> >
> > Of course, it's required to have the discussion on the mailing list (per
> > Apache rules). However, maybe it could be helpful to have a place to find
> > open discussions, with the link to the mailing list discussion thread,
> and
> > to the detailed document.
> > It could be on the website (but maybe not easy to maintain and publish),
> > or on Jira (one Jira per discussion), or a wiki.
> >
> > WDYT ?
> >
> > Regards
> > JB
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>