Filed https://issues.apache.org/jira/browse/BEAM-566 to document and implement these ideas. I'm happy to take it on.
On Wed, Aug 10, 2016 at 12:19 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Great summary. > > And good idea for a meeting (even if only mailing list counts ;)). > > Regards > JB > > > > On Aug 10, 2016, 06:09, at 06:09, Frances Perry <f...@google.com.INVALID> > wrote: > >So to summarize where I think this thread is at -- we'd like to more > >clearly lay out the expectations for larger proposals. > >- Explain what the design doc / proposal should include (like is done > >in > >https://cwiki.apache.org/confluence/display/KAFKA/ > >Kafka+Improvement+Proposals) > >- Clearly track the open proposals (potentially in JIRA with a known > >label > >and incrementing proposal IDs). > >- Set expectations around the timelines for proposals -- both to ensure > >enough feedback is gathered and perhaps inactive proposals are > >archived. > > > >Another suggestion: How about if we try resurrecting the (virtual) > >community meetings? Anything that's a deep model change or potentially > >contentious can be presented there. Often a 15 minute overview of these > >topics can be helpful context when reading the detailed proposal. > > > >On Tue, Aug 9, 2016 at 10:18 AM, Kenneth Knowles > ><k...@google.com.invalid> > >wrote: > > > >> I didn't have a specific rubric, but here are some factors: > >> > >> - Impact on users > >> - Impact on other devs (while we are incubating, this is possibly a > >big > >> deal) > >> - Backwards compatibility (not that important until stable release > >if it > >> is minor) > >> - Amount of detail needed to understand the proposal > >> - Whether the proposal needs multiple re-readings to understand > >thoroughly > >> - Whether the proposal will take a while to implement, or is > >basically a > >> one-PR thing > >> > >> I think any of these is enough to consider a BIP. I'm sure others > >will > >> think of other considerations. > >> > >> All my "no" answers are pretty mild on all categories IMO. Most of > >the > >> "yes" answers are heavy in more than one. > >> > >> So actually I didn't specifically consider whether it was a model > >change, > >> but only the impact on users and backwards compatibility. For your > >example > >> of PipelineResult#waitToFinish, if we had a stable release then I > >would > >> have said "yes" for these reasons. > >> > >> The "maybe" answers were all testing infrastructure, because they > >take a > >> while to complete and have high impact on development processes. But > >now > >> that I write these criteria down, I would change the "maybe" answers > >to > >> "no". > >> > >> Thoughts? > >> > >> Kenn > >> > >> On Tue, Aug 9, 2016 at 1:15 AM, Ismaël Mejía <ieme...@gmail.com> > >wrote: > >> > >> > Kenn, just to start the discussion, what was your criteria to > >decide what > >> > proposals are worth been a BIP ? > >> > > >> > I can clearly spot the most common case to create a BIP: Changes > >to the > >> > model / SDK (this covers most of the 'yes' in your list, with the > >> exception > >> > of Pipeline#waitToFinish). > >> > > >> > Do you guys have ideas for other criteria ? (e.g. are new runners > >and > >> DSLs > >> > worth a BIP ?, or do Infrastructure issues deserve a BIP ?). > >> > > >> > Ismael > >> > > >> > > >> > On Mon, Aug 8, 2016 at 10:05 PM, Kenneth Knowles > ><k...@google.com.invalid > >> > > >> > wrote: > >> > > >> > > +1 to the overall idea, though I would limit it to large and/or > >> long-term > >> > > proposals. > >> > > > >> > > I like: > >> > > > >> > > - JIRA for tracking: that's what it does best. > >> > > - Google Docs for detailed commenting and revision - basically a > >wiki > >> > with > >> > > easier commenting > >> > > - Beam site page for process description and list of current > >"BIPs", > >> > just > >> > > a one liner and a link to JIRA. A proposal to dev@beam could > >include a > >> > > link > >> > > to a PR against the asf-site to add the BIP. However, I would > >agree > >> with > >> > > the counter-argument that this could just be a JIRA component or > >tag. > >> > > Either one works for me. Or a page with the process that links to > >a > >> JIRA > >> > > saved search. The more formal list mostly just makes it even more > >> > visible, > >> > > right? > >> > > > >> > > I think that the number can be small. Here are examples scraped > >from > >> the > >> > > mailing list archives (in random order) and whether I would use a > >> "BIP": > >> > > > >> > > - Runner API: yes > >> > > - Serialization tech: no > >> > > - Dynamic parameters: yes > >> > > - Splittable DoFn: yes > >> > > - Scio: yes > >> > > - Pipeline#waitToFinish(), etc: no > >> > > - DoFn setup / teardown: yes > >> > > - State & Timers: yes > >> > > - Pipeline job naming changes: no > >> > > - CoGBK as primitive: yes > >> > > - New website design: no > >> > > - new DoFn: yes > >> > > - Cluster infrastructure for tests: maybe > >> > > - Beam recipes: no > >> > > - Two spark runners: no > >> > > - Nightly builds by Jenkins: maybe > >> > > > >> > > When I write them all down it really is a lot :-) > >> > > > >> > > Of course, the first thing that could be discussed in a > >[PROPOSAL] > >> thread > >> > > would be whether to file a "BIP". > >> > > > >> > > Kenn > >> > > > >> > > On Mon, Aug 8, 2016 at 8:29 AM, Lukasz Cwik > ><lc...@google.com.invalid> > >> > > wrote: > >> > > > >> > > > +1 for the cwiki approach that Aljoshca and Ismael gave > >examples of. > >> > > > > >> > > > On Mon, Aug 8, 2016 at 2:57 AM, Ismaël Mejía > ><ieme...@gmail.com> > >> > wrote: > >> > > > > >> > > > > +1 for a more formal "Improvement Proposals" with ids we can > >refer > >> > to: > >> > > > > > >> > > > > like Flink does too: > >> > > > > https://cwiki.apache.org/confluence/display/FLINK/ > >> > > > > Flink+Improvement+Proposals > >> > > > > > >> > > > > > >> > > > > On Mon, Aug 8, 2016 at 10:14 AM, Jean-Baptiste Onofré < > >> > j...@nanthrax.net > >> > > > > >> > > > > wrote: > >> > > > > > >> > > > > > Same think at Karaf: https://cwiki.apache.org/ > >> > > > confluence/display/KARAF/ > >> > > > > > > >> > > > > > Combine with Jira. > >> > > > > > > >> > > > > > Regards > >> > > > > > JB > >> > > > > > > >> > > > > > > >> > > > > > On 08/08/2016 10:03 AM, Aljoscha Krettek wrote: > >> > > > > > > >> > > > > >> Please have a look at this: > >> > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+ > >> > > > > >> Improvement+Proposals > >> > > > > >> > >> > > > > >> We recently started using this process in Flink and so far > >are > >> > quite > >> > > > > happy > >> > > > > >> with it. > >> > > > > >> > >> > > > > >> On Mon, 8 Aug 2016 at 06:52 Jean-Baptiste Onofré < > >> j...@nanthrax.net > >> > > > >> > > > > wrote: > >> > > > > >> > >> > > > > >> Good point Ben. > >> > > > > >>> > >> > > > > >>> I would say a "discussion" Jira can "evolve" to a > >> implementation > >> > > > "Jira" > >> > > > > >>> (just changing the component). > >> > > > > >>> > >> > > > > >>> WDYT ? > >> > > > > >>> > >> > > > > >>> Regards > >> > > > > >>> JB > >> > > > > >>> > >> > > > > >>> On 08/08/2016 06:50 AM, Ben Chambers wrote: > >> > > > > >>> > >> > > > > >>>> Would we use the same Jira to track the series of PRs > >> > implementing > >> > > > the > >> > > > > >>>> proposal (if accepted) or would it be discussion only > >> (possibly > >> > > > linked > >> > > > > >>>> to > >> > > > > >>>> the implementation tasks)? > >> > > > > >>>> > >> > > > > >>>> On Sun, Aug 7, 2016, 9:48 PM Frances Perry > >> > <f...@google.com.invalid > >> > > > > >> > > > > >>>> > >> > > > > >>> wrote: > >> > > > > >>> > >> > > > > >>>> > >> > > > > >>>> I'm a huge fan of keeping all the details related to a > >topic > >> in > >> > a > >> > > > > >>>>> > >> > > > > >>>> relevant > >> > > > > >>> > >> > > > > >>>> jira issue. > >> > > > > >>>>> > >> > > > > >>>>> On Sun, Aug 7, 2016 at 9:31 PM, Jean-Baptiste Onofré < > >> > > > > j...@nanthrax.net> > >> > > > > >>>>> wrote: > >> > > > > >>>>> > >> > > > > >>>>> Hi guys, > >> > > > > >>>>>> > >> > > > > >>>>>> we have now several technical discussions, sent on the > >> mailing > >> > > > list > >> > > > > >>>>>> > >> > > > > >>>>> with > >> > > > > >>> > >> > > > > >>>> link to document for details. > >> > > > > >>>>>> > >> > > > > >>>>>> I think it's not easy for people to follow the > >different > >> > > > > discussions, > >> > > > > >>>>>> > >> > > > > >>>>> and > >> > > > > >>> > >> > > > > >>>> to look for the e-mail containing the document links. > >> > > > > >>>>>> > >> > > > > >>>>>> Of course, it's required to have the discussion on the > >> mailing > >> > > > list > >> > > > > >>>>>> > >> > > > > >>>>> (per > >> > > > > >>> > >> > > > > >>>> Apache rules). However, maybe it could be helpful to > >have a > >> > place > >> > > to > >> > > > > >>>>>> > >> > > > > >>>>> find > >> > > > > >>> > >> > > > > >>>> open discussions, with the link to the mailing list > >discussion > >> > > > thread, > >> > > > > >>>>>> > >> > > > > >>>>> and > >> > > > > >>>>> > >> > > > > >>>>>> to the detailed document. > >> > > > > >>>>>> It could be on the website (but maybe not easy to > >maintain > >> and > >> > > > > >>>>>> > >> > > > > >>>>> publish), > >> > > > > >>> > >> > > > > >>>> or on Jira (one Jira per discussion), or a wiki. > >> > > > > >>>>>> > >> > > > > >>>>>> WDYT ? > >> > > > > >>>>>> > >> > > > > >>>>>> Regards > >> > > > > >>>>>> JB > >> > > > > >>>>>> -- > >> > > > > >>>>>> Jean-Baptiste Onofré > >> > > > > >>>>>> jbono...@apache.org > >> > > > > >>>>>> http://blog.nanthrax.net > >> > > > > >>>>>> Talend - http://www.talend.com > >> > > > > >>>>>> > >> > > > > >>>>>> > >> > > > > >>>>> > >> > > > > >>>> > >> > > > > >>> -- > >> > > > > >>> Jean-Baptiste Onofré > >> > > > > >>> jbono...@apache.org > >> > > > > >>> http://blog.nanthrax.net > >> > > > > >>> Talend - http://www.talend.com > >> > > > > >>> > >> > > > > >>> > >> > > > > >> > >> > > > > > -- > >> > > > > > Jean-Baptiste Onofré > >> > > > > > jbono...@apache.org > >> > > > > > http://blog.nanthrax.net > >> > > > > > Talend - http://www.talend.com > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> >
