Maybe tweaking the current process a bit is enough. I like the Docs
for having discussions but there no good as a /proper design
document/, for the following reasons:
I see design documents full of discussions and wonder:
* Who will be the *main owner* and the *co-owners* (meaning people
that are invested of bringing this forward and can *act* as
/reviewers/). I think a proposal needs especially this: ownership
* Lack of visibility of final state? Or is it superseded by another
proposal. A final state could include the votes...
* Does the proposal need amendments. An example, while implementing
the proposal, we see that something in the design was lacking and
needs to be added.
So the Docs are great, but maybe we should a few mandatory blocks and
a few rules:
* *Resolve all discussions* before switching to final state.
* If new discussions pop up, maybe an amendment needs to be made (or
correct). Corrections could be added to a *changelog* in the
beginning.
* If a new proposal supersedes on, both should be linked
* Most importantly: Who can act as *owner* end reviewers for this
proposal.
_/
_/ Alex Van Boxel
On Thu, Jan 9, 2020 at 7:59 AM Kenneth Knowles <k...@apache.org
<mailto:k...@apache.org>> wrote:
It does seem that the community would find this useful. I agree
with Robert that it has downsides and it is not appropriate all
the time.
We added https://beam.apache.org/roadmap/ a little while ago. I
think that the granularity of a BIP is about the same as the
granularity of what we would want to show to users on a roadmap on
our public site. So we sort of already have this. Perhaps we want
to formalize changes to the roadmap and only include voted upon
approved BIPs on the roadmap on the web site. The current roadmap
should be viewed as a crowd sourced bootstrap, for sure.
Imagine a roadmap that a company shares with a customer. The most
important thing is to be extremely clear about what is intended to
be built, when it is expected, and how they can follow the
developments. And for the open source community, it should be
clear what they can expect to work on and know that the project /
PMC has agreed on the feature and will not push back after some
effort has been put into it.
Kenn
On Tue, Dec 17, 2019 at 11:07 AM Jan Lukavský <je...@seznam.cz
<mailto:je...@seznam.cz>> wrote:
Hi,
I feel a "soft consensus" :) that people see some benefits of
introducing (possibly optional) process of proposing new features.
I think that in order to proceed with this we need to agree on
goals that we want to achieve. Whether the process should or
should not be optional, which form it should have, and answers
on all these other questions could be answered after that.
So, I'll try to state some issues I see with our current
approach, please feel free to correct any of them, or add any
other:
- due to the "soft consensus" approach, we actually delegate
the final responsibility of "feature acceptance" to
reviewer(s) - these might or might not be happy with that
- by splitting this into
first-consensus-then-implementation-then-review approach, we
remove the burden of responsibility of respective feature from
reviewers - they can focus only on the main purpose of the
review - that is verifying the quality of code
- as mentioned before, this brings better visibility to
(core) features
- and last but not least makes it possible to prioritize work
and build more complex long-term goals
I think it is essential to have a consensus on whether or not
these are some points we want to target (that is, we see our
current approach as sub-optimal in these areas) or not.
Jan
On 12/17/19 7:08 PM, Pablo Estrada wrote:
It seems that lots of people see benefit in a more formalized
BIP process. I think that makes sense, though I'd like to
give people the freedom to choose the medium for their design
discussions.
The projects I'm aware of usually do this through wiki-type
mediums. We have cwiki, though lots of people like working
with Gdocs' collaboration features. Are there other mediums
that could be used for this?
A possible implementation is: We could keep cwiki as the
'index' - so anyone proposing a new BIP would have to add a
new BIP entry in the cwiki, but they'd be free to link to a
Gdoc from there, or to develop the proposal in the cwiki
entry itself.
Thoughts?
Best
-P.
On Tue, Dec 17, 2019 at 9:14 AM Maximilian Michels
<m...@apache.org <mailto:m...@apache.org>> wrote:
The main benefit of BIPs I see is the visibility they
create for the
project users and contributors.
Right now, we have a long unordnered list of design
documents. Some of
the documents are not even in that list. With BIPs, we
would end up with
an ordered list "BIP-1, BIP-2, .." which reflects
important design
decisions over time.
Simply assigning an id, makes it a lot more formal. In my
eyes, the id
assignment would also require that you communicate the
changes in a way
that the community can accept the proposal, preferably
via lazy
consensus. All in all, this could help communicate
changes in Beam better.
JIRA, on the other hand, contains concrete implementation
steps and all
kinds of other changes.
Cheers,
Max
On 16.12.19 21:41, Robert Bradshaw wrote:
> Additional process is a two-edged sword: it can help
move stuff
> forward, to the correct decision, but it can also add
significant
> overhead.
>
> I think there are many proposals for which the existing
processes of
> deriving consensus (over email, possibly followed by a
formal vote or
> lazy consensus) are sufficient. However, sometimes
they're not.
> Specifically, for long-term roadmaps, it would be
useful to have them
> in a standard place that can be tracked and understood
(I don't think
> we've been able to use JIRA effectively for this here).
I also think
> there are some proposals that reach a certain level of
complexity that
> trying to address them by occasionally responding to
email threads as
> they come up is insufficient. For these latter, I think
there is a
> need for commitment for a group of people in the
community to commit
> to clearly defining and driving a solution to the
problem via a more
> formal process. Often the one making the proposal has
sufficient
> motivation, but sometimes what lacks is be
(non-sporadic) investment
> by those trying to understand, evaluate, and
incorporate the proposal.
>
> So I'm (strongly) +1 for exploring a more formal
process, but -1 on
> requiring it.
>
> On Sun, Dec 15, 2019 at 1:07 AM Jan Lukavský
<je...@seznam.cz <mailto:je...@seznam.cz>> wrote:
>>
>> Hi,
>>
>> thanks for reactions so far. I agree that there are
many questions that have to be clarified. I'd propose to
split this into two parts:
>>
>> a) first reach a consensus that we want this process
in the first place
>>
>> b) after that, we need to clarify all the details -
that will probably be somewhat iterative procedure
>>
>> I'm not sure if there is something more we need to
clarify before we can cast a vote on (a).
>>
>> Thoughts?
>>
>> Jan
>>
>> On 12/10/19 3:46 PM, Łukasz Gajowy wrote:
>>
>> +1 for formalizing the process, enhancing it and
documenting clearly.
>>
>> I noticed that Apache Airflow has a cool way of both
creating AIPs and keeping track of all of them. There is
a "Create new AIP" button on their Confluence. This way,
no AIP gets lost and all are kept in one place. Please
keep in mind that this is also the problem we want to
solve in Beam and try to keep track of all the documents
we have so far*. It's certainly good to solve that
problem too, if possible.
>>
>> Also the AIP structure is something that I find nice -
There's place for all additional resources, JIRAs,
discussion in comments and state of the proposal. Even if
we don't choose to use Confluence, we definitely could
use a similar template with all that information for our
google docs proposals or any other tool we stick to.
>>
>> Thanks!
>>
>> *thank you, Ismael and Alexey, for all the reminders
under the proposals to add them to Confluence list! :)
>>
>> wt., 10 gru 2019 o 13:29 jincheng sun
<sunjincheng...@gmail.com
<mailto:sunjincheng...@gmail.com>> napisał(a):
>>>
>>> Thanks for bring up this discussion Jan!
>>>
>>> +1 for cearly define BIP for beam.
>>>
>>> And I think would be nice to initialize a concept
document for BIP. Just a reminder: the document may contains:
>>>
>>> - How many kinds of improvement in beam.
>>> - What kind of improvement should to create a BIP.
>>> - What should be included in a BIP.
>>> - Who can create the BIP.
>>> - Who can participate in the discussion of BIP and
who can vote for BIP.
>>> - What are the possible limitations of BiP, such as
whether it is necessary to complete the dev of BIP in
one release.
>>> - How to track a BIP.
>>>
>>> Here is a question: I found out a policy[1] in beam,
but only contains the poilcy of release , my question is
does beam have something called Bylaws? Similar as Flink[1].
>>>
>>> Anyway, I like your proposals Jan :)
>>>
>>> Best,
>>> Jincheng
>>> [1] https://beam.apache.org/community/policies/
>>> [2]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals
>>>
>>>
>>> David Morávek <david.mora...@gmail.com
<mailto:david.mora...@gmail.com>> 于2019年12月10日周二
下午2:33写道：
>>>>
>>>> Hi Jan,
>>>>
>>>> I think this is more pretty much what we currently
do, just a little bit more transparent for the community.
If the process is standardized, it can open doors for
bigger contributions from people not familiar with the
process. Also it's way easier to track progress of BIPs,
than documents linked from the mailing list.
>>>>
>>>> Big +1 ;)
>>>>
>>>> D.
>>>>
>>>> On Sun, Dec 8, 2019 at 12:42 PM Jan Lukavský
<je...@seznam.cz <mailto:je...@seznam.cz>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'd like to revive a discussion that was taken some
year and a half ago
>>>>> [1], which included a concept of "BIP" (Beam
Improvement Proposal) - an
>>>>> equivalent of "FLIP" (flink), "KIP" (kafka), "SPIP"
(spark), and so on.
>>>>>
>>>>> The discussion then ended without any (public)
conclusion, so I'd like
>>>>> to pick up from there. There were questions related to:
>>>>>
>>>>> a) how does the concept of BIP differ from
simple plain JIRA?
>>>>>
>>>>> b) what does it bring to the community?
>>>>>
>>>>> I'd like to outline my point of view on both of
these aspects (they are
>>>>> related).
>>>>>
>>>>> BIP differs from JIRA by definition of a process:
>>>>>
>>>>> BIP -> vote -> consensus -> JIRA -> implementation
>>>>>
>>>>> This process (although it might seem a little
unnecessary formal) brings
>>>>> the following benefits:
>>>>>
>>>>> i) improves community's overall awareness of
planned and in-progress
>>>>> features
>>>>>
>>>>> ii) makes it possible to prioritize long-term
goals (create "roadmap"
>>>>> that was mentioned in the referred thread)
>>>>>
>>>>> iii) by casting explicit vote on each
improvement proposal diminishes
>>>>> the probability of wasted work - as opposed to our
current state, where
>>>>> it is hard to tell when there is a consensus and
what actions need to be
>>>>> done in order to reach one if there isn't
>>>>>
>>>>> iv) BIPs that eventually pass a vote can be
regarded as "to be
>>>>> included in some short term" and so new BIPs can
build upon them,
>>>>> without the risk of having to be redefined if their
dependency for
>>>>> whatever reason don't make it to the implementation
>>>>>
>>>>> Although this "process" might look rigid and
corporate, it actually
>>>>> brings better transparency and overall community
health. This is
>>>>> especially important as the community grows and
becomes more and more
>>>>> distributed. There are many, many open questions in
this proposal that
>>>>> need to be clarified, my current intent is to grab
a grasp about how the
>>>>> community feels about this.
>>>>>
>>>>> Looking forward to any comments,
>>>>>
>>>>> Jan
>>>>>
>>>>> [1]
>>>>>
https://lists.apache.org/thread.html/4e1fffa2fde8e750c6d769bf4335853ad05b360b8bd248ad119cc185%40%3Cdev.beam.apache.org%3E
>>>>>
