Re: Beam's recent community development work
I think telling people that they’re being considered as committers early on is a good idea, but AFAIK we’ve always had individual committers do that with contributors who were doing great work in various areas. We don’t have a centralized process for it though — it’s up to whoever wants to work with each contributor. Matei > On Jul 2, 2018, at 5:35 PM, Reynold Xin wrote: > > That's fair, and it's great to find high quality contributors. But I also > feel the two projects have very different background and maturity phase. > There are 1300+ contributors to Spark, and only 300 to Beam, with the vast > majority of contributions coming from a single company for Beam (based on my > cursory look at the two pages of commits on github). With the recent security > and correctness storms, I actually worry about more quality (which requires > more infrastructure) than just people adding more code to the project. > > > > On Mon, Jul 2, 2018 at 5:25 PM Holden Karau wrote: > As someone who floats a bit between both projects (as a contributor) I'd love > to see us adopt some of these techniques to be pro-active about growing our > committer-ship (I think perhaps we could do this by also moving some of the > newer committers into the PMC faster so there are more eyes out looking for > people to bring forward)? > > On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen wrote: > Worth, I think, a read and consideration from Spark folks. I'd be interested > in comments; I have a few reactions too. > > > -- Forwarded message - > From: Kenneth Knowles > Date: Sat, Jun 30, 2018 at 1:15 AM > Subject: Beam's recent community development work > To: , , Griselda Cuevas > , dev > > > Hi all, > > The ASF board suggested that we (Beam) share some of what we've been doing > for community development with d...@community.apache.org and > memb...@apache.org. So here is a long description. I have included > d...@beam.apache.org because it is the subject, really, and this is & should > be all public knowledge. > > We would love feedback! We based a lot of this on reading the community > project site, and probably could have learned even more with more study. > > # Background > > We face two problems in our contributor/committer-base: > > 1. Not enough committers to review all the code being contributed, in part > due to recent departure of a few committers > 2. We want our contributor-base (hence committer-base) to be more spread > across companies and backgrounds, for the usual Apache reasons. Our user base > is not active and varied enough to make this automatic. One solution is to > make the right software to get a varied user base, but that is a different > thread :-) so instead we have to work hard to build our community around the > software we have. > > # What we did > > ## Committer guidelines > > We published committer guidelines [1] for transparency and as an invitation. > We start by emphasizing that there are many kinds of contributions, not just > code (we have committers from community development, tech writing, training, > etc). Then we have three aspects: > > 1. ASF code of conduct > 2. ASF committer responsibilities > 3. Beam-specific committer responsibilities > > The best way to understand is to follow the link at the bottom of this email. > The important part is that you shouldn't be proposing a committer for other > reasons, and you shouldn't be blocking a committer for other reasons. > > ## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every layer > > Gris (CC'd) outlined this: people go through these phases of relationship > with our project: > > 1. aware of it > 2. interested in it / checking it out > 3. using it for real > 4. first-time contributor > 5. repeat contributor > 6. committer > 7. PMC > > As soon as we notice someone, like a user asking really deep questions, we > invite discussion on private@ on how we can move them to the next level of > engagement. > > ## Monthly cadence > > Every ~month, we call for new discussions and revisit ~all prior discussions. > This way we do not forget to keep up this effort. > > ## Individual discussions > > For each person we have a separate thread on private@. This ensures we have > quality focused discussions that lead to feedback. In collective discussions > that we used to do, we often didn't really come up with actionable feedback > and ended up not even contacting potential committers to encourage them. And > consensus was much less clear. > > ## Feedback! > > If someone is brought up for a discussion, that means they got enough > attention that we hope to engage them more. But unsolicited feedback is never > a good idea. For a potential committer, we did this: > > 1. Send an email saying something like "you were discussed as a potential > committer - do you want to become one? do you want feedback?" > 2. If they say yes (so far everyone) we send a few bullet points
Re: Beam's recent community development work
That's fair, and it's great to find high quality contributors. But I also feel the two projects have very different background and maturity phase. There are 1300+ contributors to Spark, and only 300 to Beam, with the vast majority of contributions coming from a single company for Beam (based on my cursory look at the two pages of commits on github). With the recent security and correctness storms, I actually worry about more quality (which requires more infrastructure) than just people adding more code to the project. On Mon, Jul 2, 2018 at 5:25 PM Holden Karau wrote: > As someone who floats a bit between both projects (as a contributor) I'd > love to see us adopt some of these techniques to be pro-active about > growing our committer-ship (I think perhaps we could do this by also moving > some of the newer committers into the PMC faster so there are more eyes out > looking for people to bring forward)? > > On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen wrote: > >> Worth, I think, a read and consideration from Spark folks. I'd be >> interested in comments; I have a few reactions too. >> >> >> -- Forwarded message - >> From: Kenneth Knowles >> Date: Sat, Jun 30, 2018 at 1:15 AM >> Subject: Beam's recent community development work >> To: , , Griselda Cuevas < >> g...@apache.org>, dev >> >> >> Hi all, >> >> The ASF board suggested that we (Beam) share some of what we've been >> doing for community development with d...@community.apache.org and >> memb...@apache.org. So here is a long description. I have included >> d...@beam.apache.org because it is the subject, really, and this is & >> should be all public knowledge. >> >> We would love feedback! We based a lot of this on reading the community >> project site, and probably could have learned even more with more study. >> >> # Background >> >> We face two problems in our contributor/committer-base: >> >> 1. Not enough committers to review all the code being contributed, in >> part due to recent departure of a few committers >> 2. We want our contributor-base (hence committer-base) to be more spread >> across companies and backgrounds, for the usual Apache reasons. Our user >> base is not active and varied enough to make this automatic. One solution >> is to make the right software to get a varied user base, but that is a >> different thread :-) so instead we have to work hard to build our community >> around the software we have. >> >> # What we did >> >> ## Committer guidelines >> >> We published committer guidelines [1] for transparency and as an >> invitation. We start by emphasizing that there are many kinds of >> contributions, not just code (we have committers from community >> development, tech writing, training, etc). Then we have three aspects: >> >> 1. ASF code of conduct >> 2. ASF committer responsibilities >> 3. Beam-specific committer responsibilities >> >> The best way to understand is to follow the link at the bottom of this >> email. The important part is that you shouldn't be proposing a committer >> for other reasons, and you shouldn't be blocking a committer for other >> reasons. >> >> ## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every >> layer >> >> Gris (CC'd) outlined this: people go through these phases of relationship >> with our project: >> >> 1. aware of it >> 2. interested in it / checking it out >> 3. using it for real >> 4. first-time contributor >> 5. repeat contributor >> 6. committer >> 7. PMC >> >> As soon as we notice someone, like a user asking really deep questions, >> we invite discussion on private@ on how we can move them to the next >> level of engagement. >> >> ## Monthly cadence >> >> Every ~month, we call for new discussions and revisit ~all prior >> discussions. This way we do not forget to keep up this effort. >> >> ## Individual discussions >> >> For each person we have a separate thread on private@. This ensures we >> have quality focused discussions that lead to feedback. In collective >> discussions that we used to do, we often didn't really come up with >> actionable feedback and ended up not even contacting potential committers >> to encourage them. And consensus was much less clear. >> >> ## Feedback! >> >> If someone is brought up for a discussion, that means they got enough >> attention that we hope to engage them more. But unsolicited feedback is >> never a good idea. For a potential committer, we did this: >> >> 1. Send an email saying something like "you were discussed as a potential >> committer - do you want to become one? do you want feedback?" >> 2. If they say yes (so far everyone) we send a few bullet points from the >> discussion and *most important* tie each bullet to the committer >> guidelines. If we have no feedback about which guidelines were a concern, >> that is a red flag that we are being driven by bias. >> >> We saw a *very* significant increase in engagement from those we sent >> feedback to, and the trend is that they almost all will become
Re: Beam's recent community development work
As someone who floats a bit between both projects (as a contributor) I'd love to see us adopt some of these techniques to be pro-active about growing our committer-ship (I think perhaps we could do this by also moving some of the newer committers into the PMC faster so there are more eyes out looking for people to bring forward)? On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen wrote: > Worth, I think, a read and consideration from Spark folks. I'd be > interested in comments; I have a few reactions too. > > > -- Forwarded message - > From: Kenneth Knowles > Date: Sat, Jun 30, 2018 at 1:15 AM > Subject: Beam's recent community development work > To: , , Griselda Cuevas < > g...@apache.org>, dev > > > Hi all, > > The ASF board suggested that we (Beam) share some of what we've been doing > for community development with d...@community.apache.org and > memb...@apache.org. So here is a long description. I have included > d...@beam.apache.org because it is the subject, really, and this is & > should be all public knowledge. > > We would love feedback! We based a lot of this on reading the community > project site, and probably could have learned even more with more study. > > # Background > > We face two problems in our contributor/committer-base: > > 1. Not enough committers to review all the code being contributed, in part > due to recent departure of a few committers > 2. We want our contributor-base (hence committer-base) to be more spread > across companies and backgrounds, for the usual Apache reasons. Our user > base is not active and varied enough to make this automatic. One solution > is to make the right software to get a varied user base, but that is a > different thread :-) so instead we have to work hard to build our community > around the software we have. > > # What we did > > ## Committer guidelines > > We published committer guidelines [1] for transparency and as an > invitation. We start by emphasizing that there are many kinds of > contributions, not just code (we have committers from community > development, tech writing, training, etc). Then we have three aspects: > > 1. ASF code of conduct > 2. ASF committer responsibilities > 3. Beam-specific committer responsibilities > > The best way to understand is to follow the link at the bottom of this > email. The important part is that you shouldn't be proposing a committer > for other reasons, and you shouldn't be blocking a committer for other > reasons. > > ## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every > layer > > Gris (CC'd) outlined this: people go through these phases of relationship > with our project: > > 1. aware of it > 2. interested in it / checking it out > 3. using it for real > 4. first-time contributor > 5. repeat contributor > 6. committer > 7. PMC > > As soon as we notice someone, like a user asking really deep questions, we > invite discussion on private@ on how we can move them to the next level > of engagement. > > ## Monthly cadence > > Every ~month, we call for new discussions and revisit ~all prior > discussions. This way we do not forget to keep up this effort. > > ## Individual discussions > > For each person we have a separate thread on private@. This ensures we > have quality focused discussions that lead to feedback. In collective > discussions that we used to do, we often didn't really come up with > actionable feedback and ended up not even contacting potential committers > to encourage them. And consensus was much less clear. > > ## Feedback! > > If someone is brought up for a discussion, that means they got enough > attention that we hope to engage them more. But unsolicited feedback is > never a good idea. For a potential committer, we did this: > > 1. Send an email saying something like "you were discussed as a potential > committer - do you want to become one? do you want feedback?" > 2. If they say yes (so far everyone) we send a few bullet points from the > discussion and *most important* tie each bullet to the committer > guidelines. If we have no feedback about which guidelines were a concern, > that is a red flag that we are being driven by bias. > > We saw a *very* significant increase in engagement from those we sent > feedback to, and the trend is that they almost all will become committers > over time. > > ## Results > > - Q1 we had no process and we added no new committers or PMC members. > - Q2 when we tried these new things we added 7 committers and 1 PMC > member, with ~3~4 obvious committer candidates for next time around, plus > positive feedback from other contributors, spread across five companies. > > We've had a pretty major uptick in building Beam as a result. > > ## Review-then-commit > > We are dedicated to RTC as the best way to build software. Reviews not > only make the code better, but (with apologies to ASF/GNU differences) as > RMS says "The fundamental act of friendship among programmers is the > sharing of programs" and reviews are where we
