That's fair, and it's great to find high quality contributors. But I also feel the two projects have very different background and maturity phase. There are 1300+ contributors to Spark, and only 300 to Beam, with the vast majority of contributions coming from a single company for Beam (based on my cursory look at the two pages of commits on github). With the recent security and correctness storms, I actually worry about more quality (which requires more infrastructure) than just people adding more code to the project.
On Mon, Jul 2, 2018 at 5:25 PM Holden Karau <hol...@pigscanfly.ca> wrote: > As someone who floats a bit between both projects (as a contributor) I'd > love to see us adopt some of these techniques to be pro-active about > growing our committer-ship (I think perhaps we could do this by also moving > some of the newer committers into the PMC faster so there are more eyes out > looking for people to bring forward)? > > On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen <sro...@apache.org> wrote: > >> Worth, I think, a read and consideration from Spark folks. I'd be >> interested in comments; I have a few reactions too. >> >> >> ---------- Forwarded message --------- >> From: Kenneth Knowles <k...@google.com> >> Date: Sat, Jun 30, 2018 at 1:15 AM >> Subject: Beam's recent community development work >> To: <d...@community.apache.org>, <memb...@apache.org>, Griselda Cuevas < >> g...@apache.org>, dev <d...@beam.apache.org> >> >> >> Hi all, >> >> The ASF board suggested that we (Beam) share some of what we've been >> doing for community development with d...@community.apache.org and >> memb...@apache.org. So here is a long description. I have included >> d...@beam.apache.org because it is the subject, really, and this is & >> should be all public knowledge. >> >> We would love feedback! We based a lot of this on reading the community >> project site, and probably could have learned even more with more study. >> >> # Background >> >> We face two problems in our contributor/committer-base: >> >> 1. Not enough committers to review all the code being contributed, in >> part due to recent departure of a few committers >> 2. We want our contributor-base (hence committer-base) to be more spread >> across companies and backgrounds, for the usual Apache reasons. Our user >> base is not active and varied enough to make this automatic. One solution >> is to make the right software to get a varied user base, but that is a >> different thread :-) so instead we have to work hard to build our community >> around the software we have. >> >> # What we did >> >> ## Committer guidelines >> >> We published committer guidelines [1] for transparency and as an >> invitation. We start by emphasizing that there are many kinds of >> contributions, not just code (we have committers from community >> development, tech writing, training, etc). Then we have three aspects: >> >> 1. ASF code of conduct >> 2. ASF committer responsibilities >> 3. Beam-specific committer responsibilities >> >> The best way to understand is to follow the link at the bottom of this >> email. The important part is that you shouldn't be proposing a committer >> for other reasons, and you shouldn't be blocking a committer for other >> reasons. >> >> ## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every >> layer >> >> Gris (CC'd) outlined this: people go through these phases of relationship >> with our project: >> >> 1. aware of it >> 2. interested in it / checking it out >> 3. using it for real >> 4. first-time contributor >> 5. repeat contributor >> 6. committer >> 7. PMC >> >> As soon as we notice someone, like a user asking really deep questions, >> we invite discussion on private@ on how we can move them to the next >> level of engagement. >> >> ## Monthly cadence >> >> Every ~month, we call for new discussions and revisit ~all prior >> discussions. This way we do not forget to keep up this effort. >> >> ## Individual discussions >> >> For each person we have a separate thread on private@. This ensures we >> have quality focused discussions that lead to feedback. In collective >> discussions that we used to do, we often didn't really come up with >> actionable feedback and ended up not even contacting potential committers >> to encourage them. And consensus was much less clear. >> >> ## Feedback! >> >> If someone is brought up for a discussion, that means they got enough >> attention that we hope to engage them more. But unsolicited feedback is >> never a good idea. For a potential committer, we did this: >> >> 1. Send an email saying something like "you were discussed as a potential >> committer - do you want to become one? do you want feedback?" >> 2. If they say yes (so far everyone) we send a few bullet points from the >> discussion and *most important* tie each bullet to the committer >> guidelines. If we have no feedback about which guidelines were a concern, >> that is a red flag that we are being driven by bias. >> >> We saw a *very* significant increase in engagement from those we sent >> feedback to, and the trend is that they almost all will become committers >> over time. >> >> ## Results >> >> - Q1 we had no process and we added no new committers or PMC members. >> - Q2 when we tried these new things we added 7 committers and 1 PMC >> member, with ~3~4 obvious committer candidates for next time around, plus >> positive feedback from other contributors, spread across five companies. >> >> We've had a pretty major uptick in building Beam as a result. >> >> ## Review-then-commit >> >> We are dedicated to RTC as the best way to build software. Reviews not >> only make the code better, but (with apologies to ASF/GNU differences) as >> RMS says "The fundamental act of friendship among programmers is the >> sharing of programs" and reviews are where we do that. >> >> As a minor point, we also changed our "review-then-commit" policy to >> require that *either* the reviewer or the author be a committer. Previously >> the reviewer had to be a committer. Rationale: if we trust someone as a >> committer, we should trust their choice of reviewer. This also helps the >> community, as it engages non-committers as reviewers. >> >> ---- >> >> If you made it through this long email, thanks for reading! >> >> Kenn >> >> [1] https://beam.apache.org/contribute/become-a-committer/ >> > > > > -- > Twitter: https://twitter.com/holdenkarau >
