Thanks, JD, Mike, and Adar for jumping in with more perspectives. I'm afraid that this thread might take a turn towards an unproductive argument of "tooling vs mailing lists", but I don't think that was Chris's main point. Neither do I think Chris is just trying to stir up trouble -- I approached him about being a mentor for our incubation because I've always found his advice to be unbiased, measured, and helpful towards building a good community.
To try to drive this to a useful conclusion, let me propose a few action items: 1) Let's move gerrit traffic to a different list as discussed. I think many of us already did filters like this for our own inboxes, but the point about the archives being hard to read is a good one. I have a slight preference to reuse the issues@ list instead of a new reviews@, both to keep the number of lists down, and because we often discuss and fix bugs more on gerrit than through lots of JIRA commentary. Makes sense that the filing of a bug, and its discussion/fix, would show up on the same list. If others disagree, though, I think reviews@ would be fine as well. 2) Chris also makes a good point that it's hard to extract signal from noise in the flood of gerrit traffic. Slowing down development isn't a great option, but I think we can use gerrit to our advantage here. It actually allows users the ability to selectively watch certain paths in the repository, which would be very helpful for new contributors who might for example care a lot about changes to python/* but not to our consensus implementation. Others might care a lot about design-docs/* for more architectural discussions. I'll volunteer to write up a new section in our 'contributing' guide that shows people how to selectively subscribe to the areas of code they're interested in. Chris -- do the above items seem like positive changes from your perspective? Are there any other concrete action items you think we should consider? -Todd On Thu, Mar 10, 2016 at 4:39 PM, Adar Dembo <[email protected]> wrote: > Hi Chris, > > I'm a new Apache committer and new to ASF in general. I have some > experience with other open source projects, and as a developer I value the > "advanced" tooling that many have adopted. If we assume > everything-is-over-email as the baseline, this tooling includes: > - Chat rooms for real-time communication, be it via IRC, Slack, HipChat, > etc. > - Code review tools a la Review Board. > - Complete workflow management tools a la GitHub, gerrit, etc. > - Bug report trackers a la Bugzilla, JIRA, etc. > Many of these tools are offered by Apache, so it seems like Apache's trend > is towards "the right tool for the right job" rather than "everything must > be communicated over e-mail". > > In particular, as someone who does a high volume of code review on Kudu and > other projects, I'll strongly value advances in review tooling. Taken > together, they can save me hours of time in a given week. As for design > review, Dan and I discussed this at length when we transitioned from Google > Docs to gerrit. You can see our back-and-forth here: > http://gerrit.cloudera.org:8080/#/c/2149. Generally speaking I find the > "centralized commenting" approach of a system like Google Docs or gerrit to > be more useful than an e-mail thread, especially when one wants to look > back at a design discussion that took place in the past. > > Generally speaking, I suspect that moving more of Kudu's development on > mailing lists optimizes for the casual developer who rarely contributes > patches but wishes to "stay involved" in numerous Apache projects. I don't > think we should be optimizing for this person; I'd prefer we optimize for > folks who have deliberately decided to invest their time in Kudu, because > they're thinking of using it to solve a problem, because they're already > using it, or because they find the technology to be just plain interesting. > These developers will adapt themselves to whatever workflow the project > uses, are likely to produce large contributions, and are more likely to > appreciate some of the more advanced tooling that Kudu uses. > > Personally, I don't like being asked to slow down my workflow purely on the > faith that it will spur OSS adoption. What I see is someone who is not > involved in Kudu's day-to-day activities requesting we make changes that, I > think we both agree (in your words, "Email is slow and deliberate and not > as fast or slick as gerrit etc, but that's a good thing"), will slow down > Kudu development. Further, I see a blanket dismissal of Todd's (very > reasonable) counterpoints. So I'm naturally being defensive; can you > provide more substantive arguments as to why we should move development > discussions off of tools like gerrit and onto the dev mailing list? > > > On Wed, Mar 9, 2016 at 7:00 PM, Mattmann, Chris A (3980) < > [email protected]> wrote: > > > Thanks for the reply Todd. Unfortunately it's more systematic than that. > > Apologies for top post but am on my phone. Couple points: > > > > - interested to hear from others besides you on this. No offense but I > > think it's important that project members send email here to reply. > > > > - I hand counted the 4 threads of interest. Didn't run a fancy command > but > > to be honest it's more indicative of the broader issue. Things aren't > > always solved through fancy greps and tools like gerrit. This is going to > > be a core issue with Kudu's incubation - how is someone not sitting in a > > cube working on the project who isn't on those tools like gerrit and > slack > > which don't exist at the ASF going to join on the project? > > > > - Even considering 40 threads I doubt there have only be <= 40 > *decisions* > > on the project to date. IOW they are being made somewhere but it's > unclear > > where. Email is easy to follow on a phone on the go whatever. > > > > As a mentor I would not be comfortable with Kudu being a TLP at this > point > > bc frankly projects need to use their dev list for more than automated > > discussion and big reports. Simple as that and sending a transcript of > > where convo is happening elsewhere is not going to cut it unfortunately. > > > > Email is slow and deliberate and not as fast or slick as gerrit etc, but > > that's a good thing. It allows people the time needed to read and join an > > OSS community. It's too hard to do that with Kudu right now. > > > > Cheers, > > Chris > > > > Sent from my iPhone > > > > > On Mar 9, 2016, at 6:46 PM, Todd Lipcon <[email protected]> wrote: > > > > > > Hey Chris, > > > > > > Responses inline: > > > > > > On Wed, Mar 9, 2016 at 6:08 PM, Mattmann, Chris A (3980) < > > > [email protected]> wrote: > > > > > >> Hi Team, > > >> > > >> I looked at: > > >> > > >> https://mail-archives.apache.org/mod_mbox/incubator-kudu-dev > > >> > > >> And over the last 4 months and Kudu’s inception, we have had > > >> well over 2k+ emails, and looking back I found 4 actual threads > > >> during that time (and one of which was a release VOTE thread) > > >> that wasn’t automatically generated by Gerrit. > > >> > > >> Mar 2016 438 > > >> Feb 2016 1003 > > >> Jan 2016 1143 > > >> Dec 2015 12 > > > > > > Hmm, I did a search in my inbox for: [email protected] > > -gerrit > > > -jira -"git commit" -dev-help -"svn commit" -moderate -"git push > summary" > > > and counted 30-35 threads. You're right, of course, that JIRA and > gerrit > > > eclipse the amount of email discussion, though. > > > > > > > > >> > > >> > > >> If we are going to become an ASF top level project, the project > > >> discussion has to happen on the mailing list. We had similar > > >> issues in Spark and I realize that lots of project work is assisted > > >> by tools and other technologies, but at the ASF, “if it didn’t > > >> happen on the mailing list, it didn’t happen.” More-over it’s hard > > >> to parse signal from noise in all these automated messages. Frankly > > >> I don’t really know if anything good is going on - I know things > > >> are going on, and I assume they are good, but it’s extremely hard > > >> to verify that. > > > > > > I think it's worth noting that the "automated' messages are typically > > code > > > review requests and responses, which are developer discussion. Our > > > project's culture is usually to use JIRAs and/or 'work-in-progress' > > patches > > > in gerrit to communicate when we find a bug or want an opinion on > > > something. For example, today I found a new bug > > > https://issues.apache.org/jira/browse/KUDU-1369 and wrote up a quick > > > work-in-progress for a a proposed solution and put it up at > > > http://gerrit.cloudera.org:8080/#/c/2514/ . I think it would be > > redundant > > > to also send an email to the list saying "Hey guys, I found a bug, > > here's a > > > description". > > > > > > The same goes for design discussion -- eg > > > http://gerrit.cloudera.org:8080/#/c/2443/ is a recent gerrit post that > > Dan > > > made for a new feature he's working on. In this case he also sent an > > email > > > to the dev list to point out the gerrit in case anyone missed it. I > > imagine > > > a lot of people would filter the gerrit emails out of their inbox but > not > > > direct emails to the list (gerrit provides both headers and a subject > > line > > > tag to make it easy to do) > > > > > > In terms of daily dev discussion, most of it has been happening on our > > > Slack -- eg earlier today three contributors were discussing > in-progress > > > efforts on Spark RDD integration and sharing code via that channel. > Most > > of > > > the community members we've seen so far have tended to prefer this > quick > > > back-and-forth for discussion. > > > > > > Of course any _decisions_ will be made on the mailing list. If you > think > > it > > > would be useful to send a daily slack log to the mailing list, we can > do > > > that as well. > > > > > > > > > > > >> > > >> I have a possible suggestions: > > >> > > >> * Create a [email protected] and send all automated > > >> traffic there. *-issues is one option; we could make another name for > > >> it. > > > > > > Sure, we could do that. But, isn't it just as easy for people to set > up a > > > filter for 'kudu-CR' if they want to move those messages elsewhere? Our > > > initial motivation when setting up mailing lists was to avoid having > too > > > many (makes it a pain for people to subscribe to them all). > > > > > > > > >> > > >> That will help to separate the signal from the noise in terms of > > >> dev/architectural/etc. discussions from code reviews and automated > > >> commit messages. > > >> > > >> One thing you may say is that dev/architectural discussions are > > happening > > >> but they are in Gerrit. I would then say it’s extremely difficult to > > >> separate the signal from the noise here, and as such, could be > > contributing > > >> towards making it difficult for others to join the project, something > > >> that we identified as an issue in our Incubator report. > > > > > > Right. One option is that, for patches with bigger discussion, we can > > add a > > > gerrit "reviewer" which is actually the dev mailing list. This would > > cause > > > the discussion to be CCed there, and bring it to the attention of more > > > people. Another thought is to do as you suggest above and move gerrit > > > elsewhere, and just have a policy that whenever any gerrit starts > getting > > > architectural, that we send a ping to the dev mailing list to point it > > out > > > (as Dan did with his recent design doc). > > > > > > -Todd > > > -- Todd Lipcon Software Engineer, Cloudera
