Before we get too deeply into your questions I would like to address the overarching question that you seem to be asking. Is the project healthy? I personally think that we are healthy. Could we be doing better, yes of course. I would like to grab some stats from github to try and make a point here. https://github.com/apache/storm/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c
Since 2014 we have had a large number of contributions (> 1000 commits) That looks to be about half as much as flink https://github.com/apache/flink/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c but not that far behind subversion (I know subversion on git is a bit odd) https://github.com/apache/subversion/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c I looked at many others. Hadoop core (HDFS + YARN + MR)https://github.com/apache/hadoop/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c Lucene-solrhttps://github.com/apache/lucene-solr/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c ... even Spark https://github.com/apache/spark/graphs/contributors?from=2014-01-01&to=2016-10-21&type=c In general from a pure commits bases, and even several others metrics I looked at it does not look that bad to me. I had this same concern a year or more ago and have been looking at the metrics ever since. Even the number of PMC members and contributors leaving the project is not that scary to me. I am a PMC member on multiple projects Storm, Spark, Hadoop, and Tez. My job focus has changed over the years so now I concentrate primarily on Storm. Could that change again in the future? Very possibly. This has happened for others too. My biggest concern would be more with do we have customers replacing storm with other streaming systems. If so why are they going? Do we have new customers showing up? What are the pain points that people mention about the project? The more people use the project the more people will want to contribute back is my view of this. Although if people do have suggestions about things we can change, bylaws etc, to make the project more approachable to new devs I would be happy to discuss them and adopt them. I would love to see storm get the attention that spark is seeing, and if we can find a way through marketing/features to make that happen it would be great. To me a lot of this really is marketing. Storm meets the needs of most people at Yahoo for what they want to do. Many of them have tried other solutions, especially spark streaming, and have always come back to storm because it works really well and does what they want it to do. That said I have kind of a closed view of things here. I am not out doing market research, I am looking at what our customers internally want. For us our road map is around better usability and resource utilization. Resource Aware Scheduling, elasticity, better metrics, better UI, and possibly improving the underlying workers to be better efficient. Others would need to talk about their road maps, but mostly it looks like SQL, BEAM, and a better API. - Bobby On Thursday, October 20, 2016 10:58 PM, Jungtaek Lim <[email protected]> wrote: Hi devs, If you also subscribe the user mailing list, you might see the mail 'Is trident dead? <http://mail-archives.apache.org/mod_mbox/storm-user/201610.mbox/%3CCAP3bR3xmOb=pMXa=aj--_=j=bqoakqytdwezfxysxsufdky...@mail.gmail.com%3E> '. I don't remember (and also don't mind) the origin question and following talks, but while talking, he use the word "commit-wise active" which seems to be the thing we may need to think about. (even I replied the opposite to user mailing list.) Apart from contributing features, open source project itself needs some efforts to maintain and publicize. While efforts for former is happening continuously, but I don't see great efforts for latter, for months. We have very limited resource and it could be a way to maximize our efficiency, but it is clearly showing the downsides: community is losing activity, and individual contributors left and don't come. IMHO, we may need to think about below questions clearly. There're multiple subjects so we may want to classify and may discuss. Most of questions may need to think deeply and not be easy to answer. - What's the project's goal? In other words, what's the reason we have been putting great efforts? - Are we documenting or publicizing our efforts, especially new features pretty well? - Why our heavy individual contributors or even PMCs leave the community, and why we haven't seen new heavy contributors? - How we bring back activity of Storm community: users and contributors? - How long is the release cycle of minor version of Storm so that users can take the benefit of our efforts? - Are our contribution / merge / release process pragmatic? Thanks, Jungtaek Lim (HeartSaVioR)
