On Oct 22, 2011, at 6:41 PM, Sean Owen wrote: > Thanks! good thread. > > On Sat, Oct 22, 2011 at 3:30 PM, Grant Ingersoll <gsing...@apache.org> wrote: >> 1. We aim for releases every 6 months or so >> 2. We make a best guess up front about what bug fixes will be in that >> release, but we also will, obviously, bring in other fixes as they are >> reported and dealt with >> 3. As for features, we are all committers and we all have our itches that we >> wish to scratch. I can't predict 6 months out what that will be. Nor can >> you. And if we have some project plan that prevents me from adding my >> feature b/c it isn't in this release cycle and I have to wait 3 months to do >> so, then all you've (not you, personally, but the "royal" you) done is >> discourage me from contributing it at all! So, then, I go over to Github >> and do it there. Next thing you know, my Fork on Github is truly a fork. >> Now multiply that by many others and we have no project at all. My >> experience is, and it could be wrong, is that the coordination we need often >> comes about naturally by people simply picking up the yoke of the things >> they want done and plowing forward. If others like it, they will pitch in >> and help. If they don't like it, they will either pitch in and help or be >> quiet, because ultimately, those who do the work have the say. > > That all sounds good; none of these are a problem, to me. > > I don't think anyone has suggested that someone who wants to > contribute and maintain a nice piece of code here should be refused. > The problem is rather the opposite: contributed code rots, or goes > unfinished, or goes nowhere, or doesn't begin due to this uncertainty. > It's not what people are doing, but not doing.
I agree. But I believe our fundamental issue is not lack of a plan but lack of effort by other than a few committers. http://www.ohloh.net/p/mahout/contributors You pretty much blow everyone away, which is fine until you are not happy :-) Then me, but I am decent volume for a few weeks and then disappear for a few weeks. Same w/ Ted. Naturally, that list is just showing sheer number of commits and not quality/size of a commit, so I don't think it is the be all end all of contributor metrics. Number of commits, I think, captures who is fixing bugs on a regular basis, more or less. FWIW, I also think this stuff ebbs and flows quite a bit in open source. Old contributors fade away, new ones pick up the helm. Or they don't. One thing we might try is simply looking through JIRA for patches from people who routinely show up and who are not committers and see if we can't encourage them a bit more towards committership. > > (OK: I did just suggest turning away some new code. But, there's > nothing wrong with good new maintained code: I meant that it seems > less beneficial a use of very finite attention and resources, compared > to fixing and polishing what's there now.) I don't mind turning away stuff. In fact, by all accounts, we already do. It's just a lazy consensus thing. > > I also am not suggesting a strict planning and development cycle; it's > not feasible. We are in no danger of this! There must be a happy > medium between that, and what we have now, which feels like little > planning, or even resistance to it. Fair enough. See below. > > I don't accept that this is just how open-source projects are; it > isn't, and not how I've participated in open source to date otherwise. > They need a direction, identity and leadership all the same. Those are > tougher to come by since we can only resort to soft power. > > Why would I spend time on Mahout if I can't figure out whether it's > going to go somewhere, somewhere I find interesting or useful? I am > asking myself this more and more! So, in the spirit of meritocracy and to play this out more fully, how would you like it to work? Show me how it would work. What should the plan be to get 0.6 out? Perhaps if we do that exercise, we can see how this would all go. > > >> In other words, I think we already do most of this. We just don't have >> enough people doing the actual work to accomplish all that we aim to >> accomplish. A project plan isn't going to change that. > > I think a bit more proactive project management would reveal this to > be true, indeed: yes, not nearly enough hands to make the current > scope work. Let's say we even agree on a new, right scope. How do we > label and communicate key tasks and issues? what's the framework for > agreeing those? how do we know when our rough plan is working or not? > Without some kind of loose plan, and trying to take actions for that > plan and not take actions against that plan... can we really be said > to have agreed on anything? I think I laid out my framework before, which is how it works now but with people actually being active or we spend some time encouraging new committers. I guess, however, I would ask you these questions right back. What would you like to see here? Show me the process (and it need not be perfect) that you think would work, as it sounds like you have something in mind. > > >> As for JIRA as post it notes, I think it's the only way we get reliable >> contributions from the community that are easily surfaced, findable and >> actionable. Everything else you propose above (Github, mailing list, etc.) >> does not work for that and some actually make the problem harder (Github in >> particular, if the author accepts pull requests from others). It is also >> the only way we can reliably know that patches, in what ever state, are >> clearly marked as being donated to the ASF should we choose to incorporate >> them. > > (I am not claiming patches should be submitted from Github -- I'm > suggesting that they end up in JIRA when ready.) I do just mean we > should distinguish the ideas, playground, experiments, offerings, > contributions as something different from the bits that someone's > more-or-less committed to take on, brush up, commit and maintain. OK. I would submit we already do. Ideas/playgrounds/experiments are marked in JIRA as new features with no versions. They are put on branches or Github. Stuff for versions are marked as such in JIRA and go on trunk. > > I think it's JIRA for the latter and whatever else you want for the > former. Is it that you just want JIRA for both? sure, I bet that's > workable too. How would it look in JIRA? Open, unversioned issues represent all the other stuff, versioned, in progress represents the next 1 or 2 releases. > > >> I don't think you are being anti-community. For the most part, I think you >> are just creating work for yourself. Secondly, I think you are delivering >> news that discourages contributions at a later date. That being said, I'm >> not going to stop you from doing it. I will simply reopen the ones I still >> want to keep, even if they only ever get done in my optimal world. > > I don't think it's necessarily good that you re-open them -- meaning, > I think we should agree on whether they should be open or not, > ideally. I suppose, but I essentially see it as that was what that process was. You made a blanket choice to close issues w/o "asking" and I made a choice to reopen certain ones. To me, it's the most efficient way to do so, as everything else degenerates into long discussions. > > Why do we want to reopen this hypothetical issue that nobody looked at > for 9 months? Sure, it seems a shame to think we won't do it. But I'd > say: it's shame it hasn't been done, yes. It's not helpful to hope it > might happen, and advertise to everyone that it will, since nothing > has changed. We should at some stage admit it hasn't been done, and > admit it won't be done unless something changes. We can then ask > whether we want to change something to do it, or not. I disagree that > this actually discourages anyone -- see below. > > I know, that's exaggerated. But I hope you can see the kernel of truth > in the sentiment. I get where it's coming from and think your POV is perfectly reasonable. I also know I've seen it happen on more than one occasion both here and elsewhere. ARFF support is an example here. Countless times in Lucene, Solr, etc. > > >> Practically speaking, I guess my proposal would be that we simply move to a >> mark and sweep model with every release that: >> 1. Leaves open everything by default with no version number >> 2. Moves all unfixed items for a specific version to the next one unless >> otherwise marked elsewhere. This way we have a list that we can refer to. >> 3. Marks any open ones that we deem to be DOA as "Won't Fix" > > That kind of describes what's happening now, and I am claiming this > doesn't quite work. You have open issues that may, or may not, ever go > into a version -- are they "real"? worth working on? if I do some work > and submit a patch will anyone care? You commit to releasing some for > a particular version -- unless they don't make it into that version, > in which case we commit to the next version, unless that doesn't > happen either. Why bother labeling versions then, for example? that's > not actually planning anything. > > Yes, I always comment in JIRAs that I close that, hey, there's nothing > wrong with the idea and anyone's welcome to resurrect it. > It's never happened! Pointing back to above, I'm open to trying a plan given more concrete details. How about we put together a proposal for 0.6 w/ tasks for (hypothetical) people to do? Or perhaps even the next release after that. What does that look like? > > >> I definitely agree w/ reining in scope. Ironically, doing a cull of code is >> actually increasing scope in the short term since now we need to do that as >> well as clean up issues, add features, etc. I'm for such a cull, but I >> suspect it will be less culling and more reorganization. I'd simply suggest >> that instead of trying to do some massive coordination of such an effort, >> you (again, the "royal" you) simply open up jira issues, attach a patch and, >> when you are satisfied w/ the patch, you indicate on the issue that you will >> commit within 3 days unless you hear objections. Three days later, do the >> commit. My first suggestion would be Watchmaker! > > Sounds good! my first suggestion would be for more people to get > involved in this. > > I tallied up the commit count recently and I think I am having too > much influence on the project by commit volume. It's not good per se, > and makes me play a bit of bad cop that deletes code, closes issues. I'm totally grateful for all that you do. My experience is that this stuff ebbs and flows. > > I don't ask anyone for more time. Everyone's already spending as much > time as is fun. Spending unenjoyable time isn't sustainable. I am more > asking to direct the time available a little differently. Fair enough. So, let's see what a proposal for such a thing would look like. As I don't know. I mean, I do know what a project plan looks like in closed source, but I don't in open source where you can't reliably know who is going to show up. But, like I said, I'm willing to go through the process.
