On Sat, Jun 20, 2020 at 3:19 PM Adam Lippai <a...@rigo.sk> wrote: > > I've seen better and worse examples before. > I was an active, beginner Drupal developer ~12 years ago. The Drupal > project community was very strong, particularly in Hungary where I live. > International and local IRC channels, international and local > forums+events, highly customized issue tracker and superb documentation. It > was more mature and bigger that time. On the other hand when I tried to > give back to Angular or React... Well... You are already ahead of them. > React eventually recognized the problem and they try to solve it, but a > large company's bureaucracy doesn't help that. > > My experience with Arrow is aligned with my expectations of a project of > this age or size (and in a few fields you are awesome!). Andy Grove, > xhochy, wesm, Joris were welcoming and responsive on Jira, Twitter and this > mailing list too. Ofc nobody worked for free on my ideas and I can't > develop C++ or Rust alone (yet). What I can do now is tracking the > development, the PRs (I've added a few more or less valuable, but not so > unique comments) and I'm subscribed to a few Jira issues. > > At this point I could use a gitter/IRC/slack channel for discussions - with > peers instead of core devs - and using mailing list + JIRA doesn't help > either. They are simply cumbersome, hard to navigate/search, focus is lost > when somebody is not sure what's interesting. A simpler issue tracker (eg > GitHub issues) and a super simple forum instead of mailing list would lower > the barriers. I don't think this is a priority as this setup certainly > serves your current workflows.
On this I will say: we used to have a Slack channel but it didn't work well. Only a few core developers ever looked at it and because of the general "Slackification" of open source a lot of people would join the Slack channel looking for help and be unable to get it. People also reported bugs in Slack and we would learn about them weeks after the fact, or never. I think if we added a new official communications channel for the project right now it would likely suffer the same fate. If we had 10x as many core developers then there might be enough core devs who are comfortable with the additional modality that it might make sense. We still have lots of people reporting bugs on Stack Overflow and very few core developers regularly look at the SO questions. By contrast, we nearly unfailingly respond to people on the mailing list and JIRA. So if people are looking for help they can certainly get it there. > Keep up the good work, you are amazing! I can't wait a more complete > DataFusion, group by and join for pyarrow and other dozen exciting > opportunities and features. > > tl;dr you are great, not behind, local communities/meetups are a good > opportunity (but covid...), I find Jira + mailing list hard to use > (mentally, as not core dev) > > Best regards, > Adam Lippai > > > > On Sat, Jun 20, 2020, 21:23 Wes McKinney <wesmck...@gmail.com> wrote: > > > On Sat, Jun 20, 2020 at 1:52 PM Neal Richardson > > <neal.p.richard...@gmail.com> wrote: > > > > > > Hi Suvayu, > > > Thanks for your feedback. I'm sorry to hear that you feel that you > > haven't > > > had the best experiences trying to contribute to the project. For what > > it's > > > worth, I believe that raising concerns like this _is_ itself a valuable > > > contribution. So even if you haven't gotten to the point of having a pull > > > request merged, I don't think it's accurate to say that you've been > > trying > > > unsuccessfully to contribute--you're contributing right now. > > > > > > As it turns out, just the other day I opened a JIRA issue about improving > > > the contributor guide (https://issues.apache.org/jira/browse/ARROW-9189 > > ), > > > and I'll be taking that up next week as part of our 1.0 website > > overhaul. I > > > agree that we can do a better job in helping new contributors > > participate, > > > and that many of those forms of contribution need not require lots of > > time > > > from Arrow core developers. Wes's point about the limited bandwidth to > > > provide mentorship is valid; that said, I've seen many successful cases > > of > > > first-time contributors getting the support they need. While there's > > > certainly room for improvement, I'm optimistic that we're on the right > > > track. > > > > Yes — to be clear, the core developers in my experience (myself > > included) are spending a lot of time responding to questions on JIRA, > > clarifying issues with issue reporters, and offering advice about how > > to proceed. Additionally, we spend a lot of time reviewing code and > > helping people get their patches ready to be merged. There's no way we > > would have 500+ contributors if we were not doing these things. > > > > As far as getting the help that's needed from core developers, the > > thing that helps someone like me the most is to have the "request" be > > as specific and direct as possible. In any given day I might look at > > 50-100 different issues and so if it's not clear what I need to do I > > will often move on to the next thing. Example direct requests: > > > > * Do you think $PROPOSED_APPROACH is the right one? > > * In which file(s) should I be looking to make changes? > > * Is there anything related in the codebase I can look at to learn? > > > > I'm sure we can put this advice in our contributor guide. > > > > If you ask these questions and do not get an answer, it is OK to ask again. > > > > I see six JIRA issues from Suvayu in the project > > > > * https://issues.apache.org/jira/browse/ARROW-1956 > > * https://issues.apache.org/jira/browse/ARROW-3806 > > * https://issues.apache.org/jira/browse/ARROW-4930 > > * https://issues.apache.org/jira/browse/ARROW-3792 > > * https://issues.apache.org/jira/browse/ARROW-3874 > > * https://issues.apache.org/jira/browse/ARROW-6577 > > > > There are comments in all cases and the issues were resolved in 4 out > > of 6 cases. I see one example of you asking for guidance > > (https://issues.apache.org/jira/browse/ARROW-1956) on December 29, > > 2017 while I (and presumably others) were on vacation for the New > > Year. In the future, it is OK to be more persistent. > > > > Thanks > > > > > Neal > > > > > > > > > On Sat, Jun 20, 2020 at 11:25 AM Suvayu Ali <fatkasuv...@gmail.com> > > wrote: > > > > > > > Hi Wes, others, > > > > > > > > Thank you for taking the time to draft a long response. > > > > > > > > On Sat, Jun 20, 2020 at 3:57 PM Wes McKinney <wesmck...@gmail.com> > > wrote: > > > > > > > > > > From a purely factual view, the project is successfully attracting > > and > > > > > supporting contributors. Over 500 different people have contributed > > to > > > > > the project (more than the "420" printed on GitHub because many > > people > > > > > use e-mail addresses not associated with their GitHub user names) and > > > > > that number is increasing steadily over time. > > > > > > > > This response reinforces one of my points, all this branch name change > > > > business then has nothing to do with actually getting new > > > > contributors. > > > > > > > > > We have invested greatly in providing systems to support developers > > of > > > > > the project. We have a large and complex CI setup and nowadays it > > > > > works pretty much like clockwork which is a huge change compared with > > > > > a year or two ago. > > > > > > > > Agreed, and I have learned a lot from it just by observing. > > > > > > > > > If you are looking for individualized "mentorship and guidance" > > > > > _beyond_ pointers toward what part of the project you should be > > > > > looking at to solve a problem, feedback on issues about whether or > > not > > > > > something is deemed useful or high priority or not, and feedback on > > > > > your PRs whether you are on the right track or not, I think your > > > > > expectations -- at this stage of the project -- may not be > > reasonable. > > > > > The number of regularly active developers in this project for the > > > > > parts that you have looked at is actually quite small. So you're > > > > > talking about some of the 10 people at the top of the GitHub > > > > > contributor list. It would be different if we were talking about an > > > > > older project with an order of magnitude more regularly active > > > > > developers. > > > > > > > > If pointers to you are: look at the serialisation code, then yes, I > > > > was hoping for more along the lines of look at class XYZ in file bla. > > > > I completely understand if that's not possible. That is why I never > > > > said anything before. You may not remember, during the "whether to > > > > support wheels" discussion, as I was impacted, I offered a compromise > > > > of releasing a reduced feature-set wheel with simpler dependencies, > > > > which was rejected with this exact argument. I did not counter, > > > > because it is a very reasonable position to take, and I'm in no > > > > position to "demand" anything. > > > > > > > > I only wrote today because I felt maybe now there is a willingness for > > > > newer, diverse contributors, because that's how this thread was > > > > motivated. So I stated the hurdles I have faced, and hoped instead of > > > > wasting scarce resources on superficial changes the community could > > > > address actual hurdles for new contributors like me. Obviously I > > > > misunderstood. > > > > > > > > > The area where I think we could improve the most is developer > > > > > documentation, which in a sense is "self-service guidance" in > > > > > understanding the codebases. Antoine and others have taken initiative > > > > > on this but it often goes by the way side since the number of people > > > > > with requisite knowledge to write it is small (countable on fingers > > > > > and toes if you include all the programming languages) and very short > > > > > of free cycles. > > > > > > > > I'm guessing you mean the Sphinx docs? Whatever I have managed to use > > > > Arrow for, it's thanks to those. Maybe that is my cue, when hitting a > > > > dead-end, "I should ask which source file do I look in?" > > > > > > > > Anyway, I don't want to waste anyone's time anymore. I felt there's > > > > room for feedback, I was wrong, and I withdraw from this discussion. > > > > I'll continue to lurk on the mailing list, and try to contribute when > > > > I can. > > > > > > > > Cheers and thanks for your time, > > > > > > > > -- > > > > Suvayu > > > > > > > > Open source is the future. It sets us free. > > > > > >
