Antoine, is my synopsis fair? On Thu, Oct 10, 2019 at 12:53 PM Wes McKinney <wesmck...@gmail.com> wrote:
> +1 > > On Thu, Oct 10, 2019, 2:12 PM Jacques Nadeau <jacq...@apache.org> wrote: > > > Proposed report update below. LMK your thoughts. > > > > ## Description: > > The mission of Apache Arrow is the creation and maintenance of software > > related to columnar in-memory processing and data interchange > > > > ## Issues: > > > > * We are struggling with Continuous Integration scalability as the > project > > has > > definitely outgrown what Travis CI and Appveyor can do for us. Some > > contributors have shown reluctance to submit patches they aren't sure > > about > > because they don't want to pile on the build queue. We are exploring > > alternative solutions such as Buildbot, Buildkite, and GitHub Actions > to > > provide a path to migrate away from Travis CI / Appveyor. In our > request > > to > > Infrastructure INFRA-19217, some of us were alarmed to find that an > CI/CD > > service like Buildkite may not be able to be connected to the @apache > > GitHub > > account on account of requiring admin access to repository webhooks, > but > > no > > ability to modify source code. There are workarounds (building custom > > OAuth > > bots) that could enable us to use Buildkite, but it would require extra > > development and result in a less refined experience for community > > members. > > > > > > > > ## Membership Data: > > * Apache Arrow was founded 2016-01-19 (4 years ago) > > * There are currently 48 committers and 28 PMC members in this project. > > * The Committer-to-PMC ratio is roughly 3:2. > > > > Community changes, past quarter: > > - Micah Kornfield was added to the PMC on 2019-08-21 > > - Sebastien Binet was added to the PMC on 2019-08-21 > > - Ben Kietzman was added as committer on 2019-09-07 > > - David Li was added as committer on 2019-08-30 > > - Kenta Murata was added as committer on 2019-09-05 > > - Neal Richardson was added as committer on 2019-09-05 > > - Praveen Kumar was added as committer on 2019-07-14 > > > > ## Project Activity: > > > > * The project has just made a 0.15.0 release. > > * We are discussing ways to make the Arrow libraries as accessible as > > possible > > to downstream projects for minimal use cases while allowing the > > development > > of more comprehensive "standard libraries" with larger dependency > stacks > > in > > the project > > * We plan to make a 1.0.0 release as our next major release, at which > time > > we > > will declare that the Arrow binary protocol is stable with forward and > > backward compatibility guarantees > > > > ## Community Health: > > > > * The community is continuing to grow at a great rate. We see good > adoption > > among many other projects and fast growth of key metrics. > > * Many contributors are struggling with the slowness of pre-commit CI. > > Arrow > > has a large number of different platforms and components and a complex > > build > > matrix. As new commits come in, they frequently take a long time to > > complete. The community is trying several ways to solve this. There is > > bubbling frustration in the community around the GitHub repo rules for > > using > > third party services. This is especially challenging when there are > free > > solutions to relieve the community pressure but the community is unable > > to > > access these resources. This frustration is greatest among people who > > work > > on many non-asf OSS projects which don't have such restrictive rules > > around GitHub. Some examples of ways the community has tried to > resolve > > these have included: > > * Try to use CircleCI, rejected in INFRA-15964 > > * Try to use Azure Pipelines, rejected in INFRA-17030 > > * Try to resolves Issues with Travis CI capacity: INFRA-18533 & > > https://s.apache.org/ci-capacity (no resolution beyond "find > > donations") > > * The creation of new infrastructure design (in progress but a huge > > amount of > > thankless work) > > * While the community has seen great growth in contribution (more than > 300 > > unique contributors at this point), the vast majority are casual > > contributors. The daily active committers (the workhorses of the > project > > that bear the load committing the constant PRs, more than 5000 closed > at > > this point) have been growing slower than adoption. This is despite the > > fact > > that the community has been very aggressive at being inclusive of new > > committers (with likelihood to have more than 50 in the next week). The > > community is still continuing to try to brainstorm ways to improve > this. > > >
