Thanks for pointing this out. I'd be supportive of the same type of policy for Arrow (unversioned "revolution" branches -- or separate git repositories -- where development happens in a less regimented fashion — i.e. without pull requests, issues, etc., but with periodic written community updates).
Keep in mind using https://selfserve.apache.org we can create new apache/* git repositories anytime (so we could create apache/arrow-rust2, for example) that wouldn't have the same kind of development process as trunk. On Thu, Apr 8, 2021 at 11:08 AM Julian Hyde <jh...@apache.org> wrote: > > It is a propos to mention "rules for revolutionaries", a way of making > big changes in projects by forking. It was proposed by James Duncan > Davidson, then PMC chair of Tomcat, in 2000, but remains popular > within the ASF (as evidenced by this blog post [1] from 2020 by > Bertrand Delacrétaz, ASF board member). > > Julian > > [1] > https://grep.codeconsult.ch/2020/04/07/rules-for-revolutionaries-2000-edition/ > > On Thu, Apr 8, 2021 at 7:53 AM Adam Lippai <a...@rigo.sk> wrote: > > > > Didn't this happen in the thread "[Rust] [Discuss] proposal to redesign > > Arrow crate to resolve safety violations" on 7th February before any commit > > in arrow2 (resulting in zero discussion or any objection)? > > > > Best regards, > > Adam Lippai > > > > On Thu, Apr 8, 2021 at 4:41 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > Hi Ben, > > > > > > I’m not suggesting adding any extra development process to slow things > > > down > > > on an experimental project like this. My principle objection is to the > > > lack > > > of discussion on the public record. > > > > > > A short DISCUSS email from Jorge explaining the project and soliciting > > > feedback from the community would have been enough. It would have also > > > been > > > an opportunity to set up auxiliary git repositories or branches to > > > accommodate the work on ASF channels (to enable collaboration inside the > > > community) rather than occurring as a “lone wolf” project outside the > > > community. > > > > > > On Thu, Apr 8, 2021 at 9:03 AM Benjamin Blodgett < > > > benjaminblodg...@gmail.com> > > > wrote: > > > > > > > Wes, > > > > > > > > I think we understand your administrative prerogative. I think in both > > > > Julia and Rust cases, you just have engineers that want to go fast and > > > > do > > > > very needed, deep changes for security and performance. I think, and > > > this > > > > is just my wild guess, that to go through the Apahce process (because > > > it's > > > > slow and administered by part timers) for such exploratory work would be > > > > detrimental, because it would slow said communities down. > > > > > > > > I think we should let them do their 'exploratory development branches', > > > and > > > > commit/merge back whatever they want to/feel they have got to a material > > > > contribution/end point. I think you did something similar with Pandas2 > > > and > > > > feather/arrow. > > > > > > > > Either way, we should be praising and encouraging Jorge to go fast and > > > see > > > > it as a development branch, and figure out how to remerge as we see the > > > > value of the new features. > > > > > > > > I think at this point, they are going and we should just encourage them > > > and > > > > realize there is enough support in the Rust and Julia community to deal > > > > with the merge down the road post rewrite. I do see Jorges Arrow2 is > > > > substantially better in many vectors and solves a lot of problems that > > > have > > > > been plaguing Arrow. Also we need a fundamental rewrite of APIs to make > > > a > > > > rust version of Zero copy work, which could mean we have to have a > > > slightly > > > > different arrow format in how it interacts with memory. > > > > > > > > " I do think that I was vocal enough. At some point > > > > the interactions here started to affect my wellbeing and I thus decided > > > to > > > > scale down by efforts." > > > > > > > > On Thu, Apr 8, 2021 at 6:03 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > > > > On Thu, Apr 8, 2021 at 7:49 AM Wes McKinney <wesmck...@gmail.com> > > > wrote: > > > > > > > > > > > > With both what has occurred with the Julia project and what may > > > > > > possibly be occurring (what appears to me to be occurring) with > > > > > > these > > > > > > Rust overhaul projects, is that the communities expectations with > > > > > > regards to Openness aren't being followed. > > > > > > > > > > > > If a change is significant and will affect other developers in the > > > > > > community, such as a large refactor or a large PR that will > > > > > > interfere > > > > > > with development in some portion of the codebase (example, I wrote a > > > > > > large and disruptive C++ PR last may that affected all of our > > > > > > compute > > > > > > kernels), then it needs to be discussed in a public place as soon as > > > > > > you are aware of it. The Openness tenet is about an obligation that > > > > > > individuals have to communicate with their fellow collaborators. > > > > > > > > > > For the record, the reason I'm making a fuss about this is that a > > > > > contributor proposed a contribution moratorium in the Apache project > > > > > as the result of work happening outside the community > > > > > > > > > > "would like to raise the idea of temporarily suspending major PRs > > > against > > > > > Rust > > > > > Arrow/DataFusion until the work to incorporate the two big changes for > > > > > Rust/DataFusion: > > > > > > > > > > 1. Jorge's major refactor/rewrite of the core Rust Arrow code. > > > > > ... > > > > > " > > > > > > > > > > I'm not trying to make things difficult for you all, but this looks > > > > > like the kind of thing we would like to avoid. > > > > > > > > > > > Jorge has said that "that [the community] is unwilling to change > > > > > > some > > > > > > of its processes" — if changes would involve allowing development to > > > > > > happen in private or outside the community (as what has occurred > > > > > > with > > > > > > Julia), it is simply not possible to accept these changes. We have > > > > > > said repeatedly that we would accommodate Julia's needs with respect > > > > > > to releasing. > > > > > > > > > > > > The discussions that we have had recently have centered around > > > > > > communication questions. > > > > > > > > > > > > * Mailing list discussions serve to raise awareness around matters > > > > > > of > > > > > > importance to individuals in the community > > > > > > * Issues serve to raise awareness around prospective or in progress > > > > > > projects, and to help document the community's labors > > > > > > > > > > > > Every PMC member has an obligation to ensure that communications in > > > > > > the project remain healthy and in concordance with the Apache Way so > > > > > > that we do not devolve into a small cabal of developers who > > > understand > > > > > > what is going on, while anyone outside the cabal can't understand > > > just > > > > > > by reading our public communication. Other open source projects do > > > > > > that, but we don't do it here. > > > > > > > > > > > > If the Rust developers collectively want to move everything to a > > > > > > different GitHub repository and use GitHub issues, let's go ahead > > > > > > and > > > > > > do it and live with the consequences. But these principles above > > > > > > simply must be followed. > > > > > > > > > > > > On Thu, Apr 8, 2021 at 2:32 AM Antoine Pitrou <anto...@python.org> > > > > > wrote: > > > > > > > > > > > > > > On Thu, 8 Apr 2021 00:26:57 -0700 > > > > > > > Julian Hyde <jhyde.apa...@gmail.com> wrote: > > > > > > > > Antoine, > > > > > > > > > > > > > > > > I need to correct your assertion > > > > > > > > > > > > > > > > > we develop on the side every day when we submit PRs from > > > > > > > > > forks; > > > > > > > > > it's just a matter of how much complexity is being submitted > > > > > > > > > at > > > > > once > > > > > > > > > > > > > > > > Intuitively, there seems to be a continuum between a PR > > > > > > > > developed > > > > > within a project to a major feature/codebase developed outside of it. > > > But > > > > > the ASF treats these contributions differently, and requires the > > > > > latter > > > > > kind have to go through the IP clearance process. > > > > > > > > > > > > > > > > You might wonder where the line lies. The policy [1] defines it > > > as > > > > > follows: > > > > > > > > > > > > > > > > > a substantial contribution that was not developed within the > > > > > > > > > ASF's source control system and on our public mailing lists > > > > > > > > > > > > > > Well, a PR developed in a fork was not developed within the ASF's > > > > > > > source control system. > > > > > > > > > > > > > > (and most PRs to Arrow constitute substantial contributions in my > > > > > > > experience) > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > >
