I suggest we respect Jorge's decision and give him more time/space. Considering he spent 6 months last year trying to donate arrow2 to ASF and couldn't get it through, I can see why he doesn't want to retry the same effort in a short period of time. Especially when he is working on all of these in his limited free time. I trust him to make decisions that will best push the project forward on a technical front.
As for arrow-rs, I think Andy made a good comment in the github issue suggesting that if there are community members who are motivated to maintain it, we can keep it going but probably with a less frequent release cadence. I agree with Micah that If no one is willing to take up the work, then a clear communication of maintenance only mode would better serve the existing arrow-rs users. On Wed, Jan 19, 2022 at 9:25 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > > Hi Jorge, > > Thanks for your response. > > My current hypothesis is that arrow2 will be donated to Apache Arrow, I > > just don't feel comfortable and have the energy doing so right now. > > Aside from the burden of going through the donation process (maybe other > PMCs members are willing to help here?), what would make you comfortable > with the donation? Are there specific milestones you are waiting for? If > the docs on the arrow2 repo [1] are up-to-date it seems like the main > concern is versioning, and sticking faithfully to sem-ver. It sounds like > Rust is going to move out of lock-sync versioning with the > other implementations. So maybe the gaps are small? > > There are advantages and disadvantages to having a project hosted by the > ASF, so I can understand that it might not be an easy decision. However, > as more development is done outside the ASF additional work for IP > clearance accumulates if the project ends up getting donated. This seems > like it might lead to a cycle where the timing is never quite right. > > Separately, from an outsider perspective on Rust development, I think a > community over code approach applies here. If arrow2 is decided to be the > best path forward for DataFusion, continuing to expend effort on > maintaining arrow-rs doesn't seem like a good use of people's time given > the high degree of overlap. I think it is a bad result if arrow-rs ends up > bit-rotting due to lack of maintainers (i.e. I think it would be better to > intentionally deprecate it). > > Hopefully consensus can be reached among the Rust maintainers and we can > have clear messaging for users over the future of both projects. > > Cheers, > Micah > > [1] > https://github.com/jorgecarleitao/arrow2#any-plans-to-merge-with-the-apache-arrow-project > > On Wed, Jan 19, 2022 at 10:19 AM Jorge Cardoso Leitão < > jorgecarlei...@gmail.com> wrote: > > > Hi, > > > > Thank you for raising this here and for your comments. I am very humbled by > > the feedback and adoption that arrow2 got so far. > > > > My current hypothesis is that arrow2 will be donated to Apache Arrow, I > > just don't feel comfortable and have the energy doing so right now. > > > > Thank you for your understanding, > > Jorge > > > > > > On Mon, Jan 17, 2022 at 6:21 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > Sounds good, thanks all and look forward to hearing more about this. > > > > > > To second what Micah said, a reminder to please engage with civility. > > > The ASF's code of conduct is found here [1]. We are all volunteering > > > our time to try to do what is best for the developer and user > > > communities' long-term health and success. > > > > > > [1]: https://www.apache.org/foundation/policies/conduct.html > > > > > > > > > On Mon, Jan 17, 2022 at 6:07 AM Andrew Lamb <al...@influxdata.com> > > wrote: > > > > > > > > For what it is worth, I personally would likely spend much less time > > > > maintaining arrow-rs if datafusion switched to arrow2. That discussion > > is > > > > happening independently here [1]. > > > > > > > > [1] https://github.com/apache/arrow-datafusion/issues/1532 > > > > > > > > On Sun, Jan 16, 2022 at 11:17 PM Micah Kornfield < > > emkornfi...@gmail.com> > > > > wrote: > > > > > > > > > I agree, Jorge's point of view (and anyone else who has contributed > > to > > > > > arrow2) is important here. > > > > > > > > > > One thing that isn't exactly clear to me from the linked issue is how > > > much > > > > > interest there is in the community for maintaining arrow-rs? How > > much > > > is a > > > > > donation of arrow2 a factor here? > > > > > > > > > > Also, the last time we discussed this on the mailing list I think > > > things > > > > > got contentious. Just a reminder please communicate with care. [1] > > > has > > > > > some good advice on this. > > > > > > > > > > Thanks, > > > > > Micah > > > > > > > > > > [1] https://community.apache.org/contributors/etiquette > > > > > > > > > > On Sun, Jan 16, 2022 at 6:51 PM QP Hou <houqp....@gmail.com> wrote: > > > > > > > > > > > Hi Wes, > > > > > > > > > > > > I believe what you mentioned is the plan, i.e. move arrow2 to ASF > > in > > > > > > the long run when it stabalize on its design/API and could benefit > > > > > > from a more rigorous release process. From what I have seen, the > > > > > > project is still undergoing major API changes on a monthly basis, > > so > > > > > > quick releases and fast user feedback is quite valuable. But let's > > > > > > hear Jorge's point of view on this first. > > > > > > > > > > > > On Sun, Jan 16, 2022 at 2:42 PM Wes McKinney <wesmck...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > Is there a possibility of donating arrow2 to the Arrow project > > (at > > > > > > > some point)? The main impact to development would be holding > > votes > > > on > > > > > > > releases, but this is probably a good thing long term from a > > > > > > > governance standpoint. The answer may be "not right now" and > > that's > > > > > > > fine. Having many of the same people split across projects with > > > > > > > different governance structures is less than ideal. > > > > > > > > > > > > > > On Fri, Jan 14, 2022 at 1:15 PM Andrew Lamb < > > al...@influxdata.com> > > > > > > wrote: > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > I wanted to draw your attention to two issues of significance > > to > > > the > > > > > > Rust > > > > > > > > Arrow implementations > > > > > > > > > > > > > > > > Discussion for switching DataFusion to use arrow2: > > > > > > > > https://github.com/apache/arrow-datafusion/issues/1532 > > > > > > > > > > > > > > > > Discussion for what to do with arrow if DataFusion switches to > > > use > > > > > > arrow2: > > > > > > > > https://github.com/apache/arrow-rs/issues/1176 > > > > > > > > > > > > > > > > The second is likely the most pertinent to people on this > > mailing > > > > > > list, but > > > > > > > > the first is the reason why the second has become important. > > > > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > >
