Re: Review request for ARROW-7808's PR (Dataset Java API)

I did another review mostly focused on the new allocation code. Thank you, I think this is close to mergeable once comments are addressed. If other Java committers/contributors want to have a look, please do so soon. On Thu, Jan 28, 2021 at 10:14 PM Hongze Zhang wrote: > Hi All, > > > Sorry to

Re: [Rust] Proposed PR Merge Guidelines

I'd echo Wes's broader point and also potentially encourage downstream projects to run CI against the master branch for catching breakages. I also think the initial set of recommendations do make sense for some sort of changes (when code does break APIs so more members can weigh in). Also

Re: java arrow: memory management with multiple references to same batch

I recommend that you direct these questions to u...@arrow.apache.org (https://mail-archives.apache.org/mod_mbox/arrow-user/). On Fri, Jan 29, 2021 at 7:07 AM Joris Peeters wrote: > > Hello, > > I'm writing an HTTP server in Java that provides Arrow data to users. For > performance, I keep the

Re: 3.0 release announcement blog post: contributions needed!

The Rust contributors have been working on a Rust highlights blog post [1] and I will convert this into a PR over the weekend. [1] https://docs.google.com/document/d/1HfdmVE9iQcYlWyfk68BN2PcIjAAhRFpuPLsZW6ffy3Y/edit?usp=sharing On Fri, Jan 29, 2021 at 4:23 PM Neal Richardson wrote: > Hi all, >

3.0 release announcement blog post: contributions needed!

Hi all, Sending this again: I've started a release announcement blog post: https://github.com/apache/arrow-site/pull/92 Please help fill in the sections for the various languages in the project so that we can publish this. Neal

Re: [Rust] Proposed PR Merge Guidelines

When it comes to downstream projects, it may make sense to implement some integration tests that can be triggered via Crossbow if you aren't sure whether a change will cause breakage. On Fri, Jan 29, 2021 at 1:25 PM Andrew Lamb wrote: > > Micah, it is a great question. > > I often find myself

Re: Computational Kernels: the project overview

Hello! I am trying to use the expression and compute APIs for query processing, and in my searches so far, this thread seems to be the most relevant. A lot of the operators and functions that I need in the short-term appear to be implemented, but the documentation seems sparse or at least not

Re: [Rust] Proposed PR Merge Guidelines

Micah, it is a great question. I often find myself thinking "is this PR ok to merge" and going by gut feel of if it has sufficient review and consensus. I think most of the time these decisions are sound, but at least once it was not (that particular instance I think has been since sorted

Re: [Rust] Proposed PR Merge Guidelines

One of the challenges that we face in the Rust implementation is that some parts of the project, especially DataFusion, are still moving fast, with frequent breaking changes, and because there are now multiple projects that depend on it, we need to be thoughtful about the impact of these changes

Re: [Rust] Proposed PR Merge Guidelines

Just curious what is driving the formalization of a policy? Have Rust contributions been having issues? We don't have a 2 reviewer requirement for any of the other languages as far as I know. And committers generally use their judgement if a second reviewer is necessary. Same question about

Re: [Rust] Proposed PR Merge Guidelines

Hi, Thanks a lot for writing. I like it very much. Some follow ups / clarifications: Do we differentiate between who PRed? I.e. if it is a committer, do they count for the approval or in that case do we need 2 approvals? I am more favourable to require a second approval as usual, with the idea

Re: [Rust] Proposed PR Merge Guidelines

Thanks for writing this up, Andrew. I think this looks good. One challenge for me, and I assume I am not alone, is that I generally only have time at weekends to review non-trivial PRs. I don't think there is a good solution to this problem but I will comment on the PRs that I have a particular

java arrow: memory management with multiple references to same batch

Hello, I'm writing an HTTP server in Java that provides Arrow data to users. For performance, I keep the most-recently-used Arrow batches in an in-memory cache. A batch is wrapped in a "DataBatch" Java object containing the schema and field vectors. I'm looking for a good memory management

[Rust] Proposed PR Merge Guidelines

One of the items that we discussed at Wednesday's Rust sync was "what is the criteria to merge a Rust PR". There was no conclusion at the meeting, but there was a proposal which we would like to discuss on the mailing list. *Goal*: Keep Arrow Rust PRs flowing in a timely fashion, thereby keeping

[NIGHTLY] Arrow Build Report for Job nightly-2021-01-29-0

Arrow Build Report for Job nightly-2021-01-29-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-01-29-0 Failed Tasks: - conda-linux-gcc-py36-aarch64: URL: