Re: [EXTERNAL] Re: [C++][Python]Built-in GRPC health checks in FlightServerBase

Le 04/10/2022 à 00:54, Akshaya Annavajhala (AK) a écrit : Thanks for the important clarification - reading up on UCX, it makes sense to implement a health check abstraction. A couple follow up musings: 1. "Hosting" vs "data plane" communication protocols. Clearly this isn't something worth b

RE: [EXTERNAL] Re: [C++][Python]Built-in GRPC health checks in FlightServerBase

Thanks for the important clarification - reading up on UCX, it makes sense to implement a health check abstraction. A couple follow up musings: 1. "Hosting" vs "data plane" communication protocols. Clearly this isn't something worth bundling in right now, but it seems to me that health checks f

Re: [C++][Python]Built-in GRPC health checks in FlightServerBase

Hi Akshaya, Thanks for chiming in. At first glance, I think this is a reasonable use case to cover. However, Flight does try to be independent of gRPC. So maybe the approach would be to define a "Flight health check" that would map to gRPC health check when using gRPC, and something else when n

RE: PyArrow build problem

To update others who might find this thread: If you start running into Arrow building problems upon updates to the Arrow codebase, I recommend first deleting stale build artifacts from previous builds: https://arrow.apache.org/docs/dev/developers/python.html#deleting-stale-build-artifacts .

[C++][Python]Built-in GRPC health checks in FlightServerBase

Hello, I was helpfully redirected by David from [ARROW-17920] Built-in GRPC health checks in FlightServerBase - ASF JIRA (apache.org) to the mailing list to discuss how a consumer of FlightServerBase might meaningfully implement probe endpoints

[RESULT][VOTE][RUST] Release Apache Arrow Rust 24.0.0 RC1

With 7 +1 votes (3 binding) the release is approved The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-24.0.0 It has also been published to crates.io Thank you all ! Andrew On Sun, Oct 2, 2022 at 6:46 PM Ashish wrote: > +1 (non-binding) > > verified o

Re: [DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

Hi all, Le 03/10/2022 à 17:03, Will Jones a écrit : Hi Rusty, Note we discussed Iceberg a while ago [1]. I don't think we've discussed Hudi in any depth. As I see it, we are waiting on three things: 1. Someone willing to move forward the Iceberg / Hudi integration. 2. The Iceberg and Hudi p

[Rust] Planning to cut DataFusion 13.0.0 RC 1 this week

I am planning to cut DataFusion 13.0.0 RC 1 this Friday, October 7th. I have created a tracking issue [1] to coordinate the release. If there are PRs that you feel are important to get into this release, we can discuss in this issue. Thanks, Andy. [1] https://github.com/apache/arrow-datafusion/

Re: [DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

I wanted to chime in that a current long term goal I am working towards is a Golang iceberg implementation that will also integrate with the Golang Arrow modules. I'm not sure how much desire there is for it, but I do know at least two consumers that would greatly benefit from it. But, at least fo

Re: [DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

Hi Rusty, Note we discussed Iceberg a while ago [1]. I don't think we've discussed Hudi in any depth. As I see it, we are waiting on three things: 1. Someone willing to move forward the Iceberg / Hudi integration. 2. The Iceberg and Hudi projects need native libraries that we can use. The base i

Re: [DISCUSS] Python Wheel Size

Hi Rusty, Le 02/10/2022 à 22:51, Rusty Conover a écrit : Hi Arrow Team, I'm using Apache Arrow with AWS Lambda Functions. The primary motivation is AWS Athena's user-defined functions[1]. Those functions process and return Arrow IPC segments. * The published Python wheels for Apache Arrow

[DISCUSS] Apache Iceberg / Apache Hudi support in Arrow

Hi Arrow Team, Arrow is fantastic for manipulating the Parquet file format. There is an increasing desire to have the ability to update, delete and insert the rows stored in Parquet files, but without rewriting the Parquet files in their entirety. It is not uncommon to have gigabytes/petabytes o

[DISCUSS] Python Wheel Size

Hi Arrow Team, I'm using Apache Arrow with AWS Lambda Functions. The primary motivation is AWS Athena's user-defined functions[1]. Those functions process and return Arrow IPC segments. * The published Python wheels for Apache Arrow include almost every feature of Arrow. (Gandiva, Plasma, Fligh

Re: [RELEASE] Preparing for Arrow Release version 10.0.0

Hi, An update around the next release. The plan is still to start the preparing the first Release Candidate in ~2 weeks. As a reminder this means we will be creating a new branch for 10.0.0 with a feature freeze and we will only be merging fixes for the Release. The initial blog post for the arro