Re: [Rust] Unable to read in Python or JS Arrow Stream IPC files written in Rust

2022-03-08 Thread Micah Kornfield
Hi Kyle, I'm not sure if Rust contributors monitor this list, you might have better luck opening an issue on the Rust Repo [1] [1] https://github.com/apache/arrow-rs On Sun, Feb 27, 2022 at 7:28 PM Kyle Barron wrote: > Hello! > > I've used Arrow a decent bit in Python and JS but I'm pretty new

Re: [Python][Parquet]pq.ParquetFile.read faster than pq.read_table?

2022-03-08 Thread Weston Pace
The issue was a combination of python & C++ so it isn't something we'd see in the micro benchmarks. In the macro benchmarks this regression actually did show up pretty clearly [1] but I didn't notice it in the PR comment that conbench made. Jonathan Keane raised [2] on the conbench repo which

Re: Correct usage of listener.start() when root is shared with FlightStream

2022-03-08 Thread James Duong
We see backpressure related timeouts. I'm thinking there's an issue with CallbackBackpressureStrategy relying on ServerStreamListener#isReady(). I've created https://issues.apache.org/jira/browse/ARROW-15876 for this. We're going to try a fix for this locally then if it helps create a PR. On

Re: [Python][Parquet]pq.ParquetFile.read faster than pq.read_table?

2022-03-08 Thread Wes McKinney
Since this isn't the first time this specific issue has happened in a major release, is there a way that a test or benchmark regression check could be introduced to prevent this category of problem in the future? On Thu, Feb 24, 2022 at 9:48 PM Weston Pace wrote: > > Thanks for reporting this.