Arrow sync call October 28 at 12:00 US/Eastern, 16:00 UTC

2020-10-27 Thread Neal Richardson
Hi all, Reminder that our biweekly call is coming up at https://meet.google.com/vtm-teks-phx. For those in Europe, note that the time may be an hour different than you're used to due to daylight savings time shift. All are welcome to join. Notes will be sent out to the mailing list afterward.

Re: [ANNOUNCE] New Arrow PMC chair: Wes McKinney

2020-10-27 Thread Kazuaki Ishizaki
Wes, congratulations! Kazuaki Ishizaki, Bryan Cutler wrote on 2020/10/27 09:50:12: > From: Bryan Cutler > To: dev > Date: 2020/10/27 09:50 > Subject: [EXTERNAL] Re: [ANNOUNCE] New Arrow PMC chair: Wes McKinney > > Congrats Wes, well deserved! > > On Sun, Oct 25, 2020, 10:17 PM Jorge

Re: [c++] Futures API review & help understanding benchmark result

2020-10-27 Thread Weston Pace
Please enjoy your vacation! I was able to track it down to the allocations performed when creating futures. Simply changing from Executor.Spawn to Executor.Submit explains nearly half of the performance difference. Perhaps someday an investigation can be done to look at pooled allocators. It

Re: [Rust] Blog post for 2.0.0

2020-10-27 Thread Andy Grove
The Rust blog post is now live: https://arrow.apache.org/blog/2020/10/27/rust-2.0.0-release/ On Sun, Oct 18, 2020 at 12:46 PM Fernando Herrera < fernando.j.herr...@gmail.com> wrote: > Thanks Jorge for helping me to get across the need for a user guide. The > examples you used are exactly what I

Re: [Discuss] Provide pluggable APIs to support user customized compression codec

2020-10-27 Thread Antoine Pitrou
Hi, Le 27/10/2020 à 09:55, Xie, Qi a écrit : > > The HW decompressor can't fall back automatically on SW decompression, but we > can fallback to SW in the HW library. Yes, that's what I meant :-) > How about HW-Gzip as an enhanced Gzip and still use the Compression::GZIP as >

Re: [c++] Futures API review & help understanding benchmark result

2020-10-27 Thread Antoine Pitrou
Le 27/10/2020 à 15:47, Antoine Pitrou a écrit : > > Hi Weston, > > Note: I'm on vacation, so won't be able to look at this before ~2 weeks. > > For information, there's a micro-benchmark of thread pools and task > groups in src/arrow/util/thread_pool_benchmark.cc. It should allow you >

Re: [c++] Futures API review & help understanding benchmark result

2020-10-27 Thread Antoine Pitrou
Hi Weston, Note: I'm on vacation, so won't be able to look at this before ~2 weeks. For information, there's a micro-benchmark of thread pools and task groups in src/arrow/util/thread_pool_benchmark.cc. It should allow you isolate performance concerns a bit better. Regards Antoine. Le

[NIGHTLY] Arrow Build Report for Job nightly-2020-10-27-0

2020-10-27 Thread Crossbow
Arrow Build Report for Job nightly-2020-10-27-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-27-0 Failed Tasks: - gandiva-jar-osx: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-27-0-travis-gandiva-jar-osx -

RE: [Discuss] Provide pluggable APIs to support user customized compression codec

2020-10-27 Thread Xie, Qi
Hi, The HW decompressor can't fall back automatically on SW decompression, but we can fallback to SW in the HW library. How about HW-Gzip as an enhanced Gzip and still use the Compression::GZIP as Compression::type, the end user can through some configurations to enable HW-Gzip instead of