Traffic control and cancel detect on flight do_get in python implementation

2023-07-06 Thread Wenbo Hu
Hi, I'm using arrow flight to transfer data in distributed system, but the lightning speed makes both client and server faces out of memory issue. For do_put and do_exchange method, the protocol provides stream metadata reader/writer for client/server exchange control messages along data

Re: [ANNOUNCE] New Arrow committer: Kevin Gurney

2023-07-06 Thread Jacob Wujciak-Jens
Congratulations and welcome! On Wed, Jul 5, 2023 at 10:49 PM Kevin Gurney wrote: > Thank you all for the kind words and warm welcome! > > I feel honored and excited to be part of this vibrant community! I look > forward to continuing to collaborate with all of you! > > Best Regards, > > Kevin

Re: [VOTE][RUST][DataFusion] Release DataFusion Python Bindings 27.0.0 RC1

2023-07-06 Thread Andrew Lamb
+1 (binding) Verified on x86 mac. Thank you Andy for keeping these releases going On Wed, Jul 5, 2023 at 1:13 PM Andy Grove wrote: > Hi, > > I would like to propose a release of Apache Arrow DataFusion Python > Bindings, > version 27.0.0. > > This release candidate is based on commit: >

Re: Webassembly?

2023-07-06 Thread Tim Paine
I can help, we use emscripten-compiled arrow for perspective ( https://github.com/finos/perspective) and we now compile perspective's python side for pyodide so I have an interest in a fully functional pyarrow/pandas in pyodide on an ongoing basis. Tim Paine tim.paine.nyc 908-721-1185 > On

Re: [DISCUSS][Format] Draft implementation of string view array format

2023-07-06 Thread Benjamin Kietzman
@Andrew: Restricting these arrays to a single buffer will severely decrease their utility. Since the character data is stored in multiple character buffers writing Utf8View array can proceed without resizing allocations, which is a major overhead when writing Utf8 arrays. Furthermore since the

Re: Webassembly?

2023-07-06 Thread Antoine Pitrou
Hi Joe, Thank you for working on that. The one question I have is: are you willing to help us maintain Arrow C++ on the long term? The logic you're adding in https://github.com/apache/arrow/pull/35672 is quite delicate; also I don't think anyone among us is a Webassembly expert, which