Re: [VOTE][RUST] Release Apache Arrow Rust 48.0.0 RC2

2023-10-19 Thread Will Jones
+1 Verified on M1 Mac. Thanks Raphael! On Wed, Oct 18, 2023 at 1:30 PM Andrew Lamb wrote: > +1 (binding) -- thank you Raphael > > Verified on x86 Mac > > Hint for anyone else verifying, this is RC*2* (RC1 hit an issue[1]) > > Andrew > > [1]: https://github.com/apache/arrow-rs/pull/4950 > > On

Re: [Format] C Data Interface integration testing

2023-10-19 Thread Aldrin
try the unsubscribe link at [1].[1]: https://arrow.apache.org/community/ Sent from Proton Mail for iOS On Thu, Oct 19, 2023 at 23:41, Richard Haven wrote: UNSUBSCRIBEBAJARSEANFOSGRIFIADОТПИШИHLOKOMELAOn Thu, Oct 19, 2023 at 9:56 AM Antoine Pitrou wrote:>> Hello

Re: [Format] C Data Interface integration testing

2023-10-19 Thread Richard Haven
UNSUBSCRIBE BAJARSE ANFOSGRIFIAD ОТПИШИ HLOKOMELA On Thu, Oct 19, 2023 at 9:56 AM Antoine Pitrou wrote: > > Hello again, > > Quick update: the C++, C#, Go and Java implementations now all > participate in C Data Interface integration testing. > > (this helped us fix a few interoperability

Re: [Format] C Data Interface integration testing

2023-10-19 Thread Richard Haven
> > > UNSUBSCRIBE >

Re: [Format] C Data Interface integration testing

2023-10-19 Thread Neal Richardson
Nice work! Neal On Thu, Oct 19, 2023 at 4:01 PM Antoine Pitrou wrote: > > Hello again, > > Quick update: the C++, C#, Go and Java implementations now all > participate in C Data Interface integration testing. > > (this helped us fix a few interoperability bugs, and add deterministic >

Re: [Format] C Data Interface integration testing

2023-10-19 Thread Antoine Pitrou
Hello again, Quick update: the C++, C#, Go and Java implementations now all participate in C Data Interface integration testing. (this helped us fix a few interoperability bugs, and add deterministic releasing of imported data in Go) Arrow Rust does not participate yet, but given how

Re: Apache Arrow file format

2023-10-19 Thread Aldrin
And the first paper's reference of arrow (in the references section) lists 2022 as the date of last access. Sent from Proton Mail for iOS On Thu, Oct 19, 2023 at 18:51, Aldrin wrote: For context, that second referenced paper has Wes McKinney as a co-author, so

Re: Apache Arrow file format

2023-10-19 Thread Aldrin
For context, that second referenced paper has Wes McKinney as a co-author, so they were much better positioned to say "the right things." Sent from Proton Mail for iOS On Thu, Oct 19, 2023 at 18:38, Jin Shang wrote: Honestly I don't understand why this VLDB paper [1]

Re: Apache Arrow file format

2023-10-19 Thread Jin Shang
Honestly I don't understand why this VLDB paper [1] chooses to include Feather in their evaluations. This paper studies OLAP DBMS file formats. Feather is clearly not optimized for the workload and performs badly in most of their benchmarks. This paper also has several inaccurate or outdated

Re: Apache Arrow file format

2023-10-19 Thread Roman Shaposhnik
On Wed, Oct 18, 2023 at 11:20 PM Andrew Lamb wrote: > > If you are looking for a more formal discussion and empirical analysis of > the differences, I suggest reading "A Deep Dive into Common Open Formats > for Analytical DBMSs" [1], a VLDB 2023 (runner up best paper!) that > compares and

Arrow R Package Development Sync Call - Thursday 19th October

2023-10-19 Thread Nic Crane
The fortnightly Arrow R package dev community call is on Thursday 19th October at 16:30 UTC (12:30 ET). Video call link: https://meet.google.com/ghw-qfvv-cjb The meeting notes can be found here; please feel free to add items to the agenda:

Re: [VOTE][Format] C data interface format strings for Utf8View and BinaryView

2023-10-19 Thread Joris Van den Bossche
+1 On Wed, 18 Oct 2023 at 23:33, Jonathan Keane wrote: > > +1 > > -Jon > > > On Wed, Oct 18, 2023 at 2:26 PM Felipe Oliveira Carvalho < > felipe...@gmail.com> wrote: > > > +1 > > > > On Wed, Oct 18, 2023 at 2:49 PM Dewey Dunnington > > wrote: > > > > > +1! > > > > > > On Wed, Oct 18, 2023 at

Re: Apache Arrow file format

2023-10-19 Thread Jacek Pliszka
There is a note there explaining what they understand by it but further down the line they do not make such distinction. The fact that parquet can be better in-memory format than arrow for certain common uses is something I haven't thought of and is eye-opening for me, admittedly so because I am