Hi Jorge,
> The results are a bit surprising: reading 2^20 rows of 3 byte strings is
> ~6x faster than the official Avro Rust implementation and ~20x faster vs
> "fastavro"
This sentence is a little bit hard to parse. Is a row of 3 strings or a
row of 1 string consisting of 3 bytes? Was the
Rust allows to easily swap the global allocator to e.g. mimalloc or
snmalloc, even without the library supporting to change the allocator. In
my experience this indeed helps with allocation heavy code (I have seen
changes of up to 30%).
Best regards,
Daniël
On Sun, Oct 31, 2021, 18:15 Adam
+1 (binding).
Verified on Ubuntu 20.04.3 LTS.
On Fri, Oct 29, 2021 at 8:07 AM Andrew Lamb wrote:
> Hi,
>
> I would like to propose a release of Apache Arrow Rust Implementation,
> version 6.1.0.
>
> This release candidate is based on commit:
> 03d4f4cd3f4cdf39d97a1028471bd9cfbed9 [1]
>
>
Hi Jorge,
Just an idea: Do the Avro libs support different allocators? Maybe using a
different one (e.g. mimalloc) would yield more similar results by working
around the fragmentation you described.
This wouldn't change the fact that they are relatively slow, however it
could allow you better
Hi,
I am reporting back a conclusion that I recently arrived at when adding
support for reading Avro to Arrow.
Avro is a storage format that does not have an associated in-memory format.
In Rust, the official implementation deserializes an enum, in Python to a
vector of Object, and I suspect in
