Hi Neville, Thanks for the update. One question, we have "gold" files for 0.14.0 checked into the test-data repo and run integration tests on those to ensure we can read them in a few implementations. Does Rust at least read those correctly?
Thanks, Micah On Fri, Jul 10, 2020 at 1:03 PM Neville Dipale <nevilled...@gmail.com> wrote: > Good day Arrow devs, > > I've spent a few evenings looking into the issues that we're experiencing > with Rust integration testing. > In summary, none of our tests pass (zero batch doesn't count :) ). > This is mainly because of changes from the legacy padding in the 0.15.0 > release, which we never made in Rust ( > https://issues.apache.org/jira/browse/ARROW-6313). > I honestly didn't see this at the time. > > Anyways, the implication is that Rust is using the 'legacy' format of > 4-byte alignment, and fails to parse and read the ipc::Message into > RecordBatches. > > I'm so far struggling with the fixes in Rust, so this work might take long > and likely won't make it to 1.0.0. I'll open JIRAs for whatever work is > necessary. > > The Rust implementation appears to be behind with quite some implementation > details, so I'll try catching up by the next release. > > Regards > Neville >
