Re: Status of Rust Integration Testing

Hi Neville, Thanks for the update. One question, we have "gold" files for 0.14.0 checked into the test-data repo and run integration tests on those to ensure we can read them in a few implementations. Does Rust at least read those correctly? Thanks, Micah On Fri, Jul 10, 2020 at 1:03 PM

Re: How can I use "C" API of arrow in other projects?

The command is exactly what I need. Thanks a lot. Song > 2020年7月11日 上午6:21，Sutou Kouhei 写道： > > Hi, > > Here is the smallest command line to build with c_glib: > > gcc -o XXX XXX.c $(pkg-config --cflags --libs arrow-glib) > > We don't provide tutorial yet. Sorry. > > We provide only

Re: How can I use "C" API of arrow in other projects?

Hi, Here is the smallest command line to build with c_glib: gcc -o XXX XXX.c $(pkg-config --cflags --libs arrow-glib) We don't provide tutorial yet. Sorry. We provide only API reference https://arrow.apache.org/docs/c_glib/ and some examples

Status of Rust Integration Testing

Good day Arrow devs, I've spent a few evenings looking into the issues that we're experiencing with Rust integration testing. In summary, none of our tests pass (zero batch doesn't count :) ). This is mainly because of changes from the legacy padding in the 0.15.0 release, which we never made in

Re: Upcoming JS fixes and release timeline

Just to be more specific. Since most JavaScript packages follow semantic versioning that means that a change from 1.0.0 to 2.0.0 would imply that there were breaking changes in the API (i.e. not backwards compatible). By default, when declaring a dependency on a package that has a 1.X release,

Re: Upcoming JS fixes and release timeline

Hey Micah, npm allows you to set the version to anything you wish, but semantic versioning[1] is the convention. A few large-ish packages don't follow this (closure-compiler uses a timestamp as its version), but the tooling strongly nudges package owners and consumers towards semver. 1.0.0

RE: [EXTERNAL] Re: .NET support for Arrow

I agree with Adam, the more usage and feedback we can get the better on the .NET Library. > However there is no library for C# listed anywhere else in the > documentation. We have some XML style doc comments in the code. It would be great if we could generate a website/markdown from those XML

Re: language independent representation of filter expressions

Thanks for clarifying! And yes, my bad on the typo. I meant to say format/Schema.fbs On 2020/07/10 04:27:50, Micah Kornfield wrote: > Hi Patrick, > > > I'm working with Steve on this issue. Can you please share what you have > > in mind for something more general than Gandiva's serialized

Re: .NET support for Arrow

Hi Yash, My organisation is using the C# library for a product we are working on.  However, we are using a fork which includes a number of bug-fixes for issues that would have otherwise blocked us. I've raised a few PRs to fix these upstream. I think it's fair to say that the C# library is

[NIGHTLY] Arrow Build Report for Job nightly-2020-07-10-0

Arrow Build Report for Job nightly-2020-07-10-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-07-10-0 Failed Tasks: - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-07-10-0-travis-homebrew-cpp -

How can I use "C" API of arrow in other projects?

Hi all, I’m working on a project which uses libarrow to scan and compute. The project is implemented by “C”, I have found that arrow contains c_glib as the API for “C” code. I can compile and install c_glib, but have no idea how to use it. My project is organized by Makefiles, do I have to

Re: What's a good starting point?

Thanks Micah, that was very helpful! ARROW-7278 looks like a good place to dig in =] On Fri, Jul 10, 2020 at 7:33 AM Micah Kornfield wrote: > Hi Chris, > I don't think I've seen a formal roadmap for either Gandiva or Flight > (others might have more context). What you described is certainly

Fwd: Property-driven Parquet encryption

Sorry, Micah, and thanks again. Cheers, Gidon -- Forwarded message - From: Gidon Gershinsky Date: Fri, Jul 10, 2020 at 10:41 AM Subject: Re: Property-driven Parquet encryption To: dev , Hi Michah, Thanks! I was hoping for community feedback, it's better to discuss these

Re: Property-driven Parquet encryption

Hi Michah, Thanks! I was hoping for community feedback, it's better to discuss these things now, than during the rull request review. > > > * 1. "kms_client_class" This sounds like it might be a very Java centric > approach. Have you given consideration to how this can be used in > C++/Python?

Re: Writing very large rowgroups to Apache Parquet

Hi, I wasn't aware of the fact that jemalloc mmap automatically for larger allocations. And I didn't yet test this. The approach could be different in that we would know which parts of the buffers are going to be used next (the buffers are appendonly) and which parts won't be needed until

Re: What's a good starting point?

Hi Chris, I don't think I've seen a formal roadmap for either Gandiva or Flight (others might have more context). What you described is certainly how a lot of work gets done. There has been a slightly more formal roadmap proposed for datasets, dataframe and C++ query engine but that is the

Re: [DISCUSS] [C++] custom allocator for large objects

Sorry for the delay. Clearing through my inbox backlog ... We should double check the code, but one thing that has bitten me in the past with variable-width data is the binary array builder ReserveData call [1], does not act the same way Reserve works. The former only grows the buffer by the