There's a bunch more to do here; I'm going to try to get to a reasonable starting point soon so that follow-up work can be addressed in smaller incremental patches.
This is a partial C++ implementation of the Flight RPC system initially proposed by Jacques in ARROW-249. As in Java, I had to dig into gRPC and Protocol Buffers internals to ensure that * On write, memory is only copied once into the outgoing gRPC buffer * On read, no memory is copied The way that I tricked gRPC into circumventing the built-in protobuf serde paths might look a bit hacky, but after digging around in the library a bunch I've convinced myself that it's the best and perhaps only way to accomplish this. Luckily, the message that's being serialized/deserialized is pretty opaque to the rest of the gRPC system, and it's controlled by the `SerializationTraits<T>` class. So you can take a gRPC stream reader and make it create any kind of type you want, even if the input data is a protocol buffer. Some things that won't be addressed in this patch, as scope is too large: * gRPC build toolchain issues (this is rather complex, I will create follow-up issues) * Security / encryption, and authentication issues. I have only implemented an insecure server * Integration with Travis CI * Python bindings API is preliminary and I expect to be the subject of iteration to make general and fast over the next several months. [ Full content available at: https://github.com/apache/arrow/pull/2547 ] This message was relayed via gitbox.apache.org for [email protected]
