hi Maxim, What you're describing is presently an "off-label" use of Flight but it's perfectly valid if you're willing to roll up your sleeves a bit. Since we've focused on dealing with the columnar format as a multi-message stream, you'll find the APIs in Java, C++, and Python to be oriented around that. As far as documentation I would suggest you look at the code and associated patches for the C++ part
https://github.com/apache/arrow/commits/master/cpp/src/arrow/flight If you want to send and receive tensors/ndarrays, the easiest thing to do is to embed the data in arrow::BinaryArray (this is possible with zero copy with a little finagling -- essentially a column with 1 cell containing the tensor data). I have recently developed ExtensionType machinery to make this easier, but there is some engineering work needed to make things really simple for the use case. Please feel free to open some JIRA issues to describe your requirements in more detail. Thanks Wes On Fri, Mar 15, 2019 at 7:01 AM Maxim Novikov <[email protected]> wrote: > > Hello, > I am evaluating Arrow RPC for usage with multidimensional Tensors. > In my case, mostly 2D/3D images with multiple channels. > > Let's say I want to develop a service that applies a gaussian blur on a 2D > image. > From my understanding expected workflow is the following: > 1. DoPut $key, image data > 2. DoAction (blur $key) > 2.1. Get image data from $key > 2.2. Apply blur > 2.3. Encode Tensor to result > 3. Recv result on the client > 4. Decode result > Is it correct approach or am I misinterpreting the purpose of this > API/endpoints? > > I didn't find any documentation except for a couple of slides and > Flight.proto definition. I understand that docs are planned in the future, > but maybe there is an issue/pull-request from which I can get more > information about design decisions and expected use cases? > -- > Maksim Novikov, *Developer* > *EMBL*
