Thanks for the feedback! Collective operations: unfortunately, these look very different from the model Flight provides (which is just RPC). If there's interest, we could consider implementing or exposing them in the future, or looking at making sure Arrow's IPC APIs play well with the MPI APIs.
Non-blocking operations: I was thinking about this. In Flight, this would probably mean async APIs, which have been discussed before (either here or on JIRA). The UCX APIs lend themselves naturally to implementing an async API and I would like to explore this further. Serialization: what you probably want is GetRecordBatchPayload[1] and related functions, which is also what Flight uses as part of its zero-copy optimizations. This function will allocate a buffer for the IPC metadata and return that along with a list of buffers to be written; you can then individually send the buffers. You do have to remember to add padding yourself. (It's what this proof-of-concept does: [2]) @Antoine: sorry, I was a little loose with things there. I don't see an HTTP/2 implementation for UCX, unfortunately. [1]: https://github.com/apache/arrow/blob/06b10133e486ff736e657f79ffad7b029150cfcd/cpp/src/arrow/ipc/writer.h#L389 [2]: https://github.com/lidavidm/arrow/blob/cf804e3505b6dab996d03f8fab658aea02504090/cpp/src/arrow/flight/transport/ucx/ucx_internal.cc#L341 On Tue, Dec 28, 2021, at 15:35, Antoine Pitrou wrote: > > Le 28/12/2021 à 20:09, David Li a écrit : > > > > Antoine/Micah raised the possibility of extending gRPC instead. That would > > be preferable, frankly, given otherwise we'd might have to re-implement a > > lot of what gRPC and HTTP2 provide by ourselves. However, the necessary > > proposal stalled and was dropped without much discussion: > > https://groups.google.com/g/grpc-io/c/oIbBfPVO0lY > > I'm not sure whether I proposed extending gRPC :-) Is there an HTTP2 > implementation above UCX? If so, we could devise a Flight > implementation over REST/HTTP2, which might also make the TCP backend > faster than with gRPC. > > Regards > > Antoine. >