I am planning on doing something similar, but I don't have a concrete reference to point you to.
In my case, I am treating the metadata as something that is re-usable (e.g. the FlightInfo is not something that expires after the first use). This makes the most sense when the information in FlightInfo is something that directly names the data objects you need to retrieve. For me, I am only expecting Flight communications to come from a library that I have written for use by others, therefore I will have full control of the interactions between a FlightClient and a FlightServer. As you point out, your main concern should probably be protocol compatibility. If you will have control of the client side of communications, then I think there are minimal concerns other than how you design what a Ticket or FlightInfo contains. Aldrin Montana Computer Science PhD Student UC Santa Cruz On Fri, Feb 3, 2023 at 2:18 PM Nate Jones <nate.jo...@twosigma.com> wrote: > Hello, > > We've been using the Flight protocol similar to the way that the read path > is described in documentation< > https://arrow.apache.org/docs/format/Flight.html#downloading-data>. That > is, services have a separate metadata server (at least logically separated > such that a network round trip occurs for GetFlightInfo), which returns > FlightInfo to be used to access data server(s). We follow a similar pattern > for writes and exchanges, as well. > > While the separate metadata concept is crucial for certain applications, > we think other use cases could be made much simpler by skipping the > metadata step altogether - in this case, clients would craft their own > Tickets and talk directly to data servers for reads, writes, and exchanges. > This would be nice when we're just looking for a "normal gRPC streaming > call but with the benefits of Flight." For example, some services have a > metadata server that returns FlightInfo that simply points clients back to > itself, resulting in an unnecessary round trip since the GetFlightInfo is > essentially a “noop” here. > > I notice in the docs the statement "Of course, applications may ignore > compatibility and simply treat the Flight RPC methods as low-level building > blocks for their own purposes." Despite this, I wanted to reach out to see > if there are any reference use cases that use Flight in this way. Are there > any concerns that come to mind when adapting the Flight pattern like this? > > Thanks, > Nate >
