The idea is that the client would reuse the existing connection, in which case the protocol and such are implicit. (If the client doesn't have a connection anymore, it can't use the fallback anyways.)
I suppose this has the advantage that you could "fall back" to a known hostname with a different protocol, but I'm not sure that always applies anyways. (Correct me if I'm wrong Matt, but as I recall, UCX addresses aren't hostnames but rather opaque byte blobs, for instance.) If we do prefer this, to avoid overloading the hostname, there's also the informal convention of using + in the scheme, so it could be arrow-flight-fallback+grpc+tls://, arrow-flight-fallback+http://, etc. On Mon, Feb 12, 2024, at 17:03, Joel Lubinitsky wrote: > Thanks for clarifying. > > Given the relationship between these two proposals, would it also be > necessary to distinguish the scheme (or schemes) supported by the > originating Flight RPC service? > > If that is the case, it may be preferred to use the "host" portion of the > URI rather than the "scheme" to denote the location of the data. In this > scenario, the host "0.0.0.0" could be used. This IP address is defined in > IETF RFC1122 [1] as "This host on this network", which seems most > consistent with the intended use-case. There are some caveats to this usage > but in my experience it's not uncommon for protocols to extend the > definition of this address in their own usage. > > A benefit of this convention is that the scheme remains available in the > URI to specify the transport available. For example, the following list of > locations may be included in the response: > > ["grpc://0.0.0.0", "ucx://0.0.0.0", "grpc://1.2.3.4", <other_locations>...] > > This would indicate that grpc and ucx transport is available from the > current service, grpc is available at 1.2.3.4, and possibly more > combinations of scheme/host. > > [1] https://datatracker.ietf.org/doc/html/rfc1122#section-3.2.1.3 > > On Mon, Feb 12, 2024 at 2:53 PM David Li <lidav...@apache.org> wrote: > >> Ah, while I was thinking of it as useful for a fallback, I'm not >> specifying it that way. Better ideas for names would be appreciated. >> >> The actual precedence has never been specified. All endpoints are >> equivalent, so clients may use what is "best". For instance, with Matt >> Topol's concurrent proposal, a GPU-enabled client may preferentially try >> UCX endpoints while other clients may choose to ignore them entirely (e.g. >> because they don't have UCX installed). >> >> In practice the ADBC/JDBC drivers just scan the list left to right and try >> each endpoint in turn for lack of a better heuristic. >> >> On Mon, Feb 12, 2024, at 14:28, Joel Lubinitsky wrote: >> > Thanks for proposing this David. >> > >> > I think the ability to include the Flight RPC service itself in the list >> of >> > endpoints from which data can be fetched is a helpful addition. >> > >> > The current choice of name for the URI (arrow-flight-fallback://) seems >> to >> > imply that there is an order of precedence that should be considered in >> the >> > list of URI’s. Specifically, as a developer receiving the list of >> locations >> > I might assume that I should try fetching from other locations first. If >> > those do not succeed, I may try the original service as a fallback. >> > >> > Are these the intended semantics? If so, is there a way to include the >> > original service in the list of locations without the implied precedence? >> > >> > Thanks, >> > Joel >> > >> > On Mon, Feb 12, 2024 at 11:52 James Duong <james.du...@improving.com >> .invalid> >> > wrote: >> > >> >> This seems like a good idea, and also improves consistency with clients >> >> that erroneously assumed that the service endpoint was always in the >> list >> >> of endpoints. >> >> >> >> From: Antoine Pitrou <anto...@python.org> >> >> Date: Monday, February 12, 2024 at 6:05 AM >> >> To: dev@arrow.apache.org <dev@arrow.apache.org> >> >> Subject: Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme >> >> >> >> Hello, >> >> >> >> This looks fine to me. >> >> >> >> Regards >> >> >> >> Antoine. >> >> >> >> >> >> Le 12/02/2024 à 14:46, David Li a écrit : >> >> > Hello, >> >> > >> >> > I'd like to propose a slight update to Flight RPC to make Flight SQL >> >> work better in different deployment scenarios. Comments on the doc >> would >> >> be appreciated: >> >> > >> >> > >> >> >> https://docs.google.com/document/d/1g9M9FmsZhkewlT1mLibuceQO8ugI0-fqumVAXKFjVGg/edit?usp=sharing >> >> > >> >> > The gist is that FlightEndpoint allows specifying either (1) a list of >> >> concrete URIs to fetch data from or (2) no URIs, meaning to fetch from >> the >> >> Flight RPC service itself; but it would be useful to combine both >> behaviors >> >> (try these concrete URIs and fall back to the Flight RPC service itself) >> >> without requiring the service to know its own public address. >> >> > >> >> > Best, >> >> > David >> >> >>
