Ah, thanks for the clarification Jacques - we should perhaps clarify that in the spec itself.
On Fri, Mar 11, 2022, at 16:37, Jacques Nadeau wrote: > The intention was that a flight consumer could use any of the location values > and they produced equivalent data. Similar to the concept of block location > in the hdfs api. This could allow a single flight dataset to be served from > multiple locations (and/or protocols) and temhe consumer can choose the > optimal path of access (to maximize locality, etc). > > It was never meant to be application specified. > > On Fri, Mar 11, 2022, 1:24 PM Kyle Porter <[email protected]> wrote: >> I would second that - let's just make it part of implementing Flight SQL in >> terms of how this should be interpreted. I don't have a specific opinion on >> what the best interpretation would be, however. >> >> *Kyle Porter* >> CEO >> Bit Quill Technologies Inc. >> Office: +1.778.331.3355 | Direct: +1.604.441.7318 | [email protected] >> https://www.bitquill.com >> >> This email message is for the sole use of the intended recipient(s) and may >> contain confidential and privileged information. Any unauthorized review, >> use, disclosure, or distribution is prohibited. If you are not the intended >> recipient, please contact the sender by reply email and destroy all copies >> of the original message. Thank you. >> >> >> On Fri, Mar 11, 2022 at 1:21 PM David Li <[email protected]> wrote: >>> __ >>> Are existing services really a concern? Implementing Flight SQL means >>> you're in control already, and I'd rather see a single clear specification >>> over a lot of toggles/flags. >>> >>> On Fri, Mar 11, 2022, at 16:08, James Duong wrote: >>>> I propose for Flight SQL, we make this an always existing GetSqlInfo >>>> property, so clients can understand the server's intent. (As opposed to >>>> mandating behavior one way or the other, which could make Flight SQL >>>> harder to adapt to existing Flight services if we pick the wrong way). >>>> >>>> With regular Flight, the application will already be tuned to the server's >>>> behavior so there's no need for metadata around this there. >>>> >>>> On Fri, Mar 11, 2022 at 1:02 PM Kyle Porter <[email protected]> wrote: >>>>> It seems like we should probably specify the behaviour for Flight SQL, at >>>>> the very least, to ensure that the behaviour is consistent, which is one >>>>> of the goals of Flight SQL. >>>>> >>>>> *Kyle Porter* >>>>> CEO >>>>> Bit Quill Technologies Inc. >>>>> Office: +1.778.331.3355 | Direct: +1.604.441.7318 | [email protected] >>>>> https://www.bitquill.com >>>>> >>>>> This email message is for the sole use of the intended recipient(s) and >>>>> may contain confidential and privileged information. Any unauthorized >>>>> review, use, disclosure, or distribution is prohibited. If you are not >>>>> the intended recipient, please contact the sender by reply email and >>>>> destroy all copies of the original message. Thank you. >>>>> >>>>> >>>>> On Fri, Mar 11, 2022 at 1:01 PM David Li <[email protected]> wrote: >>>>>> __ >>>>>> The perhaps unsatisfactory answer is that it is up to the application >>>>>> how to interpret this. For instance, it could be used to indicate >>>>>> multiple servers that can all handle the request, allowing the client to >>>>>> pick one at its choosing (say, for load balancing, or because the client >>>>>> only supports certain protocols). >>>>>> >>>>>> On Fri, Mar 11, 2022, at 15:58, James Duong wrote: >>>>>>> This seems inconsistent. >>>>>>> >>>>>>> In this c++ performance test it looks like it just gets the first >>>>>>> location in every endpoint (though it gets data from every endpoint in >>>>>>> parallel): >>>>>>> https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/cpp/src/arrow/flight/flight_benchmark.cc#L297 >>>>>>> >>>>>>> The protobuf definition is a bit unclear too. It says the client can >>>>>>> redeem the ticket at every location, but doesn't say the client >>>>>>> must retrieve data at every location to get the whole dataset (so it's >>>>>>> not clear if this is for redundancy or partitioning): >>>>>>> https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/format/Flight.proto#L283 >>>>>>> >>>>>>> The Java and C++ examples posted also do not handle the case where the >>>>>>> location is empty. >>>>>>> >>>>>>> On Fri, Mar 11, 2022 at 12:53 PM Rafael Telles <[email protected]> >>>>>>> wrote: >>>>>>>> Hi guys, >>>>>>>> >>>>>>>> What is the user supposed to do with the Location list in a >>>>>>>> FlightEndpoint? >>>>>>>> Looking at this integration test >>>>>>>> (https://github.com/apache/arrow/blob/dc2e0b2e44fdaa3d5ad0bb358ff8ce9db3bc7416/java/flight/flight-integration-tests/src/main/java/org/apache/arrow/flight/integration/tests/IntegrationTestClient.java#L151) >>>>>>>> it's iterating on Location list and issuing multiple getStream() >>>>>>>> calls with the same ticket. >>>>>>>> >>>>>>>> Best Regards, >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *James Duong* >>>>>>> Lead Software Developer >>>>>>> Bit Quill Technologies Inc. >>>>>>> Direct: +1.604.562.6082 | [email protected] >>>>>>> https://www.bitquilltech.com >>>>>>> >>>>>>> >>>>>>> This email message is for the sole use of the intended recipient(s) and >>>>>>> may contain confidential and privileged information. Any unauthorized >>>>>>> review, use, disclosure, or distribution is prohibited. If you are not >>>>>>> the intended recipient, please contact the sender by reply email and >>>>>>> destroy all copies of the original message. Thank you. >>>>>> >>>> >>>> >>>> -- >>>> *James Duong* >>>> Lead Software Developer >>>> Bit Quill Technologies Inc. >>>> Direct: +1.604.562.6082 | [email protected] >>>> https://www.bitquilltech.com >>>> >>>> >>>> This email message is for the sole use of the intended recipient(s) and >>>> may contain confidential and privileged information. Any unauthorized >>>> review, use, disclosure, or distribution is prohibited. If you are not >>>> the intended recipient, please contact the sender by reply email and >>>> destroy all copies of the original message. Thank you. >>>
