Hi Tiffany, I was just curious about the status of the FlightSQL proposal. Given that the 4.0.0 development cycle will be concluding soon, it doesn't seem FlightSQL will make it, but is there interest in trying to get it into 5.0.0?
Similarly, there was the related effort to expand Flight's auth capabilities which also seems to have been inactive lately - is there anything we can do to help keep that moving forward as well? Best, David On 2021/02/09 01:52:45, Hoi Ling Tiffany Lam <tiffanyla...@gmail.com> wrote: > HI David, > > I will be taking over the work for Flight SQL. Last week, I created a draft > PR to Arrow Main <https://github.com/apache/arrow/pull/9368> that addresses > the original comments on this PR > <https://github.com/ryannicholson/arrow/pull/2>. It would be amazing if you > could take a look at the draft PR. The work on Flight SQL is temporarily > paused on my end, I will be resuming the work next week. I am also going to > send out an email to the dev mailing list once the work on FLight SQL > resumes next week. Hopefully I will be able to concentrate on Flight SQL > from next week onwards. Exciting times ahead! > > Cheers, > Tiffany Lam (She/Her) > > On Mon, Feb 8, 2021 at 10:16 AM David Li <lidav...@apache.org> wrote: > > > Hi Ryan et. al., > > > > Following up here- I saw Dremio gave a presentation about this project > > recently, and also, over the past several months, the authentication work > > has been proceeding. Given that, what does the status of the Flight SQL > > project itself look like now? I see there are some unanswered comments in > > the doc and in the draft PR, would another round of comments be useful if > > those were addressed? And the talk mentioned a release in the next few > > months, are there any details you can share there? > > > > Thanks, > > David > > > > On Thu, Oct 8, 2020, at 13:28, Ryan Nicholson wrote: > > > Thanks Wes, > > > > > > Apologies for the delay. The next steps which we were going to pursue > > once > > > the authentication redesign direction was solid included a session > > concept. > > > The initial proposal we would put forth would likely include these > > concepts > > > via use of headers. I agree that with sessions and databases in mind, a > > > close operation would be beneficial. In thinking through this earlier it > > > wasn't clear to me if it would make sense to implement "close" as an > > Action > > > but it also didn't seem to fit as a new first class Flight RPC > > altogether. > > > > > > Ryan > > > > > > > > > > > > On Thu, Oct 1, 2020 at 3:44 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > > I just reviewed the FlightSQL.proto on > > > > https://github.com/ryannicholson/arrow/pull/2. It might be a good idea > > > > to document somewhere how some of the various SQL concepts (like > > > > sessions) are intended to be handled in Flight (which doesn't have > > > > these concepts built in). Does it make sense to have > > > > session-lifetime-related RPCs (start/close session, check session > > > > status, include a session bytes token with requests, etc.)? > > > > > > > > On Tue, Sep 29, 2020 at 6:53 PM Ryan Nicholson <rnichol...@gmail.com> > > > > wrote: > > > > > > > > > > Hello Andy and Wes, > > > > > > > > > > Thanks for reaching out. > > > > > > > > > > Later this week I have time freeing up to start on Flight-SQL. Would > > > > either > > > > > of you mind examining the approach in the POC [1] as described in the > > > > > "Proof of Concept" section of the proposal document [2]? > > > > > > > > > > I would certainly be happy to chat about this with anyone interested > > in > > > > > participating too. > > > > > > > > > > Thank you, > > > > > Ryan > > > > > > > > > > [1] https://github.com/ryannicholson/arrow/pull/2 > > > > > [2] > > > > > > > > > > > https://docs.google.com/document/d/1WQz32bDF06GgMdEYyzhakqUigBZkALFwDF2y1x3DTAI > > > > > > > > > > > > > > > On Tue, Sep 29, 2020 at 8:52 AM Andy Grove <andygrov...@gmail.com> > > > > wrote: > > > > > > > > > > > I am also interested in helping with this effort. > > > > > > > > > > > > Andy. > > > > > > > > > > > > On Mon, Sep 28, 2020, 1:58 PM Wes McKinney <wesmck...@gmail.com> > > > > wrote: > > > > > > > > > > > > > hi Ryan, > > > > > > > > > > > > > > I'm interested in enabling this effort to move forward -- is > > there > > > > any > > > > > > > part where feedback or other help could be useful? > > > > > > > > > > > > > > Thanks, > > > > > > > Wes > > > > > > > > > > > > > > On Wed, Sep 2, 2020 at 4:45 PM Wes McKinney <wesmck...@gmail.com > > > > > > > wrote: > > > > > > > > > > > > > > > > I haven't been able to scrutinize the details greatly, but I am > > > > > > > > supportive of this effort, otherwise we may end up with many > > > > database > > > > > > > > systems and vendors implementing Flight in different ways. It > > > > would be > > > > > > > > nice if there was a built-in SQL client layer that we ship in > > C++ > > > > (and > > > > > > > > its bindings), Java, and others. > > > > > > > > > > > > > > > > On Wed, Sep 2, 2020 at 10:24 AM Ryan Nicholson < > > > > rnichol...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hello everyone, > > > > > > > > > > > > > > > > > > Here is a link to the slides > > > > > > > > > < > > > > > > > > > > > > > > > > > > > https://docs.google.com/presentation/d/1JcOKA1yKYTNodQl5Z1BtK3dhJLx-Vu_RSAqWF_wiJH8/edit?usp=sharing > > > > > > > > > > > > > > > > > I would like to discuss this morning. > > > > > > > > > > > > > > > > > > Take care, > > > > > > > > > Ryan Nicholson > > > > > > > > > > > > > > > > > > On Tue, Sep 1, 2020 at 4:02 PM Ryan Nicholson < > > > > rnichol...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Thank you to all who have viewed and given feedback in the > > > > posted > > > > > > > material > > > > > > > > > > so far. I am working on responses as well as preparing some > > > > > > material > > > > > > > which > > > > > > > > > > I would like to share for discussion in the sync call > > > > tomorrow. I > > > > > > > will post > > > > > > > > > > the material here ahead of time. > > > > > > > > > > > > > > > > > > > > Take care, > > > > > > > > > > Ryan Nicholson > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 31, 2020 at 2:21 PM Ryan Nicholson < > > > > > > rnichol...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> Hello everyone, > > > > > > > > > >> > > > > > > > > > >> I would like to propose the following specification to > > ease > > > > > > > adoption of > > > > > > > > > >> SQL based clients and backends while leveraging data > > streams > > > > for > > > > > > > > > >> transporting data. This specification entails a series of > > > > protobuf > > > > > > > messages > > > > > > > > > >> to be used in the opaque messaging framework payloads to > > > > enable > > > > > > > browsing of > > > > > > > > > >> database metadata and statement style operations with > > > > scaffolding > > > > > > > available > > > > > > > > > >> to help implementers provide logic for their backend. > > > > > > > > > >> > > > > > > > > > >> Please take a look at the following material: > > > > > > > > > >> > > > > > > > > > >> - ARROW-9825 < > > > > https://issues.apache.org/jira/browse/ARROW-9825 > > > > > > > > > > > > > > has > > > > > > > > > >> more details around the goals of the proposal. > > > > > > > > > >> - Flight SQL Proposal > > > > > > > > > >> < > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1WQz32bDF06GgMdEYyzhakqUigBZkALFwDF2y1x3DTAI/edit?usp=sharing > > > > > > > > > > > > > > is > > > > > > > > > >> a more detailed document describing new messages and > > how > > > > they > > > > > > > would be used. > > > > > > > > > >> - https://github.com/ryannicholson/arrow/pull/2 on my > > > > Arrow > > > > > > fork > > > > > > > > > >> shows a sample implementation with a catalog browsing > > > > action > > > > > > and > > > > > > > a prepared > > > > > > > > > >> statement query workflow. There are details in the > > proposal > > > > > > > linking to > > > > > > > > > >> specific areas of the PR. > > > > > > > > > >> > > > > > > > > > >> Thank you for your time, > > > > > > > > > >> Ryan Nicholson > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
