This sounds great, but I had one question: Read the initial ADBC proposal and it mentioned that OLTP was not a targeted usecase If this work is intended to take on the role of a sort of standard ABI/SDK, does that mean that building OLTP-oriented drivers/tooling with it is off the table?
On Wed, Jun 1, 2022 at 11:11 AM Wes McKinney <wesmck...@gmail.com> wrote: > I went ahead and created > > https://github.com/apache/arrow-adbc > > I directed issue comments / PRs to issues@ > > On Tue, May 31, 2022 at 8:49 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > > I think spinning up a new repository while this exploratory work > > progresses is a fine idea — perhaps apache/arrow-dbc / arrow-adbc or > > similar (the name can always be changed later). That would bubble up > > discussions in a way that's easier for people to follow (watching your > > fork isn't ideal!). If it makes sense to move code later, it can > > always be moved. > > > > > > On Tue, May 31, 2022 at 1:02 PM David Li <lidav...@apache.org> wrote: > > > > > > Some updates: > > > > > > The proposal is being updated based on feedback from contributors to > DuckDB and DBI. We've been using GitHub issues on the fork to discuss the > API design and how to implement data ingestion/bound parameters: > https://github.com/lidavidm/arrow/issues > > > > > > If anyone has suggestions/ideas/questions, or would like to jump in as > well, please feel free to chime in there too. > > > > > > I have also been wondering if we might want to plan to split off a new > repo for this work? In particular, some components might be easiest to > consume if they didn't also have a hard dependency on the Arrow C++ > libraries. And we could use the repo to manage contributed drivers (some of > which may individually leverage the Arrow libraries). Of course, > maintaining a parallel build system, setting up releases, etc. is also a > lot of work. > > > > > > -David > > > > > > On Tue, Apr 26, 2022, at 15:01, Wes McKinney wrote: > > > > I don't have major new things to add on this topic except that I've > > > > long had the aspiration of creating something like Python's DBAPI 2.0 > > > > [1] at the C or C++ level to enable a measure of API standardization > > > > for Arrow-native read/write interfaces with database drivers. It > seems > > > > like a natural complement to the wire-protocol standardization work > > > > with FlightSQL. I had previously brought in some code that I had > > > > worked on related to interfacing with the HiveServer2 wire protocol > > > > (for Hive and Impala, or other HS2-compatible query engines) with the > > > > intention of prototyping but never was able to find the time. > > > > > > > > From an external messaging standpoint, one thing that will be > > > > important is to assert that this is not intended to displace or > > > > deprecate ODBC or JDBC drivers. In fact, I would hope that the > > > > Arrow-native APIs could be added somehow to existing driver libraries > > > > where it made sense, so that if they are used in an application that > > > > uses Arrow, they can opt in to using the Arrow-based APIs for getting > > > > result sets, or doing bulk inserts, etc. > > > > > > > > [1]: https://peps.python.org/pep-0249/ > > > > > > > > On Tue, Apr 26, 2022 at 12:36 PM Antoine Pitrou <anto...@python.org> > wrote: > > > >> > > > >> > > > >> Do we want something more flexible than dlopen() and runtime symbol > > > >> lookup (a mechanism which constrains the way you can organize and > > > >> distribute drivers)? > > > >> > > > >> For example, perhaps we could expose an API struct of function > pointers > > > >> that could be obtained through driver-specific means. > > > >> > > > >> > > > >> Le 26/04/2022 à 18:29, David Li a écrit : > > > >> > Hello, > > > >> > > > > >> > In light of recent efforts around Flight SQL, projects like pgeon > [1], and long-standing tickets/discussions about database support in Arrow > [2], it seems there's an opportunity to define standard database interfaces > for Arrow that could unify these efforts. So we've put together a proposal > for "ADBC", a common Arrow-based database client API: > > > >> > > > > >> > > https://docs.google.com/document/d/1t7NrC76SyxL_OffATmjzZs2xcj1owdUsIF2WKL_Zw1U/edit#heading=h.r6o6j2navi4c > > > >> > > > > >> > A common API and implementations could help combine/simplify > client-side projects like pgeon, or what DBI is considering [3], and help > them take advantage of developments like Flight SQL and existing columnar > APIs. > > > >> > > > > >> > We'd appreciate any feedback. (Comments should be open, please > let me know if not.) > > > >> > > > > >> > [1]: https://github.com/0x0L/pgeon > > > >> > [2]: https://issues.apache.org/jira/browse/ARROW-11670 > > > >> > [3]: https://github.com/r-dbi/dbi3/issues/48 > > > >> > > > > >> > Thanks, > > > >> > David >
