Re: Arrow Flight RPC stuck

2024-11-04 Thread David Li
Hi Susmit, Since you submitted a Github issue as well [1] maybe we can consolidate discussion there? [1]: https://github.com/apache/arrow/issues/44560 Best, David On Tue, Oct 29, 2024, at 22:08, Susmit Sarkar wrote: > Hi Team, > > We are writing polyglot sdk wrappers on top of Flight Client, w

Re: [ANNOUNCE] New Arrow PMC member: Curt Hagenlocher

2024-10-30 Thread David Li
Congrats Curt! Thanks for the help with ADBC! On Thu, Oct 31, 2024, at 08:26, Jacob Wujciak-Jens wrote: > Congratulations! > > Am Do., 31. Okt. 2024 um 00:06 Uhr schrieb Neal Richardson < > neal.p.richard...@gmail.com>: > >> Congratulations! >> >> On Wed, Oct 30, 2024 at 6:20 PM Ian Cook wrote: >

Re: [C++][Flight] Status of the ODBC driver PR

2024-10-28 Thread David Li
Thanks for the update JB, that sounds reasonable to me (and then we can finally close out the code donation form). On Mon, Oct 28, 2024, at 23:10, Jean-Baptiste Onofré wrote: > Hi everyone, > > I did a sync-up with James about the Arrow Flight ODBC driver PR > (https://github.com/apache/arrow/pul

Re: [VOTE] Add Async C Data Interface

2024-10-28 Thread David Li
+1 (binding) for me On Sat, Oct 26, 2024, at 10:39, Ian Cook wrote: > Oh ok, thanks Matt, I understand. > > In that case I am +1 on the proposal but I would like to see notes added to > the documentation to make this clearer to readers. I created an issue for > this: https://github.com/apache/arro

Re: Suggestions on deploying Arrow Flight gRPC Server

2024-10-27 Thread David Li
I think that's a bit broad of a question to answer. In general, you are best off following advice meant for gRPC and your particular environment (Kubernetes or otherwise); Flight does not change that aspect of gRPC. If you have specific questions about gRPC vis-a-vis Flight (e.g. how to enable a

Re: [VOTE] Release Apache Arrow 18.0.0 - RC0

2024-10-23 Thread David Li
+1 (binding) I'm having some issue with wheels, but I believe this is just a verification script issue [1]: + python -c 'import pyarrow._s3fs' Traceback (most recent call last): File "", line 1, in ImportError: /tmp/arrow-test/miniforge/envs/conda-wheel-3.9-manylinux_2_28_x86_64/lib/python3.

Re: [ANNOUNCE] New Arrow committer: Rossi Sun

2024-10-22 Thread David Li
Welcome Rossi! On Wed, Oct 23, 2024, at 11:41, wish maple wrote: > Congrats Ruoxi! > > Best, > Xuwei Fu > > Felipe Oliveira Carvalho 于2024年10月23日周三 08:18写道: > >> Great news! Congratulations. >> >> — >> Felipe >> >> On Tue, 22 Oct 2024 at 16:03 Weston Pace wrote: >> >> > On behalf of the Arrow PM

Re: [VOTE][Go] Release Apache Arrow Go 18.0.0 RC0

2024-10-16 Thread David Li
+1 (binding) Tested on Debian bookworm/x86_64 On Thu, Oct 17, 2024, at 10:58, Dewey Dunnington wrote: > +1! (binding) > > I ran `dev/release/verify_rc.sh 18.0.0 0` on MacOS 14.6 (go 1.23.2) > > Cheers, > > -dewey > > On Wed, Oct 16, 2024 at 7:13 PM Sutou Kouhei wrote: >> >> +1 (binding) >> >> I

Re: Query of Arrow Flight SQL with S3 as a storage for parquet files

2024-10-16 Thread David Li
If your clients are sending full SQL queries to be executed, and you need to execute them against S3 on the server, why not consider something like Apache DataFusion or DuckDB to implement that part instead of building the query parser/engine yourself? (There are probably already examples of wra

Re: [VOTE] Release Apache Arrow nanoarrow 0.6.0

2024-10-08 Thread David Li
+1 (binding) Tested with conda-forge, Debian 12, x86_64 On Wed, Oct 9, 2024, at 11:30, Dewey Dunnington wrote: > Hello, > > I would like to propose the following release candidate (rc0) of > Apache Arrow nanoarrow [0] version 0.6.0. This release consists of 114 > resolved GitHub issues from 10 co

Re: [ANNOUNCE] New Arrow committer: Will Ayd

2024-10-05 Thread David Li
Welcome, Will! On Wed, Oct 2, 2024, at 23:25, Gang Wu wrote: > Congrats and welcome! > > Best regards, > Gang > > On Wed, Oct 2, 2024 at 10:16 PM Vibhatha Abeykoon > wrote: > >> Congratulations, Will! >> >> On Wed, Oct 2, 2024 at 3:18 PM Joris Van den Bossche < >> jorisvandenboss...@gmail.com> wr

Re: [EXTERNAL] Re: How to return auth token back to client?

2024-09-16 Thread David Li
row/flight/FlightClient.java#L197 On Tue, Sep 17, 2024, at 10:18, Zhao, Chuan wrote: > Yes, so what’s the easiest way to send user/password to arrow flight > server then get the token back using Java? > > From: David Li > Date: Monday, September 16, 2024 at 5:07 PM > To: Chuan Zhao , dev@

Re: [EXTERNAL] Re: How to return auth token back to client?

2024-09-16 Thread David Li
I think something is getting mixed up here. You are already sending the token back to the client, but it sounds like the actual question is how to extract the token from the client? I don't believe the JDBC driver offers any way to do this currently. On Tue, Sep 17, 2024, at 08:35, Zhao, Chuan

Re: [RESULT][VOTE] Release Apache Arrow ADBC 14 - RC0

2024-09-05 Thread David Li
] Publish release blog post [4] [1]: Rust upload was done by hand this time around [2]: https://github.com/conda-forge/arrow-adbc-split-feedstock/pull/27 [3]: https://github.com/apache/arrow-adbc/pull/2125 [4]: https://github.com/apache/arrow-site/pull/540 On Thu, Sep 5, 2024, at 10:28, David Li wrote

[RESULT][VOTE] Release Apache Arrow ADBC 14 - RC0

2024-09-04 Thread David Li
The vote passes with 4 binding +1 votes, no other votes. Binding votes: - Matt Topol - Raúl Cumplido - Sutou Kouhei - David Li I will begin on the release tasks next. On Thu, Sep 5, 2024, at 03:41, Matt Topol wrote: > My vote: +1 (binding) > > On Mon, Sep 2, 2024 at 10:42 PM Sutou Kouh

Re: [VOTE] Release Apache Arrow ADBC 14 - RC0

2024-09-02 Thread David Li
My vote: +1 (binding) On Fri, Aug 30, 2024, at 19:44, Raúl Cumplido wrote: > +1 (binding) > > Successfully validated release candidate with: > $ USE_CONDA=1 ./dev/release/verify-release-candidate.sh 14 0 > > On Ubuntu 22.04 > > Raúl > > El vie, 30 ago 2024 a

[VOTE] Release Apache Arrow ADBC 14 - RC0

2024-08-29 Thread David Li
Hello, I would like to propose the following release candidate (RC0) of Apache Arrow ADBC version 14. This is a release consisting of 27 resolved GitHub issues [1]. The subcomponents are versioned independently: - C/C++/GLib/Go/Python/Ruby: 1.2.0 - C#: 0.14.0 - Java: 0.14.0 - R: 0.14.0 - Rust:

Re: [VOTE][Julia] Release Apache Arrow Julia 2.7.3 RC1

2024-08-28 Thread David Li
+1 (binding) Tested on Debian Bookworm On Thu, Aug 29, 2024, at 09:15, Sutou Kouhei wrote: > +1 (binding) > > I ran the following command line on Debian GNU/Linux sid: > > VERIFY_FORCE_USE_JULIA_BINARY=1 dev/release/verify_rc.sh 2.7.3 1 > > > Thanks, > -- > kou > > In <20240829.085933.65207404

Re: [VOTE] Split Go release process

2024-08-25 Thread David Li
+1 (binding) On Mon, Aug 26, 2024, at 11:49, Sutou Kouhei wrote: > +1 (binding) > > In <20240826.113702.1496714896073155680@clear-code.com> > "[VOTE] Split Go release process" on Mon, 26 Aug 2024 11:37:02 +0900 (JST), > Sutou Kouhei wrote: > >> Hi, >> >> I would like to propose splitting

Re: [EXTERNAL] Re: Question about arrow flight authentication

2024-08-07 Thread David Li
t_ NULL. On Wed, Jul 24, 2024, at 11:41, Zhao, Chuan wrote: > Hi David, > > Here are the files: > https://gist.github.com/bigchuan79/a1efa7d981aa74a3d8f61beb2344f91a > > Thanks! > > -Chuan > > From: David Li > Date: Tuesday, July 23, 2024 at 6:21 PM > To: d

Re: [VOTE][Format] Bool8 Canonical Extension Type

2024-08-05 Thread David Li
+1 (binding) On Tue, Aug 6, 2024, at 10:17, Sutou Kouhei wrote: > +1 (binding) > > In > "[VOTE][Format] Bool8 Canonical Extension Type" on Mon, 5 Aug 2024 > 08:59:42 -0400, > Joel Lubinitsky wrote: > >> Hello Devs, >> >> I would like to propose a new canonical extension type: Bool8 >> >>

[RESULT][VOTE][Format] Opaque canonical extension type

2024-07-28 Thread David Li
...@app.fastmail.com> >> "[VOTE][Format] Opaque canonical extension type" on Wed, 24 Jul 2024 >> 14:33:01 +0900, >> "David Li" wrote: >> >> > Hello, >> > >> > I'd like to propose the 'Opaque' canonical ext

[VOTE][Format] Opaque canonical extension type

2024-07-23 Thread David Li
Hello, I'd like to propose the 'Opaque' canonical extension type. Prior discussion can be found at [1] and the proposal and implementations for C++, Go, Java, and Python can be found at [2]. The proposal is additionally reproduced below. The vote will be open for at least 72 hours. [ ] +1 Acce

Re: Question about arrow flight authentication

2024-07-23 Thread David Li
Hi Chuan, The attachment didn't make it through - could you post a Gist or something? Thanks, David On Wed, Jul 24, 2024, at 08:06, Zhao, Chuan wrote: > Hi, > > I am from Teradata and I am working on a POC related to Arrow Flight. > Basically, I wanted to send metadata request from the client

Re: Unsupported/Other Type

2024-07-22 Thread David Li
ed more eyes on it :-) >> >> Can other people go and take a look at David's PR below? >> >> >> Le 25/05/2024 à 04:47, David Li a écrit : >> > I've put up a draft PR here: https://github.com/apache/arrow/pull/41823 >>

Re: [DISCUSS] Deprecate UCX transport for Arrow Flight in favor of Dissociated IPC Protocol

2024-07-17 Thread David Li
> >> [1] https://github.com/apache/arrow/pull/43297 >> >> El mié, 19 jun 2024 a las 11:29, Raúl Cumplido () >> escribió: >> > >> > Hi, >> > >> > I would like to discuss deprecation of the UCX transport for Arrow >> > Flight (ARROW_WITH_UC

Re: [VOTE] Release Apache Arrow 17.0.0 - RC2

2024-07-15 Thread David Li
+1 (binding) Tested on Debian 12/x86_64 On Mon, Jul 15, 2024, at 15:31, Gang Wu wrote: > +1 (non-binding) > > Verified C++ on my M1 Mac by running: > - TEST_DEFAULT=0 TEST_CPP=1 ./verify-release-candidate.sh 17.0.0 2 > > BTW, I ran into this issue as well: > https://github.com/apache/arrow/issues

Re: [RESULT][VOTE] Release Apache Arrow ADBC 13 - RC0

2024-07-04 Thread David Li
/conda-forge/arrow-adbc-split-feedstock/pull/24 [2]: https://github.com/apache/arrow-site/pull/533 On Fri, Jul 5, 2024, at 10:22, David Li wrote: > I've hit a snag with C#: it appears the packages got built with the > wrong version number (0.13.0-SNAPSHOT instead of 0.13.0) [1]. For t

[ANNOUNCE] Apache Arrow ADBC 13 released

2024-07-04 Thread David Li
The Apache Arrow community is pleased to announce the 13th release of the Apache Arrow ADBC libraries. It includes 24 resolved GitHub issues ([1]). Individual components are versioned separately: some packages are on version 0.13.0 and others are now version 1.1.0, with the release as a whole on

Re: [RESULT][VOTE] Release Apache Arrow ADBC 13 - RC0

2024-07-04 Thread David Li
nnounce the new release [ ] Remove old artifacts [ ] Bump versions [ ] Publish release blog post On Fri, Jul 5, 2024, at 09:29, David Li wrote: > The vote passes with 3 binding, 1 non-binding +1 votes. Thanks all! > > Binding: Raúl Cumplido, Sutou Kouhei, Matt Topol > Non-binding: Dane Pit

[RESULT][VOTE] Release Apache Arrow ADBC 13 - RC0

2024-07-04 Thread David Li
t; > --- a/dev/release/verify-yum.sh >> > +++ b/dev/release/verify-yum.sh >> > @@ -170,6 +170,7 @@ echo "::endgroup::" >> > >> > echo "::group::Test ADBC Arrow GLib" >> > >> > +${install_command} --enablerepo=epel arro

[VOTE] Release Apache Arrow ADBC 13 - RC0

2024-07-01 Thread David Li
Hello, I would like to propose the following release candidate (RC0) of Apache Arrow ADBC version 13. This is a release consisting of 24 resolved GitHub issues [1]. The subcomponents are versioned independently: - C/C++/GLib/Go/Python/Ruby: 1.1.0 - C#: 0.13.0 - Java: 0.13.0 - R: 0.13.0 - Rust:

Re: Unsupported/Other Type

2024-05-24 Thread David Li
I've put up a draft PR here: https://github.com/apache/arrow/pull/41823 On Wed, Apr 17, 2024, at 23:34, David Li wrote: > Yes, this would be for an extension type. > > On Wed, Apr 17, 2024, at 23:25, Weston Pace wrote: >>> people generally find use in Arrow schemas indepen

Re: [VOTE] Release Apache Arrow nanoarrow 0.5.0

2024-05-22 Thread David Li
+1 (binding) Tested on Debian 12 'bookworm' On Thu, May 23, 2024, at 11:03, Sutou Kouhei wrote: > +1 (binding) > > I ran the following command line on Debian GNU/Linux sid: > > dev/release/verify-release-candidate.sh 0.5.0 0 > > with: > > * Apache Arrow C++ main > * gcc (Debian 13.2.0-23) 1

Re: [VOTE] Release Apache Arrow ADBC 12 - RC4

2024-05-21 Thread David Li
row-glib-devel may be installed. We may need to add > "Conflicts: libarrow-glib-devel" to Apache Arrow's > arrow-glib-devel to resolve this case automatically. Anyway, > this is not a ADBC problem. So it's not a blocker. > > > > Thanks, > -- > ko

[ANNOUNCE] Apache Arrow ADBC 12 released

2024-05-20 Thread David Li
The Apache Arrow community is pleased to announce the 12th release of the Apache Arrow ADBC libraries. It includes 56 resolved GitHub issues ([1]). Individual components are versioned separately: some packages are on version 0.12.0 and others are now version 1.0.0, with the release as a whole on

Re: [VOTE] Release Apache Arrow ADBC 12 - RC4

2024-05-20 Thread David Li
The vote passes with 4 binding, 2 non-binding +1 votes. +1 (binding): Raúl Cumplido, Dewey Dunnington, Weston Pace, David Li +1 (non-binding): Jean-Baptiste Onofré, Vibhatha Abeykoon I will take care of the post-release tasks next. On Tue, May 21, 2024, at 00:32, Weston Pace wrote: >

Re: [VOTE] Release Apache Arrow ADBC 12 - RC4

2024-05-20 Thread David Li
6 AM Jean-Baptiste Onofré > wrote: >> >> +1 (non binding) >> >> Testing on MacOS M2. >> >> Regards >> JB >> >> On Wed, May 15, 2024 at 7:00 AM David Li wrote: >> > >> > Hello, >> > >> > I would like to propo

[VOTE] Release Apache Arrow ADBC 12 - RC4

2024-05-14 Thread David Li
Hello, I would like to propose the following release candidate (RC4) of Apache Arrow ADBC version 12. This is a release consisting of 56 resolved GitHub issues [1]. Please note that the versioning scheme has changed. This is the 12th release of ADBC, and so is called version "12". The subcomp

Re: [VOTE] Release Apache Arrow 16.1.0 - RC1

2024-05-09 Thread David Li
+1 (binding) Tested sources with Conda on Debian 12/x86_64 (binaries failed due to download flakiness) On Fri, May 10, 2024, at 07:02, Rok Mihevc wrote: > +1 (non-binding) > > Ran: > TEST_DEFAULT=0 TEST_SOURCE=1 ./verify-release-candidate.sh 16.1.0 1 > On Ubuntu 22.04.1 x86_64 > > Thanks for the

[VOTE] Release Apache Arrow ADBC 12 - RC3

2024-05-08 Thread David Li
Hello, I would like to propose the following release candidate (RC3) of Apache Arrow ADBC version 12. This is a release consisting of 56 resolved GitHub issues [1]. Please note that the versioning scheme has changed. This is the 12th release of ADBC, and so is called version "12". The subcomp

Re: [VOTE] Release Apache Arrow ADBC 12 - RC0

2024-05-08 Thread David Li
It appears manylinux aarch64 wheels didn't get built, so rc1 will be incoming On Wed, May 8, 2024, at 14:44, David Li wrote: > Hello, > > I would like to propose the following release candidate (RC0) of Apache > Arrow ADBC version 12. This is a release consisting of 48 resolved

[VOTE] Release Apache Arrow ADBC 12 - RC0

2024-05-07 Thread David Li
Hello, I would like to propose the following release candidate (RC0) of Apache Arrow ADBC version 12. This is a release consisting of 48 resolved GitHub issues [1]. Please note that the versioning scheme has changed. This is the 12th release of ADBC, and so is called version "12". The subcomp

Re: [ANNOUNCE] New Arrow committer: Dane Pitkin

2024-05-07 Thread David Li
Congrats Dane! On Wed, May 8, 2024, at 09:46, Felipe Oliveira Carvalho wrote: > Great news. Congratulations Dane! > > On Tue, May 7, 2024 at 7:57 PM Vibhatha Abeykoon wrote: >> >> Congratulations Dane!!! >> >> Vibhatha Abeykoon >> >> >> On Wed, May 8, 2024 at 4:02 AM Jacob Wujciak wrote: >> >> >

Re: [VOTE][Format] JSON canonical extension type

2024-04-29 Thread David Li
+1 (binding) assuming we explicitly state RFC-8259 On Tue, Apr 30, 2024, at 08:02, Matt Topol wrote: > +1 (binding) > > On Mon, Apr 29, 2024 at 5:36 PM Ian Cook wrote: > >> +1 (non-binding) >> >> I added a comment in the PR suggesting that we explicitly refer to RFC-8259 >> in CanonicalExtension

Re: [DISCUSSION] New Flags for Arrow C Interface Schema

2024-04-23 Thread David Li
For scalars: Arrow doesn't define scalars. They're an implementation concept. (They may be a *useful* one, but if we want to define them more generally, that's a separate discussion.) For UDFs: UDFs are a system-specific interface. Presumably, that interface can encode whether an Arrow array is

Re: ADBC - OS-level driver manager

2024-04-23 Thread David Li
I'd rather not hard code it directly into the manager, both because this may surprise applications that don't want it and would be inflexible for applications who are looking to use it, but providing an additional list of search paths that (say) Excel can configure + some platform-specific guida

Re: [DISCUSS] Versioning and releases for apache/arrow components

2024-04-22 Thread David Li
Another possibility I'd like to float is doing this in ADBC first? My primary motivation is (1) from Joris's list: I'd like to bump a few components (Snowflake, maybe SQLite) to a "stable" version while leaving the others behind, and in this context I think it'd be much more helpful to users to

Re: Unsupported/Other Type

2024-04-17 Thread David Li
gt;> Ah! Well, I think this could be an interesting proposal, but someone >> should put a more formal proposal, perhaps as a draft PR. >> >> Regards >> >> Antoine. >> >> >> Le 17/04/2024 à 11:57, David Li a écrit : >> > For an unsupported/ot

Re: [VOTE] Release Apache Arrow 16.0.0 - RC0

2024-04-17 Thread David Li
+1 tested sources on Debian 12, x86-64 On Wed, Apr 17, 2024, at 18:14, Raúl Cumplido wrote: > Hi, > > Just a minor note, the binary verification for > verify-rc-binaries-wheels-windows failed with [1]. > This can be avoided by implementing the solution proposed in this > comment by Kou [2]. See m

Re: Unsupported/Other Type

2024-04-17 Thread David Li
On Wed, Apr 17, 2024, at 20:09, Antoine Pitrou wrote: > Ah! Well, I think this could be an interesting proposal, but someone > should put a more formal proposal, perhaps as a draft PR. > > Regards > > Antoine. > > > Le 17/04/2024 à 11:57, David Li a écrit : >> For a

Re: Unsupported/Other Type

2024-04-17 Thread David Li
For an unsupported/other extension type. On Wed, Apr 17, 2024, at 18:32, Antoine Pitrou wrote: > What is "this proposal"? > > > Le 17/04/2024 à 10:38, David Li a écrit : >> Should I take it that this proposal is dead in the water? While we could >> define our

Re: Unsupported/Other Type

2024-04-17 Thread David Li
Should I take it that this proposal is dead in the water? While we could define our own Unknown/Other type for say the ADBC PostgreSQL driver it might be useful to have a singular type for consumers to latch on to. On Fri, Apr 12, 2024, at 07:32, David Li wrote: > I think an "Other&qu

Re: Personal feedback on your last release on Apache Arrow ADBC 0.11.0

2024-04-17 Thread David Li
Hi Christofer, Sutou Kouhei is part of the PMC. Additionally, there is a result email: https://lists.apache.org/thread/gb5k69pd3k6lnbzw978fm7ppx1p9cx15 On Wed, Apr 17, 2024, at 16:52, Christofer Dutz wrote: > Hi all, > > while reviewing your projects activity in the last quarter as part of > m

Re: [ANNOUNCE] New Arrow committer: Sarah Gilmore

2024-04-11 Thread David Li
Congrats Sarah! On Fri, Apr 12, 2024, at 06:04, Joris Van den Bossche wrote: > Congrats! > > On Thu, 11 Apr 2024 at 22:56, Sarah Gilmore > wrote: >> >> Thank you everyone! It's been awesome working with everyone and look >> forwarding to continuing to do so! 😄 >>

Re: Unsupported/Other Type

2024-04-11 Thread David Li
I think an "Other" extension type is slightly different than an arbitrary extension type, though: the latter may be understood downstream but the former represents a point at which a component explicitly declares it does not know how to handle a field. In this example, the PostgreSQL ADBC driver

Re: Unsupported/Other Type

2024-04-10 Thread David Li
I think this should be an extension type, yes. It could be parametrized on the storage type; the other system might at least know that one type is based on another (e.g. a user defined type). Type metadata can be preserved in the extension type's metadata. I think it would be good to have stand

Re: [DISCUSS] Versioning and releases for apache/arrow components

2024-04-09 Thread David Li
Java has JNI parts, but I think they do not necessarily need to release at the same time as C++, especially since the JAR bundles the libraries; Java could just pick up the latest version of the C++ library whenever it releases. It would make it harder if the next step is to also decouple the re

[RESULT][VOTE] Bulk ingestion support for Flight SQL (vote #2)

2024-04-08 Thread David Li
The vote passes with 3 binding, 4 non-binding +1 votes. Thanks Joel! On Sat, Apr 6, 2024, at 17:56, Andrew Lamb wrote: > +1 > > On Sat, Apr 6, 2024 at 3:48 AM wish maple wrote: > >> +1 (non binding) >> >> Best, >> Xuwei Fu >> ulk ingestion support for

Re: [VOTE] Add new info codes and options keys to ADBC specification

2024-04-07 Thread David Li
+1 On Sat, Apr 6, 2024, at 22:20, Matt Topol wrote: > +1 > > On Sat, Apr 6, 2024, 4:54 AM Andrew Lamb wrote: > >> +1 >> >> On Fri, Apr 5, 2024 at 9:55 PM Jacob Wujciak >> wrote: >> >> > + 1 (non-binding) >> > >> > Am Sa., 6. Apr. 2024 um 01:57 Uhr schrieb Joel Lubinitsky < >> > joell...@gmail.co

[VOTE] Bulk ingestion support for Flight SQL (vote #2)

2024-04-05 Thread David Li
Hello, Joel Lubinitsky has proposed adding bulk ingestion support to Arrow Flight SQL [1]. This provides a path for uploading an Arrow dataset to a Flight SQL server to create or append to a table, without having to know the specifics of the SQL or Substrait support on the server. The functiona

Re: [Format][ADBC] GetObjects semantics for system objects

2024-04-02 Thread David Li
I don't see why we should exclude them; I would also caution against treating the driver behavior (especially only drivers in one language) as a reference. On Tue, Apr 2, 2024, at 23:04, Joel Lubinitsky wrote: > Hi, > > The ADBC spec does not currently define whether system > catalogs/schemas/tab

Re: [RESULT][VOTE] Release Apache Arrow ADBC 0.11.0 - RC0

2024-04-02 Thread David Li
] Publish release blog post On Sun, Mar 31, 2024, at 11:18, David Li wrote: > The vote carries with 3 binding, 3 non-binding +1 votes. > > I will handle the post-release tasks. @Dewey I would appreciate help > with CRAN as usual! > > [x] Close the GitHub milestone/project > [x

Re: [ANNOUNCE] New Committer Joel Lubinitsky

2024-04-01 Thread David Li
Congrats Joel! On Tue, Apr 2, 2024, at 05:42, Weston Pace wrote: > Congratulations Joel! > > On Mon, Apr 1, 2024 at 1:16 PM Bryce Mecum wrote: > >> Congrats, Joel! >> >> On Mon, Apr 1, 2024 at 6:59 AM Matt Topol wrote: >> > >> > On behalf of the Arrow PMC, I'm happy to announce that Joel Lubinit

[ANNOUNCE] Apache Arrow ADBC 0.11.0 released

2024-03-31 Thread David Li
The Apache Arrow community is pleased to announce the 0.11.0 release of the Apache Arrow ADBC libraries. It includes 36 resolved GitHub issues ([1]). The release is available now from [2] and [3]. Release notes are available at: https://github.com/apache/arrow-adbc/blob/apache-arrow-adbc-0.11.0

[RESULT][VOTE] Release Apache Arrow ADBC 0.11.0 - RC0

2024-03-31 Thread David Li
[ ] Announce the new release [ ] Remove old artifacts [ ] Bump versions [ ] Publish release blog post On Sat, Mar 30, 2024, at 02:33, Jean-Baptiste Onofré wrote: > +1 (non binding) > > Regards > JB > > On Thu, Mar 28, 2024 at 4:07 PM David Li wrote: >> >> Hello, >

Re: [VOTE] Protocol for Dissociated Arrow IPC Transports

2024-03-31 Thread David Li
t; [ ] -1 Do not accept this proposal because... >> > >> > Thanks everyone! >> > >> > On Wed, Mar 27, 2024 at 7:51 PM Benjamin Kietzman >> > wrote: >> > >> > > +1 >> > > >> > > On Tue, Mar 26, 2024, 18:3

[VOTE] Release Apache Arrow ADBC 0.11.0 - RC0

2024-03-28 Thread David Li
Hello, I would like to propose the following release candidate (RC0) of Apache Arrow ADBC version 0.11.0. This is a release consisting of 36 resolved GitHub issues [1]. This release candidate is based on commit: 3cb5825bf551ae93d0e9ed2f64be226b569b27a7 [2] The source release rc0 is hosted at

Re: [DISCUSS] Flight SQL Experimental status in docs

2024-03-21 Thread David Li
There's a ticket for it already if anyone would like to help On Thu, Mar 21, 2024, at 12:02, James Duong wrote: > Hi, > > We fairly recently merged a PR to remove the experimental status for > Flight SQL from maven: https://github.com/apache/arrow/pull/39040 > > However our public documentation s

Re: [VOTE] Stateless prepared statements in FlightSQL

2024-03-21 Thread David Li
+1 Thank you Adam! On Thu, Mar 21, 2024, at 10:07, Andrew Lamb wrote: > +1 (binding) > > I reviewed the spec proposal and the rust implementation and I think they > look good to go. I am not as confident on the golang implementation, but > the comments on the Go PR look like there are no objecti

Re: [VOTE] Protocol for Dissociated Arrow IPC Transports

2024-03-21 Thread David Li
I think let's try again. Would it be reasonable to declare this 'experimental' for the time being, just as we did with Flight/Flight SQL/etc? On Tue, Mar 19, 2024, at 15:24, Matt Topol wrote: > Hey All, It's been another month and we've gotten a whole bunch of feedback > and engagement on the doc

Re: [ANNOUNCE] New Arrow committer: Bryce Mecum

2024-03-18 Thread David Li
Congrats Bryce! On Mon, Mar 18, 2024, at 08:52, Ian Cook wrote: > Congratulations Bryce! > > Ian > > On Sun, Mar 17, 2024 at 22:24 Nic Crane wrote: > >> On behalf of the Arrow PMC, I'm happy to announce that Bryce Mecum has >> accepted an invitation to become a committer on Apache Arrow. Welcome,

Re: [VOTE] Release Apache Arrow 15.0.2 - RC3

2024-03-16 Thread David Li
+1 tested on Debian Linux/Conda/x86_64 On Sat, Mar 16, 2024, at 13:43, Ruoxi Sun wrote: > +1 (non-binding) > > On my Intel Mac, OS version Sonoma 14.2.1 (23C71). > > TEST_DEFAULT=0 TEST_GO=1 TEST_CPP=1 ./verify-release-candidate.sh 15.0.2 3 > > I also tried to verify Python > > TEST_DEFAULT=0 TES

Re: [VOTE] Release Apache Arrow 15.0.1 - RC0

2024-03-05 Thread David Li
+1 Tested on Debian 12/x86_64 with Conda I had some issues with APT/YUM packages but figure it's due to my local setup On Tue, Mar 5, 2024, at 01:46, Jean-Baptiste Onofré wrote: > +1 (non binding) > > I checked: > - signature and hashes look good > - build OK > - ASF headers present > - No probl

[RESULT][VOTE] Flight RPC: add 'fallback' URI scheme

2024-03-04 Thread David Li
' URI scheme" on Tue, 27 Feb 2024 > 09:01:36 -0500, > "David Li" wrote: > >> I would like to propose a 'reuse connection' URI scheme for Flight RPC. This >> proposal was previously discussed at [1]. A candidate implementation for >

Re: [DISCUSS] [FlightSQL] Supporting parameters and prepared statements with a stateless server

2024-03-02 Thread David Li
The proposal looks good to me. Thanks! On Fri, Mar 1, 2024, at 20:11, Adam Curtis wrote: > Hello, > > We would like to support prepared statements with bind parameters with > a stateless service. This was discussed previously on the mailing list > [1]. The original ticket outlining the proposed de

Re: [VOTE] Flight RPC: add 'fallback' URI scheme

2024-03-01 Thread David Li
My vote: +1 On Wed, Feb 28, 2024, at 15:50, Joel Lubinitsky wrote: > +1 > > On Wed, Feb 28, 2024 at 3:22 PM Andrew Lamb wrote: > >> +1 >> >> >> On Tue, Feb 27, 2024 at 9:06 AM David Li wrote: >> >> > I would like to propose a 'reuse conne

Re: [VOTE] Move Arrow DataFusion Subproject to new Top Level Apache Project

2024-03-01 Thread David Li
+1 On Fri, Mar 1, 2024, at 12:06, Jorge Cardoso Leitão wrote: > +1 - great work!!! > > On Fri, Mar 1, 2024 at 5:49 PM Micah Kornfield > wrote: > >> +1 (binding) >> >> On Friday, March 1, 2024, Uwe L. Korn wrote: >> >> > +1 (binding) >> > >> > On Fri, Mar 1, 2024, at 2:37 PM, Andy Grove wrote: >>

[VOTE] Flight RPC: add 'fallback' URI scheme

2024-02-27 Thread David Li
use... [1]: https://lists.apache.org/thread/pc9fs0hf8t5ylj9os00r9vg8d2xv2npz [2]: https://github.com/apache/arrow/pull/40084 On Tue, Feb 20, 2024, at 14:14, David Li wrote: > Thanks for the comments - I've updated the implementation [1] and added > Go + integration tests. If this all checks

Re: [RESULT][VOTE] Release Apache Arrow ADBC 0.10.0 - RC1

2024-02-22 Thread David Li
/pull/21 [2]: https://github.com/apache/arrow-adbc/pull/1560 [3]: https://github.com/apache/arrow-site/pull/477 On Thu, Feb 22, 2024, at 09:12, David Li wrote: > The vote passes with 4 binding, 2 non-binding +1 votes. > > I'll take care of the release tasks. > > On Wed, Feb 21,

[ANNOUNCE] Apache Arrow ADBC 0.10.0 released

2024-02-22 Thread David Li
The Apache Arrow community is pleased to announce the 0.10.0 release of the Apache Arrow ADBC libraries. It includes 31 resolved GitHub issues ([1]). The release is available now from [2] and [3]. Release notes are available at: https://github.com/apache/arrow-adbc/blob/apache-arrow-adbc-0.10.0

[RESULT][VOTE] Release Apache Arrow ADBC 0.10.0 - RC1

2024-02-22 Thread David Li
> +1! >> >> I ran USE_CONDA=1 dev/release/verify-release-candidate.sh 0.10.0 1 on >> MacOS Sonoma (M1). >> >> On Tue, Feb 20, 2024 at 9:43 AM Jean-Baptiste Onofré >> wrote: >> > >> > +1 (non binding) >> > >> > I quickly te

Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-20 Thread David Li
case is great, but agree with the other > reviewers that the name may be confusing. I left some notes on the ticket > > Andrew > > On Wed, Feb 14, 2024 at 3:52 PM David Li wrote: > >> I've put up a candidate implementation sans integration test [1]. >> >>

[RESULT][VOTE] Explicit session management for Flight RPC

2024-02-20 Thread David Li
The vote passes with 3 binding, 3 non-binding +1 votes. Thanks all! On Mon, Feb 19, 2024, at 09:16, David Li wrote: > My vote: +1 > > On Sun, Feb 18, 2024, at 07:06, Joel Lubinitsky wrote: >> +1 >> >> On Fri, Feb 16, 2024 at 1:07 PM Andrew Lamb wrote: >> >>

Re: [VOTE] Explicit session management for Flight RPC

2024-02-19 Thread David Li
My vote: +1 On Sun, Feb 18, 2024, at 07:06, Joel Lubinitsky wrote: > +1 > > On Fri, Feb 16, 2024 at 1:07 PM Andrew Lamb wrote: > >> +1 >> >> On Fri, Feb 16, 2024 at 1:46 AM Jean-Baptiste Onofré >> wrote: >> >> > +1 >> > >> &g

Re: [VOTE] Release Apache Arrow ADBC 0.10.0 - RC1

2024-02-19 Thread David Li
My vote: +1 Tested on macOS/AArch64 On Mon, Feb 19, 2024, at 08:24, Raúl Cumplido wrote: > +1 > > I have verified successfully on Ubuntu 22.04 with: > USE_CONDA=1 dev/release/verify-release-candidate.sh 0.10.0 1 > > El dom, 18 feb 2024 a las 21:48, David Li () escribió: &g

[VOTE] Release Apache Arrow ADBC 0.10.0 - RC1

2024-02-18 Thread David Li
Hello, I would like to propose the following release candidate (RC1) of Apache Arrow ADBC version 0.10.0. This is a release consisting of 30 resolved GitHub issues [1]. This release candidate is based on commit: 9a8e44cc62f23a68ffc0d3d4c7362214b221bea0 [2] The source release rc1 is hosted at

Re: Flight RPC: Add total_records and total_bytes to FlightEndpoint

2024-02-15 Thread David Li
Hi Taeyun, Is this related to the previous thread about fetching a part of a result set? I think it's reasonable to have these fields. If others agree the next step would be to create a PR with an implementation for review/voting. Best, David On Wed, Feb 14, 2024, at 18:58, 김태연 (Taeyun Kim)

Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-14 Thread David Li
ofré wrote: > Hi David, > > It's reasonable. I think we can start with your initial proposal (it > sounds fine to me) and we can always improve step by step. > > Thanks ! > Regards > JB > > On Tue, Feb 13, 2024 at 4:53 PM David Li wrote: >> >> I'm g

[VOTE] Explicit session management for Flight RPC

2024-02-14 Thread David Li
Paul Nienaber would like to propose explicit session management for Flight RPC. This proposal was previously discussed at [1]. A candidate implementation for C++ and Java is at [2]. The vote will be open for at least 72 hours. [ ] +1 [ ] +0 [ ] -1 Do not accept this proposal because... [1]:

Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-13 Thread David Li
ow one would switch from e.g. grpc+tls to > http without an explicit server location (unless both Flight servers are > hosted under the same port?). So the "+" proposal seems a bit weird. > > > Le 12/02/2024 à 23:39, David Li a écrit : >> The idea is that the cl

Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-12 Thread David Li
rpc://0.0.0.0", "ucx://0.0.0.0", "grpc://1.2.3.4", ...] > > This would indicate that grpc and ucx transport is available from the > current service, grpc is available at 1.2.3.4, and possibly more > combinations of scheme/host. > > [1] https://datatracker.

Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-12 Thread David Li
2024 at 6:05 AM >> To: dev@arrow.apache.org >> Subject: Re: [DISCUSS] Flight RPC: add 'fallback' URI scheme >> >> Hello, >> >> This looks fine to me. >> >> Regards >> >> Antoine. >> >> >> Le 12/02/2024 à 14

Re: DISCUSS: [FlightSQL] Catalog support

2024-02-12 Thread David Li
ov 15, 2023, at 15:31, David Li wrote: > Hi all, > > Paul and company have been working on this feature for Flight SQL via > support for explicit sessions in Flight SQL. Feedback would be much > appreciated on the PR, especially before tackling the second > implementation and eventua

[DISCUSS] Flight RPC: add 'fallback' URI scheme

2024-02-12 Thread David Li
Hello, I'd like to propose a slight update to Flight RPC to make Flight SQL work better in different deployment scenarios. Comments on the doc would be appreciated: https://docs.google.com/document/d/1g9M9FmsZhkewlT1mLibuceQO8ugI0-fqumVAXKFjVGg/edit?usp=sharing The gist is that FlightEndpoint

Re: [DISCUSS] Proposal to expand Arrow Communications

2024-02-02 Thread David Li
I like this new direction, and I think it'll be actually viable unlike the Flight-UCX work that was attempted a couple years ago. I think the hardcoded endpoints in Flight RPC are difficult for other projects (including Flight SQL!) to build on top of, and we would serve users better by - stand

Arrow community meeting January 31 at 17:00 UTC

2024-01-31 Thread David Li
Hello all, Apologies for the late notice. I believe Ian is busy this week, but we will still be having our usual call at our usual time (i.e., right when I'm sending this). Zoom meeting URL: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Meeting ID: 876 4903 3008 Passcode: 9

Re: [VOTE] Release Apache Arrow nanoarrow 0.4.0 - RC0

2024-01-29 Thread David Li
+1 (binding) Tested on Debian Linux 'bookworm' On Mon, Jan 29, 2024, at 10:45, Dane Pitkin wrote: > +1 (non-binding) > > Verified on MacOS 14 using conda. > > On Mon, Jan 29, 2024 at 10:11 AM Dewey Dunnington > wrote: > >> Hello, >> >> I would like to propose the following release candidate (rc

Re: [VOTE] Accept donation of Comet Spark native engine

2024-01-27 Thread David Li
+1 (binding) On Sat, Jan 27, 2024, at 13:03, L. C. Hsieh wrote: > +1 (binding) > > On Sat, Jan 27, 2024 at 8:10 AM Andrew Lamb wrote: >> >> +1 (binding) >> >> This is super exciting >> >> On Sat, Jan 27, 2024 at 11:00 AM Daniël Heres wrote: >> >> > +1 (binding). Awesome addition to the DataFusio

Re: [VOTE] Release Apache Arrow 15.0.0 - RC1

2024-01-19 Thread David Li
+1 (binding) Verified sources on Debian 12 'bookworm'. I had issues with binaries but that was because of AlmaLinux failing to verify its own GPG key for some reason. On Thu, Jan 18, 2024, at 04:40, Raúl Cumplido wrote: > El mié, 17 ene 2024 a las 23:37, Matt Topol () > escribió: >> >> Yea, I c

  1   2   3   4   5   6   7   8   9   >