Re: [PR] GH-46157: [C++] Move test utility RunEndEncodeTableColumns that uses REE to test_util on compute instead of common gtest_util [arrow]

2025-04-22 Thread via GitHub
kou commented on code in PR #46161: URL: https://github.com/apache/arrow/pull/46161#discussion_r2055383157 ## cpp/src/arrow/compute/kernels/test_util.h: ## @@ -0,0 +1,30 @@ +// Licensed to the Apache Software Foundation (ASF) under one Review Comment: Ah, we want to prevent

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
kou commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055370133 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm pyth

Re: [PR] GH-46087: [FlightSQL] Allow returning column remarks in FlightSQL's CommandGetTables [arrow]

2025-04-22 Thread via GitHub
mateuszrzeszutek commented on PR #46110: URL: https://github.com/apache/arrow/pull/46110#issuecomment-2823246444 > I believe if we want to add this to the spec we should have implementations in at least one other language (preferably both Go/Java if possible, though) and vote on it, as triv

Re: [PR] GH-46087: [FlightSQL] Allow returning column remarks in FlightSQL's CommandGetTables [arrow]

2025-04-22 Thread via GitHub
mateuszrzeszutek commented on code in PR #46110: URL: https://github.com/apache/arrow/pull/46110#discussion_r2055369358 ## format/FlightSql.proto: ## @@ -1212,6 +1212,7 @@ message CommandGetDbSchemas { * - ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is c

Re: [PR] GH-46157: [C++] Move test utility RunEndEncodeTableColumns that uses REE to test_util on compute instead of common gtest_util [arrow]

2025-04-22 Thread via GitHub
kou commented on code in PR #46161: URL: https://github.com/apache/arrow/pull/46161#discussion_r2055366980 ## cpp/src/arrow/compute/kernels/test_util.h: ## @@ -0,0 +1,30 @@ +// Licensed to the Apache Software Foundation (ASF) under one Review Comment: I think that we need to

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055352618 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm

Re: [PR] GH-46157: [C++] Move test utility RunEndEncodeTableColumns that uses REE to test_util on compute instead of common gtest_util [arrow]

2025-04-22 Thread via GitHub
pitrou commented on code in PR #46161: URL: https://github.com/apache/arrow/pull/46161#discussion_r2055348868 ## cpp/src/arrow/compute/kernels/test_util.h: ## @@ -0,0 +1,30 @@ +// Licensed to the Apache Software Foundation (ASF) under one Review Comment: If `test_util_intern

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#issuecomment-2823214062 Or rather, explicitly install a version newer than 3.12 (in Ubuntu 22.04) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#issuecomment-2823212975 Hmm, that's an odd failure. I guess we need to explicitly install protoc somewhere. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
kou commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055344114 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm pyth

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055338777 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm

Re: [PR] GH-725: Added ExtensionReader [arrow-java]

2025-04-22 Thread via GitHub
github-actions[bot] commented on PR #726: URL: https://github.com/apache/arrow-java/pull/726#issuecomment-2823193285 Thank you for opening a pull request! Please label the PR with one or more of: - bug-fix - chore - dependencies - documentation - enhancement

[PR] GH-725: Added ExtensionReader [arrow-java]

2025-04-22 Thread via GitHub
xxlaykxx opened a new pull request, #726: URL: https://github.com/apache/arrow-java/pull/726 ## What's Changed ExtensionReader was added to support reading extension types from a complex vector. It contains **read(ExtensionHolder)** method for reading to the holder. And **readObject*

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
kou commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055327980 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm pyth

Re: [PR] chore(rust): bump the arrow-datafusion group in /rust with 2 updates [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm merged PR #2732: URL: https://github.com/apache/arrow-adbc/pull/2732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [I] [Python] Boolean array basic biops results are wrong! [arrow]

2025-04-22 Thread via GitHub
cmpute commented on issue #46187: URL: https://github.com/apache/arrow/issues/46187#issuecomment-2823159042 Thanks for your reply! If the 'and' keyword is not correct, why does BooleanArray support it? -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] [Python] Impossible to import pyarrow module in python. Generates this "ImportError: DLL load failed: The specified procedure could not be found." [arrow]

2025-04-22 Thread via GitHub
mir-math commented on issue #26138: URL: https://github.com/apache/arrow/issues/26138#issuecomment-2823157793 Same error ("ImportError: DLL load failed") here. Solved by downgrading pyarrow from 19.x.x to 15.0.2 (both were installed via pip). While researching made several clean inst

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2055302972 ## .github/workflows/nightly-verify.yml: ## @@ -201,3 +205,31 @@ jobs: run: | pushd arrow-adbc docker compose run -e PYTHON=3.12 --rm

Re: [PR] GH-45957: [Python] Expose `allow_delayed_open` on S3FileSystem [arrow]

2025-04-22 Thread via GitHub
AlenkaF commented on PR #46078: URL: https://github.com/apache/arrow/pull/46078#issuecomment-2823016243 @raulcd, whenever you get a chance, a second pair of eyes would be great. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

Re: [PR] GH-45957: [Python] Expose `allow_delayed_open` on S3FileSystem [arrow]

2025-04-22 Thread via GitHub
AlenkaF commented on code in PR #46078: URL: https://github.com/apache/arrow/pull/46078#discussion_r2055217577 ## python/pyarrow/tests/test_fs.py: ## @@ -1230,7 +1230,8 @@ def test_s3_options(pickle_module): assert pickle_module.loads(pickle_module.dumps(fs2)) == fs2 a

Re: [PR] fix(go/adbc/driver/snowflake): implement ability to set database options after initialization [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm merged PR #2728: URL: https://github.com/apache/arrow-adbc/pull/2728 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [I] [Python] Error thrown when multiplying decimal numbers [arrow]

2025-04-22 Thread via GitHub
sfc-gh-gfrere commented on issue #43252: URL: https://github.com/apache/arrow/issues/43252#issuecomment-2822965772 Hi @khwilson — are you referring to this check? https://github.com/apache/arrow/blob/0cc4855de73777524f1419a7e11ab15ba2abd645/cpp/src/gandiva/decimal_type_util.cc#L59. This

Re: [PR] GH-37891: [C++] Refine class parquet::encryption::FileSystemKeyMaterialStore [arrow]

2025-04-22 Thread via GitHub
wgtmac commented on PR #46202: URL: https://github.com/apache/arrow/pull/46202#issuecomment-2822962291 Yes, I think we should not create similar PRs for a single issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] GH-46085: [C++][FS][Azure] Treat a 403 when getting container properties like the container exists [arrow]

2025-04-22 Thread via GitHub
kou commented on code in PR #46095: URL: https://github.com/apache/arrow/pull/46095#discussion_r2055174926 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -570,6 +570,17 @@ bool IsContainerNotFound(const Storage::StorageException& e) { return false; } +bool IsForbidden(cons

Re: [PR] GH-37891: [C++] Refine class parquet::encryption::FileSystemKeyMaterialStore [arrow]

2025-04-22 Thread via GitHub
kapoisu commented on PR #46202: URL: https://github.com/apache/arrow/pull/46202#issuecomment-2822932333 > Are these the only places that we can apply it? Any other similar place in the parquet subdirectory? I'm sure there are more (e.g. FileKeyWrapper). Further changes can be include

Re: [PR] GH-46061: [Python] Replace setup.py invocations with modern commands [arrow]

2025-04-22 Thread via GitHub
WillAyd commented on PR #46062: URL: https://github.com/apache/arrow/pull/46062#issuecomment-2822904813 The current crossbow failures seem unrelated to this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] GH-46157: [C++] Move test utility RunEndEncodeTableColumns that uses REE to test_util on compute instead of common gtest_util [arrow]

2025-04-22 Thread via GitHub
kou commented on code in PR #46161: URL: https://github.com/apache/arrow/pull/46161#discussion_r2055121029 ## cpp/src/arrow/compute/kernels/test_util_test.cc: ## @@ -0,0 +1,38 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agree

Re: [I] [C++] Error linking Parquet [arrow]

2025-04-22 Thread via GitHub
wgtmac commented on issue #46188: URL: https://github.com/apache/arrow/issues/46188#issuecomment-2822847220 Could you please share your full build log with the precise steps to reproduce it? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] GH-46209: [Documentation][C++][Compute] Documentation for row table [arrow]

2025-04-22 Thread via GitHub
zanmato1984 commented on PR #46210: URL: https://github.com/apache/arrow/pull/46210#issuecomment-2822845865 @pitrou @amoeba @mapleFU mind to take a look? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] [Parquet][C++] Logical types with sort order UNKNOWN are missing null_count statistics [arrow]

2025-04-22 Thread via GitHub
wgtmac commented on issue #46205: URL: https://github.com/apache/arrow/issues/46205#issuecomment-2822845340 The current `parquet::Statistics` implementation is tied to `TypeDefinedOrder`. We will also add a `IEEE754TotalOrder` as proposed by https://github.com/apache/parquet-format/pull/221

[PR] GH-46209: [Documentation][C++][Compute] Documentation for row table [arrow]

2025-04-22 Thread via GitHub
zanmato1984 opened a new pull request, #46210: URL: https://github.com/apache/arrow/pull/46210 ### Rationale for this change ### What changes are included in this PR? Documentation for row table. ### Are these changes tested? No need. ### Are there any user-

Re: [PR] GH-587: [Release] Add .env description to dev/release/README.md [arrow-java]

2025-04-22 Thread via GitHub
kou merged PR #724: URL: https://github.com/apache/arrow-java/pull/724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.o

Re: [PR] GH-698: Improve and fix Avro read consumers [arrow-java]

2025-04-22 Thread via GitHub
lidavidm merged PR #718: URL: https://github.com/apache/arrow-java/pull/718 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm merged PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] GH-37891: [C++] Refine class parquet::encryption::FileSystemKeyMaterialStore [arrow]

2025-04-22 Thread via GitHub
wgtmac commented on PR #46202: URL: https://github.com/apache/arrow/pull/46202#issuecomment-2822809864 Are these the only places that we can apply it? Any other similar place in the parquet subdirectory? -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] GH-587: [Release] Add .env description to dev/release/README.md [arrow-java]

2025-04-22 Thread via GitHub
github-actions[bot] commented on PR #724: URL: https://github.com/apache/arrow-java/pull/724#issuecomment-2822790151 Thank you for opening a pull request! Please label the PR with one or more of: - bug-fix - chore - dependencies - documentation - enhancement

Re: [I] [Release] 18.3.0 [arrow-java]

2025-04-22 Thread via GitHub
kou commented on issue #723: URL: https://github.com/apache/arrow-java/issues/723#issuecomment-2822790111 > I'll do [#587](https://github.com/apache/arrow-java/issues/587) but it's for release manager. We don't need to add it to 18.3.0. #724 -- This is an automated message from the

[PR] GH-587: [Release] Add .env description to dev/release/README.md [arrow-java]

2025-04-22 Thread via GitHub
kou opened a new pull request, #724: URL: https://github.com/apache/arrow-java/pull/724 ## What's Changed Add missing `dev/release/.env` description to `dev/release/README.md`. This also moves `GH_TOKEN` to `dev/release/.env`. Closes #587. -- This is an automated m

Re: [PR] feat(csharp/src/Drivers/BigQuery): Add support for AAD/Entra authentication [arrow-adbc]

2025-04-22 Thread via GitHub
davidhcoe commented on code in PR #2655: URL: https://github.com/apache/arrow-adbc/pull/2655#discussion_r2055084479 ## csharp/src/Drivers/BigQuery/BigQueryConnection.cs: ## @@ -337,33 +419,49 @@ private IArrowArray[] GetCatalogs( StringArray.Builder catalogNameBuild

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736#issuecomment-2822780316 LGTM, will merge after CI - thanks Felipe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] feat(csharp/src/Drivers/Databricks) Add option to enable using direct results for queries [arrow-adbc]

2025-04-22 Thread via GitHub
alexguo-db opened a new pull request, #2737: URL: https://github.com/apache/arrow-adbc/pull/2737 - Add option to set EnableDirectResults, which sends getDirectResults in the Thrift execute statement request - If getDirectResults is set in the request, then directResults is set on the res

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736#issuecomment-2822772006 @lidavidm if you're OK with the comment update, please merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] fix(go/adbc/driver/snowflake): implement ability to set database options after initialization [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on PR #2728: URL: https://github.com/apache/arrow-adbc/pull/2728#issuecomment-2822761711 > LGTM > > Please update the PR description, it seems this does a few other minor things beyond just implementing PostInitOptions I guess you're referring to the additio

Re: [PR] GH-46197: [C++] Tests use legacy timezones [arrow]

2025-04-22 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #46201: URL: https://github.com/apache/arrow/pull/46201#issuecomment-2822764248 After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 0cc4855de73777524f1419a7e11ab15ba2abd645. There were no

Re: [PR] fix(go/adbc/driver/snowflake): implement ability to set database options after initialization [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on code in PR #2728: URL: https://github.com/apache/arrow-adbc/pull/2728#discussion_r2055069567 ## go/adbc/driver/snowflake/snowflake_database.go: ## @@ -156,284 +162,300 @@ func (d *databaseImpl) SetOptions(cnOptions map[string]string) error {

Re: [PR] fix(go/adbc/driver/snowflake): implement ability to set database options after initialization [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on code in PR #2728: URL: https://github.com/apache/arrow-adbc/pull/2728#discussion_r2055069397 ## go/adbc/driver/snowflake/snowflake_database.go: ## @@ -156,284 +162,300 @@ func (d *databaseImpl) SetOptions(cnOptions map[string]string) error {

Re: [PR] GH-46087: [FlightSQL] Allow returning column remarks in FlightSQL's CommandGetTables [arrow]

2025-04-22 Thread via GitHub
lidavidm commented on code in PR #46110: URL: https://github.com/apache/arrow/pull/46110#discussion_r2055056063 ## format/FlightSql.proto: ## @@ -1212,6 +1212,7 @@ message CommandGetDbSchemas { * - ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is case-sens

Re: [PR] GH-46087: [FlightSQL] Allow returning column remarks in FlightSQL's CommandGetTables [arrow]

2025-04-22 Thread via GitHub
lidavidm commented on PR #46110: URL: https://github.com/apache/arrow/pull/46110#issuecomment-2822733906 I believe if we want to add this to the spec we should have implementations in at least one other language (preferably both Go/Java if possible, though) and vote on it, as trivial as it

Re: [I] [Release] 18.3.0 [arrow-java]

2025-04-22 Thread via GitHub
lidavidm commented on issue #723: URL: https://github.com/apache/arrow-java/issues/723#issuecomment-2822732665 I'll merge https://github.com/apache/arrow-java/pull/718 today for 18.3 as @martin-traverse has been plugging away at the Avro improvements for a while Otherwise, I went thro

Re: [PR] GH-698: Improve and fix Avro read consumers [arrow-java]

2025-04-22 Thread via GitHub
martin-traverse commented on PR #718: URL: https://github.com/apache/arrow-java/pull/718#issuecomment-2822728753 > Looks like there are some lint errors to be fixed Apologies - I have reapplied spotless, should be ok now! -- This is an automated message from the Apache Git Service.

Re: [PR] GH-721: Allow using 1GB+ data buffers in variable width vectors [arrow-java]

2025-04-22 Thread via GitHub
lidavidm merged PR #722: URL: https://github.com/apache/arrow-java/pull/722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [I] [Java] VectorSchemaRoot#addVector() cannot add a vector to the end of the current vector collection [arrow-java]

2025-04-22 Thread via GitHub
lidavidm commented on issue #301: URL: https://github.com/apache/arrow-java/issues/301#issuecomment-2822711065 I suppose you can always create a new root, but given the existence of the method I would support fixing this too. PRs are welcome. -- This is an automated message from the Apach

Re: [PR] build(c): Remove unnecessary test branching from Meson configuration [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm merged PR #2734: URL: https://github.com/apache/arrow-adbc/pull/2734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736#issuecomment-2822708652 It seems we should update that comment, though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] chore(rust): make Arrow version selection more flexible [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm commented on PR #2525: URL: https://github.com/apache/arrow-adbc/pull/2525#issuecomment-2822702472 Why can't we split workspaces in this repo? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] fix(c): Add libdl as dependency of driver manager in Meson [arrow-adbc]

2025-04-22 Thread via GitHub
lidavidm merged PR #2735: URL: https://github.com/apache/arrow-adbc/pull/2735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] GH-45957: [Python] Expose `allow_delayed_open` on S3FileSystem [arrow]

2025-04-22 Thread via GitHub
gmcrocetti commented on code in PR #46078: URL: https://github.com/apache/arrow/pull/46078#discussion_r2055003856 ## python/pyarrow/tests/test_fs.py: ## @@ -1230,7 +1230,8 @@ def test_s3_options(pickle_module): assert pickle_module.loads(pickle_module.dumps(fs2)) == fs2

Re: [PR] GH-45957: [Python] Expose `allow_delayed_open` on S3FileSystem [arrow]

2025-04-22 Thread via GitHub
gmcrocetti commented on code in PR #46078: URL: https://github.com/apache/arrow/pull/46078#discussion_r2055003856 ## python/pyarrow/tests/test_fs.py: ## @@ -1230,7 +1230,8 @@ def test_s3_options(pickle_module): assert pickle_module.loads(pickle_module.dumps(fs2)) == fs2

Re: [I] Add end-user logging and tracing for drivers [arrow-adbc]

2025-04-22 Thread via GitHub
birschick-bq commented on issue #2210: URL: https://github.com/apache/arrow-adbc/issues/2210#issuecomment-2822609900 @CurtHagenlocher > Is there a standard format for specifying a receiver and all its arguments which could be passed as e.g. an ADBC option value? ## Via pro

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add prefetch for direct result [arrow-adbc]

2025-04-22 Thread via GitHub
eric-wang-1990 closed pull request #2666: feat(csharp/src/Drivers/Apache/Spark): Add prefetch for direct result URL: https://github.com/apache/arrow-adbc/pull/2666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): Add prefetch for direct result [arrow-adbc]

2025-04-22 Thread via GitHub
eric-wang-1990 commented on PR #2666: URL: https://github.com/apache/arrow-adbc/pull/2666#issuecomment-2822578914 We will not check this in, based on @birschick-bq analysis before. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [I] [C++][Packaging] Remove pin for grpc-cpp in conda_env_cpp.txt [arrow]

2025-04-22 Thread via GitHub
kou commented on issue #46137: URL: https://github.com/apache/arrow/issues/46137#issuecomment-2822571889 Could you try debug print? ```diff diff --git a/cpp/cmake_modules/ThirdpartyToolchain.cmake b/cpp/cmake_modules/ThirdpartyToolchain.cmake index 1af347914f..2ef62f83f5 100644

Re: [PR] ci: verify source using OS-installed dependencies [arrow-adbc]

2025-04-22 Thread via GitHub
kou commented on code in PR #2718: URL: https://github.com/apache/arrow-adbc/pull/2718#discussion_r2054929718 ## ci/scripts/verify_ubuntu.sh: ## @@ -0,0 +1,104 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [I] [Release] 18.3.0 [arrow-java]

2025-04-22 Thread via GitHub
kou commented on issue #723: URL: https://github.com/apache/arrow-java/issues/723#issuecomment-2822567287 I'll do https://github.com/apache/arrow-java/issues/587 but it's for release manager. We don't need to add it to 18.3.0. -- This is an automated message from the Apache Git Service. T

Re: [PR] Variant Support for Arrow and Parquet [DRAFT] [arrow-rs]

2025-04-22 Thread via GitHub
PinkCrow007 commented on PR #7404: URL: https://github.com/apache/arrow-rs/pull/7404#issuecomment-2822560319 Thanks everyone for the helpful suggestions! Just a quick update — I’ve adjusted the design to use ExtensionType over **Struct** in Arrow and **GroupType** in Parquet. Read/write Par

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736#issuecomment-2822532180 https://github.com/apache/arrow-adbc/issues/2715#issuecomment-2817443865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on PR #2736: URL: https://github.com/apache/arrow-adbc/pull/2736#issuecomment-2822519118 > Is it specified somewhere that all driver implementations must be thread-safe? > > This comment explains the rationale: > > https://github.com/apache/arrow-adbc/blob/

Re: [PR] WIP: [Release] Verify release-20.0.0-rc2 [arrow]

2025-04-22 Thread via GitHub
kou commented on PR #46208: URL: https://github.com/apache/arrow/pull/46208#issuecomment-2822503492 Revision: apache-arrow-20.0.0-rc2 Submitted crossbow builds: [ursacomputing/crossbow @ release-20.0.0-rc2-0](https://github.com/ursacomputing/crossbow/branches/all?query=release-20.0.0-

Re: [PR] WIP: [Release] Verify release-20.0.0-rc2 [arrow]

2025-04-22 Thread via GitHub
github-actions[bot] commented on PR #46208: URL: https://github.com/apache/arrow/pull/46208#issuecomment-2822500946 Revision: 3ad0370a04ccdae638755b94c3c31c8760a11193 Submitted crossbow builds: [ursacomputing/crossbow @ actions-6870de9441](https://github.com/ursacomputing/crossbow/bra

Re: [PR] WIP: [Release] Verify release-20.0.0-rc2 [arrow]

2025-04-22 Thread via GitHub
kou commented on PR #46208: URL: https://github.com/apache/arrow/pull/46208#issuecomment-2822495621 @github-actions crossbow submit --group verify-rc-source --param release=20.0.0 --param rc=2 -- This is an automated message from the Apache Git Service. To respond to the message, please

[PR] WIP: [Release] Verify release-20.0.0-rc2 [arrow]

2025-04-22 Thread via GitHub
kou opened a new pull request, #46208: URL: https://github.com/apache/arrow/pull/46208 PR to verify Release Candidate -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] feat(parquet): add variant encoder/decoder [arrow-go]

2025-04-22 Thread via GitHub
sfc-gh-mbojanczyk commented on code in PR #344: URL: https://github.com/apache/arrow-go/pull/344#discussion_r2054836925 ## parquet/variants/util.go: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] feat(parquet): add variant encoder/decoder [arrow-go]

2025-04-22 Thread via GitHub
sfc-gh-mbojanczyk commented on code in PR #344: URL: https://github.com/apache/arrow-go/pull/344#discussion_r2054827281 ## parquet/variants/util.go: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [PR] GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs [arrow]

2025-04-22 Thread via GitHub
amoeba commented on PR #46180: URL: https://github.com/apache/arrow/pull/46180#issuecomment-2822414180 Latest docs preview: - User guide: http://crossbow.voltrondata.com/pr_docs/46180/cpp/arrays.html#fromjson-helpers - API docs: http://crossbow.voltrondata.com/pr_docs/46180/cpp/ap

Re: [PR] feat(parquet): add variant encoder/decoder [arrow-go]

2025-04-22 Thread via GitHub
sfc-gh-mbojanczyk commented on code in PR #344: URL: https://github.com/apache/arrow-go/pull/344#discussion_r2054810924 ## parquet/variants/util.go: ## @@ -0,0 +1,154 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. S

Re: [I] [C++] Rename arrow::util::StringBuilder and move to internal namespace [arrow]

2025-04-22 Thread via GitHub
bkietz commented on issue #46207: URL: https://github.com/apache/arrow/issues/46207#issuecomment-2822392479 It's worth noting that `arrow::util::StringBuilderRecursive()` (and possibly `arrow::util::StringBuilder` itself) is a helper function which is no longer necessary in C++17 at all; we

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): add user agent entry + thrift version for spark http connections [arrow-adbc]

2025-04-22 Thread via GitHub
CurtHagenlocher merged PR #2711: URL: https://github.com/apache/arrow-adbc/pull/2711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

[PR] fix(rust/core): remove the Mutex around the FFI driver object [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv opened a new pull request, #2736: URL: https://github.com/apache/arrow-adbc/pull/2736 Fixes #2715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): add user agent entry + thrift version for spark http connections [arrow-adbc]

2025-04-22 Thread via GitHub
toddmeng-db commented on code in PR #2711: URL: https://github.com/apache/arrow-adbc/pull/2711#discussion_r2054791186 ## csharp/test/Drivers/Apache/Spark/SparkHttpConnectionUserAgentTest.cs: ## @@ -0,0 +1,148 @@ +/* +* Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] chore(rust): make Arrow version selection more flexible [arrow-adbc]

2025-04-22 Thread via GitHub
mbrobbel commented on PR #2525: URL: https://github.com/apache/arrow-adbc/pull/2525#issuecomment-2822379254 > I don't know how to implement the next steps here. I'm maintaining a fork with changes like this to be able to use `adbc_core` as I intend to update `adbc_core` much more frequently

Re: [PR] chore(rust): make Arrow version selection more flexible [arrow-adbc]

2025-04-22 Thread via GitHub
felipecrv commented on PR #2525: URL: https://github.com/apache/arrow-adbc/pull/2525#issuecomment-2822351726 I don't know how to implement the next steps here. I'm maintaining a fork with changes like this to be able to use `adbc_core` as I intend to update `adbc_core` much more frequently

Re: [I] [C++] Use System Provided Flatbuffers [arrow]

2025-04-22 Thread via GitHub
WillAyd commented on issue #45636: URL: https://github.com/apache/arrow/issues/45636#issuecomment-2822341831 Closing as I don't think it is valuable to do this - flatbuffers requires the version used to create the proto files is exactly the same version used to compile them. It seems highly

Re: [PR] clean: Use Meson disabler objects instead of conditions [arrow-nanoarrow]

2025-04-22 Thread via GitHub
WillAyd merged PR #744: URL: https://github.com/apache/arrow-nanoarrow/pull/744 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow

Re: [PR] fix(c): Add libdl as dependency of driver manager in Meson [arrow-adbc]

2025-04-22 Thread via GitHub
WillAyd commented on PR #2735: URL: https://github.com/apache/arrow-adbc/pull/2735#issuecomment-2822287910 Ah OK - I see this is required when using the `compilers` package in a conda environment. Looks like the system installed compiler provides this anyway, but the conda one needs this ex

Re: [I] Add end-user logging and tracing for drivers [arrow-adbc]

2025-04-22 Thread via GitHub
birschick-bq commented on issue #2210: URL: https://github.com/apache/arrow-adbc/issues/2210#issuecomment-2822273722 @CurtHagenlocher > Is there a standard format for specifying a receiver and all its arguments which could be passed as e.g. an ADBC option value? I believe the

[PR] clean: Use Meson disabler objects instead of conditions [arrow-nanoarrow]

2025-04-22 Thread via GitHub
WillAyd opened a new pull request, #744: URL: https://github.com/apache/arrow-nanoarrow/pull/744 This is a minor usability enhancement to use Meson's `disabler()` object rather than repeating a lot of if statements to enable/disable tests -- This is an automated message from the Apache Gi

[PR] fix(c): Add libdl as dependency of driver manager in Meson [arrow-adbc]

2025-04-22 Thread via GitHub
WillAyd opened a new pull request, #2735: URL: https://github.com/apache/arrow-adbc/pull/2735 Not sure how this was working before, but on main I get the following issues without this: ``` adbc_driver_manager.cc:(.text.AdbcLoadDriver+0xbe): undefined reference to `dlopen' /hom

Re: [PR] GH-46193: [Flight][Format] Extend Flight Location URI Semantics [arrow]

2025-04-22 Thread via GitHub
paleolimbot commented on code in PR #46194: URL: https://github.com/apache/arrow/pull/46194#discussion_r2054684518 ## docs/source/format/Flight.rst: ## @@ -369,6 +369,61 @@ string, so the obvious candidates are not compatible. The chosen representation can be parsed by both i

[PR] chore(c): Remove unnecessary test branching from Meson configuration [arrow-adbc]

2025-04-22 Thread via GitHub
WillAyd opened a new pull request, #2734: URL: https://github.com/apache/arrow-adbc/pull/2734 The gtest_main_dep and gmock_dep dependencies are set to `disabler()` when the test option is not enabled in the top level configuration. Therefore, all of these subsequent `if get_option('tests')`

Re: [PR] GH-46193: [Flight][Format] Extend Flight Location URI Semantics [arrow]

2025-04-22 Thread via GitHub
paleolimbot commented on code in PR #46194: URL: https://github.com/apache/arrow/pull/46194#discussion_r2054672903 ## docs/source/format/Flight.rst: ## @@ -369,6 +369,61 @@ string, so the obvious candidates are not compatible. The chosen representation can be parsed by both i

Re: [PR] GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs [arrow]

2025-04-22 Thread via GitHub
amoeba commented on code in PR #46180: URL: https://github.com/apache/arrow/pull/46180#discussion_r2054658604 ## cpp/src/arrow/util/from_json.h: ## @@ -32,40 +32,52 @@ namespace arrow { class Array; class DataType; -namespace ipc { -namespace internal { -namespace json { +na

Re: [PR] feat(csharp/src/Drivers/Apache/Spark): add user agent entry + thrift version for spark http connections [arrow-adbc]

2025-04-22 Thread via GitHub
CurtHagenlocher commented on code in PR #2711: URL: https://github.com/apache/arrow-adbc/pull/2711#discussion_r2054671728 ## csharp/test/Drivers/Apache/Spark/SparkHttpConnectionUserAgentTest.cs: ## @@ -0,0 +1,148 @@ +/* +* Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs [arrow]

2025-04-22 Thread via GitHub
github-actions[bot] commented on PR #46180: URL: https://github.com/apache/arrow/pull/46180#issuecomment-2822164447 Revision: 11da4d40f05e97ad744df58c3d4b0db267e6cbb0 Submitted crossbow builds: [ursacomputing/crossbow @ actions-bfb9efe751](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs [arrow]

2025-04-22 Thread via GitHub
amoeba commented on PR #46180: URL: https://github.com/apache/arrow/pull/46180#issuecomment-2822159340 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] GH-46061: [Python] Replace setup.py invocations with modern commands [arrow]

2025-04-22 Thread via GitHub
github-actions[bot] commented on PR #46062: URL: https://github.com/apache/arrow/pull/46062#issuecomment-2822153123 Revision: 6f248fdcd49c6d82267caf3dfc9a29eb5e1afa37 Submitted crossbow builds: [ursacomputing/crossbow @ actions-401a69339f](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-46061: [Python] Replace setup.py invocations with modern commands [arrow]

2025-04-22 Thread via GitHub
WillAyd commented on PR #46062: URL: https://github.com/apache/arrow/pull/46062#issuecomment-2822146528 @github-actions crossbow submit -g wheel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] feat(csharp): Add retry-after behavior for 503 responses in Spark ADBC driver [arrow-adbc]

2025-04-22 Thread via GitHub
CurtHagenlocher merged PR #2664: URL: https://github.com/apache/arrow-adbc/pull/2664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] feat(csharp/src/Drivers/BigQuery): Add support for AAD/Entra authentication [arrow-adbc]

2025-04-22 Thread via GitHub
CurtHagenlocher commented on code in PR #2655: URL: https://github.com/apache/arrow-adbc/pull/2655#discussion_r2054534280 ## csharp/src/Drivers/BigQuery/BigQueryConnection.cs: ## @@ -58,14 +57,44 @@ public class BigQueryConnection : AdbcConnection public BigQueryConne

Re: [PR] feat(csharp/src/Drivers/BigQuery): Add support for AAD/Entra authentication [arrow-adbc]

2025-04-22 Thread via GitHub
CurtHagenlocher commented on PR #2655: URL: https://github.com/apache/arrow-adbc/pull/2655#issuecomment-2822032366 > My biggest concern around this change is a lack of clarity about concurrency issues. I had been specifically concerned about a token expiring while we were downloadin

Re: [PR] GH-45908: [C++][Docs] Expose basic {Array,...}FromJSON helpers as public APIs [arrow]

2025-04-22 Thread via GitHub
EnricoMi commented on code in PR #46180: URL: https://github.com/apache/arrow/pull/46180#discussion_r2054553719 ## cpp/src/arrow/util/from_json.h: ## @@ -32,40 +32,52 @@ namespace arrow { class Array; class DataType; -namespace ipc { -namespace internal { -namespace json { +

  1   2   3   >