Re: [I] [C++] CI job ubuntu-cpp-emscripten is failing [arrow]

2024-04-30 Thread via GitHub
jorisvandenbossche commented on issue #41463: URL: https://github.com/apache/arrow/issues/41463#issuecomment-2088050893 Issue resolved by pull request 41461 https://github.com/apache/arrow/pull/41461 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] GH-41463: [C++] Skip TestConcurrentFillFromScalar for platforms without threading support [arrow]

2024-04-30 Thread via GitHub
jorisvandenbossche merged PR #41461: URL: https://github.com/apache/arrow/pull/41461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] GH-41463: [C++] Skip TestConcurrentFillFromScalar for platforms without threading support [arrow]

2024-04-30 Thread via GitHub
jorisvandenbossche commented on PR #41461: URL: https://github.com/apache/arrow/pull/41461#issuecomment-2088051094 Thanks @zanmato1984 for the quick follow-up -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] GH-41471: [Java] Fix performance uber-jar [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41473: URL: https://github.com/apache/arrow/pull/41473#issuecomment-2088048443 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 6b278be178975fe7174b961a3bf33502acb79295. There were no

Re: [I] [Python] SEGFAULT when casting a slice of a fixed with binary array to binary [arrow]

2024-04-30 Thread via GitHub
llama90 commented on issue #41306: URL: https://github.com/apache/arrow/issues/41306#issuecomment-2088048467 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
jorisvandenbossche commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585935670 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in ro

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
kou commented on code in PR #40392: URL: https://github.com/apache/arrow/pull/40392#discussion_r1585922763 ## cpp/src/arrow/ipc/metadata_internal.cc: ## @@ -477,7 +477,9 @@ static Status GetDictionaryEncoding(FBB& fbb, const std::shared_ptr& fiel static KeyValueOffset Append

Re: [PR] GH-41306: [C++] Check to avoid copying when NullBitmapBuffer is Null [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41452: URL: https://github.com/apache/arrow/pull/41452#issuecomment-2088007786 :warning: GitHub issue #41306 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] [C++] Wrong and low inefficient expression execution for [if/else, case/when ... etc] expression [arrow]

2024-04-30 Thread via GitHub
ZhangHuiGui commented on issue #41094: URL: https://github.com/apache/arrow/issues/41094#issuecomment-2088002703 It would be very grateful if you move this forward. > This can't be done properly when `if_then_else`, `case_when`, `and`, `or` **are implemented as functions** like in Arr

Re: [PR] GH-41418: [C++] Add [Large]ListView and Map nested types for scalar_if_else's kernel functions [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41419: URL: https://github.com/apache/arrow/pull/41419#issuecomment-2087983911 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 0d7fac0d49eae7f139735c3e7c9256fc304a698a. There were no

Re: [I] `cast` kernel support for `StringViewArray` and `BinaryViewArray` [arrow-rs]

2024-04-30 Thread via GitHub
RinChanNOWWW commented on issue #5508: URL: https://github.com/apache/arrow-rs/issues/5508#issuecomment-2087927159 I want to discuss about casting from ViewArray to ByteArray. As we know, we can use ViewArray for random access of byte buffers. So, when converting ViewArray to ByteArra

Re: [PR] GH-41183: [C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41295: URL: https://github.com/apache/arrow/pull/41295#issuecomment-2087901702 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 5e986be59f08135d2fdaeb819c87120b0bf7436a. There was 1 b

Re: [PR] GH-41105: [Python][Docs] Update PyArrow installation docs for conda package split [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41135: URL: https://github.com/apache/arrow/pull/41135#issuecomment-2087886700 Revision: be37b52ba33cb577f2e693147045db9e5bc7e7e7 Submitted crossbow builds: [ursacomputing/crossbow @ actions-c33978f0b1](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41105: [Python][Docs] Update PyArrow installation docs for conda package split [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #41135: URL: https://github.com/apache/arrow/pull/41135#issuecomment-2087885173 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] GH-41105: [Python][Docs] Update PyArrow installation docs for conda package split [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #41135: URL: https://github.com/apache/arrow/pull/41135#issuecomment-2087885072 Oh, that's really neat. I added a subsection about custom selections but let me know if you think it needs higher billing. Next steps on this PR are: - I want to wrap up http

Re: [PR] GH-41105: [Python][Docs] Update PyArrow installation docs for conda package split [arrow]

2024-04-30 Thread via GitHub
amoeba commented on code in PR #41135: URL: https://github.com/apache/arrow/pull/41135#discussion_r1585809186 ## docs/source/python/install.rst: ## @@ -93,3 +100,41 @@ a custom path to the database from Python: >>> import pyarrow as pa >>> pa.set_timezone_db_path("cust

Re: [PR] GH-41105: [Python][Docs] Update PyArrow installation docs for conda package split [arrow]

2024-04-30 Thread via GitHub
amoeba commented on code in PR #41135: URL: https://github.com/apache/arrow/pull/41135#discussion_r1585808437 ## docs/source/python/install.rst: ## @@ -93,3 +100,41 @@ a custom path to the database from Python: >>> import pyarrow as pa >>> pa.set_timezone_db_path("cust

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087845901 Revision: 9843835e8c34776283292bc95f3f7c663a63a84e Submitted crossbow builds: [ursacomputing/crossbow @ actions-2d9e138827](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087844006 @github-actions crossbow submit test-r-clang-sanitizer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087841375 I've pushed up a fix to address the includes issue and the crossbow jobs looked clear (failures are unrelated and the test we expect to pass does indeed pass). I've marked this as r

Re: [PR] GH-39990: [Docs][CI] Add sphinx-lint for docs linting [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40022: URL: https://github.com/apache/arrow/pull/40022#issuecomment-2087833246 Merged. There were two failing CI checks but both were unrelated (recent pyarrow asof join failures, both have their own issues). Thanks for the review here @kou and @jorisvandenboss

Re: [PR] GH-39990: [Docs][CI] Add sphinx-lint for docs linting [arrow]

2024-04-30 Thread via GitHub
amoeba merged PR #40022: URL: https://github.com/apache/arrow/pull/40022 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087829337 Odd, we are getting an undefined behavior sanitizer error but it appears to be coming from `purrr` which didn't show up in a recent run of [the old rhub image version](https://github.co

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087796050 Revision: c3af79dbaabf59cd61fee65cdd9e47a47a9bdd62 Submitted crossbow builds: [ursacomputing/crossbow @ actions-08f54365f0](https://github.com/ursacomputing/crossbow/bra

Re: [PR] [C++] Add test for flatbuffers serialization [arrow]

2024-04-30 Thread via GitHub
amoeba closed pull request #41018: [C++] Add test for flatbuffers serialization URL: https://github.com/apache/arrow/pull/41018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] GH-40361: [C++] Make flatbuffers serialization more deterministic [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40392: URL: https://github.com/apache/arrow/pull/40392#issuecomment-2087794194 @github-actions crossbow submit -g cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] feat(go/adbc/driver/flightsql): support stateless prepared statements [arrow-adbc]

2024-04-30 Thread via GitHub
lidavidm commented on code in PR #1796: URL: https://github.com/apache/arrow-adbc/pull/1796#discussion_r1585756679 ## go/adbc/driver/flightsql/flightsql_adbc_server_test.go: ## @@ -39,13 +39,13 @@ import ( driver "github.com/apache/arrow-adbc/go/adbc/driver/flightsql"

Re: [PR] feat(go/adbc/driver/flightsql): support stateless prepared statements [arrow-adbc]

2024-04-30 Thread via GitHub
zeroshade commented on code in PR #1796: URL: https://github.com/apache/arrow-adbc/pull/1796#discussion_r1585753806 ## go/adbc/driver/flightsql/flightsql_adbc_server_test.go: ## @@ -39,13 +39,13 @@ import ( driver "github.com/apache/arrow-adbc/go/adbc/driver/flightsql"

Re: [PR] GH-39990: [Docs][CI] Add sphinx-lint for docs linting [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #40022: URL: https://github.com/apache/arrow/pull/40022#issuecomment-2087788496 I rebased, pushed two fixes in recent changes and will merge this once CI passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
ianmcook commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087788097 I successfully tested this Julia server with all the other client examples 🎉 Just one small thing: The int64 columns created in the server example are non-nullable (they hav

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
ianmcook commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087779988 Great, thanks. It works fine for me if I reduce `total_records` to `10_000_000` -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] GH-41343: [C++][CMake] Remove unused ARROW_NO_DEPRECATED_API [arrow]

2024-04-30 Thread via GitHub
kou commented on code in PR #41345: URL: https://github.com/apache/arrow/pull/41345#discussion_r1585737790 ## docs/source/developers/cpp/building.rst: ## @@ -627,9 +627,10 @@ outputs like: Deprecations and API Changes -We use the compiler definit

Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]

2024-04-30 Thread via GitHub
amoeba commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2087772418 I ended up reverting the changes which fixes the output to appease the linter here. This is how things look now: ![Screenshot 2024-04-30 at 4 34 25  PM](https://github.com/apache/arr

Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2087771604 ``` Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-2ff4f49ee9-github-preview-docs', 'refs/tags/actions-2ff4f49ee9-github-

Re: [PR] GH-41410: [C++][FS][Azure][Docs] Add AzureFileSystem to Filesystems API reference [arrow]

2024-04-30 Thread via GitHub
kou commented on PR #41411: URL: https://github.com/apache/arrow/pull/41411#issuecomment-2087770298 @github-actions crossbow submit preview-docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] GH-41407: [C++] Use static method to fill scalar scratch space to prevent ub [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41421: URL: https://github.com/apache/arrow/pull/41421#issuecomment-2087763939 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 0ef7351986ee8b967e210d0f9c7a9c8e4d4038fd. There were no

Re: [PR] feat(go/adbc/driver/flightsql): support stateless prepared statements [arrow-adbc]

2024-04-30 Thread via GitHub
lidavidm commented on code in PR #1796: URL: https://github.com/apache/arrow-adbc/pull/1796#discussion_r1585717089 ## go/adbc/driver/flightsql/flightsql_adbc_server_test.go: ## @@ -39,13 +39,13 @@ import ( driver "github.com/apache/arrow-adbc/go/adbc/driver/flightsql"

Re: [I] Directly read ADBC to Spark Dataframe [arrow-adbc]

2024-04-30 Thread via GitHub
HaoXuAI commented on issue #1801: URL: https://github.com/apache/arrow-adbc/issues/1801#issuecomment-2087750446 make sense. let me post it in the Spark repo. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Directly read ADBC to Spark Dataframe [arrow-adbc]

2024-04-30 Thread via GitHub
lidavidm commented on issue #1801: URL: https://github.com/apache/arrow-adbc/issues/1801#issuecomment-2087748054 I don't believe anyone is working on this. Best to take it to the Spark community. The ADBC driver for postgres, snowflake in Java just wraps JDBC. It doesn't provide any

Re: [I] Directly read ADBC to Spark Dataframe [arrow-adbc]

2024-04-30 Thread via GitHub
HaoXuAI commented on issue #1801: URL: https://github.com/apache/arrow-adbc/issues/1801#issuecomment-2087746496 > I think this should be a Spark feature request? > > What I would like to do here is provide a JNI driver that can leverage the better-optimized postgresql/snowflake driver

Re: [PR] GH-41470: [C++] Reuse deduplication logic for direct registration [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41466: URL: https://github.com/apache/arrow/pull/41466#issuecomment-2087746360 Revision: 2fd3fcc6e9c2bbd4b105f14306f0ea8bd7d59cec Submitted crossbow builds: [ursacomputing/crossbow @ actions-319edba519](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41470: [C++] Reuse deduplication logic for direct registration [arrow]

2024-04-30 Thread via GitHub
lidavidm commented on code in PR #41466: URL: https://github.com/apache/arrow/pull/41466#discussion_r1585706584 ## dev/tasks/java-jars/github.yml: ## @@ -80,7 +80,7 @@ jobs: fail-fast: false matrix: platform: - - { runs_on: ["macos-latest"], arch:

Re: [PR] GH-41470: [C++] Reuse deduplication logic for direct registration [arrow]

2024-04-30 Thread via GitHub
lidavidm commented on PR #41466: URL: https://github.com/apache/arrow/pull/41466#issuecomment-2087744100 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] [R][CI] CRAN-style openssl not being picked up [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on issue #41426: URL: https://github.com/apache/arrow/issues/41426#issuecomment-2087737009 I believe this is also causing the builds we are hosting to be slightly busted. I just tried installing a nightly on arm64 and found that gzip wasn't enabled. Which makes sen

Re: [I] c: define async version of ArrowArrayStream [arrow-adbc]

2024-04-30 Thread via GitHub
CurtHagenlocher commented on issue #811: URL: https://github.com/apache/arrow-adbc/issues/811#issuecomment-2087736516 I've been starting to play around with this, and the management of the data structures is a bit annoying. It also isn't very consistent with other parts of the Arrow C API w

Re: [I] [Java] Performance uberjar is empty [arrow]

2024-04-30 Thread via GitHub
lidavidm commented on issue #41471: URL: https://github.com/apache/arrow/issues/41471#issuecomment-2087733430 Issue resolved by pull request 41473 https://github.com/apache/arrow/pull/41473 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] GH-41471: [Java] Fix performance uber-jar [arrow]

2024-04-30 Thread via GitHub
lidavidm merged PR #41473: URL: https://github.com/apache/arrow/pull/41473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087726815 Revision: cb4ecea8e2d532c0fd55ba96d2290b05eefa398e Submitted crossbow builds: [ursacomputing/crossbow @ actions-1cccdf27a2](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087724853 @github-actions crossbow submit test-r-clang-sanitizer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [I] [C++] Wrong and low inefficient expression execution for [if/else, case/when ... etc] expression [arrow]

2024-04-30 Thread via GitHub
felipecrv commented on issue #41094: URL: https://github.com/apache/arrow/issues/41094#issuecomment-2087716483 > Because if-else related expressions have a special execution order, should we extend ExecuteScalarIfElseExpression to handle such expressions separately? The solution shoul

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
progger-dev commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585672675 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in row-major

[PR] Add Meson build with Werror [arrow-nanoarrow]

2024-04-30 Thread via GitHub
WillAyd opened a new pull request, #448: URL: https://github.com/apache/arrow-nanoarrow/pull/448 This should be a relatively easy way to enforce -Werror in CI; moved the Meson build from a scheduled job to be part of the normal CI runs. Looks like there are still a few more upstream i

Re: [PR] test(go/adbc/driver/snowflake): missed a quote [arrow-adbc]

2024-04-30 Thread via GitHub
lidavidm merged PR #1799: URL: https://github.com/apache/arrow-adbc/pull/1799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
simsurace commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087673497 Yes, on macos there is a known issue that I think will be fixed in the next releases https://github.com/JuliaLang/julia/issues/54225 -- This is an automated message from the Apa

Re: [PR] feat(rust): add the driver exporter [arrow-adbc]

2024-04-30 Thread via GitHub
lidavidm merged PR #1756: URL: https://github.com/apache/arrow-adbc/pull/1756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.a

Re: [I] [C++] Support [Large]ListView and Map types for if_else, case_when, and coalesce [arrow]

2024-04-30 Thread via GitHub
felipecrv commented on issue #41418: URL: https://github.com/apache/arrow/issues/41418#issuecomment-2087588066 Issue resolved by pull request 41419 https://github.com/apache/arrow/pull/41419 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] GH-41418: [C++] Add [Large]ListView and Map nested types for scalar_if_else's kernel functions [arrow]

2024-04-30 Thread via GitHub
felipecrv merged PR #41419: URL: https://github.com/apache/arrow/pull/41419 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087567540 Revision: 9664a4cb9ba56cff09c2c6ef324354e042c48809 Submitted crossbow builds: [ursacomputing/crossbow @ actions-6816ab39a7](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41183: [C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings [arrow]

2024-04-30 Thread via GitHub
felipecrv merged PR #41295: URL: https://github.com/apache/arrow/pull/41295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

Re: [I] [C++][Python] Expose recursive flatten for lists on list_flatten kernel function and pyarrow bindings [arrow]

2024-04-30 Thread via GitHub
felipecrv commented on issue #41183: URL: https://github.com/apache/arrow/issues/41183#issuecomment-2087570299 Issue resolved by pull request 41295 https://github.com/apache/arrow/pull/41295 -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087561061 @github-actions crossbow submit test-r-clang-sanitizer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
ianmcook commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087559070 @simsurace I'm not able to get the server example working on macOS. It starts successfully, but when a client connects to it (any client), it throws an error: ```console % ju

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087548745 Revision: 3185879e1a6a76ba19443b386a3400eb8c9a8df4 Submitted crossbow builds: [ursacomputing/crossbow @ actions-09f0dd36e9](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-41450: [R][CI] rhub/container follow ons [arrow]

2024-04-30 Thread via GitHub
jonkeane commented on PR #41451: URL: https://github.com/apache/arrow/pull/41451#issuecomment-2087539845 @github-actions crossbow submit test-r-clang-sanitizer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
ianmcook commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087523137 On closer inspection: actually `batches` is not an `Arrow.Table` — it's a `Vector{Arrow.Table}`. Considering that, the added layer of nesting in the output schema makes more sense.

Re: [PR] GH-41400: [MATLAB] Bump `libmexclass` version to commit `ca3cea6` [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41436: URL: https://github.com/apache/arrow/pull/41436#issuecomment-2087474324 After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit e22197f39e41446789dcc52e931995fe20a784a4. There was 1 b

Re: [PR] GH-39858: [C++][Device] Add Copy/View slice functions to MemoryManager [arrow]

2024-04-30 Thread via GitHub
alanstoate commented on PR #41477: URL: https://github.com/apache/arrow/pull/41477#issuecomment-2087419948 Hi @zeroshade I've had a go at adding these MemoryManager methods, just a few q's: - I've added all the copy/view methods in MemoryManager as slice versions I'm not sure if this

[PR] GH-39858: [C++][Device] Add Copy/View slice functions to MemoryManager [arrow]

2024-04-30 Thread via GitHub
alanstoate opened a new pull request, #41477: URL: https://github.com/apache/arrow/pull/41477 ### Rationale for this change Currently ```MemoryManager``` objects define functionality to Copy or View entire buffers. Occasionally there is the need to only copy a single value or slice from

Re: [PR] GH-41430: [Docs] Use sphinxcontrib-mermaid instead of generating images from .mmd [arrow]

2024-04-30 Thread via GitHub
zeroshade commented on code in PR #41455: URL: https://github.com/apache/arrow/pull/41455#discussion_r1585545169 ## ci/docker/linux-apt-docs.dockerfile: ## @@ -48,16 +61,20 @@ RUN apt-get update -y && \ libxml2-dev \ meson \ ninja-build \ +node

Re: [PR] GH-40494: [Go] add support for protobuf messages [arrow]

2024-04-30 Thread via GitHub
tscottcoombes1 commented on code in PR #40496: URL: https://github.com/apache/arrow/pull/40496#discussion_r1585533839 ## go/arrow/util/protobuf_reflect_test.go: ## @@ -0,0 +1,180 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [PR] GH-41343: [C++][CMake] Remove unused ARROW_NO_DEPRECATED_API [arrow]

2024-04-30 Thread via GitHub
pitrou commented on code in PR #41345: URL: https://github.com/apache/arrow/pull/41345#discussion_r1585531547 ## docs/source/developers/cpp/building.rst: ## @@ -624,13 +624,6 @@ outputs like: ../src/arrow/ipc/ipc-read-write-test.cc:574 code: writer->WriteRecordBatch(batch)

Re: [PR] GH-40494: [Go] add support for protobuf messages [arrow]

2024-04-30 Thread via GitHub
zeroshade commented on code in PR #40496: URL: https://github.com/apache/arrow/pull/40496#discussion_r1585518232 ## go/arrow/util/protobuf_reflect_test.go: ## @@ -0,0 +1,180 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreeme

Re: [PR] Add Julia example [WIP] [arrow-experiments]

2024-04-30 Thread via GitHub
simsurace commented on PR #29: URL: https://github.com/apache/arrow-experiments/pull/29#issuecomment-2087261216 Hmm ok, I think I'm still confused about the nomenclature. Looking at the other implementations (e.g. Python), record batches seem to be small tables (i.e. 4096 rows), so wouldn't

Re: [PR] GH-40078: [C++] Import/Export ArrowDeviceArrayStream [arrow]

2024-04-30 Thread via GitHub
zeroshade commented on PR #40807: URL: https://github.com/apache/arrow/pull/40807#issuecomment-2087247081 I was able to narrow the failure down to attempting to read a parquet file containing extension types with registered extensions causing the segfault. i.e.: either this line https://gi

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
rok commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585499203 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in row-major/C-conti

Re: [PR] GH-40494: [Go] add support for protobuf messages [arrow]

2024-04-30 Thread via GitHub
tscottcoombes1 commented on code in PR #40496: URL: https://github.com/apache/arrow/pull/40496#discussion_r1585494010 ## go/arrow/util/protobuf_reflect_test.go: ## @@ -0,0 +1,180 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [I] [C++] Add functionality to MemoryManager for copying a slice of a buffer [arrow]

2024-04-30 Thread via GitHub
alanstoate commented on issue #39858: URL: https://github.com/apache/arrow/issues/39858#issuecomment-2087113259 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [I] [Python] Windows fatal exception: access violation [arrow]

2024-04-30 Thread via GitHub
dburton-influxdata commented on issue #40100: URL: https://github.com/apache/arrow/issues/40100#issuecomment-2086991962 @amoeba Bryce, have you been able to identify any next steps? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] feat(python): Allow creation of dictionary and list types [arrow-nanoarrow]

2024-04-30 Thread via GitHub
paleolimbot commented on code in PR #445: URL: https://github.com/apache/arrow-nanoarrow/pull/445#discussion_r1585451995 ## python/src/nanoarrow/schema.py: ## @@ -343,6 +347,34 @@ def scale(self) -> int: return self._c_schema_view.decimal_scale +@property +d

Re: [I] [Flight] Flight error status for quota exceeded [arrow]

2024-04-30 Thread via GitHub
janosik47 commented on issue #35888: URL: https://github.com/apache/arrow/issues/35888#issuecomment-2086915243 Any chance to have this fixed? It feels more like a bug rather than an enhancement. Silently replacing the RESOURCE_EXHAUSTED with INVALID_ARGUMENT changes the meaning of th

Re: [PR] feat(python): Allow creation of dictionary and list types [arrow-nanoarrow]

2024-04-30 Thread via GitHub
paleolimbot commented on code in PR #445: URL: https://github.com/apache/arrow-nanoarrow/pull/445#discussion_r1585448475 ## python/src/nanoarrow/schema.py: ## @@ -957,14 +988,113 @@ def struct(fields, nullable=True) -> Schema: >>> import nanoarrow as na >>> na.struct([

Re: [I] Bus errors when writing `DataFrame` [arrow-julia]

2024-04-30 Thread via GitHub
sprig commented on issue #474: URL: https://github.com/apache/arrow-julia/issues/474#issuecomment-2086886538 Can confirm the same issue that @stuartthomas25 identified. My quite obvious workaround is to write the table to a temporary file and then overwrite the original. However, julia `mv`

Re: [PR] GH-40078: [C++] Import/Export ArrowDeviceArrayStream [arrow]

2024-04-30 Thread via GitHub
zeroshade commented on PR #40807: URL: https://github.com/apache/arrow/pull/40807#issuecomment-2086853686 @paleolimbot Damn i was hoping that the rebase I just did would solve those segfaults :frowning: Guess i'm gonna have to dig in and figure out what the hell is segfaulting there -- T

Re: [PR] GH-40342: [Python] Fix pickling of LocalFileSystem for cython 2 [arrow]

2024-04-30 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #41459: URL: https://github.com/apache/arrow/pull/41459#issuecomment-2086848648 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit b609de374c7c00e1537eb8092e1ff2db718d2b61. There were no

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
pitrou commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585434862 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in row-major/C-co

Re: [I] [R] arrow in R not dealing correctly with int64 [arrow]

2024-04-30 Thread via GitHub
rafapereirabr commented on issue #41474: URL: https://github.com/apache/arrow/issues/41474#issuecomment-2086688708 Here's my `sessionInfo()` ``` > sessionInfo() R version 4.1.3 (2022-03-10) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Re: [PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]

2024-04-30 Thread via GitHub
pitrou commented on PR #41335: URL: https://github.com/apache/arrow/pull/41335#issuecomment-2086688378 > The stack space might be storing pointers to itself, so it can't grow dynamically and preserve referential integrity. IIUC, this can be circumvented by using something like a `std

Re: [PR] refactor(python): Reorganize strategies for building arrays [arrow-nanoarrow]

2024-04-30 Thread via GitHub
paleolimbot merged PR #444: URL: https://github.com/apache/arrow-nanoarrow/pull/444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

Re: [I] [Ruby] 16.0.0 install failed on macOS [arrow]

2024-04-30 Thread via GitHub
amoeba commented on issue #41448: URL: https://github.com/apache/arrow/issues/41448#issuecomment-2086583661 Hey all, my first issue was not having brew link'd glib but after doing that I ran into other issues so I'll file another ticket if I think what I'm running into isn't just my setup (

[PR] chore(python): Remove C sources from wheels [arrow-nanoarrow]

2024-04-30 Thread via GitHub
paleolimbot opened a new pull request, #447: URL: https://github.com/apache/arrow-nanoarrow/pull/447 Even though the binaries were quite small, nanoarrow's wheels were ~4MB! This is because the flatcc headers and cython generated code was rather extensive and had ended up in the built wheel

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
progger-dev commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585322513 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in row-major

[PR] feat(go/adbc/driver/snowflake): add quoted identifier ignore case option [arrow-adbc]

2024-04-30 Thread via GitHub
zeroshade opened a new pull request, #1800: URL: https://github.com/apache/arrow-adbc/pull/1800 closes #1766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] feat(rust): add the driver exporter [arrow-adbc]

2024-04-30 Thread via GitHub
alexandreyc commented on PR #1756: URL: https://github.com/apache/arrow-adbc/pull/1756#issuecomment-2086445527 @lidavidm I think this PR can be merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] GH-41471: [Java] Fix performance uber-jar [arrow]

2024-04-30 Thread via GitHub
github-actions[bot] commented on PR #41473: URL: https://github.com/apache/arrow/pull/41473#issuecomment-2086422742 :warning: GitHub issue #41471 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-41471: [Java] Fix performance uber-jar [arrow]

2024-04-30 Thread via GitHub
laurentgo opened a new pull request, #41473: URL: https://github.com/apache/arrow/pull/41473 ### Rationale for this change Performance `benchmarks.jar` uber-jar is mostly empty and is missing critical metadata information which would allow someone to run performance benchmarks using

Re: [PR] GH-41407: [C++] Use static method to fill scalar scratch space to prevent ub [arrow]

2024-04-30 Thread via GitHub
bkietz merged PR #41421: URL: https://github.com/apache/arrow/pull/41421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [I] [C++][R] R Sanitizer nightly error [arrow]

2024-04-30 Thread via GitHub
bkietz commented on issue #41407: URL: https://github.com/apache/arrow/issues/41407#issuecomment-2086310835 Issue resolved by pull request 41421 https://github.com/apache/arrow/pull/41421 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] GH-41334: [C++][Acero] Add env var to tune the size of the temp stack [arrow]

2024-04-30 Thread via GitHub
bkietz commented on PR #41335: URL: https://github.com/apache/arrow/pull/41335#issuecomment-2086291405 Sorry for the confusion! I think per-node thread local temp stacks are what was eventually intended but that refactor was not completed. -- This is an automated message from the Apache G

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
jorisvandenbossche commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585269512 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in ro

Re: [PR] GH-41256: [Format][Docs] Add a canonical extension type specification for JSON [arrow]

2024-04-30 Thread via GitHub
rok commented on code in PR #41257: URL: https://github.com/apache/arrow/pull/41257#discussion_r1585212898 ## docs/source/format/CanonicalExtensions.rst: ## @@ -251,6 +251,27 @@ Variable shape tensor Values inside each **data** tensor element are stored in row-major/C-conti

  1   2   3   >