[GitHub] [arrow] jorisvandenbossche merged pull request #36550: GH-34884: [Python]: Support pickling pyarrow.dataset PartitioningFactory objects

2023-07-09 Thread via GitHub
jorisvandenbossche merged PR #36550: URL: https://github.com/apache/arrow/pull/36550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

[GitHub] [arrow] github-actions[bot] commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1628260284 Revision: 5ee0e2691a78779c7158e8584b89a1c300c308e5 Submitted crossbow builds: [ursacomputing/crossbow @ actions-13781510df](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
kou commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1628255858 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow] github-actions[bot] commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1628234315 Revision: e79c31e547544790451661715dfae482e24b0d74 Submitted crossbow builds: [ursacomputing/crossbow @ actions-32865d51a3](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
kou commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1628229865 @github-actions crossbow submit r-binary-packages homebrew-r-autobrew -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36460: GH-35943: [Dev] Ensure link issue works when PR body is empty

2023-07-09 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #36460: URL: https://github.com/apache/arrow/pull/36460#issuecomment-1628164486 Conbench analyzed the 6 benchmark runs on commit `0df414e1`. There were 3 benchmark results indicating a performance regression: - Commit Run on `ursa-think

[GitHub] [arrow] github-actions[bot] commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1628163661 Revision: 42dd6231a2eca2dbe94f5b5fca14b72d3c7cc4e5 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1cf795e683](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
kou commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1628159682 @github-actions crossbow submit r-binary-packages homebrew-r-autobrew -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] kou merged pull request #36584: MINOR: [C++] Cleanup FindgRPCAlt

2023-07-09 Thread via GitHub
kou merged PR #36584: URL: https://github.com/apache/arrow/pull/36584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow-ballista] yahoNanJing commented on issue #803: [Problem] How to deploy multiple schedulers on standalone mode but not docker

2023-07-09 Thread via GitHub
yahoNanJing commented on issue #803: URL: https://github.com/apache/arrow-ballista/issues/803#issuecomment-1628072182 Hi @smallzhongfeng, could you explain the reason of using multiple schedulers? Is it just for HA or worried about the performance of single scheduler for task scheduling? If

[GitHub] [arrow-rs] jayzhan211 commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257684164 ## arrow-cast/src/cast.rs: ## @@ -3680,6 +3682,45 @@ fn cast_primitive_to_list( Ok(list_array) } +/// Wraps a list array with another list array, using the

[GitHub] [arrow] assignUser commented on a diff in pull request #36584: MINOR: [C++] Cleanup FindgRPCAlt

2023-07-09 Thread via GitHub
assignUser commented on code in PR #36584: URL: https://github.com/apache/arrow/pull/36584#discussion_r1257683765 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -36,7 +36,7 @@ if(GRPCPP_PC_FOUND) # gRPC's pkg-config file neglects to specify pthreads. find_package(Threads RE

[GitHub] [arrow] github-actions[bot] commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1628056381 Revision: 50e206a77a93bcd24cd220596e8656fc804d67dd Submitted crossbow builds: [ursacomputing/crossbow @ actions-43ffb53e8f](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on a diff in pull request #36584: MINOR: [C++] Cleanup FindgRPCAlt

2023-07-09 Thread via GitHub
kou commented on code in PR #36584: URL: https://github.com/apache/arrow/pull/36584#discussion_r1257681400 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -36,7 +36,7 @@ if(GRPCPP_PC_FOUND) # gRPC's pkg-config file neglects to specify pthreads. find_package(Threads REQUIRED)

[GitHub] [arrow] kou commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
kou commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1628052503 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow] assignUser commented on a diff in pull request #36584: MINOR: [C++] Cleanup FindgRPCAlt

2023-07-09 Thread via GitHub
assignUser commented on code in PR #36584: URL: https://github.com/apache/arrow/pull/36584#discussion_r1257677054 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -36,7 +36,6 @@ if(GRPCPP_PC_FOUND) # gRPC's pkg-config file neglects to specify pthreads. find_package(Threads RE

[GitHub] [arrow] kou commented on a diff in pull request #36584: MINOR: [C++] Cleanup FindgrpcAlt

2023-07-09 Thread via GitHub
kou commented on code in PR #36584: URL: https://github.com/apache/arrow/pull/36584#discussion_r1257675031 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -36,7 +36,6 @@ if(GRPCPP_PC_FOUND) # gRPC's pkg-config file neglects to specify pthreads. find_package(Threads REQUIRED)

[GitHub] [arrow] assignUser opened a new pull request, #36584: MINOR: [C==

2023-07-09 Thread via GitHub
assignUser opened a new pull request, #36584: URL: https://github.com/apache/arrow/pull/36584 ### Rationale for this change ### What changes are included in this PR? ### Are these changes tested? ### Are there any user-facing changes?

[GitHub] [arrow] github-actions[bot] commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1627988504 Revision: 9ea58239e42adee031f57bb26df76165f3a2e336 Submitted crossbow builds: [ursacomputing/crossbow @ actions-eafc75adab](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on a diff in pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
kou commented on code in PR #36581: URL: https://github.com/apache/arrow/pull/36581#discussion_r1257657341 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -45,6 +47,7 @@ if(GRPCPP_PC_FOUND) HINTS ${GRPCPP_PC_STATIC_LIBRARY_DIRS}) list(APPEND GRPCPP_LINK_

[GitHub] [arrow-datafusion] 3AceShowHand closed issue #6877: run `arrow-datafusion/datafusion/sql` failed

2023-07-09 Thread via GitHub
3AceShowHand closed issue #6877: run `arrow-datafusion/datafusion/sql` failed URL: https://github.com/apache/arrow-datafusion/issues/6877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow-datafusion] 3AceShowHand commented on issue #6877: run `arrow-datafusion/datafusion/sql` failed

2023-07-09 Thread via GitHub
3AceShowHand commented on issue #6877: URL: https://github.com/apache/arrow-datafusion/issues/6877#issuecomment-1627987056 > I wonder if it works if you do `cargo update`? Yes, it's fixed after `cargo update`, thanks for your help, I will close this issue. -- This is an automated

[GitHub] [arrow] kou commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
kou commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1627986526 @github-actions crossbow submit r-binary-packages homebrew-r-autobrew -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] kou merged pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
kou merged PR #36583: URL: https://github.com/apache/arrow/pull/36583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] kou commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
kou commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627983772 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[GitHub] [arrow] assignUser commented on a diff in pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
assignUser commented on code in PR #36581: URL: https://github.com/apache/arrow/pull/36581#discussion_r1257654614 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -45,6 +47,7 @@ if(GRPCPP_PC_FOUND) HINTS ${GRPCPP_PC_STATIC_LIBRARY_DIRS}) list(APPEND GRPCP

[GitHub] [arrow] assignUser commented on a diff in pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
assignUser commented on code in PR #36581: URL: https://github.com/apache/arrow/pull/36581#discussion_r1257652834 ## cpp/cmake_modules/FindgRPCAlt.cmake: ## @@ -45,6 +47,7 @@ if(GRPCPP_PC_FOUND) HINTS ${GRPCPP_PC_STATIC_LIBRARY_DIRS}) list(APPEND GRPCP

[GitHub] [arrow-datafusion] waynexia merged pull request #6894: Minor: deleted duplicated substrait integration test

2023-07-09 Thread via GitHub
waynexia merged PR #6894: URL: https://github.com/apache/arrow-datafusion/pull/6894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@a

[GitHub] [arrow] kou merged pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
kou merged PR #36581: URL: https://github.com/apache/arrow/pull/36581 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow-ballista] dependabot[bot] opened a new pull request, #840: Bump tough-cookie from 4.1.2 to 4.1.3 in /ballista/scheduler/ui

2023-07-09 Thread via GitHub
dependabot[bot] opened a new pull request, #840: URL: https://github.com/apache/arrow-ballista/pull/840 Bumps [tough-cookie](https://github.com/salesforce/tough-cookie) from 4.1.2 to 4.1.3. Release notes Sourced from https://github.com/salesforce/tough-cookie/releases";>tough-cooki

[GitHub] [arrow-rs] jayzhan211 commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257644741 ## arrow-cast/src/cast.rs: ## @@ -3680,6 +3682,45 @@ fn cast_primitive_to_list( Ok(list_array) } +/// Wraps a list array with another list array, using the

[GitHub] [arrow-rs] jayzhan211 commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257632011 ## arrow-cast/src/cast.rs: ## @@ -3680,6 +3682,45 @@ fn cast_primitive_to_list( Ok(list_array) } +/// Wraps a list array with another list array, using the

[GitHub] [arrow] github-actions[bot] commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1627951625 Revision: 5962489d730bf5bbc4c0f04e6c2f04613aa6dd46 Submitted crossbow builds: [ursacomputing/crossbow @ actions-517c0fb001](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars

2023-07-09 Thread via GitHub
kou commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1627950308 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow-rs] jayzhan211 commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257622080 ## arrow-cast/src/cast.rs: ## @@ -3680,6 +3682,45 @@ fn cast_primitive_to_list( Ok(list_array) } +/// Wraps a list array with another list array, using the

[GitHub] [arrow] github-actions[bot] commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627922103 Revision: aac3d8d6cb5672d3fc195f3abcdc756ed9f21e09 Submitted crossbow builds: [ursacomputing/crossbow @ actions-556ee658e7](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
kou commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627918749 @github-actions crossbow submit homebrew-cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36465: GH-23870: [Python] Ensure parquet.write_to_dataset doesn't create empty files for non-observed dictionary (category) value

2023-07-09 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #36465: URL: https://github.com/apache/arrow/pull/36465#issuecomment-1627901928 Conbench analyzed the 6 benchmark runs on commit `20d5c310`. There were 3 benchmark results indicating a performance regression: - Commit Run on `ursa-think

[GitHub] [arrow-rs] eitsupi commented on a diff in pull request #4490: ci: verify MSRV on CI

2023-07-09 Thread via GitHub
eitsupi commented on code in PR #4490: URL: https://github.com/apache/arrow-rs/pull/4490#discussion_r1257591195 ## object_store/Cargo.toml: ## @@ -25,6 +25,8 @@ description = "A generic object store interface for uniformly interacting with A keywords = ["object", "storage", "c

[GitHub] [arrow] github-actions[bot] commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627880635 Revision: e663975d5369e29d3765442415546bb594b18eca Submitted crossbow builds: [ursacomputing/crossbow @ actions-22a9472159](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] github-actions[bot] commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627879755 :warning: GitHub issue #36582 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow] kou commented on pull request #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
kou commented on PR #36583: URL: https://github.com/apache/arrow/pull/36583#issuecomment-1627879589 @github-actions crossbow submit homebrew-cpp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow] kou opened a new pull request, #36583: GH-36582: [CI][C++][Homebrew] Backport the latest formula changes

2023-07-09 Thread via GitHub
kou opened a new pull request, #36583: URL: https://github.com/apache/arrow/pull/36583

[GitHub] [arrow-rs] jayzhan211 commented on pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#issuecomment-1627878162 I only consider list-to-list casting in this PR. For non-list-to-list cases, do you mean something like casting a primitive array to a nested list array? -- This is an automated mes

[GitHub] [arrow-rs] jayzhan211 commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
jayzhan211 commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257571959 ## arrow-schema/src/datatype.rs: ## @@ -407,6 +407,19 @@ impl DataType { } } +/// Returns the number of dimensions if the data type is nested (L

[GitHub] [arrow-datafusion] djouallah commented on issue #5646: TPCH, Query 18 and 17 very slow

2023-07-09 Thread via GitHub
djouallah commented on issue #5646: URL: https://github.com/apache/arrow-datafusion/issues/5646#issuecomment-1627857495 i am doing this experimentation using fabric notebook, datafusion doing alright, would love really to start seeing numbers for 8 cores, make it works first then fast late

[GitHub] [arrow-datafusion] alamb commented on pull request #6895: Minor: Add FixedSizeBinaryTest

2023-07-09 Thread via GitHub
alamb commented on PR #6895: URL: https://github.com/apache/arrow-datafusion/pull/6895#issuecomment-1627854452 > Possibly needs this change that was just merged into Arrow? https://github.com/apache/arrow-rs/pull/4492 I am pretty sure I did (previously the error message was different

[GitHub] [arrow-datafusion] alamb commented on issue #5646: TPCH, Query 18 and 17 very slow

2023-07-09 Thread via GitHub
alamb commented on issue #5646: URL: https://github.com/apache/arrow-datafusion/issues/5646#issuecomment-1627854135 I expect Q17 to go about 2x faster and use much less memory when we merge our most recent work -- see https://github.com/apache/arrow-datafusion/pull/6800#issuecomment-162741

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #6801: parallel csv scan

2023-07-09 Thread via GitHub
alamb commented on code in PR #6801: URL: https://github.com/apache/arrow-datafusion/pull/6801#discussion_r1257562874 ## datafusion/core/src/datasource/physical_plan/csv.rs: ## @@ -270,14 +297,223 @@ impl CsvOpener { } } +/// Returns the position of the first newline in

[GitHub] [arrow-datafusion] maxburke commented on pull request #6895: Minor: Add FixedSizeBinaryTest

2023-07-09 Thread via GitHub
maxburke commented on PR #6895: URL: https://github.com/apache/arrow-datafusion/pull/6895#issuecomment-1627853512 > @maxburke sadly it seems like your change in #6891 isn't sufficient -- the tests in this PR now fail with > > > query error DataFusion error: Arrow error: Compute error

[GitHub] [arrow-datafusion] alamb commented on pull request #6891: Add FixedSizeBinary support to binary_op_dyn_scalar

2023-07-09 Thread via GitHub
alamb commented on PR #6891: URL: https://github.com/apache/arrow-datafusion/pull/6891#issuecomment-1627853209 FYI this PR may not be enough to compared fixed size binary -- see https://github.com/apache/arrow-datafusion/pull/6895 for details -- This is an automated message from the Apac

[GitHub] [arrow-datafusion] alamb commented on pull request #6895: Minor: Add FixedSizeBinaryTest

2023-07-09 Thread via GitHub
alamb commented on PR #6895: URL: https://github.com/apache/arrow-datafusion/pull/6895#issuecomment-1627853109 @maxburke sadly it seems like your change in https://github.com/apache/arrow-datafusion/pull/6891 isn't sufficient -- the tests in this PR now fail with > query error Dat

[GitHub] [arrow-datafusion] alamb closed issue #6890: FixedSizeBinary support for binary_array_op_dyn_scalar

2023-07-09 Thread via GitHub
alamb closed issue #6890: FixedSizeBinary support for binary_array_op_dyn_scalar URL: https://github.com/apache/arrow-datafusion/issues/6890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [arrow-datafusion] alamb merged pull request #6891: Add FixedSizeBinary support to binary_op_dyn_scalar

2023-07-09 Thread via GitHub
alamb merged PR #6891: URL: https://github.com/apache/arrow-datafusion/pull/6891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-datafusion] djouallah commented on issue #5646: TPCH, Query 18 and 17 very slow

2023-07-09 Thread via GitHub
djouallah commented on issue #5646: URL: https://github.com/apache/arrow-datafusion/issues/5646#issuecomment-1627851766 using Python_datafusion 27, unfortunately still issues with Query 17, my VM has 8 cores and 64 GB of RAM, Query 17 got OOM ![image](https://github.com/apache/arrow

[GitHub] [arrow] github-actions[bot] commented on pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36581: URL: https://github.com/apache/arrow/pull/36581#issuecomment-1627839299 Revision: 7a33aae217456369652a7551fb36ec076948d1cf Submitted crossbow builds: [ursacomputing/crossbow @ actions-f185f0e942](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
kou commented on PR #36581: URL: https://github.com/apache/arrow/pull/36581#issuecomment-1627837399 @github-actions crossbow submit -g nightly-tests -g nightly-packaging -g nightly-release -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [arrow] kou opened a new pull request, #36581: GH-36479: [C++][FlightRPC] Use gRPC version detected by find_package()

2023-07-09 Thread via GitHub
kou opened a new pull request, #36581: URL: https://github.com/apache/arrow/pull/36581 ### Rationale for this change We don't need to use `try_compile()` by using gRPC version detected by `find_package()`. ### What changes are included in this PR? Use gRPC version detec

[GitHub] [arrow] github-actions[bot] commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars.

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1627831620 Revision: 76ec84101d5216bbb3c3a823ddc7b5ead1cb760c Submitted crossbow builds: [ursacomputing/crossbow @ actions-0f05e128b9](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] github-actions[bot] commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1627831035 Revision: a1e0265edf9e47f9f577625f5367fbb4e8c36e55 Submitted crossbow builds: [ursacomputing/crossbow @ actions-36db938171](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars.

2023-07-09 Thread via GitHub
kou commented on PR #36487: URL: https://github.com/apache/arrow/pull/36487#issuecomment-1627830880 @github-actions crossbow submit java-jars -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [arrow] kou commented on a diff in pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars.

2023-07-09 Thread via GitHub
kou commented on code in PR #36487: URL: https://github.com/apache/arrow/pull/36487#discussion_r1257545041 ## dev/tasks/java-jars/github.yml: ## @@ -22,8 +22,18 @@ jobs: build-cpp-ubuntu: -name: Build C++ libraries Ubuntu -runs-on: ubuntu-latest +{% set arch =

[GitHub] [arrow] kou commented on pull request #36522: GH-36456: [CI][R] Unlink system OpenSSL to avoid mixing OpenSSL versions

2023-07-09 Thread via GitHub
kou commented on PR #36522: URL: https://github.com/apache/arrow/pull/36522#issuecomment-1627830567 @github-actions crossbow submit r-binary-packages homebrew-r-autobrew -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36162: GH-21761: [Python] accept pyarrow scalars in array constructor

2023-07-09 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #36162: URL: https://github.com/apache/arrow/pull/36162#issuecomment-1627829784 Conbench analyzed the 6 benchmark runs on commit `b116b8ab`. There were 4 benchmark results indicating a performance regression: - Commit Run on `ursa-think

[GitHub] [arrow-datafusion] maxburke opened a new issue, #6897: Fix for issue #6595 has broken existing working queries

2023-07-09 Thread via GitHub
maxburke opened a new issue, #6897: URL: https://github.com/apache/arrow-datafusion/issues/6897 ### Describe the bug After upgrading to Datafusion 27.0.0 we noticed some of our regression tests were failing. We bisected the commit that introduced the break to 36123ee0, which is the f

[GitHub] [arrow] github-actions[bot] commented on pull request #36580: GH-36482: [C++][CI] Fix sporadic test failures in AsofJoinBasicTest

2023-07-09 Thread via GitHub
github-actions[bot] commented on PR #36580: URL: https://github.com/apache/arrow/pull/36580#issuecomment-1627822634 Revision: 58a648b4e500750ffa1fe97abd672c3d4280c954 Submitted crossbow builds: [ursacomputing/crossbow @ actions-cdacd2d9b4](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] kou commented on pull request #36580: GH-36482: [C++][CI] Fix sporadic test failures in AsofJoinBasicTest

2023-07-09 Thread via GitHub
kou commented on PR #36580: URL: https://github.com/apache/arrow/pull/36580#issuecomment-1627822011 @github-actions crossbow submit verify-rc-source-*macos* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] kou merged pull request #36579: GH-36556: [CI][C++] Enable S3 in Valgrind build

2023-07-09 Thread via GitHub
kou merged PR #36579: URL: https://github.com/apache/arrow/pull/36579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow] kou commented on a diff in pull request #35656: GH-33321: [Python] Support converting to non-nano datetime64 for pandas >= 2.0

2023-07-09 Thread via GitHub
kou commented on code in PR #35656: URL: https://github.com/apache/arrow/pull/35656#discussion_r1257540254 ## python/pyarrow/tests/test_pandas.py: ## @@ -4179,20 +4258,20 @@ def test_to_pandas_extension_dtypes_mapping(): assert isinstance(result['a'].dtype, pd.PeriodDtype)

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4490: ci: verify MSRV on CI

2023-07-09 Thread via GitHub
tustvold commented on code in PR #4490: URL: https://github.com/apache/arrow-rs/pull/4490#discussion_r1257534161 ## object_store/Cargo.toml: ## @@ -25,6 +25,8 @@ description = "A generic object store interface for uniformly interacting with A keywords = ["object", "storage", "

[GitHub] [arrow-rs] tustvold commented on pull request #4494: Add negate kernels (#4488)

2023-07-09 Thread via GitHub
tustvold commented on PR #4494: URL: https://github.com/apache/arrow-rs/pull/4494#issuecomment-1627805985 Will add interval tests shortly, realise I forgot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow-rs] tustvold opened a new pull request, #4494: Add negate kernels (#4488)

2023-07-09 Thread via GitHub
tustvold opened a new pull request, #4494: URL: https://github.com/apache/arrow-rs/pull/4494 # Which issue does this PR close? Closes #4488 # Rationale for this change # What changes are included in this PR? # Are there any user-facing chan

[GitHub] [arrow-datafusion] maxburke commented on pull request #6891: Add FixedSizeBinary support to binary_op_dyn_scalar

2023-07-09 Thread via GitHub
maxburke commented on PR #6891: URL: https://github.com/apache/arrow-datafusion/pull/6891#issuecomment-1627789123 > Thanks @maxburke -- the code change looks good to me, but I think there are unrelated changes in this PR for some reason. Once that is resolved I think this PR is good to go.

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36468: MINOR: [Dev] Remove westonpace from compute reviews and add to acero reviews in CODEOWNERS

2023-07-09 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #36468: URL: https://github.com/apache/arrow/pull/36468#issuecomment-1627785881 Conbench analyzed the 6 benchmark runs on commit `947a446c`. There was 1 benchmark result indicating a performance regression: - Commit Run on `ursa-thinkce

[GitHub] [arrow-datafusion] 2010YOUY01 commented on pull request #6801: parallel csv scan

2023-07-09 Thread via GitHub
2010YOUY01 commented on PR #6801: URL: https://github.com/apache/arrow-datafusion/pull/6801#issuecomment-1627778667 memo for myself: update comments/docs in configurations -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [arrow] benibus commented on pull request #36575: GH-36311: [C++] Fix integer overflows in `utf8_slice_codeunits`

2023-07-09 Thread via GitHub
benibus commented on PR #36575: URL: https://github.com/apache/arrow/pull/36575#issuecomment-1627778208 @pitrou I haven't added any python tests (although I can if necessary) - but the original examples should be working now as well. ```python >>> pa.compute.utf8_slice_codeunits(f"AB{c

[GitHub] [arrow-datafusion] izveigor commented on pull request #6787: Minor: add test cases with columns for math expressions

2023-07-09 Thread via GitHub
izveigor commented on PR #6787: URL: https://github.com/apache/arrow-datafusion/pull/6787#issuecomment-162631 @alamb Honestly I forgot about this PR. Yes there were some problems with giving different results using different OS (the last digits did not match). I have solved the probl

[GitHub] [arrow-datafusion] 2010YOUY01 commented on a diff in pull request #6801: parallel csv scan

2023-07-09 Thread via GitHub
2010YOUY01 commented on code in PR #6801: URL: https://github.com/apache/arrow-datafusion/pull/6801#discussion_r1257516647 ## datafusion/core/src/datasource/physical_plan/csv.rs: ## @@ -270,14 +297,223 @@ impl CsvOpener { } } +/// Returns the position of the first newlin

[GitHub] [arrow-datafusion] 2010YOUY01 commented on pull request #6801: parallel csv scan

2023-07-09 Thread via GitHub
2010YOUY01 commented on PR #6801: URL: https://github.com/apache/arrow-datafusion/pull/6801#issuecomment-162282 > Thank you @2010YOUY01 -- I tried this out again and it does indeed go (much!) faster -- 3x faster in my initial testing. 👏 Thank you for the review ❤️ I will update

[GitHub] [arrow-datafusion] alamb commented on issue #6887: `make_array` does not properly support nulls

2023-07-09 Thread via GitHub
alamb commented on issue #6887: URL: https://github.com/apache/arrow-datafusion/issues/6887#issuecomment-1627773340 I am going to try and fix this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] jp0317 commented on pull request #36510: PARQUET-2321: [C++] allow customized buffer size when creating ArrowInputStream for a column PageReader

2023-07-09 Thread via GitHub
jp0317 commented on PR #36510: URL: https://github.com/apache/arrow/pull/36510#issuecomment-1627768067 Thanks for the comments. Would allowing an optional column-specific read properties instead of optional buffer size be better? This column-specific read properties can be the `ColumnReader

[GitHub] [arrow-datafusion] alamb commented on pull request #6787: Minor: add test cases with columns for math expressions

2023-07-09 Thread via GitHub
alamb commented on PR #6787: URL: https://github.com/apache/arrow-datafusion/pull/6787#issuecomment-1627764489 @izveigor what became of this PR? Would you like some help pushing it over the line? -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow-datafusion] alamb commented on pull request #6874: Make streaming_merge public

2023-07-09 Thread via GitHub
alamb commented on PR #6874: URL: https://github.com/apache/arrow-datafusion/pull/6874#issuecomment-1627764194 > Thank you all! Thanks again for the contribution @kazuyukitanimura -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow-datafusion] alamb merged pull request #6864: feat: column support for `array_dims`, `array_ndims`, `cardinality` and `array_length`

2023-07-09 Thread via GitHub
alamb merged PR #6864: URL: https://github.com/apache/arrow-datafusion/pull/6864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow] jonathanswenson commented on a diff in pull request #36487: GH-36469: [Java] distribute linux aarch64 libs with mavencentral jars.

2023-07-09 Thread via GitHub
jonathanswenson commented on code in PR #36487: URL: https://github.com/apache/arrow/pull/36487#discussion_r1257496898 ## dev/tasks/java-jars/github.yml: ## @@ -22,8 +22,22 @@ jobs: build-cpp-ubuntu: -name: Build C++ libraries Ubuntu -runs-on: ubuntu-latest +{%

[GitHub] [arrow-datafusion] alamb commented on pull request #6893: Minor: Add TPCH scale factor 10 to bench.sh

2023-07-09 Thread via GitHub
alamb commented on PR #6893: URL: https://github.com/apache/arrow-datafusion/pull/6893#issuecomment-1627736384 > What about increasing the number of iterations as well? This also reduces the variance in the results. That is a good idea 🤔 It turns out that DataFusion blew up my mach

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #6891: Add FixedSizeBinary support to binary_op_dyn_scalar

2023-07-09 Thread via GitHub
alamb commented on code in PR #6891: URL: https://github.com/apache/arrow-datafusion/pull/6891#discussion_r1257494785 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -1005,6 +1005,7 @@ macro_rules! binary_array_op_dyn_scalar { ScalarValue::LargeUtf8(v)

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #6895: Minor: Add FixedSizeBinaryTest

2023-07-09 Thread via GitHub
alamb commented on code in PR #6895: URL: https://github.com/apache/arrow-datafusion/pull/6895#discussion_r1257494735 ## datafusion/core/tests/sqllogictests/test_files/binary.slt: ## @@ -73,3 +73,57 @@ GROUP BY column1; 1 1 1 + +statement ok +drop table t; + +# +#

[GitHub] [arrow-datafusion] alamb opened a new pull request, #6895: Minor: Add FixedSizeBinaryTest

2023-07-09 Thread via GitHub
alamb opened a new pull request, #6895: URL: https://github.com/apache/arrow-datafusion/pull/6895 # Which issue does this PR close? Related to https://github.com/apache/arrow-datafusion/pull/6891 # Rationale for this change https://github.com/apache/arrow-datafusion/pull/

[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36466: GH-36450: [CI][Python] Upload wheel artifacts for Windows

2023-07-09 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #36466: URL: https://github.com/apache/arrow/pull/36466#issuecomment-1627733266 Conbench analyzed the 6 benchmark runs on commit `1876d581`. There were 2 benchmark results indicating a performance regression: - Commit Run on `arm64-m6g-

[GitHub] [arrow-datafusion] alamb commented on pull request #6868: feat: implement substrait join filter support

2023-07-09 Thread via GitHub
alamb commented on PR #6868: URL: https://github.com/apache/arrow-datafusion/pull/6868#issuecomment-1627726701 > I just realized we have two different case files 🤣 It looks like this is a mistake from https://github.com/apache/arrow-datafusion/pull/6604. I'll reorganize those test cases la

[GitHub] [arrow-rs] eitsupi commented on a diff in pull request #4490: ci: verify MSRV on CI

2023-07-09 Thread via GitHub
eitsupi commented on code in PR #4490: URL: https://github.com/apache/arrow-rs/pull/4490#discussion_r1257489483 ## object_store/Cargo.toml: ## @@ -25,6 +25,8 @@ description = "A generic object store interface for uniformly interacting with A keywords = ["object", "storage", "c

[GitHub] [arrow-datafusion] alamb opened a new pull request, #6894: Minor: deleted duplicated substrait integration test

2023-07-09 Thread via GitHub
alamb opened a new pull request, #6894: URL: https://github.com/apache/arrow-datafusion/pull/6894 # Which issue does this PR close? Noticed by @nseekhao https://github.com/apache/arrow-datafusion/pull/6868#issuecomment-1624255211 and @waynexia in https://github.com/apache/arrow-dat

[GitHub] [arrow-datafusion] Dandandan commented on pull request #6893: Minor: Add TPCH scale factor 10 to bench.sh

2023-07-09 Thread via GitHub
Dandandan commented on PR #6893: URL: https://github.com/apache/arrow-datafusion/pull/6893#issuecomment-1627724630 What about increasing the number of iterations as well? This also reduces the variance in the results. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [arrow-datafusion] alamb closed issue #6866: Substrait: Add support for joins with filter

2023-07-09 Thread via GitHub
alamb closed issue #6866: Substrait: Add support for joins with filter URL: https://github.com/apache/arrow-datafusion/issues/6866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [arrow-datafusion] alamb merged pull request #6868: feat: implement substrait join filter support

2023-07-09 Thread via GitHub
alamb merged PR #6868: URL: https://github.com/apache/arrow-datafusion/pull/6868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-datafusion] alamb commented on pull request #6864: feat: column support for `array_dims`, `array_ndims`, `cardinality` and `array_length`

2023-07-09 Thread via GitHub
alamb commented on PR #6864: URL: https://github.com/apache/arrow-datafusion/pull/6864#issuecomment-1627724272 Thank you @jayzhan211 for the review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4484: Support nested list casting

2023-07-09 Thread via GitHub
tustvold commented on code in PR #4484: URL: https://github.com/apache/arrow-rs/pull/4484#discussion_r1257486684 ## arrow-cast/src/cast.rs: ## @@ -3680,6 +3682,45 @@ fn cast_primitive_to_list( Ok(list_array) } +/// Wraps a list array with another list array, using the sp

[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4490: ci: verify MSRV on CI

2023-07-09 Thread via GitHub
tustvold commented on code in PR #4490: URL: https://github.com/apache/arrow-rs/pull/4490#discussion_r1257486437 ## object_store/Cargo.toml: ## @@ -25,6 +25,8 @@ description = "A generic object store interface for uniformly interacting with A keywords = ["object", "storage", "

[GitHub] [arrow-datafusion] alamb merged pull request #6872: Support array concatenation for arrays with different dimensions

2023-07-09 Thread via GitHub
alamb merged PR #6872: URL: https://github.com/apache/arrow-datafusion/pull/6872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

  1   2   >