[PR] CrossJoin Refactor [arrow-datafusion]

2024-03-27 Thread via GitHub
berkaysynnada opened a new pull request, #9830: URL: https://github.com/apache/arrow-datafusion/pull/9830 ## Which issue does this PR close? Closes #. ## Rationale for this change `CrossJoinExec` does some unnecessary operations, and does not preserve the

Re: [PR] GH-40431: [C++] Try to check/alloc the TempVectorStack size as HashBatch needed [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #40484: URL: https://github.com/apache/arrow/pull/40484#discussion_r1542401276 ## cpp/src/arrow/compute/util.cc: ## @@ -32,10 +32,10 @@ using internal::CpuInfo; namespace util { void TempVectorStack::alloc(uint32_t num_bytes, uint8_t** data, int*

Re: [PR] GH-40431: [C++] Try to check/alloc the TempVectorStack size as HashBatch needed [arrow]

2024-03-27 Thread via GitHub
mapleFU commented on PR #40484: URL: https://github.com/apache/arrow/pull/40484#issuecomment-2024516075 Thanks @zanmato1984 cc @felipecrv @pitrou do we have some reviewers for ASOF join? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Implement LimitPushDown for ExecutionPlan [arrow-datafusion]

2024-03-27 Thread via GitHub
mustafasrepo commented on code in PR #9815: URL: https://github.com/apache/arrow-datafusion/pull/9815#discussion_r1542392108 ## datafusion/physical-plan/src/coalesce_batches.rs: ## @@ -83,6 +83,9 @@ impl CoalesceBatchesExec { input.execution_mode(),

Re: [PR] Implement LimitPushDown for ExecutionPlan [arrow-datafusion]

2024-03-27 Thread via GitHub
mustafasrepo commented on code in PR #9815: URL: https://github.com/apache/arrow-datafusion/pull/9815#discussion_r1542390079 ## datafusion/sqllogictest/test_files/repartition.slt: ## @@ -121,9 +121,9 @@ Limit: skip=0, fetch=5 --Filter: sink_table.c3 > Int16(0) TableScan: s

Re: [PR] GH-40282: [Python] Use C++ type traits [arrow]

2024-03-27 Thread via GitHub
llama90 commented on code in PR #40761: URL: https://github.com/apache/arrow/pull/40761#discussion_r1542381779 ## python/pyarrow/type_traits.pxi: ## @@ -0,0 +1,394 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See th

Re: [PR] Implement LimitPushDown for ExecutionPlan [arrow-datafusion]

2024-03-27 Thread via GitHub
mustafasrepo commented on PR #9815: URL: https://github.com/apache/arrow-datafusion/pull/9815#issuecomment-2024488357 Output partition number of the `UnionExec` is the sum of partition number of its inputs. Hence we cannot rely on after `UnionExec` output partitioning is 1. -- This is an

Re: [I] [Python][Parquet] Memory leak still showed on parquet.write_table and Table.from_pandas [arrow]

2024-03-27 Thread via GitHub
kyle-ip commented on issue #40738: URL: https://github.com/apache/arrow/issues/40738#issuecomment-2024484407 @guozhans Thank you very much. Your information helps me a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Adding complex expressions projections for Subquery [arrow-datafusion]

2024-03-27 Thread via GitHub
mustafasrepo commented on PR #9719: URL: https://github.com/apache/arrow-datafusion/pull/9719#issuecomment-2024475111 > @mustafasrepo Hi If you have a moment, I would be honored to have your review on my updated one. Sure, I will look into this PR. -- This is an automated message

Re: [I] Versions >32.0.0 on PyPI have broken substrait support [arrow-datafusion]

2024-03-27 Thread via GitHub
l1t1 commented on issue #9823: URL: https://github.com/apache/arrow-datafusion/issues/9823#issuecomment-2024473335 35 win64 is ok ```python >>> from datafusion import substrait as ss >>> dir(ss) ['__builtins__', '__cached__', '__doc__', '__file__', '__getattr__', '__loader__

Re: [I] [Ruby][Parquet] Maintaining Parquet Writer Ruby [arrow]

2024-03-27 Thread via GitHub
kou commented on issue #40798: URL: https://github.com/apache/arrow/issues/40798#issuecomment-2024464767 How about using `Parquet::ArrowFileWriter#write_table`? ```ruby require "parquet" schema = Arrow::Schema.new(a: :int64, b: :int64,

Re: [I] [Python][Parquet] Memory leak still showed on parquet.write_table and Table.from_pandas [arrow]

2024-03-27 Thread via GitHub
guozhans commented on issue #40738: URL: https://github.com/apache/arrow/issues/40738#issuecomment-2024457991 I closed this issue, and see above comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] [Python][Parquet] Memory leak still showed on parquet.write_table and Table.from_pandas [arrow]

2024-03-27 Thread via GitHub
guozhans commented on issue #40738: URL: https://github.com/apache/arrow/issues/40738#issuecomment-2024456764 Hi @kyle-ip, I had Arrow 14.0.0 and 16.0.0 DEV version installed in different folders before, and i am not aware of the old version until that day. I removed Arrow 14.0.0 com

Re: [PR] GH-40431: [C++] Try to check/alloc the TempVectorStack size as HashBatch needed [arrow]

2024-03-27 Thread via GitHub
ZhangHuiGui commented on code in PR #40484: URL: https://github.com/apache/arrow/pull/40484#discussion_r1542348539 ## cpp/src/arrow/compute/key_hash_test.cc: ## @@ -311,5 +311,38 @@ TEST(VectorHash, FixedLengthTailByteSafety) { HashFixedLengthFrom(/*key_length=*/19, /*num_row

Re: [PR] GH-40773: [Java] add `DENSEUNION` case to StructWriters, resolves #40773 [arrow]

2024-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #40809: URL: https://github.com/apache/arrow/pull/40809#issuecomment-2024451025 After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 515c61dd617e65c01a6e40e570487ad4ae9f151c. There were no

Re: [I] [R] write_dataset returns nothing [arrow]

2024-03-27 Thread via GitHub
amoeba commented on issue #40723: URL: https://github.com/apache/arrow/issues/40723#issuecomment-2024449558 I wonder if we could hook things up so it returns a Dataset object for the _serialized_ Dataset, instead of the input dataset. I too am not sure about the use case of putting `write_d

Re: [PR] GH-40282: [Python] Use C++ type traits [arrow]

2024-03-27 Thread via GitHub
llama90 commented on code in PR #40761: URL: https://github.com/apache/arrow/pull/40761#discussion_r1542343616 ## python/pyarrow/types.py: ## @@ -64,17 +82,17 @@ def is_boolean(t): @doc(is_null, datatype="any integer") def is_integer(t): -return t.id in _INTEGER_TYPES +

Re: [PR] GH-39069: [C++][FS][Azure] Use the generic filesystem tests [arrow]

2024-03-27 Thread via GitHub
kou commented on PR #40567: URL: https://github.com/apache/arrow/pull/40567#issuecomment-2024448174 The generic filesystem tests are passed with Azurite but failed with hierarchical name space enabled Azure account: ```text [==] Running 26 tests from 1 test suite. [-

Re: [PR] GH-39069: [C++][FS][Azure] Use the generic filesystem tests [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #40567: URL: https://github.com/apache/arrow/pull/40567#discussion_r1542342251 ## cpp/src/arrow/filesystem/test_util.h: ## @@ -168,6 +168,8 @@ class ARROW_TESTING_EXPORT GenericFileSystemTest { virtual bool allow_write_file_over_dir() const { retur

Re: [PR] GH-40631: [C++] Add lost conjunctions back in FoldConstants and GuaranteeConjunctionMembers [arrow]

2024-03-27 Thread via GitHub
ZhangHuiGui commented on code in PR #40688: URL: https://github.com/apache/arrow/pull/40688#discussion_r1542341993 ## cpp/src/arrow/compute/expression.cc: ## @@ -874,7 +874,7 @@ Result FoldConstants(Expression expr) { } } -if (call->function_name ==

Re: [PR] GH-39069: [C++][FS][Azure] Use the generic filesystem tests [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #40567: URL: https://github.com/apache/arrow/pull/40567#discussion_r1542317802 ## cpp/src/arrow/filesystem/azurefs.cc: ## @@ -1803,37 +1805,63 @@ class AzureFileSystem::Impl { const AzureLocation& location, bool recursive)

Re: [PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
amoeba commented on PR #40857: URL: https://github.com/apache/arrow/pull/40857#issuecomment-2024408534 Thanks @cyb70289 and @mapleFU! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
cyb70289 commented on PR #40857: URL: https://github.com/apache/arrow/pull/40857#issuecomment-2024408269 Thanks @amoeba ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
cyb70289 merged PR #40857: URL: https://github.com/apache/arrow/pull/40857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apac

[PR] GH-40858: [R] Remove dangling commas from codegen.R [arrow]

2024-03-27 Thread via GitHub
amoeba opened a new pull request, #40859: URL: https://github.com/apache/arrow/pull/40859 ### Rationale for this change This is a draft PR fixing https://github.com/apache/arrow/issues/40858, though I'm not sure how or why this broke (or worked correctly). Fixes #40858 #

Re: [PR] GH-40858: [R] Remove dangling commas from codegen.R [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #40859: URL: https://github.com/apache/arrow/pull/40859#issuecomment-2024407336 :warning: GitHub issue #40858 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [I] [GLib] apache-arrow-glib 15.0.2 failed to build [arrow]

2024-03-27 Thread via GitHub
kou commented on issue #40674: URL: https://github.com/apache/arrow/issues/40674#issuecomment-2024371670 Issue resolved by pull request 40736 https://github.com/apache/arrow/pull/40736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-40674: [GLib] Don't assume gint64 and int64_t use the same type [arrow]

2024-03-27 Thread via GitHub
kou merged PR #40736: URL: https://github.com/apache/arrow/pull/40736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
mapleFU commented on PR #40857: URL: https://github.com/apache/arrow/pull/40857#issuecomment-2024356148 Looks ok to me, but I'm not so familiar with arm. Will wait for yibo's comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] MINOR: [Docs] Fix broken link in acero/options.h docstring [arrow]

2024-03-27 Thread via GitHub
amoeba commented on PR #40811: URL: https://github.com/apache/arrow/pull/40811#issuecomment-2024340065 Thanks for taking a look at this @mapleFU. Merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] MINOR: [Docs] Fix broken link in acero/options.h docstring [arrow]

2024-03-27 Thread via GitHub
amoeba merged PR #40811: URL: https://github.com/apache/arrow/pull/40811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache

Re: [PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #40857: URL: https://github.com/apache/arrow/pull/40857#issuecomment-2024338347 :warning: GitHub issue #40806 **has been automatically assigned in GitHub** to PR creator. -- This is an automated message from the Apache Git Service. To respond to the mes

[PR] GH-40806: [C++] Correctly report asimd/neon in GetRuntimeInfo [arrow]

2024-03-27 Thread via GitHub
amoeba opened a new pull request, #40857: URL: https://github.com/apache/arrow/pull/40857 ### What changes are included in this PR? New case to conditional in `MakeSimdLevelString` which makes `GetRuntimeInfo` report correctly on respective CPUs. I chose to have it report "neon". Low

Re: [PR] GH-40775: [Benchmarking][Java] Fix conbench timeout [arrow]

2024-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #40786: URL: https://github.com/apache/arrow/pull/40786#issuecomment-2024312839 Thanks for your patience. Conbench analyzed the 0 benchmarking runs that have been run so far on PR commit 63916d2df1ee6ef59b2983d4ba8ed6ee3bc3a513. None of the s

Re: [I] [Go] Need to update README.md [arrow]

2024-03-27 Thread via GitHub
llama90 commented on issue #40847: URL: https://github.com/apache/arrow/issues/40847#issuecomment-2024310568 FYI, This directory(`go/arrow/internal/cpu`) was deleted in a [past commit](https://github.com/apache/arrow/commit/6240eae2189d938b4f98b8d8235146baed0246c5). It seems that the CPU di

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 in /java [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #40525: URL: https://github.com/apache/arrow/pull/40525#issuecomment-2024303243 Revision: a1a700d0d603fd4e5654b654026b15254abcdf56 Submitted crossbow builds: [ursacomputing/crossbow @ actions-3c324bb675](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 in /java [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40525: URL: https://github.com/apache/arrow/pull/40525#issuecomment-2024301573 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] GH-40431: [C++] Try to check/alloc the TempVectorStack size as HashBatch needed [arrow]

2024-03-27 Thread via GitHub
zanmato1984 commented on code in PR #40484: URL: https://github.com/apache/arrow/pull/40484#discussion_r1542242894 ## cpp/src/arrow/compute/key_hash_test.cc: ## @@ -311,5 +311,38 @@ TEST(VectorHash, FixedLengthTailByteSafety) { HashFixedLengthFrom(/*key_length=*/19, /*num_row

Re: [PR] Expose `parser` on DFParser to enable user controlled parsing [arrow-datafusion]

2024-03-27 Thread via GitHub
alamb commented on PR #9729: URL: https://github.com/apache/arrow-datafusion/pull/9729#issuecomment-2024294610 Thanks again @tshauck -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] Add extension plugin to parse SQL into logical plan / user defined SQL parser [arrow-datafusion]

2024-03-27 Thread via GitHub
alamb closed issue #533: Add extension plugin to parse SQL into logical plan / user defined SQL parser URL: https://github.com/apache/arrow-datafusion/issues/533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Expose `parser` on DFParser to enable user controlled parsing [arrow-datafusion]

2024-03-27 Thread via GitHub
alamb merged PR #9729: URL: https://github.com/apache/arrow-datafusion/pull/9729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

Re: [PR] MINOR: [CPP] Fix typo in parquet/column_writer.cc [arrow]

2024-03-27 Thread via GitHub
mapleFU commented on PR #40856: URL: https://github.com/apache/arrow/pull/40856#issuecomment-2024287289 Would you mind create an issue for that? Since it changes a method name here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[PR] MINOR: [CPP] Fix typo in parquet/column_writer.cc [arrow]

2024-03-27 Thread via GitHub
RoboSchmied opened a new pull request, #40856: URL: https://github.com/apache/arrow/pull/40856 fix typo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [I] [C++][Parquet] Encryption: FileKeyUnwrapper remove or deprecate ctor with key_material_store [arrow]

2024-03-27 Thread via GitHub
adamreeve commented on issue #40846: URL: https://github.com/apache/arrow/issues/40846#issuecomment-2024274895 Marking it as deprecated seems sensible to me too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 in /java [arrow]

2024-03-27 Thread via GitHub
dependabot[bot] commented on PR #40525: URL: https://github.com/apache/arrow/pull/40525#issuecomment-2024265905 Looks like this PR is already up-to-date with main! If you'd still like to recreate it from scratch, overwriting any edits, you can request `@dependabot recreate`. -- This is a

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 in /java [arrow]

2024-03-27 Thread via GitHub
kou commented on PR #40525: URL: https://github.com/apache/arrow/pull/40525#issuecomment-2024260295 https://github.com/dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] GH-40507: [C++][ORC] Upgrade ORC to 2.0.0 [arrow]

2024-03-27 Thread via GitHub
wgtmac commented on PR #40508: URL: https://github.com/apache/arrow/pull/40508#issuecomment-2024256175 Just reverted the java module change. Thanks for the reminder! @pitrou -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] GH-38849: [C++][Parquet]: Add support for list view and large list view [arrow]

2024-03-27 Thread via GitHub
wgtmac commented on PR #38850: URL: https://github.com/apache/arrow/pull/38850#issuecomment-2024256902 Sorry for missing this! I will take a look recently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] GH-40507: [C++][ORC] Upgrade ORC to 2.0.0 [arrow]

2024-03-27 Thread via GitHub
wgtmac commented on code in PR #40508: URL: https://github.com/apache/arrow/pull/40508#discussion_r1542216589 ## java/adapter/orc/pom.xml: ## @@ -38,7 +38,7 @@ org.apache.orc orc-core -1.9.2 +2.0.0 Review Comment: `

Re: [PR] GH-23221: [C++] Add support for building with Emscripten [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #37821: URL: https://github.com/apache/arrow/pull/37821#discussion_r1542213516 ## cpp/src/arrow/util/cancel_test.cc: ## Review Comment: Could you add `#include "arrow/util/future.h"` for `ARROW_ENABLE_THREADING`? ## cpp/cmake_modules/S

Re: [I] [C++][Parquet] Encryption: FileKeyUnwrapper remove or deprecate ctor with key_material_store [arrow]

2024-03-27 Thread via GitHub
wgtmac commented on issue #40846: URL: https://github.com/apache/arrow/issues/40846#issuecomment-2024253319 +1 for marking it as deprecated. We can remove it after several major releases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] MINOR: [C++] Remove misleading comment on FileKeyUnwrapper constructor [arrow]

2024-03-27 Thread via GitHub
conbench-apache-arrow[bot] commented on PR #40808: URL: https://github.com/apache/arrow/pull/40808#issuecomment-2024252290 After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit dc2c5c66f5234a92169da76613399135786dbffb. There were 3

Re: [PR] GH-40431: [C++] Try to check/alloc the TempVectorStack size as HashBatch needed [arrow]

2024-03-27 Thread via GitHub
ZhangHuiGui commented on PR #40484: URL: https://github.com/apache/arrow/pull/40484#issuecomment-2024250297 > My one last suggestion :) Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] GH-40816: [C++] Security checks and relaxing hashjoin batch rows size. [arrow]

2024-03-27 Thread via GitHub
Light-City commented on PR #40817: URL: https://github.com/apache/arrow/pull/40817#issuecomment-2024246728 > Is there any situation where these checks might fail or is this purely checking internal invariants? https://github.com/apache/arrow/pull/40817#issuecomment-2023191322 -- Th

Re: [PR] GH-40816: [C++] Security checks and relaxing hashjoin batch rows size. [arrow]

2024-03-27 Thread via GitHub
Light-City commented on code in PR #40817: URL: https://github.com/apache/arrow/pull/40817#discussion_r1542209756 ## cpp/src/arrow/acero/partition_util.h: ## @@ -62,7 +62,7 @@ class PartitionSort { template static void Eval(int64_t num_rows, int num_prtns, uint16_t* prtn_

Re: [PR] GH-23221: [C++] Add support for building with Emscripten [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #37821: URL: https://github.com/apache/arrow/pull/37821#issuecomment-2024244909 Revision: 07f22685f1e5502d1bb204c21450c8f3fd72399e Submitted crossbow builds: [ursacomputing/crossbow @ actions-c79c396028](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-23221: [C++] Add support for building with Emscripten [arrow]

2024-03-27 Thread via GitHub
kou commented on PR #37821: URL: https://github.com/apache/arrow/pull/37821#issuecomment-2024243364 @github-actions crossbow submit test-ubuntu-22.04-cpp-emscripten -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] GH-40816: [C++] Security checks and relaxing hashjoin batch rows size. [arrow]

2024-03-27 Thread via GitHub
Light-City commented on code in PR #40817: URL: https://github.com/apache/arrow/pull/40817#discussion_r1542205887 ## cpp/src/arrow/acero/partition_util.h: ## @@ -62,7 +62,7 @@ class PartitionSort { template static void Eval(int64_t num_rows, int num_prtns, uint16_t* prtn_

Re: [PR] Adding complex expressions projections for Subquery [arrow-datafusion]

2024-03-27 Thread via GitHub
Lordworms commented on PR #9719: URL: https://github.com/apache/arrow-datafusion/pull/9719#issuecomment-2024221509 @mustafasrepo Hi If you have a moment, I would be honored to have your review on my updated one. -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] GH-40733: [Go] Require Go 1.21 or later [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #40848: URL: https://github.com/apache/arrow/pull/40848#discussion_r1542191873 ## go/arrow/bitutil/bitutil_bytes_go1.19.go: ## Review Comment: Can we remove this because we drop support for Go 1.19? -- This is an automated message from the Ap

Re: [PR] Implement LimitPushDown for ExecutionPlan [arrow-datafusion]

2024-03-27 Thread via GitHub
Lordworms commented on PR #9815: URL: https://github.com/apache/arrow-datafusion/pull/9815#issuecomment-2024219176 I got stuck in a plan like this https://github.com/apache/arrow-datafusion/assets/48054792/6c3b0325-1167-410f-af3a-ca0bfec0854a";> the panic error lies in https://git

Re: [PR] GH-40814: [C++] Thirdparty: bump zstd to 1.5.6 [arrow]

2024-03-27 Thread via GitHub
kou merged PR #40837: URL: https://github.com/apache/arrow/pull/40837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [I] [C++] Thirdparty: Do we need bump zstd to 1.5.6? [arrow]

2024-03-27 Thread via GitHub
kou commented on issue #40814: URL: https://github.com/apache/arrow/issues/40814#issuecomment-2024215724 Issue resolved by pull request 40837 https://github.com/apache/arrow/pull/40837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-20213: [C++] Implement cast to/from halffloat [arrow]

2024-03-27 Thread via GitHub
kou commented on PR #40067: URL: https://github.com/apache/arrow/pull/40067#issuecomment-2024208133 Could you rebase on main? The failure was fixed by #40794. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] GH-40819: [Java] Adding Spotless to Algorithm module [arrow]

2024-03-27 Thread via GitHub
kou commented on code in PR #40835: URL: https://github.com/apache/arrow/pull/40835#discussion_r1542181268 ## java/algorithm/pom.xml: ## @@ -50,4 +55,68 @@ + + + + spotless-jdk11+ + +[11,] + + + + +com.

Re: [I] [Dev][C++][Python][R] Use pre-commit for clang-format [arrow]

2024-03-27 Thread via GitHub
kou commented on issue #40586: URL: https://github.com/apache/arrow/issues/40586#issuecomment-2024204194 Issue resolved by pull request 40587 https://github.com/apache/arrow/pull/40587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] GH-40586: [Dev][C++][Python][R] Use pre-commit for clang-format [arrow]

2024-03-27 Thread via GitHub
kou merged PR #40587: URL: https://github.com/apache/arrow/pull/40587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

Re: [PR] Implement LimitPushDown for ExecutionPlan [arrow-datafusion]

2024-03-27 Thread via GitHub
Lordworms commented on code in PR #9815: URL: https://github.com/apache/arrow-datafusion/pull/9815#discussion_r1542179422 ## datafusion/core/src/physical_optimizer/limit_pushdown.rs: ## @@ -0,0 +1,127 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

Re: [PR] [WIP] Analyzer: Arc vs Cow [arrow-datafusion]

2024-03-27 Thread via GitHub
comphead commented on PR #9824: URL: https://github.com/apache/arrow-datafusion/pull/9824#issuecomment-2024189261 @alamb @jayzhan211 I can see some improvements for the `Arc` let me know folks if you wanna proceed with this -- This is an automated message from the Apache Git Service. To

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542174266 ## cpp/src/arrow/extension/tensor_extension_array_test.cc: ## @@ -165,47 +170,51 @@ TEST_F(TestExtensionType, CreateFromArray) { ASSERT_EQ(ext_arr->null_count(), 0); }

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542174125 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542172903 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-40716: [Java][Integration] Fix test_package_java in verification scripts [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #40724: URL: https://github.com/apache/arrow/pull/40724#issuecomment-2024183053 Revision: 8848b6b38908348e334d56919582e93d6dbc60e4 Submitted crossbow builds: [ursacomputing/crossbow @ actions-e07aa5a2fb](https://github.com/ursacomputing/crossbow/bra

Re: [PR] GH-40716: [Java][Integration] Fix test_package_java in verification scripts [arrow]

2024-03-27 Thread via GitHub
danepitkin commented on PR #40724: URL: https://github.com/apache/arrow/pull/40724#issuecomment-2024181140 @github-actions crossbow submit verify-rc-source-integration* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugin-tools:maven-plugin-annotations from 3.6.0 to 3.11.0 in /java [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40524: URL: https://github.com/apache/arrow/pull/40524#issuecomment-2024179380 This should be good to merge? or shall we verify the crossbows` once more? cc @lidavidm @danepitkin -- This is an automated message from the Apache Git Service. To respond to the m

Re: [PR] GH-40775: Fix conbench timeout for java benchmarks [arrow]

2024-03-27 Thread via GitHub
ursabot commented on PR #40786: URL: https://github.com/apache/arrow/pull/40786#issuecomment-2024179314 Benchmark runs are scheduled for commit 63916d2df1ee6ef59b2983d4ba8ed6ee3bc3a513. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be po

Re: [PR] GH-40775: Fix conbench timeout for java benchmarks [arrow]

2024-03-27 Thread via GitHub
danepitkin commented on PR #40786: URL: https://github.com/apache/arrow/pull/40786#issuecomment-2024179276 @ursabot please benchmark lang=Java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: [Java] Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 in /java [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40525: URL: https://github.com/apache/arrow/pull/40525#issuecomment-2024179058 @kou could we re-run the failing CI, seems like it is not related. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] MINOR: [Java] Bump org.apache.orc:orc-core from 1.9.2 to 2.0.0 in /java [arrow]

2024-03-27 Thread via GitHub
dependabot[bot] commented on PR #40779: URL: https://github.com/apache/arrow/pull/40779#issuecomment-2024174853 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] MINOR: [Java] Bump org.apache.orc:orc-core from 1.9.2 to 2.0.0 in /java [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40779: URL: https://github.com/apache/arrow/pull/40779#issuecomment-2024174822 closing this as we are tracking it https://github.com/apache/arrow/issues/38051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] MINOR: [Java] Bump org.apache.orc:orc-core from 1.9.2 to 2.0.0 in /java [arrow]

2024-03-27 Thread via GitHub
vibhatha closed pull request #40779: MINOR: [Java] Bump org.apache.orc:orc-core from 1.9.2 to 2.0.0 in /java URL: https://github.com/apache/arrow/pull/40779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] MINOR: [Java] Fix maven-checkstyle-plugin configuration [arrow]

2024-03-27 Thread via GitHub
github-actions[bot] commented on PR #40850: URL: https://github.com/apache/arrow/pull/40850#issuecomment-2024173121 Revision: 1588a21122890cf09d8e0ce4ccc74b8b4cf3e326 Submitted crossbow builds: [ursacomputing/crossbow @ actions-1a040d372b](https://github.com/ursacomputing/crossbow/bra

Re: [PR] MINOR: [Java] Fix maven-checkstyle-plugin configuration [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40850: URL: https://github.com/apache/arrow/pull/40850#issuecomment-2024172951 Thanks @laurentgo for creating this PR, I kicked the crossbows to validate the change. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542164378 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542164240 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] MINOR: [Java] Fix maven-checkstyle-plugin configuration [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40850: URL: https://github.com/apache/arrow/pull/40850#issuecomment-2024171406 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542163349 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542162612 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542162406 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

[PR] Correct example code for column (#5560) [arrow-rs]

2024-03-27 Thread via GitHub
zgershkoff opened a new pull request, #5561: URL: https://github.com/apache/arrow-rs/pull/5561 # Which issue does this PR close? Closes #5560 . # Rationale for this change The example is incorrect as it is. This changes the assertion to reflect the actual behavio

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542160937 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542161085 ## cpp/src/arrow/extension/variable_shape_tensor.cc: ## @@ -0,0 +1,341 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agr

Re: [I] [Benchmarking] [Java] new `java.lang.OutOfMemoryError` in Java benchmarks after local build cache change [arrow]

2024-03-27 Thread via GitHub
danepitkin commented on issue #40775: URL: https://github.com/apache/arrow/issues/40775#issuecomment-2024166929 Nvm, I think this is okay. It's configured to run for 5 iterations 10 seconds each. We are just printing out billions of debug lines so just need to disable DEBUG logging. -- T

Re: [PR] GH-40843: [Java] Cleanup protobuf-maven-plugin usage [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40844: URL: https://github.com/apache/arrow/pull/40844#issuecomment-2024166066 @lidavidm seems like the failing CIs not related, could we re-run and verify? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542160044 ## cpp/src/arrow/extension/variable_shape_tensor.h: ## @@ -0,0 +1,110 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agre

Re: [PR] GH-40843: [Java] Cleanup protobuf-maven-plugin usage [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on PR #40844: URL: https://github.com/apache/arrow/pull/40844#issuecomment-2024165717 @github-actions crossbow submit -g java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542159505 ## cpp/src/arrow/extension/tensor_internal.h: ## @@ -42,4 +47,46 @@ Status IsPermutationValid(const std::vector& permutation) { return Status::OK(); } +ARROW_EXPORT

Re: [PR] GH-40819: [Java] Adding Spotless to Algorithm module [arrow]

2024-03-27 Thread via GitHub
vibhatha commented on code in PR #40835: URL: https://github.com/apache/arrow/pull/40835#discussion_r1542158874 ## java/algorithm/src/main/java/org/apache/arrow/algorithm/search/VectorRangeSearcher.java: ## @@ -1,108 +1,108 @@ -/* - * Licensed to the Apache Software Foundation (

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542158156 ## cpp/src/arrow/extension/tensor_internal.h: ## @@ -42,4 +47,46 @@ Status IsPermutationValid(const std::vector& permutation) { return Status::OK(); } +ARROW_EXPORT

Re: [PR] GH-38007: [C++] Add VariableShapeTensor implementation [arrow]

2024-03-27 Thread via GitHub
rok commented on code in PR #38008: URL: https://github.com/apache/arrow/pull/38008#discussion_r1542158037 ## cpp/src/arrow/extension/tensor_internal.h: ## @@ -42,4 +47,46 @@ Status IsPermutationValid(const std::vector& permutation) { return Status::OK(); } +ARROW_EXPORT

  1   2   3   4   >