[GitHub] [arrow] kou commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

kou commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673239409 @kszucs It seems that this should be included into 1.0.1. What do you think about this? This is an automated message

[GitHub] [arrow] kou closed pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

kou closed pull request #7937: URL: https://github.com/apache/arrow/pull/7937 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [arrow] chaokunyang commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

chaokunyang commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673228598 Hi @kou , I've updated the pr, could you help review again and merge it? This is an automated message from the

[GitHub] [arrow] liyafan82 commented on pull request #7826: ARROW-9495: [C++] Equality assertions don't handle Inf / -Inf properly

liyafan82 commented on pull request #7826: URL: https://github.com/apache/arrow/pull/7826#issuecomment-673213609 @kiszk Thanks a lot for your feedback. @pitrou could you please give some comments? This is an automated mes

[GitHub] [arrow] liyafan82 commented on pull request #7326: ARROW-9010: [Java] Framework and interface changes for RecordBatch IPC buffer compression

liyafan82 commented on pull request #7326: URL: https://github.com/apache/arrow/pull/7326#issuecomment-673213128 The conda integration test is failing, because the default no compression option is not supported by the specification. Maybe we need to start a discussion in the ML. ---

[GitHub] [arrow] liyafan82 commented on a change in pull request #7837: ARROW-9554: [Java] FixedWidthInPlaceVectorSorter sometimes produces wrong result

liyafan82 commented on a change in pull request #7837: URL: https://github.com/apache/arrow/pull/7837#discussion_r469660938 ## File path: java/algorithm/src/main/java/org/apache/arrow/algorithm/sort/VariableWidthOutOfPlaceVectorSorter.java ## @@ -45,6 +46,22 @@ public void sor

[GitHub] [arrow] kou commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

kou commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673204636 Thanks. How about this? ```diff diff --git a/python/pyarrow/__init__.py b/python/pyarrow/__init__.py index e011ee74f..9e22cc013 100644 --- a/python/pyarrow/__init__.p

[GitHub] [arrow] chaokunyang removed a comment on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

chaokunyang removed a comment on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673202525 ``` def get_symlink_path(hard_path): return '.'.join((hard_path.split('.')[0], 'dylib')) ``` `get_symlink_path("/Users/xxx/anaconda3/envs/ray/lib/python

[GitHub] [arrow] chaokunyang commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

chaokunyang commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673202954 ```python def get_symlink_path(hard_path): return '.'.join((hard_path.split('.')[0], 'dylib')) ``` `get_symlink_path("/Users/xxx/anaconda3/envs/ray/lib/python3.

[GitHub] [arrow] chaokunyang commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

chaokunyang commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673202525 ``` def get_symlink_path(hard_path): return '.'.join((hard_path.split('.')[0], 'dylib')) ``` `get_symlink_path("/Users/xxx/anaconda3/envs/ray/lib/python3.6/site

[GitHub] [arrow] jianxind commented on pull request #7871: ARROW-9605: [C++] Speed up aggregate min/max compute kernels on integer types

jianxind commented on pull request #7871: URL: https://github.com/apache/arrow/pull/7871#issuecomment-673192812 > @ursabot benchmark --suite-filter=arrow-compute-aggregate-benchmark --benchmark-filter=MinMax Below is the results for null_percent 0.01% and 0% on https://ci.ursalabs.o

[GitHub] [arrow] jianxind commented on a change in pull request #7940: ARROW-9702: [C++] Move bpacking SIMD to runtime path.

jianxind commented on a change in pull request #7940: URL: https://github.com/apache/arrow/pull/7940#discussion_r469638237 ## File path: cpp/src/arrow/util/bpacking.cc ## @@ -0,0 +1,174 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor li

[GitHub] [arrow] corleyma commented on pull request #7803: ARROW-9517: [C++/Python] Add support for temporary credentials to S3Options

corleyma commented on pull request #7803: URL: https://github.com/apache/arrow/pull/7803#issuecomment-673186485 @github-actions crossbow submit -g conda This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] jianxind commented on pull request #7871: ARROW-9605: [C++] Speed up aggregate min/max compute kernels on integer types

jianxind commented on pull request #7871: URL: https://github.com/apache/arrow/pull/7871#issuecomment-673186171 @ursabot benchmark --suite-filter=arrow-compute-aggregate-benchmark --benchmark-filter=MinMax This is an automat

[GitHub] [arrow] jianxind removed a comment on pull request #7871: ARROW-9605: [C++] Speed up aggregate min/max compute kernels on integer types

jianxind removed a comment on pull request #7871: URL: https://github.com/apache/arrow/pull/7871#issuecomment-673185847 @ursabot benchmark --suite-filter=arrow-compute-aggregate-benchmark --benchmark_filter="MinMax" This is

[GitHub] [arrow] jianxind commented on pull request #7871: ARROW-9605: [C++] Speed up aggregate min/max compute kernels on integer types

jianxind commented on pull request #7871: URL: https://github.com/apache/arrow/pull/7871#issuecomment-673185847 @ursabot benchmark --suite-filter=arrow-compute-aggregate-benchmark --benchmark_filter="MinMax" This is an autom

[GitHub] [arrow] ursabot commented on pull request #7871: ARROW-9605: [C++] Speed up aggregate min/max compute kernels on integer types

ursabot commented on pull request #7871: URL: https://github.com/apache/arrow/pull/7871#issuecomment-673185853 ``` no such option: --benchmark_filter ``` This is an automated message from the Apache Git Service. To resp

[GitHub] [arrow] github-actions[bot] commented on pull request #7951: ARROW-9716: [Rust] [DataFusion] Implement limit on concurrent threads in MergeExec

github-actions[bot] commented on pull request #7951: URL: https://github.com/apache/arrow/pull/7951#issuecomment-673183472 https://issues.apache.org/jira/browse/ARROW-9716 This is an automated message from the Apache Git Serv

[GitHub] [arrow] jianxind commented on pull request #7940: ARROW-9702: [C++] Move bpacking SIMD to runtime path.

jianxind commented on pull request #7940: URL: https://github.com/apache/arrow/pull/7940#issuecomment-673183390 > @ursabot benchmark --suite-filter=parquet-encoding-benchmark --benchmark-filter=BM_PlainDecodingBoolean Below is the results on this action https://ci.ursalabs.org/#/bui

[GitHub] [arrow] andygrove commented on a change in pull request #7951: ARROW-9716: [Rust] [DataFusion] Implement limit on concurrent threads in MergeExec

andygrove commented on a change in pull request #7951: URL: https://github.com/apache/arrow/pull/7951#discussion_r469629508 ## File path: rust/datafusion/src/execution/context.rs ## @@ -80,9 +82,16 @@ fn tuple_err(value: (Result, Result)) -> Result<(T, R)> { impl ExecutionCon

[GitHub] [arrow] andygrove commented on pull request #7951: ARROW-9716: [Rust] [DataFusion] Implement limit on concurrent threads in MergeExec

andygrove commented on pull request #7951: URL: https://github.com/apache/arrow/pull/7951#issuecomment-673173406 @alamb This is a quick version of what we were discussing This is an automated message from the Apache Git Servi

[GitHub] [arrow] andygrove opened a new pull request #7951: ARROW-9716: [Rust] [DataFusion] Implement limit on concurrent threads

andygrove opened a new pull request #7951: URL: https://github.com/apache/arrow/pull/7951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] nealrichardson closed pull request #7950: ARROW-9715: [R] changelog/doc updates for 1.0.1

nealrichardson closed pull request #7950: URL: https://github.com/apache/arrow/pull/7950 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] github-actions[bot] commented on pull request #7950: ARROW-9715: [R] changelog/doc updates for 1.0.1

github-actions[bot] commented on pull request #7950: URL: https://github.com/apache/arrow/pull/7950#issuecomment-673159838 https://issues.apache.org/jira/browse/ARROW-9715 This is an automated message from the Apache Git Serv

[GitHub] [arrow] nealrichardson opened a new pull request #7950: ARROW-9715: [R] changelog/doc updates for 1.0.1

nealrichardson opened a new pull request #7950: URL: https://github.com/apache/arrow/pull/7950 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow] andygrove commented on pull request #7946: ARROW-9711: [Rust] Add new benchmark derived from TPC-H

andygrove commented on pull request #7946: URL: https://github.com/apache/arrow/pull/7946#issuecomment-673157623 @jorgecarleitao @alamb fyi This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] andygrove commented on pull request #7946: ARROW-9711: [Rust] Add new benchmark derived from TPC-H

andygrove commented on pull request #7946: URL: https://github.com/apache/arrow/pull/7946#issuecomment-673156833 @wesm I believe that we can use TPC tests under their fair use policy [1] but we need to be careful to refer to them as "derived from" TPC since they are not official TPC tests.

[GitHub] [arrow] github-actions[bot] commented on pull request #7949: ARROW-9714: [Rust] [DataFusion] Implement type coercion rule for limit and sort

github-actions[bot] commented on pull request #7949: URL: https://github.com/apache/arrow/pull/7949#issuecomment-673155789 https://issues.apache.org/jira/browse/ARROW-9714 This is an automated message from the Apache Git Serv

[GitHub] [arrow] andygrove opened a new pull request #7949: ARROW-9714: [Rust] [DataFusion] Implement type coercion rule for limit and sort

andygrove opened a new pull request #7949: URL: https://github.com/apache/arrow/pull/7949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #7948: ARROW-9713: [Rust] [DataFusion] Remove explicit panics

github-actions[bot] commented on pull request #7948: URL: https://github.com/apache/arrow/pull/7948#issuecomment-673151990 https://issues.apache.org/jira/browse/ARROW-9713 This is an automated message from the Apache Git Serv

[GitHub] [arrow] andygrove opened a new pull request #7948: ARROW-9713: [Rust] [DataFusion] Remove explicit panics

andygrove opened a new pull request #7948: URL: https://github.com/apache/arrow/pull/7948 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #7947: ARROW-9712: [Rust] [DataFusion] Fix parquet error handling and general code improvements

github-actions[bot] commented on pull request #7947: URL: https://github.com/apache/arrow/pull/7947#issuecomment-673146937 https://issues.apache.org/jira/browse/ARROW-9712 This is an automated message from the Apache Git Serv

[GitHub] [arrow] andygrove commented on pull request #7947: ARROW-9712: [Rust] [DataFusion] Fix parquet error handling and general code improvements

andygrove commented on pull request #7947: URL: https://github.com/apache/arrow/pull/7947#issuecomment-673143723 @alamb @jorgecarleitao This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow] andygrove opened a new pull request #7947: ARROW-9712: [Rust] [DataFusion] Fix parquet error handling and general code improvements

andygrove opened a new pull request #7947: URL: https://github.com/apache/arrow/pull/7947 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] kou commented on a change in pull request #7938: ARROW-9701: [CI][Java] Add a job for s390x Java on TravisCI

kou commented on a change in pull request #7938: URL: https://github.com/apache/arrow/pull/7938#discussion_r469565710 ## File path: .travis.yml ## @@ -81,24 +137,6 @@ before_install: install: - pip3 install -e dev/archery[docker] -script: Review comment: How about

[GitHub] [arrow] github-actions[bot] commented on pull request #7946: ARROW-9711: [Rust] Add new benchmark derived from TPC-H [DRAFT]

github-actions[bot] commented on pull request #7946: URL: https://github.com/apache/arrow/pull/7946#issuecomment-673126940 https://issues.apache.org/jira/browse/ARROW-9711 This is an automated message from the Apache Git Serv

[GitHub] [arrow] andygrove commented on pull request #7946: ARROW-9711: [Rust] Add new benchmark derived from TPC-H [DRAFT]

andygrove commented on pull request #7946: URL: https://github.com/apache/arrow/pull/7946#issuecomment-673126018 @jorgecarleitao @alamb fyi This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] andygrove opened a new pull request #7946: ARROW-9711: [Rust] Add new benchmark derived from TPC-H [DRAFT]

andygrove opened a new pull request #7946: URL: https://github.com/apache/arrow/pull/7946 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] alamb commented on pull request #7880: ARROW-9619: [Rust] [DataFusion] Add predicate push-down

alamb commented on pull request #7880: URL: https://github.com/apache/arrow/pull/7880#issuecomment-673117865 I'll try and check it out carefully tomorrow morning (US EST time) On Wed, Aug 12, 2020 at 3:51 PM Jorge Leitao wrote: > Any of you @alamb

[GitHub] [arrow] kou commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

kou commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-673116538 Could you show a `.dylib` path that causes this problem? It seems that the current implementation works well for `libarrow.100.dylib` that is included in https://files.pythonho

[GitHub] [arrow] kou commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

kou commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673105431 I suggested the renaming approach because using release build libarrow.so without `NDEBUG` increases needless (empty) `CheckXXX()` function calls. If it's acceptable, just removing

[GitHub] [arrow] github-actions[bot] commented on pull request #7945: ARROW-9710: [C++] Improve performance of Decimal128::ToString by 8x, and make the implementation reusable for Decimal256.

github-actions[bot] commented on pull request #7945: URL: https://github.com/apache/arrow/pull/7945#issuecomment-673083492 https://issues.apache.org/jira/browse/ARROW-9710 This is an automated message from the Apache Git Serv

[GitHub] [arrow] jorgecarleitao edited a comment on pull request #7797: ARROW-4189 [Rust] Added coverage report.

jorgecarleitao edited a comment on pull request #7797: URL: https://github.com/apache/arrow/pull/7797#issuecomment-673082183 @andygrove , unfortunately do not benefit from caching here, as this needs to be built from scratch. I am not very happy with the build time (+10m of build time).

[GitHub] [arrow] jorgecarleitao commented on pull request #7797: ARROW-4189 [Rust] Added coverage report.

jorgecarleitao commented on pull request #7797: URL: https://github.com/apache/arrow/pull/7797#issuecomment-673082183 @andygrove , unfortunately do not benefit from caching here, as this needs to be built from scratch. I am not very happy with the build time (+10m of build time). I

[GitHub] [arrow] MingyuZhong opened a new pull request #7945: ARROW-9710: [C++] Improve performance of Decimal128::ToString by 8x, and make the implementation reusable for Decimal256.

MingyuZhong opened a new pull request #7945: URL: https://github.com/apache/arrow/pull/7945 Added a benchmark to decimal_benchmark. ``` Running release/arrow-decimal-benchmark Run on (12 X 4500 MHz CPU s) CPU Caches: L1 Data 32 KiB (x6) L1 Instruction 32 KiB (x6)

[GitHub] [arrow] andygrove closed pull request #7934: ARROW-9695: [Rust] Improve comments on LogicalPlan enum variants

andygrove closed pull request #7934: URL: https://github.com/apache/arrow/pull/7934 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove closed pull request #7936: ARROW-9679: [Rust] [DataFusion] More efficient creation of final batch from HashAggregateExec

andygrove closed pull request #7936: URL: https://github.com/apache/arrow/pull/7936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] jorgecarleitao commented on pull request #7880: ARROW-9619: [Rust] [DataFusion] Add predicate push-down

jorgecarleitao commented on pull request #7880: URL: https://github.com/apache/arrow/pull/7880#issuecomment-673076257 Any of you @alamb @houqp @nevi-me @paddyhoran could help out here? I think that this does significantly speeds querying for anything more complex, as we run aggregations an

[GitHub] [arrow] jorgecarleitao commented on pull request #7934: ARROW-9695: [Rust] Improve comments on LogicalPlan enum variants

jorgecarleitao commented on pull request #7934: URL: https://github.com/apache/arrow/pull/7934#issuecomment-673073884 LGTM also. Thanks a lot! This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [arrow] jorgecarleitao commented on pull request #7936: ARROW-9679: [Rust] [DataFusion] More efficient creation of final batch from HashAggregateExec

jorgecarleitao commented on pull request #7936: URL: https://github.com/apache/arrow/pull/7936#issuecomment-673072087 LGTM! Thanks a lot for this, nice cleanup! I closed #7687 in favor of this one as the overall is too high to salvage. -

[GitHub] [arrow] corleyma commented on pull request #7803: ARROW-9517: [C++/Python] Add support for temporary credentials to S3Options

corleyma commented on pull request #7803: URL: https://github.com/apache/arrow/pull/7803#issuecomment-673070138 @pitrou any idea what the cadence is for rebuilding those Docker images used by the CI system (or whether we can use our own base image)? --

[GitHub] [arrow] jorgecarleitao closed pull request #7687: ARROW-9382: [Rust][DataFusion] Simplified hash aggregations and added Boolean type

jorgecarleitao closed pull request #7687: URL: https://github.com/apache/arrow/pull/7687 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] jorgecarleitao commented on pull request #7687: ARROW-9382: [Rust][DataFusion] Simplified hash aggregations and added Boolean type

jorgecarleitao commented on pull request #7687: URL: https://github.com/apache/arrow/pull/7687#issuecomment-673068765 Closing in preference of #7936 . This is an automated message from the Apache Git Service. To respond to th

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673067583 Hey look at that, it's happy now 😄 This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [arrow] jorgecarleitao commented on pull request #7876: ARROW-9615: [Rust] Added kernel to compute length of a string.

jorgecarleitao commented on pull request #7876: URL: https://github.com/apache/arrow/pull/7876#issuecomment-673065004 Thank you @nevi-me , @jhorstmann and @paddyhoran for the reviews, much appreciated! This is an automated m

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673051995 I'll submit this for review as is if that's okay with you. This is an automated message from the Apache Git Servic

[GitHub] [arrow] pitrou closed pull request #7870: ARROW-9604: [C++] Add aggregate min/max benchmark

pitrou closed pull request #7870: URL: https://github.com/apache/arrow/pull/7870 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] pitrou commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

pitrou commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673049070 If you installed [archery](https://arrow.apache.org/docs/developers/archery.html), you can run `archery lint --cmake-format`. Otherwise we can do it for you.

[GitHub] [arrow] bdunlay edited a comment on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay edited a comment on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673047928 I will push a new change with that newline removed, but the linter is going to complain anyway because there are lots of linting issues that are not compliant with the lint

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673047928 I will push a new change with that newline removed, but the linter is going to complain anyway because there are lots of changes that are not compliant with the linter in that cma

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673039693 Think the linter is mad at me for an extra newline in the cmake file. Will remove that w/ an update once CI completes. ---

[GitHub] [arrow] eerhardt commented on a change in pull request #7654: ARROW-8581: [C#] Accept and return DateTime from DateXXArray

eerhardt commented on a change in pull request #7654: URL: https://github.com/apache/arrow/pull/7654#discussion_r469456287 ## File path: csharp/src/Apache.Arrow/Arrays/Date64Array.cs ## @@ -15,56 +15,103 @@ using Apache.Arrow.Types; using System; -using System.Collections.G

[GitHub] [arrow] kiszk commented on pull request #7826: ARROW-9495: [C++] Equality assertions don't handle Inf / -Inf properly

kiszk commented on pull request #7826: URL: https://github.com/apache/arrow/pull/7826#issuecomment-673034321 Looks good to me This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow] pitrou commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

pitrou commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673017463 Ah, yes, you are right, in those cases the `#ifdef` is pointless. This is an automated message from the Apache Git

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673016929 For example the lines you pointed out to me: [src/arrow/util/trie.h](https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/trie.h#L106-L108) If this is a debug buil

[GitHub] [arrow] pitrou commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

pitrou commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673015852 @bdunlay Where exactly? Do you have an example? This is an automated message from the Apache Git Service. To respon

[GitHub] [arrow] bdunlay commented on pull request #7939: ARROW-9698: [C++] Remove -DNDEBUG flag leak in .pc file

bdunlay commented on pull request #7939: URL: https://github.com/apache/arrow/pull/7939#issuecomment-673015197 @pitrou Why are the asserts wrapped in NDEBUG? Isn't that redundant? This is an automated message from the Apache

[GitHub] [arrow] pitrou commented on a change in pull request #7907: ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to partition base_dir

pitrou commented on a change in pull request #7907: URL: https://github.com/apache/arrow/pull/7907#discussion_r469419589 ## File path: python/pyarrow/fs.py ## @@ -117,6 +116,9 @@ def get_type_name(self): protocol = protocol[0] return "fsspec+{0}".format(pr

[GitHub] [arrow] pitrou commented on a change in pull request #7907: ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to partition base_dir

pitrou commented on a change in pull request #7907: URL: https://github.com/apache/arrow/pull/7907#discussion_r469418747 ## File path: python/pyarrow/fs.py ## @@ -117,6 +116,9 @@ def get_type_name(self): protocol = protocol[0] return "fsspec+{0}".format(pr

[GitHub] [arrow] pitrou commented on a change in pull request #7870: ARROW-9604: [C++] Add aggregate min/max benchmark

pitrou commented on a change in pull request #7870: URL: https://github.com/apache/arrow/pull/7870#discussion_r469412632 ## File path: cpp/src/arrow/compute/kernels/aggregate_benchmark.cc ## @@ -369,5 +369,51 @@ MODE_KERNEL_BENCHMARK(ModeKernelInt16, Int16Type); MODE_KERNEL_BE

[GitHub] [arrow] pitrou commented on a change in pull request #7940: ARROW-9702: [C++] Move bpacking SIMD to runtime path.

pitrou commented on a change in pull request #7940: URL: https://github.com/apache/arrow/pull/7940#discussion_r469406589 ## File path: cpp/src/arrow/util/bpacking.cc ## @@ -0,0 +1,174 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor lice

[GitHub] [arrow] kiszk commented on a change in pull request #7837: ARROW-9554: [Java] FixedWidthInPlaceVectorSorter sometimes produces wrong result

kiszk commented on a change in pull request #7837: URL: https://github.com/apache/arrow/pull/7837#discussion_r469408080 ## File path: java/algorithm/src/main/java/org/apache/arrow/algorithm/sort/VariableWidthOutOfPlaceVectorSorter.java ## @@ -45,6 +46,22 @@ public void sortOut

[GitHub] [arrow] pitrou edited a comment on pull request #7940: ARROW-9702: [C++] Move bpacking SIMD to runtime path.

pitrou edited a comment on pull request #7940: URL: https://github.com/apache/arrow/pull/7940#issuecomment-672989561 Here are some benchmarks on an AMD Ryzen CPU (AVX2-enabled): * before: ``` BM_PlainDecodingBoolean/1024 344 ns 344 ns 2030607 bytes_per_second=

[GitHub] [arrow] pitrou commented on pull request #7940: ARROW-9702: [C++] Move bpacking SIMD to runtime path.

pitrou commented on pull request #7940: URL: https://github.com/apache/arrow/pull/7940#issuecomment-672989561 Here are some benchmarks on an AMD Ryzen CPU (AVX2-enabled): * before: ``` BM_PlainDecodingBoolean/1024 344 ns 344 ns 2030607 bytes_per_second=2.77389

[GitHub] [arrow] pitrou commented on a change in pull request #7898: ARROW-9642: [C++] Let MakeBuilder refer DictionaryType's index_type for deciding the starting bit width of the indices

pitrou commented on a change in pull request #7898: URL: https://github.com/apache/arrow/pull/7898#discussion_r469387808 ## File path: cpp/src/arrow/array/array_test.cc ## @@ -2029,6 +2029,19 @@ TEST_F(TestAdaptiveIntBuilder, TestAppendNulls) { } } +TEST(TestAdaptiveIntBu

[GitHub] [arrow] github-actions[bot] commented on pull request #7944: ARROW-9709: [Java] Test cases in arrow-vector takes care of endianness

github-actions[bot] commented on pull request #7944: URL: https://github.com/apache/arrow/pull/7944#issuecomment-672974799 https://issues.apache.org/jira/browse/ARROW-9709 This is an automated message from the Apache Git Serv

[GitHub] [arrow] kiszk opened a new pull request #7944: ARROW-9709: [Java] Test cases in arrow-vector takes care of endianness

kiszk opened a new pull request #7944: URL: https://github.com/apache/arrow/pull/7944 This PR aims to correctly take care of endianness in test cases in arrow-vector. In the current code, they assume only a little-endian platform. --

[GitHub] [arrow] github-actions[bot] commented on pull request #7922: [Packaging][Python] Update wheel dependency files

github-actions[bot] commented on pull request #7922: URL: https://github.com/apache/arrow/pull/7922#issuecomment-672963266 Revision: 2bacf8c61c9bbdf6718db9d8cba0c0b2883b9cd4 Submitted crossbow builds: [ursa-labs/crossbow @ actions-480](https://github.com/ursa-labs/crossbow/branches/a

[GitHub] [arrow] github-actions[bot] commented on pull request #7942: ARROW-9704: [Java] TestEndianness.testLittleEndian supports little- and big-endian platforms

github-actions[bot] commented on pull request #7942: URL: https://github.com/apache/arrow/pull/7942#issuecomment-672959448 https://issues.apache.org/jira/browse/ARROW-9704 This is an automated message from the Apache Git Serv

[GitHub] [arrow] github-actions[bot] commented on pull request #7937: ARROW-9700: [Python] fix create_library_symlinks for macos

github-actions[bot] commented on pull request #7937: URL: https://github.com/apache/arrow/pull/7937#issuecomment-672959452 https://issues.apache.org/jira/browse/ARROW-9700 This is an automated message from the Apache Git Serv

[GitHub] [arrow] github-actions[bot] commented on pull request #7943: ARROW-9706: [Java] Tests of TestLargeListVector correctly read offset

github-actions[bot] commented on pull request #7943: URL: https://github.com/apache/arrow/pull/7943#issuecomment-672959449 https://issues.apache.org/jira/browse/ARROW-9706 This is an automated message from the Apache Git Serv

[GitHub] [arrow] kiszk opened a new pull request #7943: ARROW-9704: [Java] Tests of TestLargeListVector correctly read offset

kiszk opened a new pull request #7943: URL: https://github.com/apache/arrow/pull/7943 This PR aims to correctly read offset as 8-byte in tests of `TestLargeListVector` while they currently read offset as only the first 4-byte. The current code works only for a little-endian platform. ---

[GitHub] [arrow] pitrou commented on pull request #7922: [Packaging][Python] Update wheel dependency files

pitrou commented on pull request #7922: URL: https://github.com/apache/arrow/pull/7922#issuecomment-672957058 @github-actions crossbow submit wheel-win-cp36m This is an automated message from the Apache Git Service. To respon

[GitHub] [arrow] kiszk opened a new pull request #7942: ARROW-9704: [Java] TestEndianness.testLittleEndian supports little- and big-endian platforms

kiszk opened a new pull request #7942: URL: https://github.com/apache/arrow/pull/7942 This PR aims to take care of data layout on little and big endian platforms in `TestEndianness.testLittleEndian` This is an automated mess

[GitHub] [arrow] pitrou commented on a change in pull request #7907: ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to partition base_dir

pitrou commented on a change in pull request #7907: URL: https://github.com/apache/arrow/pull/7907#discussion_r469351892 ## File path: python/pyarrow/fs.py ## @@ -117,6 +116,9 @@ def get_type_name(self): protocol = protocol[0] return "fsspec+{0}".format(pr

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #7907: ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to partition base_dir

jorisvandenbossche commented on a change in pull request #7907: URL: https://github.com/apache/arrow/pull/7907#discussion_r469348756 ## File path: python/pyarrow/fs.py ## @@ -117,6 +116,9 @@ def get_type_name(self): protocol = protocol[0] return "fsspec+{0

[GitHub] [arrow] jhorstmann commented on pull request #7854: ARROW-9583: [Rust] Fix offsets in result of arithmetic kernels

jhorstmann commented on pull request #7854: URL: https://github.com/apache/arrow/pull/7854#issuecomment-672940720 Thanks @paddyhoran, I will then look into removing the offsets requirements and into the simd kernels next. I think the `chunked_exact`/`remainder` pattern would work quite wel

[GitHub] [arrow] markfarnan commented on pull request #6731: [WIP] ARROW-8601: [Go][Flight] Added implementation of FlightDataWriter

markfarnan commented on pull request #6731: URL: https://github.com/apache/arrow/pull/6731#issuecomment-672933678 I would be very interested in using this if it moves forward.. This is an automated message from the Apache G

[GitHub] [arrow] markfarnan edited a comment on pull request #6731: [WIP] ARROW-8601: [Go][Flight] Added implementation of FlightDataWriter

markfarnan edited a comment on pull request #6731: URL: https://github.com/apache/arrow/pull/6731#issuecomment-672933678 I would be very interested in using this if it moves forward..(Native Go, and from WASM) This is an

[GitHub] [arrow] pitrou closed pull request #7909: ARROW-9659: [C++] Fix RecordBatchStreamReader when source is CudaBufferReader

pitrou closed pull request #7909: URL: https://github.com/apache/arrow/pull/7909 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] pitrou commented on pull request #7909: ARROW-9659: [C++] Fix RecordBatchStreamReader when source is CudaBufferReader

pitrou commented on pull request #7909: URL: https://github.com/apache/arrow/pull/7909#issuecomment-672932842 Green Travis-CI build: https://travis-ci.org/github/pitrou/arrow/builds/717245892 This is an automated message fro

[GitHub] [arrow] andygrove commented on a change in pull request #7936: ARROW-9679: [Rust] [DataFusion] More efficient creation of final batch from HashAggregateExec

andygrove commented on a change in pull request #7936: URL: https://github.com/apache/arrow/pull/7936#discussion_r469332693 ## File path: rust/datafusion/src/execution/physical_plan/hash_aggregate.rs ## @@ -677,6 +532,152 @@ impl RecordBatchReader for HashAggregateIterator {

[GitHub] [arrow] pitrou commented on a change in pull request #7907: ARROW-9644: [C++][Dataset] Don't apply ignore_prefixes to partition base_dir

pitrou commented on a change in pull request #7907: URL: https://github.com/apache/arrow/pull/7907#discussion_r469330525 ## File path: python/pyarrow/fs.py ## @@ -117,6 +116,9 @@ def get_type_name(self): protocol = protocol[0] return "fsspec+{0}".format(pr

[GitHub] [arrow] andygrove commented on a change in pull request #7936: ARROW-9679: [Rust] [DataFusion] More efficient creation of final batch from HashAggregateExec

andygrove commented on a change in pull request #7936: URL: https://github.com/apache/arrow/pull/7936#discussion_r469309756 ## File path: rust/datafusion/src/execution/physical_plan/hash_aggregate.rs ## @@ -677,6 +532,152 @@ impl RecordBatchReader for HashAggregateIterator {

[GitHub] [arrow] andygrove commented on pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

andygrove commented on pull request #7916: URL: https://github.com/apache/arrow/pull/7916#issuecomment-672910543 @wqc200 I think this is a good change to make. Please see my comments though. This is an automated message from

[GitHub] [arrow] andygrove commented on a change in pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

andygrove commented on a change in pull request #7916: URL: https://github.com/apache/arrow/pull/7916#discussion_r469307058 ## File path: rust/datafusion/src/sql/parser.rs ## @@ -77,20 +77,20 @@ pub struct DFParser { impl DFParser { /// Parse the specified tokens -p

[GitHub] [arrow] andygrove commented on a change in pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

andygrove commented on a change in pull request #7916: URL: https://github.com/apache/arrow/pull/7916#discussion_r469306474 ## File path: rust/datafusion/src/sql/parser.rs ## @@ -77,20 +77,20 @@ pub struct DFParser { impl DFParser { /// Parse the specified tokens -p

[GitHub] [arrow] andygrove commented on a change in pull request #7916: ARROW-9673: [Rust] [DataFusion] Add a param "dialect" for DFParser::parse_sql

andygrove commented on a change in pull request #7916: URL: https://github.com/apache/arrow/pull/7916#discussion_r469305811 ## File path: rust/datafusion/src/sql/planner.rs ## @@ -815,7 +816,8 @@ mod tests { fn logical_plan(sql: &str) -> Result { let planner = S

[GitHub] [arrow] mrkn commented on a change in pull request #7898: ARROW-9642: [C++] Let MakeBuilder refer DictionaryType's index_type for deciding the starting bit width of the indices

mrkn commented on a change in pull request #7898: URL: https://github.com/apache/arrow/pull/7898#discussion_r469298987 ## File path: cpp/src/arrow/array/builder_adaptive.h ## @@ -122,9 +122,10 @@ class ARROW_EXPORT AdaptiveUIntBuilder : public internal::AdaptiveIntBuilderBase

[GitHub] [arrow] mrkn commented on a change in pull request #7898: ARROW-9642: [C++] Let MakeBuilder refer DictionaryType's index_type for deciding the starting bit width of the indices

mrkn commented on a change in pull request #7898: URL: https://github.com/apache/arrow/pull/7898#discussion_r469298490 ## File path: cpp/src/arrow/array/builder_dict.h ## @@ -409,6 +409,10 @@ class DictionaryBuilder : public internal::DictionaryBuilderBase; using BASE::BASE

  1   2   >