[GitHub] [arrow-datafusion] Igosuki commented on issue #985: CI test failing on master due to clippy

2021-09-09 Thread GitBox
Igosuki commented on issue #985: URL: https://github.com/apache/arrow-datafusion/issues/985#issuecomment-916662971 @alamb Rebased the #910 on top of this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] roee88 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
roee88 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705915890 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] roee88 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
roee88 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705915890 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] roee88 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
roee88 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705915890 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] zhztheplayer commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
zhztheplayer commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705890425 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [arrow] zhztheplayer commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
zhztheplayer commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705905547 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [arrow] edponce commented on a change in pull request #11080: ARROW-13390: [C++] Implement coalesce for remaining types

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11080: URL: https://github.com/apache/arrow/pull/11080#discussion_r705892971 ## File path: cpp/src/arrow/compute/kernels/scalar_if_else.cc ## @@ -1988,6 +1988,65 @@ Status ExecBinaryCoalesce(KernelContext* ctx, Datum left, Datum r

[GitHub] [arrow] edponce commented on a change in pull request #11080: ARROW-13390: [C++] Implement coalesce for remaining types

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11080: URL: https://github.com/apache/arrow/pull/11080#discussion_r705892680 ## File path: cpp/src/arrow/compute/kernels/scalar_if_else.cc ## @@ -1988,6 +1988,65 @@ Status ExecBinaryCoalesce(KernelContext* ctx, Datum left, Datum r

[GitHub] [arrow] zhztheplayer commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
zhztheplayer commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705890425 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [arrow] edponce commented on a change in pull request #11123: ARROW-13655: [C++][Parquet] Disable Thrift message size protections

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11123: URL: https://github.com/apache/arrow/pull/11123#discussion_r705886136 ## File path: cpp/src/parquet/thrift_internal.h ## @@ -363,12 +364,26 @@ static inline format::EncryptionAlgorithm ToThrift(EncryptionAlgorithm encryptio

[GitHub] [arrow] roee88 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
roee88 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705882401 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] roee88 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
roee88 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705882401 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705878505 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705878505 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705876241 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705875814 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705872442 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705872442 ## File path: cpp/src/arrow/compute/kernels/vector_nested.cc ## @@ -76,6 +76,24 @@ struct ListParentIndicesArray { Status Visit(const LargeListType&

[GitHub] [arrow] edponce commented on a change in pull request #11127: ARROW-13877: [C++] Support FixedSizeList in generic list kernels

2021-09-09 Thread GitBox
edponce commented on a change in pull request #11127: URL: https://github.com/apache/arrow/pull/11127#discussion_r705858924 ## File path: cpp/src/arrow/compute/kernels/scalar_nested.cc ## @@ -55,6 +55,23 @@ Status ListValueLength(KernelContext* ctx, const ExecBatch& batch, Dat

[GitHub] [arrow] liyafan82 commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
liyafan82 commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705858673 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [arrow] aocsa commented on pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on pull request #11019: URL: https://github.com/apache/arrow/pull/11019#issuecomment-916583022 Thanks again for the comments @lidavidm. Besides the minor latest changes addressing the feedback I filed follow-up JIRA issues: [C++][Compute] Improve top_k/bottom_k Selec

[GitHub] [arrow] edponce commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-09-09 Thread GitBox
edponce commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r705850030 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -852,24 +853,243 @@ struct LogbChecked { } }; +struct RoundUtil { + template

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705848345 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1799,736 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705848282 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1784,711 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705847646 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1784,711 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] zhztheplayer commented on a change in pull request #11067: ARROW-12965: [Java] C Data Interface implementation

2021-09-09 Thread GitBox
zhztheplayer commented on a change in pull request #11067: URL: https://github.com/apache/arrow/pull/11067#discussion_r705846900 ## File path: java/ffi/src/main/java/org/apache/arrow/ffi/FFIReferenceManager.java ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [arrow] shanhuuang commented on a change in pull request #10978: PARQUET-492: [C++][Parquet] Basic support for reading DELTA_BYTE_ARRAY data.

2021-09-09 Thread GitBox
shanhuuang commented on a change in pull request #10978: URL: https://github.com/apache/arrow/pull/10978#discussion_r705843544 ## File path: cpp/src/parquet/encoding.cc ## @@ -2281,46 +2346,130 @@ class DeltaByteArrayDecoder : public DecoderImpl, : DecoderImpl(descr, Enc

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705840347 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1799,736 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705840347 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1799,736 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705838641 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -1778,6 +1784,711 @@ class SortIndicesMetaFunction : public MetaFunction { } }; +// -

[GitHub] [arrow] aocsa commented on a change in pull request #11019: ARROW-1565: [C++] Implement TopK/BottomK

2021-09-09 Thread GitBox
aocsa commented on a change in pull request #11019: URL: https://github.com/apache/arrow/pull/11019#discussion_r705837507 ## File path: cpp/src/arrow/compute/kernels/select_k_test.cc ## @@ -0,0 +1,737 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow-datafusion] b41sh commented on pull request #870: Add support for PostgreSQL regex match

2021-09-09 Thread GitBox
b41sh commented on pull request #870: URL: https://github.com/apache/arrow-datafusion/pull/870#issuecomment-916563374 > The python test failures have been resolved. One more rebase and I bet this one can finally be merged in. Super kudos for keeping with it @b41sh -- thank you I hav

[GitHub] [arrow-cookbook] drabastomek commented on a change in pull request #70: Adding anonymous flag to s3

2021-09-09 Thread GitBox
drabastomek commented on a change in pull request #70: URL: https://github.com/apache/arrow-cookbook/pull/70#discussion_r705807569 ## File path: python/source/io.rst ## @@ -419,7 +422,8 @@ by ``month`` using .. testcode:: -dataset = ds.dataset("s3://ursa-labs-taxi-data

[GitHub] [arrow] edponce commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-09-09 Thread GitBox
edponce commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r705805393 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -852,24 +853,243 @@ struct LogbChecked { } }; +struct RoundUtil { + template

[GitHub] [arrow] kou commented on pull request #10913: ARROW-13607: [C++] Add Skyhook to Arrow

2021-09-09 Thread GitBox
kou commented on pull request #10913: URL: https://github.com/apache/arrow/pull/10913#issuecomment-916525712 @github-actions rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [arrow] kou closed pull request #11112: ARROW-13942: [Dev] Update cmake_format usage in autotune comment bot

2021-09-09 Thread GitBox
kou closed pull request #2: URL: https://github.com/apache/arrow/pull/2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...

[GitHub] [arrow] kou commented on pull request #11112: ARROW-13942: [Dev] Update cmake_format usage in autotune comment bot

2021-09-09 Thread GitBox
kou commented on pull request #2: URL: https://github.com/apache/arrow/pull/2#issuecomment-916523336 Thanks for the review! I'll merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] augustoasilva commented on a change in pull request #11053: ARROW-13829: [C++][Gandiva] Implement GREATEST and LEAST Hive functions on Gandiva

2021-09-09 Thread GitBox
augustoasilva commented on a change in pull request #11053: URL: https://github.com/apache/arrow/pull/11053#discussion_r705802067 ## File path: cpp/src/gandiva/precompiled/arithmetic_ops_test.cc ## @@ -101,6 +101,34 @@ TEST(TestArithmeticOps, TestDiv) { context.Reset(); }

[GitHub] [arrow] nealrichardson commented on a change in pull request #11108: ARROW-13778: [R] Handle complex summarize expressions

2021-09-09 Thread GitBox
nealrichardson commented on a change in pull request #11108: URL: https://github.com/apache/arrow/pull/11108#discussion_r705801480 ## File path: r/R/dplyr-summarize.R ## @@ -81,3 +118,109 @@ summarize_projection <- function(.data) { format_aggregation <- function(x) { paste

[GitHub] [arrow] nealrichardson commented on a change in pull request #11108: ARROW-13778: [R] Handle complex summarize expressions

2021-09-09 Thread GitBox
nealrichardson commented on a change in pull request #11108: URL: https://github.com/apache/arrow/pull/11108#discussion_r705801196 ## File path: r/R/dplyr-summarize.R ## @@ -81,3 +118,109 @@ summarize_projection <- function(.data) { format_aggregation <- function(x) { paste

[GitHub] [arrow] nealrichardson commented on a change in pull request #11108: ARROW-13778: [R] Handle complex summarize expressions

2021-09-09 Thread GitBox
nealrichardson commented on a change in pull request #11108: URL: https://github.com/apache/arrow/pull/11108#discussion_r70583 ## File path: r/R/dplyr-summarize.R ## @@ -81,3 +118,109 @@ summarize_projection <- function(.data) { format_aggregation <- function(x) { paste

[GitHub] [arrow] augustoasilva commented on a change in pull request #11052: ARROW-13828: [C++][Gandiva] Implement SOUNDEX Hive functions on Gandiva

2021-09-09 Thread GitBox
augustoasilva commented on a change in pull request #11052: URL: https://github.com/apache/arrow/pull/11052#discussion_r705780612 ## File path: cpp/src/gandiva/gdv_function_stubs.cc ## @@ -606,6 +606,53 @@ const char* gdv_fn_upper_utf8(int64_t context, const char* data, int32_

[GitHub] [arrow] augustoasilva commented on a change in pull request #11052: ARROW-13828: [C++][Gandiva] Implement SOUNDEX Hive functions on Gandiva

2021-09-09 Thread GitBox
augustoasilva commented on a change in pull request #11052: URL: https://github.com/apache/arrow/pull/11052#discussion_r705780316 ## File path: cpp/src/gandiva/gdv_function_stubs.cc ## @@ -606,6 +606,53 @@ const char* gdv_fn_upper_utf8(int64_t context, const char* data, int32_

[GitHub] [arrow-cookbook] westonpace commented on a change in pull request #70: Adding anonymous flag to s3

2021-09-09 Thread GitBox
westonpace commented on a change in pull request #70: URL: https://github.com/apache/arrow-cookbook/pull/70#discussion_r705753936 ## File path: python/source/io.rst ## @@ -419,7 +422,8 @@ by ``month`` using .. testcode:: -dataset = ds.dataset("s3://ursa-labs-taxi-data/

[GitHub] [arrow-cookbook] drabastomek opened a new pull request #70: Adding anonymous flag to s3

2021-09-09 Thread GitBox
drabastomek opened a new pull request #70: URL: https://github.com/apache/arrow-cookbook/pull/70 Without the `anonymous=True` flag I was not able to read data from the `ursa-labs-taxi-data` S3 bucket. I updated the S3 section so this should work every time. -- This is an automated messa

[GitHub] [arrow] edponce commented on a change in pull request #10349: ARROW-12744: [C++][Compute] Add rounding kernel

2021-09-09 Thread GitBox
edponce commented on a change in pull request #10349: URL: https://github.com/apache/arrow/pull/10349#discussion_r705731494 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -852,24 +853,243 @@ struct LogbChecked { } }; +struct RoundUtil { + template

[GitHub] [arrow-rs] alamb commented on pull request #491: Minimal MapArray support

2021-09-09 Thread GitBox
alamb commented on pull request #491: URL: https://github.com/apache/arrow-rs/pull/491#issuecomment-916463758 As @houqp mentions, I think this one can not be included into 5.x without breaking compatibility. Specifically, the introduction of `DataType::Map` will cause existing code

[GitHub] [arrow-datafusion] alamb commented on a change in pull request #984: Update DataFusion to arrow 6.0 (WIP)

2021-09-09 Thread GitBox
alamb commented on a change in pull request #984: URL: https://github.com/apache/arrow-datafusion/pull/984#discussion_r705580786 ## File path: ballista/rust/core/src/serde/logical_plan/to_proto.rs ## @@ -407,6 +407,9 @@ impl From<&DataType> for protobuf::arrow_type::ArrowTypeE

[GitHub] [arrow] ursabot edited a comment on pull request #11118: ARROW-13940: [R] Turn on multithreading with Arrow engine queries

2021-09-09 Thread GitBox
ursabot edited a comment on pull request #8: URL: https://github.com/apache/arrow/pull/8#issuecomment-916112151 Benchmark runs are scheduled for baseline = 4b5ed4eb5583cf24d8daff05a865c8d1cb616576 and contender = a007be32fe21fc680068bdc5021317090a6cebc2. Results will be available a

[GitHub] [arrow-datafusion] alamb commented on pull request #870: Add support for PostgreSQL regex match

2021-09-09 Thread GitBox
alamb commented on pull request #870: URL: https://github.com/apache/arrow-datafusion/pull/870#issuecomment-916461902 The python test failures have been resolved. One more rebase and I bet this one can finally be merged in. Super kudos for keeping with it @b41sh -- thank you -- This is

[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #761: Cherry pick Optimize array::transform::utils::set_bits to active_release

2021-09-09 Thread GitBox
codecov-commenter edited a comment on pull request #761: URL: https://github.com/apache/arrow-rs/pull/761#issuecomment-916461374 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/761?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_ter

[GitHub] [arrow-rs] matthewmturner commented on issue #208: flight_data_from_arrow_batch sends too much data

2021-09-09 Thread GitBox
matthewmturner commented on issue #208: URL: https://github.com/apache/arrow-rs/issues/208#issuecomment-916461549 Thank you @alamb , very helpful. I will review and let you know if any questions. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow-rs] codecov-commenter commented on pull request #761: Cherry pick Optimize array::transform::utils::set_bits to active_release

2021-09-09 Thread GitBox
codecov-commenter commented on pull request #761: URL: https://github.com/apache/arrow-rs/pull/761#issuecomment-916461374 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/761?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+A

[GitHub] [arrow-rs] alamb edited a comment on issue #746: Examples (or guidance) on parquet usage

2021-09-09 Thread GitBox
alamb edited a comment on issue #746: URL: https://github.com/apache/arrow-rs/issues/746#issuecomment-916460502 As mentioned above, the challenge for log writing is that it is often requires "schema evolution" -- aka new fields may appear in subsequent messages. This requires some non tri

[GitHub] [arrow-rs] alamb commented on issue #746: Examples (or guidance) on parquet usage

2021-09-09 Thread GitBox
alamb commented on issue #746: URL: https://github.com/apache/arrow-rs/issues/746#issuecomment-916460502 As mentioned above, the challenge for log writing is that it is often requires "schema evolution" -- aka new fields may appear in subsequent messages. This requires some non trivial en

[GitHub] [arrow-rs] alamb merged pull request #749: Update Bitmap::len to return bits rather than bytes

2021-09-09 Thread GitBox
alamb merged pull request #749: URL: https://github.com/apache/arrow-rs/pull/749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb closed issue #730: `Bitmap::len` returns the number of bytes, not bits.

2021-09-09 Thread GitBox
alamb closed issue #730: URL: https://github.com/apache/arrow-rs/issues/730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow-rs] alamb opened a new pull request #761: Cherry pick Optimize array::transform::utils::set_bits to active_release

2021-09-09 Thread GitBox
alamb opened a new pull request #761: URL: https://github.com/apache/arrow-rs/pull/761 Automatic cherry-pick of 4221099 * Originally appeared in https://github.com/apache/arrow-rs/pull/716: Optimize array::transform::utils::set_bits -- This is an automated message from the Apache

[GitHub] [arrow-rs] alamb commented on pull request #716: Optimize array::transform::utils::set_bits

2021-09-09 Thread GitBox
alamb commented on pull request #716: URL: https://github.com/apache/arrow-rs/pull/716#issuecomment-916451919 > @alamb anything that is left to do here in your opinion? Nope -- thanks @mathiaspeters-sig I am happy with @nevi-me 's review. I am sorry for the delay in merging I ha

[GitHub] [arrow-rs] alamb merged pull request #716: Optimize array::transform::utils::set_bits

2021-09-09 Thread GitBox
alamb merged pull request #716: URL: https://github.com/apache/arrow-rs/pull/716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb closed issue #397: Optimize MutableArrayData::extend for null buffers

2021-09-09 Thread GitBox
alamb closed issue #397: URL: https://github.com/apache/arrow-rs/issues/397 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow-rs] alamb commented on issue #500: `minute` and `second` temporal kernels do not respect timezone

2021-09-09 Thread GitBox
alamb commented on issue #500: URL: https://github.com/apache/arrow-rs/issues/500#issuecomment-916449815 @sum12 this also sounds reasonable to me. As you have probably seen the spec for what can be in the timezone string is here: https://github.com/apache/arrow/blob/master/format/S

[GitHub] [arrow] github-actions[bot] commented on pull request #11128: ARROW-13967: [Go] Implement Concatenate function for array.Interface

2021-09-09 Thread GitBox
github-actions[bot] commented on pull request #11128: URL: https://github.com/apache/arrow/pull/11128#issuecomment-916446286 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] zeroshade opened a new pull request #11128: ARROW-13967: [Go] Implement Concatenate function for array.Interface

2021-09-09 Thread GitBox
zeroshade opened a new pull request #11128: URL: https://github.com/apache/arrow/pull/11128 This is needed for adding nested types to the `MakeArrayFromScalar` function in ARROW-13789 (#11024) @emkornfield @sbinet @fsaintjacques -- This is an automated message from the Apache Git

[GitHub] [arrow-rs] alamb commented on pull request #750: Added PartialEq to RecordBatch

2021-09-09 Thread GitBox
alamb commented on pull request #750: URL: https://github.com/apache/arrow-rs/pull/750#issuecomment-916443472 @matthewmturner -- I think trying to add `#[derive(PartialEq)]` on `RecordBatch` is probably a good place to start -- it would compare two `RecordBatch`es field by field which see

[GitHub] [arrow-rs] alamb merged pull request #757: Cherry pick 0e7c4c5f: make chrono time emit converted type

2021-09-09 Thread GitBox
alamb merged pull request #757: URL: https://github.com/apache/arrow-rs/pull/757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb opened a new pull request #760: Manual Cherrypick: fix: new_null_array for structs (#736)

2021-09-09 Thread GitBox
alamb opened a new pull request #760: URL: https://github.com/apache/arrow-rs/pull/760 Manual cherry pick of 5af340c556c0d765113b5b4683f369790e612ede which originally appeared in https://github.com/apache/arrow-rs/pull/736 from @bjchambers -- This is an automated message from the Apach

[GitHub] [arrow] jonkeane commented on a change in pull request #11108: ARROW-13778: [R] Handle complex summarize expressions

2021-09-09 Thread GitBox
jonkeane commented on a change in pull request #11108: URL: https://github.com/apache/arrow/pull/11108#discussion_r705692706 ## File path: r/R/dplyr-summarize.R ## @@ -42,33 +49,63 @@ summarise.arrow_dplyr_query <- function(.data, ..., .engine = c("arrow", "duckdb } summaris

[GitHub] [arrow-rs] alamb opened a new pull request #759: Cherry pick Add `append_nulls` and `append_trusted_len_iter` to `PrimitiveBuilder` to active_release

2021-09-09 Thread GitBox
alamb opened a new pull request #759: URL: https://github.com/apache/arrow-rs/pull/759 Automatic cherry-pick of 7675164 * Originally appeared in https://github.com/apache/arrow-rs/pull/728: Add `append_nulls` and `append_trusted_len_iter` to `PrimitiveBuilder` -- This is an automa

[GitHub] [arrow-rs] alamb closed issue #734: `new_null_array` creates invalid struct arrays

2021-09-09 Thread GitBox
alamb closed issue #734: URL: https://github.com/apache/arrow-rs/issues/734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow-rs] alamb commented on issue #734: `new_null_array` creates invalid struct arrays

2021-09-09 Thread GitBox
alamb commented on issue #734: URL: https://github.com/apache/arrow-rs/issues/734#issuecomment-916437589 Closed in https://github.com/apache/arrow-rs/pull/736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [arrow-rs] alamb opened a new pull request #758: Cherry pick fix: Scalar math operations on slices to active_release

2021-09-09 Thread GitBox
alamb opened a new pull request #758: URL: https://github.com/apache/arrow-rs/pull/758 Automatic cherry-pick of 72d7c5b * Originally appeared in https://github.com/apache/arrow-rs/pull/743: fix: Scalar math operations on slices -- This is an automated message from the Apache Git S

[GitHub] [arrow-rs] alamb commented on pull request #731: fix: Allow parquet to be compiled without arrow (fix --no-default-features)

2021-09-09 Thread GitBox
alamb commented on pull request #731: URL: https://github.com/apache/arrow-rs/pull/731#issuecomment-916436847 I don't plan to backport this to arrow 5.4 (it will be included in 6.0) so as to avoid causing downstream projects to break -- This is an automated message from the Apache Git Se

[GitHub] [arrow-rs] matthewmturner commented on pull request #750: Added PartialEq to RecordBatch

2021-09-09 Thread GitBox
matthewmturner commented on pull request #750: URL: https://github.com/apache/arrow-rs/pull/750#issuecomment-916432884 Thx both - I will add check for column names. @jorgecarleitao regarding adding the `PartialEq` to `#derive()` - i had looked into this but saw the following in the d

[GitHub] [arrow-datafusion] alamb merged pull request #986: Rename NthValue::{first_value,last_value,nth_value} to satisfy clippy in Rust 1.55

2021-09-09 Thread GitBox
alamb merged pull request #986: URL: https://github.com/apache/arrow-datafusion/pull/986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-un

[GitHub] [arrow-datafusion] alamb closed issue #985: CI test failing on master due to clippy

2021-09-09 Thread GitBox
alamb closed issue #985: URL: https://github.com/apache/arrow-datafusion/issues/985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubsc

[GitHub] [arrow-datafusion] alamb commented on pull request #986: Rename NthValue::{first_value,last_value,nth_value} to satisfy clippy in Rust 1.55

2021-09-09 Thread GitBox
alamb commented on pull request #986: URL: https://github.com/apache/arrow-datafusion/pull/986#issuecomment-916430726 I am going to merge this in to get CI green on master again; The actual change has already been discussed in the context of https://github.com/apache/arrow-datafusion/pull/

[GitHub] [arrow-datafusion] alamb commented on pull request #910: Avro Table Provider

2021-09-09 Thread GitBox
alamb commented on pull request #910: URL: https://github.com/apache/arrow-datafusion/pull/910#issuecomment-916430044 Thanks @Igosuki - I also just got back. I think this PR is close. I plan to review it again carefully tomorrow. -- This is an automated message from the Apache Git Serv

[GitHub] [arrow-datafusion] alamb commented on pull request #965: Move CBOs to physical plan

2021-09-09 Thread GitBox
alamb commented on pull request #965: URL: https://github.com/apache/arrow-datafusion/pull/965#issuecomment-916428018 FWIW I think running a cost based optimizer on the physical plan is a good idea. I plan to review this PR carefully tomorrow -- This is an automated message from the Apac

[GitHub] [arrow-rs] codecov-commenter edited a comment on pull request #757: Cherry pick 0e7c4c5f: make chrono time emit converted type

2021-09-09 Thread GitBox
codecov-commenter edited a comment on pull request #757: URL: https://github.com/apache/arrow-rs/pull/757#issuecomment-916401553 # [Codecov](https://codecov.io/gh/apache/arrow-rs/pull/757?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_ter

[GitHub] [arrow-rs] alamb commented on pull request #750: Added PartialEq to RecordBatch

2021-09-09 Thread GitBox
alamb commented on pull request #750: URL: https://github.com/apache/arrow-rs/pull/750#issuecomment-916423940 Checking for different column names is an excellent point @jorgecarleitao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow-rs] alamb commented on issue #208: flight_data_from_arrow_batch sends too much data

2021-09-09 Thread GitBox
alamb commented on issue #208: URL: https://github.com/apache/arrow-rs/issues/208#issuecomment-916422638 Hi @matthewmturner -- thanks! Some additional context is that we encountered this issue while working on IOx -- see details https://github.com/influxdata/influxdb_iox/issues/113

[GitHub] [arrow-rs] jorgecarleitao commented on pull request #750: Added PartialEq to RecordBatch

2021-09-09 Thread GitBox
jorgecarleitao commented on pull request #750: URL: https://github.com/apache/arrow-rs/pull/750#issuecomment-916420840 Can't we add `PartialEq` to `#derive()`? It seems to me that this PR ignores differences in the schema (e.g. column names) -- This is an automated message from the Apach

[GitHub] [arrow] kou commented on pull request #11112: ARROW-13942: [Dev] Update cmake_format usage in autotune comment bot

2021-09-09 Thread GitBox
kou commented on pull request #2: URL: https://github.com/apache/arrow/pull/2#issuecomment-916420458 @kszucs Do you want to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [arrow-rs] jorgecarleitao commented on pull request #749: Update Bitmap::len to return bits

2021-09-09 Thread GitBox
jorgecarleitao commented on pull request #749: URL: https://github.com/apache/arrow-rs/pull/749#issuecomment-916419977 I would not consider this backward incompatibility but a bug fix: imo `Bitmap::len` should be in bits, not bytes. We actually found this while assuming that the len was in

[GitHub] [arrow-rs] alamb commented on pull request #757: Cherry pick 0e7c4c5f: make chrono time emit converted type

2021-09-09 Thread GitBox
alamb commented on pull request #757: URL: https://github.com/apache/arrow-rs/pull/757#issuecomment-916415003 @xrl FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [arrow-rs] alamb merged pull request #756: Cherry pick fix: Comparisons against scalar slices to active_release

2021-09-09 Thread GitBox
alamb merged pull request #756: URL: https://github.com/apache/arrow-rs/pull/756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb merged pull request #755: Cherry pick fix: Handle slices in unary kernel to active_release

2021-09-09 Thread GitBox
alamb merged pull request #755: URL: https://github.com/apache/arrow-rs/pull/755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb merged pull request #754: Cherry pick 6ee9bc03 - Add a note on rust compiler testing and compatibility #726

2021-09-09 Thread GitBox
alamb merged pull request #754: URL: https://github.com/apache/arrow-rs/pull/754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb merged pull request #753: Cherry pick Fix decimal value_as_string to active_release

2021-09-09 Thread GitBox
alamb merged pull request #753: URL: https://github.com/apache/arrow-rs/pull/753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb merged pull request #752: Cherry pick Fix decimal repr in parquet schema printer to active_release

2021-09-09 Thread GitBox
alamb merged pull request #752: URL: https://github.com/apache/arrow-rs/pull/752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-rs] alamb merged pull request #743: fix: Scalar math operations on slices

2021-09-09 Thread GitBox
alamb merged pull request #743: URL: https://github.com/apache/arrow-rs/pull/743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr..

[GitHub] [arrow-datafusion] mmuru commented on pull request #981: [Python] Add support to perform sql query on in-memory datasource.

2021-09-09 Thread GitBox
mmuru commented on pull request #981: URL: https://github.com/apache/arrow-datafusion/pull/981#issuecomment-916412342 @houqp: I fixed both rebase and black errors. Please, review and provide your feedback. Thanks. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow-rs] alamb closed issue #742: Arithmetic with scalars doesn't work on slices

2021-09-09 Thread GitBox
alamb closed issue #742: URL: https://github.com/apache/arrow-rs/issues/742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow-rs] alamb commented on issue #742: Arithmetic with scalars doesn't work on slices

2021-09-09 Thread GitBox
alamb commented on issue #742: URL: https://github.com/apache/arrow-rs/issues/742#issuecomment-916411225 Wow -- thank you for the train of fixes @bjchambers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [arrow-rs] alamb commented on a change in pull request #745: fix: Support length on slices with null

2021-09-09 Thread GitBox
alamb commented on a change in pull request #745: URL: https://github.com/apache/arrow-rs/pull/745#discussion_r705669376 ## File path: arrow/src/compute/kernels/length.rs ## @@ -243,18 +242,13 @@ mod tests { /// Tests with an offset #[test] fn length_offsets() ->

[GitHub] [arrow] asfgit closed pull request #11126: ARROW-13964: MINOR: [Go][Parquet] remove base bitmap reader/writer from parquet module, use arrow bitutil ones

2021-09-09 Thread GitBox
asfgit closed pull request #11126: URL: https://github.com/apache/arrow/pull/11126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow-rs] alamb commented on a change in pull request #749: Update Bitmap::len to return bits

2021-09-09 Thread GitBox
alamb commented on a change in pull request #749: URL: https://github.com/apache/arrow-rs/pull/749#discussion_r705663275 ## File path: arrow/src/bitmap.rs ## @@ -45,7 +45,7 @@ impl Bitmap { } pub fn len(&self) -> usize { Review comment: Can we also add a doc

[GitHub] [arrow-rs] matthewmturner commented on a change in pull request #750: Added PartialEq to RecordBatch

2021-09-09 Thread GitBox
matthewmturner commented on a change in pull request #750: URL: https://github.com/apache/arrow-rs/pull/750#discussion_r705662484 ## File path: arrow/src/record_batch.rs ## @@ -419,6 +419,18 @@ impl From<&StructArray> for RecordBatch { } } +impl PartialEq for RecordBatc

[GitHub] [arrow] nealrichardson closed pull request #11118: ARROW-13940: [R] Turn on multithreading with Arrow engine queries

2021-09-09 Thread GitBox
nealrichardson closed pull request #8: URL: https://github.com/apache/arrow/pull/8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

  1   2   3   4   >