[GitHub] [arrow] cyb70289 commented on issue #9735: [Golang] Create ipc format for ipc Reader

2021-03-17 Thread GitBox
cyb70289 commented on issue #9735: URL: https://github.com/apache/arrow/issues/9735#issuecomment-801634282 > ` subscribe first to send an e-mail` > Sorry, because the first time use github. Please guide me know-how ! @hunght3101, mail list is the preferred way for issues

[GitHub] [arrow] projjal commented on a change in pull request #9707: ARROW-11984: [C++] Implement SHA128 and SHA256 functions in Gandiva module - WIP

2021-03-17 Thread GitBox
projjal commented on a change in pull request #9707: URL: https://github.com/apache/arrow/pull/9707#discussion_r596529641 ## File path: cpp/src/gandiva/gdv_function_stubs.cc ## @@ -122,6 +123,133 @@ int32_t gdv_fn_populate_varlen_vector(int64_t context_ptr, int8_t* data_ptr,

[GitHub] [arrow] liyafan82 commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
liyafan82 commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-801600759 > If you've already started [ARROW-11899](https://issues.apache.org/jira/browse/ARROW-11899) then I'll let you finish it up, hopefully it isn't too much work. We are discussing

[GitHub] [arrow] github-actions[bot] commented on pull request #9745: ARROW-11703: [R] Implement dplyr::arrange() [WIP]

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9745: URL: https://github.com/apache/arrow/pull/9745#issuecomment-801599144 https://issues.apache.org/jira/browse/ARROW-11703 This is an automated message from the Apache Git

[GitHub] [arrow] ianmcook opened a new pull request #9745: ARROW-11703: [R] Implement dplyr::arrange() [WIP]

2021-03-17 Thread GitBox
ianmcook opened a new pull request #9745: URL: https://github.com/apache/arrow/pull/9745 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] emkornfield commented on a change in pull request #9147: ARROW-11177: [Java] ArrowMessage failed to parse compressed grpc stream

2021-03-17 Thread GitBox
emkornfield commented on a change in pull request #9147: URL: https://github.com/apache/arrow/pull/9147#discussion_r596528147 ## File path: java/flight/flight-core/src/main/java/org/apache/arrow/flight/ArrowMessage.java ## @@ -332,6 +333,23 @@ private static ArrowMessage

[GitHub] [arrow] nealrichardson closed pull request #9741: ARROW-12005: [R] Fix a bash typo in configure

2021-03-17 Thread GitBox
nealrichardson closed pull request #9741: URL: https://github.com/apache/arrow/pull/9741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] emkornfield commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
emkornfield commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-801585216 If you've already started ARROW-11899 then I'll let you finish it up, hopefully it isn't too much work. We are discussing on the ML the path forward for LZ4 in general, once

[GitHub] [arrow] cyb70289 commented on a change in pull request #9728: ARROW-10250: [C++][FlightRPC] Consistently use FlightClientOptions::Defaults

2021-03-17 Thread GitBox
cyb70289 commented on a change in pull request #9728: URL: https://github.com/apache/arrow/pull/9728#discussion_r596522463 ## File path: cpp/src/arrow/flight/client.h ## @@ -94,8 +94,6 @@ class ARROW_FLIGHT_EXPORT FlightWriteSizeStatusDetail : public arrow::StatusDeta

[GitHub] [arrow] liyafan82 commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
liyafan82 commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-801583143 > +1 thank you. @liyafan82 did you have plans to work on the follow-up items or ZSTD? Otherwise I can take them up. > > @HedgehogCode any thoughts on how to procede for

[GitHub] [arrow] github-actions[bot] commented on pull request #9744: ARROW-12012: [Java][JDBC] Fix BinaryConsumer reallocation

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9744: URL: https://github.com/apache/arrow/pull/9744#issuecomment-801571749 https://issues.apache.org/jira/browse/ARROW-12012 This is an automated message from the Apache Git

[GitHub] [arrow] zxf opened a new pull request #9744: ARROW-12012: [Java][JDBC] Fix BinaryConsumer reallocation

2021-03-17 Thread GitBox
zxf opened a new pull request #9744: URL: https://github.com/apache/arrow/pull/9744 [ARROW-12012](https://issues.apache.org/jira/browse/ARROW-12012) An exception will be thrown when BinaryConsumer consumes a large amount or a lot of data.

[GitHub] [arrow] github-actions[bot] commented on pull request #9743: first informative error msg for lz4 error

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9743: URL: https://github.com/apache/arrow/pull/9743#issuecomment-801510996 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then

[GitHub] [arrow] github-actions[bot] commented on pull request #9741: ARROW-12005: [R] Fix a bash typo in configure

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9741: URL: https://github.com/apache/arrow/pull/9741#issuecomment-801499897 Revision: e2c22c680aee40a3eadeb3ec583c560372734171 Submitted crossbow builds: [ursacomputing/crossbow @

[GitHub] [arrow] github-actions[bot] commented on pull request #9741: ARROW-12005: [R] Fix a bash typo in configure

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9741: URL: https://github.com/apache/arrow/pull/9741#issuecomment-801490891 https://issues.apache.org/jira/browse/ARROW-12005 This is an automated message from the Apache Git

[GitHub] [arrow] seddonm1 commented on pull request #9682: ARROW-7364: [Rust] Add cast options to cast kernel [WIP]

2021-03-17 Thread GitBox
seddonm1 commented on pull request #9682: URL: https://github.com/apache/arrow/pull/9682#issuecomment-801488963 FYI https://github.com/ballista-compute/sqlparser-rs/pull/299 has been raised to add TRY_CAST to the parser.

[GitHub] [arrow] seddonm1 commented on pull request #9243: ARROW-11298: [Rust][DataFusion] Implement Postgres String Functions [Splitting to separate PRs]

2021-03-17 Thread GitBox
seddonm1 commented on pull request #9243: URL: https://github.com/apache/arrow/pull/9243#issuecomment-801488843 Closed after splitting. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] seddonm1 closed pull request #9243: ARROW-11298: [Rust][DataFusion] Implement Postgres String Functions [Splitting to separate PRs]

2021-03-17 Thread GitBox
seddonm1 closed pull request #9243: URL: https://github.com/apache/arrow/pull/9243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] bkietz commented on pull request #9621: ARROW-11591: [C++][Compute] Grouped aggregation

2021-03-17 Thread GitBox
bkietz commented on pull request #9621: URL: https://github.com/apache/arrow/pull/9621#issuecomment-801466302 @pitrou PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] bkietz commented on a change in pull request #9621: ARROW-11591: [C++][Compute] Grouped aggregation

2021-03-17 Thread GitBox
bkietz commented on a change in pull request #9621: URL: https://github.com/apache/arrow/pull/9621#discussion_r596407348 ## File path: cpp/src/arrow/compute/kernels/aggregate_basic.cc ## @@ -229,6 +604,710 @@ std::unique_ptr AllInit(KernelContext*, const KernelInitArgs& args)

[GitHub] [arrow] bkietz commented on a change in pull request #9621: ARROW-11591: [C++][Compute] Grouped aggregation

2021-03-17 Thread GitBox
bkietz commented on a change in pull request #9621: URL: https://github.com/apache/arrow/pull/9621#discussion_r596406429 ## File path: cpp/src/arrow/compute/kernels/aggregate_basic.cc ## @@ -229,6 +604,710 @@ std::unique_ptr AllInit(KernelContext*, const KernelInitArgs& args)

[GitHub] [arrow] returnString commented on a change in pull request #9710: ARROW-11969: [Rust][DataFusion] Improve Examples in documentation

2021-03-17 Thread GitBox
returnString commented on a change in pull request #9710: URL: https://github.com/apache/arrow/pull/9710#discussion_r596404956 ## File path: rust/datafusion/README.md ## @@ -58,6 +58,49 @@ Here are some of the projects known to use DataFusion: (if you know of another

[GitHub] [arrow] returnString commented on a change in pull request #9710: ARROW-11969: [Rust][DataFusion] Improve Examples in documentation

2021-03-17 Thread GitBox
returnString commented on a change in pull request #9710: URL: https://github.com/apache/arrow/pull/9710#discussion_r596404956 ## File path: rust/datafusion/README.md ## @@ -58,6 +58,49 @@ Here are some of the projects known to use DataFusion: (if you know of another

[GitHub] [arrow] returnString commented on a change in pull request #9704: ARROW-11964: [Rust][DataFusion] Extend constant folding and parquet filtering support

2021-03-17 Thread GitBox
returnString commented on a change in pull request #9704: URL: https://github.com/apache/arrow/pull/9704#discussion_r596399314 ## File path: rust/datafusion/src/optimizer/constant_folding.rs ## @@ -188,6 +188,97 @@ impl<'a> ExprRewriter for ConstantRewriter<'a> {

[GitHub] [arrow] pachamaltese opened a new pull request #9743: first informative error msg for lz4 error

2021-03-17 Thread GitBox
pachamaltese opened a new pull request #9743: URL: https://github.com/apache/arrow/pull/9743 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] jonkeane commented on pull request #9741: ARROW-12005: [R] Fix a bash typo in configure

2021-03-17 Thread GitBox
jonkeane commented on pull request #9741: URL: https://github.com/apache/arrow/pull/9741#issuecomment-801425608 @github-actions crossbow submit test-r-install-local This is an automated message from the Apache Git Service.

[GitHub] [arrow] github-actions[bot] commented on pull request #9740: ARROW-12003: [R] Fix NOTE re undefined global function group_by_drop_default

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9740: URL: https://github.com/apache/arrow/pull/9740#issuecomment-801401711 https://issues.apache.org/jira/browse/ARROW-12003 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson closed pull request #9740: ARROW-12003: [R] Fix NOTE re undefined global function group_by_drop_default

2021-03-17 Thread GitBox
nealrichardson closed pull request #9740: URL: https://github.com/apache/arrow/pull/9740 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] pitrou commented on pull request #9742: [WIP] ARROW-11928: [C++] Execution engine API

2021-03-17 Thread GitBox
pitrou commented on pull request #9742: URL: https://github.com/apache/arrow/pull/9742#issuecomment-801355489 @bkietz @wesm Here is an initial stab at the exec node API. Only base classes are present. This is an automated

[GitHub] [arrow] pitrou opened a new pull request #9742: [WIP] ARROW-11928: [C++] Execution engine API

2021-03-17 Thread GitBox
pitrou opened a new pull request #9742: URL: https://github.com/apache/arrow/pull/9742 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] jonkeane opened a new pull request #9741: ARROW-12005: [R] Fix a bash typo in configure

2021-03-17 Thread GitBox
jonkeane opened a new pull request #9741: URL: https://github.com/apache/arrow/pull/9741 Without the quotes, we get ` parse error: condition expected: !=` This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] emkornfield closed pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
emkornfield closed pull request #8949: URL: https://github.com/apache/arrow/pull/8949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] emkornfield commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
emkornfield commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-801332194 +1 thank you. @liyafan82 did you have plans to work on the follow-up items or ZSTD? Otherwise I can take them up. @hedgehogcode any thoughts on how to procede for LZ4?

[GitHub] [arrow] emkornfield closed pull request #9421: ARROW-11066: [FlightRPC][Java] Make zero-copy writes a configurable option

2021-03-17 Thread GitBox
emkornfield closed pull request #9421: URL: https://github.com/apache/arrow/pull/9421 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] emkornfield commented on pull request #9421: ARROW-11066: [FlightRPC][Java] Make zero-copy writes a configurable option

2021-03-17 Thread GitBox
emkornfield commented on pull request #9421: URL: https://github.com/apache/arrow/pull/9421#issuecomment-801327752 +1 merging. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [arrow] emkornfield commented on a change in pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
emkornfield commented on a change in pull request #9671: URL: https://github.com/apache/arrow/pull/9671#discussion_r596278550 ## File path: go/parquet/LICENSE.txt ## @@ -0,0 +1,1987 @@ + Review comment: yeah, looking back it seems like we needed to copy to make this

[GitHub] [arrow] pitrou commented on pull request #9629: ARROW-11838: [C++] Support IPC reads with shared dictionaries.

2021-03-17 Thread GitBox
pitrou commented on pull request #9629: URL: https://github.com/apache/arrow/pull/9629#issuecomment-801280256 You need to update the git submodule as part of this PR. This is an automated message from the Apache Git Service.

[GitHub] [arrow] zeroshade commented on pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
zeroshade commented on pull request #9671: URL: https://github.com/apache/arrow/pull/9671#issuecomment-801275287 rebased branch with master, hopefully that fixes the failing check :) This is an automated message from the

[GitHub] [arrow] sbinet commented on a change in pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
sbinet commented on a change in pull request #9671: URL: https://github.com/apache/arrow/pull/9671#discussion_r596236416 ## File path: go/parquet/internal/bmi/_lib/bitmap_bmi2.s ## @@ -0,0 +1,174 @@ + .text Review comment: ok. fine by me.

[GitHub] [arrow] zeroshade commented on a change in pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
zeroshade commented on a change in pull request #9671: URL: https://github.com/apache/arrow/pull/9671#discussion_r596233932 ## File path: go/parquet/internal/bmi/_lib/bitmap_bmi2.s ## @@ -0,0 +1,174 @@ + .text Review comment: Well, the assembly in the _lib

[GitHub] [arrow] jmgpeeters commented on pull request #9629: ARROW-11838: [C++] Support IPC reads with shared dictionaries.

2021-03-17 Thread GitBox
jmgpeeters commented on pull request #9629: URL: https://github.com/apache/arrow/pull/9629#issuecomment-801261388 Hm, for some reason the github integration test checked out an older version of the arrow-testing data, Run ci/scripts/util_checkout.sh Submodule

[GitHub] [arrow] sbinet commented on a change in pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
sbinet commented on a change in pull request #9671: URL: https://github.com/apache/arrow/pull/9671#discussion_r596219612 ## File path: go/parquet/internal/bmi/_lib/bitmap_bmi2.s ## @@ -0,0 +1,174 @@ + .text Review comment: well, removing those would require

[GitHub] [arrow] sbinet commented on a change in pull request #9671: ARROW-7905: [Go][Parquet] Initial Chunk of Parquet port to Go

2021-03-17 Thread GitBox
sbinet commented on a change in pull request #9671: URL: https://github.com/apache/arrow/pull/9671#discussion_r596217949 ## File path: go/parquet/internal/bmi/_lib/bitmap_bmi2.c ## @@ -0,0 +1,38 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more

[GitHub] [arrow] mm0708 commented on issue #8099: Specifying schema when using write_feather?

2021-03-17 Thread GitBox
mm0708 commented on issue #8099: URL: https://github.com/apache/arrow/issues/8099#issuecomment-801249604 I dropped the column causing issues because it wasn't essential This is an automated message from the Apache Git

[GitHub] [arrow] kdkavanagh commented on issue #8099: Specifying schema when using write_feather?

2021-03-17 Thread GitBox
kdkavanagh commented on issue #8099: URL: https://github.com/apache/arrow/issues/8099#issuecomment-801240233 @mm0708 what did you conclude here? I have this same question/problem This is an automated message from the Apache

[GitHub] [arrow] pitrou closed pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou closed pull request #9738: URL: https://github.com/apache/arrow/pull/9738 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou commented on pull request #9738: URL: https://github.com/apache/arrow/pull/9738#issuecomment-801225326 @dianaclarke You may want to enable Travis-CI on your Arrow fork for faster CI results. (in this case this doesn't matter, since Travis-CI doesn't run Python tests)

[GitHub] [arrow] pitrou commented on pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou commented on pull request #9738: URL: https://github.com/apache/arrow/pull/9738#issuecomment-801224615 Green CI run at https://github.com/dianaclarke/arrow/runs/2131383253 This is an automated message from the Apache

[GitHub] [arrow] ianmcook opened a new pull request #9740: ARROW-12003: [R] Fix NOTE re undefined global function group_by_drop_default

2021-03-17 Thread GitBox
ianmcook opened a new pull request #9740: URL: https://github.com/apache/arrow/pull/9740 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] github-actions[bot] commented on pull request #9739: ARROW-12000: [Documentation] Add note about deviation from style guide on struct/classes

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9739: URL: https://github.com/apache/arrow/pull/9739#issuecomment-801195182 https://issues.apache.org/jira/browse/ARROW-12000 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson closed pull request #9733: ARROW-11996: [R] Make r/configure run successfully on Solaris

2021-03-17 Thread GitBox
nealrichardson closed pull request #9733: URL: https://github.com/apache/arrow/pull/9733 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [arrow] lidavidm commented on a change in pull request #9725: ARROW-8631: [C++][Python][Dataset] Add ReadOptions to CsvFileFormat, expose options to python

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9725: URL: https://github.com/apache/arrow/pull/9725#discussion_r596093793 ## File path: cpp/src/arrow/dataset/file_csv.h ## @@ -37,6 +37,11 @@ class ARROW_DS_EXPORT CsvFileFormat : public FileFormat { public: /// Options

[GitHub] [arrow] bkietz commented on a change in pull request #9725: ARROW-8631: [C++][Python][Dataset] Add ReadOptions to CsvFileFormat, expose options to python

2021-03-17 Thread GitBox
bkietz commented on a change in pull request #9725: URL: https://github.com/apache/arrow/pull/9725#discussion_r596084650 ## File path: cpp/src/arrow/dataset/dataset_internal.h ## @@ -185,5 +185,14 @@ inline bool operator==(const SubtreeImpl::Encoded& l, const

[GitHub] [arrow] bkietz commented on a change in pull request #9725: ARROW-8631: [C++][Python][Dataset] Add ReadOptions to CsvFileFormat, expose options to python

2021-03-17 Thread GitBox
bkietz commented on a change in pull request #9725: URL: https://github.com/apache/arrow/pull/9725#discussion_r596092120 ## File path: cpp/src/arrow/dataset/file_csv.h ## @@ -37,6 +37,11 @@ class ARROW_DS_EXPORT CsvFileFormat : public FileFormat { public: /// Options

[GitHub] [arrow] lidavidm commented on a change in pull request #9715: ARROW-11745: [C++] WIP: Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596088841 ## File path: cpp/src/arrow/testing/random.cc ## @@ -558,5 +584,248 @@ std::shared_ptr RandomArrayGenerator::ArrayOf(std::shared_ptr t return

[GitHub] [arrow] lidavidm commented on a change in pull request #9715: ARROW-11745: [C++] Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596087724 ## File path: cpp/src/arrow/testing/random.cc ## @@ -558,5 +584,248 @@ std::shared_ptr RandomArrayGenerator::ArrayOf(std::shared_ptr t return

[GitHub] [arrow] pitrou closed pull request #8023: ARROW-9318: [C++] Parquet encryption key management

2021-03-17 Thread GitBox
pitrou closed pull request #8023: URL: https://github.com/apache/arrow/pull/8023 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on pull request #8023: ARROW-9318: [C++] Parquet encryption key management

2021-03-17 Thread GitBox
pitrou commented on pull request #8023: URL: https://github.com/apache/arrow/pull/8023#issuecomment-801144014 I'll merge then. Thank you @thamht4190 and @ggershinsky for contributing this! This is an automated message from

[GitHub] [arrow] lidavidm commented on a change in pull request #9715: ARROW-11745: [C++] Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596086831 ## File path: cpp/src/arrow/testing/random.cc ## @@ -558,5 +584,248 @@ std::shared_ptr RandomArrayGenerator::ArrayOf(std::shared_ptr t return

[GitHub] [arrow] lidavidm commented on a change in pull request #9715: ARROW-11745: [C++] Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596085481 ## File path: cpp/src/arrow/testing/random.h ## @@ -358,6 +362,10 @@ class ARROW_TESTING_EXPORT RandomArrayGenerator { std::default_random_engine

[GitHub] [arrow] pitrou commented on a change in pull request #9730: ARROW-9878: [Python] Document caveats of to_pandas(self_destruct=True)

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9730: URL: https://github.com/apache/arrow/pull/9730#discussion_r596085147 ## File path: docs/source/python/pandas.rst ## @@ -293,3 +293,19 @@ Used together, the call will yield significantly lower memory usage in some

[GitHub] [arrow] lidavidm commented on a change in pull request #9715: ARROW-11745: [C++] Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596084830 ## File path: cpp/src/arrow/testing/random.h ## @@ -358,6 +362,10 @@ class ARROW_TESTING_EXPORT RandomArrayGenerator { std::default_random_engine

[GitHub] [arrow] lidavidm commented on a change in pull request #9730: ARROW-9878: [Python] Document caveats of to_pandas(self_destruct=True)

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9730: URL: https://github.com/apache/arrow/pull/9730#discussion_r596083089 ## File path: docs/source/python/pandas.rst ## @@ -293,3 +293,19 @@ Used together, the call will yield significantly lower memory usage in some

[GitHub] [arrow] lidavidm commented on a change in pull request #9730: ARROW-9878: [Python] Document caveats of to_pandas(self_destruct=True)

2021-03-17 Thread GitBox
lidavidm commented on a change in pull request #9730: URL: https://github.com/apache/arrow/pull/9730#discussion_r596081637 ## File path: docs/source/python/pandas.rst ## @@ -293,3 +293,19 @@ Used together, the call will yield significantly lower memory usage in some

[GitHub] [arrow] bkietz commented on a change in pull request #9715: ARROW-11745: [C++] Add helper to generate random record batches by schema

2021-03-17 Thread GitBox
bkietz commented on a change in pull request #9715: URL: https://github.com/apache/arrow/pull/9715#discussion_r596029757 ## File path: cpp/src/arrow/testing/random.h ## @@ -358,6 +362,10 @@ class ARROW_TESTING_EXPORT RandomArrayGenerator { std::default_random_engine

[GitHub] [arrow] pitrou commented on a change in pull request #8468: ARROW-10306: [C++] Add string replacement kernel

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #8468: URL: https://github.com/apache/arrow/pull/8468#discussion_r596070474 ## File path: cpp/src/arrow/compute/kernels/scalar_string.cc ## @@ -1194,6 +1198,197 @@ void AddSplit(FunctionRegistry* registry) { #endif } +//

[GitHub] [arrow] pitrou closed pull request #9720: ARROW-11976: [C++] Fix sporadic TSAN error with GatingTask

2021-03-17 Thread GitBox
pitrou closed pull request #9720: URL: https://github.com/apache/arrow/pull/9720 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on a change in pull request #9730: ARROW-9878: [Python] Document caveats of to_pandas(self_destruct=True)

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9730: URL: https://github.com/apache/arrow/pull/9730#discussion_r596063732 ## File path: docs/source/python/pandas.rst ## @@ -293,3 +293,19 @@ Used together, the call will yield significantly lower memory usage in some

[GitHub] [arrow] pitrou commented on a change in pull request #9730: ARROW-9878: [Python] Document caveats of to_pandas(self_destruct=True)

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9730: URL: https://github.com/apache/arrow/pull/9730#discussion_r596063175 ## File path: docs/source/python/pandas.rst ## @@ -293,3 +293,19 @@ Used together, the call will yield significantly lower memory usage in some

[GitHub] [arrow] pitrou commented on pull request #9739: ARROW-12000: [Documentation] Add note about deviation from style guide on struct/classes

2021-03-17 Thread GitBox
pitrou commented on pull request #9739: URL: https://github.com/apache/arrow/pull/9739#issuecomment-801118593 This doesn't seem exact. We have `struct` types in public headers (such as `filesystem.h`). I think @wesm 's rationale seems more faithful to reality: > In the public headers

[GitHub] [arrow] dianaclarke commented on a change in pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
dianaclarke commented on a change in pull request #9738: URL: https://github.com/apache/arrow/pull/9738#discussion_r596059886 ## File path: python/pyarrow/tests/test_table.py ## @@ -271,7 +271,9 @@ def ne(xarrs, yarrs): eq([a, c], [d]) ne([c, a], [a, c]) -

[GitHub] [arrow] dianaclarke commented on a change in pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
dianaclarke commented on a change in pull request #9738: URL: https://github.com/apache/arrow/pull/9738#discussion_r596057813 ## File path: python/pyarrow/tests/test_table.py ## @@ -271,7 +271,9 @@ def ne(xarrs, yarrs): eq([a, c], [d]) ne([c, a], [a, c]) -

[GitHub] [arrow] pitrou commented on a change in pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9738: URL: https://github.com/apache/arrow/pull/9738#discussion_r596043491 ## File path: python/pyarrow/tests/test_table.py ## @@ -271,7 +271,9 @@ def ne(xarrs, yarrs): eq([a, c], [d]) ne([c, a], [a, c]) -assert not

[GitHub] [arrow] pitrou commented on a change in pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9738: URL: https://github.com/apache/arrow/pull/9738#discussion_r596043491 ## File path: python/pyarrow/tests/test_table.py ## @@ -271,7 +271,9 @@ def ne(xarrs, yarrs): eq([a, c], [d]) ne([c, a], [a, c]) -assert not

[GitHub] [arrow] dianaclarke commented on a change in pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
dianaclarke commented on a change in pull request #9738: URL: https://github.com/apache/arrow/pull/9738#discussion_r596042150 ## File path: python/pyarrow/tests/test_table.py ## @@ -271,7 +271,9 @@ def ne(xarrs, yarrs): eq([a, c], [d]) ne([c, a], [a, c]) -

[GitHub] [arrow] bkietz closed pull request #9685: ARROW-10372: [Dataset][C++][Python][R] Support reading compressed CSV

2021-03-17 Thread GitBox
bkietz closed pull request #9685: URL: https://github.com/apache/arrow/pull/9685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] dianaclarke commented on pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
dianaclarke commented on pull request #9738: URL: https://github.com/apache/arrow/pull/9738#issuecomment-801073738 Oh, thanks a ton @pitrou! I'm working on the additional failing tests now. This is an automated

[GitHub] [arrow] pitrou commented on pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
pitrou commented on pull request #9738: URL: https://github.com/apache/arrow/pull/9738#issuecomment-801062229 This is a recurring problem, so instead of testing for None manually we could let Cython do it for us: ```diff diff --git a/python/pyarrow/lib.pyx b/python/pyarrow/lib.pyx

[GitHub] [arrow] dianaclarke commented on a change in pull request #9739: ARROW-12000: [Documentation] Add note about deviation from style guide on struct/classes

2021-03-17 Thread GitBox
dianaclarke commented on a change in pull request #9739: URL: https://github.com/apache/arrow/pull/9739#discussion_r595979695 ## File path: docs/source/developers/cpp/development.rst ## @@ -77,6 +77,10 @@ This project follows `Google's C++ Style Guide * We relax the line

[GitHub] [arrow] westonpace opened a new pull request #9739: ARROW-12000: [Documentation] Add note about deviation from style guide on struct/classes

2021-03-17 Thread GitBox
westonpace opened a new pull request #9739: URL: https://github.com/apache/arrow/pull/9739 ![docs](https://user-images.githubusercontent.com/1696093/111465419-2b1e8880-86c6-11eb-98d5-5e5b873c224c.png) This is an

[GitHub] [arrow] lidavidm commented on pull request #9421: ARROW-11066: [FlightRPC][Java] Make zero-copy writes a configurable option

2021-03-17 Thread GitBox
lidavidm commented on pull request #9421: URL: https://github.com/apache/arrow/pull/9421#issuecomment-801015523 I think we should be all good (obviously let's keep an eye out for if it regresses again though), thanks for taking a look.

[GitHub] [arrow] lidavidm commented on pull request #9147: ARROW-11177: [Java] ArrowMessage failed to parse compressed grpc stream

2021-03-17 Thread GitBox
lidavidm commented on pull request #9147: URL: https://github.com/apache/arrow/pull/9147#issuecomment-801014998 I still believe we should not be polling available(). This is an automated message from the Apache Git Service.

[GitHub] [arrow] lidavidm commented on pull request #9728: ARROW-10250: [C++][FlightRPC] Consistently use FlightClientOptions::Defaults

2021-03-17 Thread GitBox
lidavidm commented on pull request #9728: URL: https://github.com/apache/arrow/pull/9728#issuecomment-801014690 Done, sorry (I thought I ran archery locally…) This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
github-actions[bot] commented on pull request #9738: URL: https://github.com/apache/arrow/pull/9738#issuecomment-801014571 https://issues.apache.org/jira/browse/ARROW-11997 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs closed pull request #9713: ARROW-11971: [Packaging] Vcpkg patch doesn't apply on windows due to line endings

2021-03-17 Thread GitBox
kszucs closed pull request #9713: URL: https://github.com/apache/arrow/pull/9713 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] kszucs commented on pull request #9713: ARROW-11971: [Packaging] Vcpkg patch doesn't apply on windows due to line endings

2021-03-17 Thread GitBox
kszucs commented on pull request #9713: URL: https://github.com/apache/arrow/pull/9713#issuecomment-800999549 CI error is unrelated, merging. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] dianaclarke opened a new pull request #9738: ARROW-11997: [Python] concat_tables crashes python interpreter

2021-03-17 Thread GitBox
dianaclarke opened a new pull request #9738: URL: https://github.com/apache/arrow/pull/9738 Before: >>> import pyarrow >>> pyarrow.concat_tables([None]) Bus error: 10 Python quit unexpectedly After: >>> import pyarrow >>> pyarrow.concat_tables([None])

[GitHub] [arrow] westonpace commented on pull request #9720: ARROW-11976: [C++] Fix sporadic TSAN error with GatingTask

2021-03-17 Thread GitBox
westonpace commented on pull request #9720: URL: https://github.com/apache/arrow/pull/9720#issuecomment-800985902 Thanks for cleaning this up. LGTM. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] pitrou closed pull request #9678: ARROW-11907: [C++] Use our own executor in S3FileSystem

2021-03-17 Thread GitBox
pitrou closed pull request #9678: URL: https://github.com/apache/arrow/pull/9678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on pull request #9728: ARROW-10250: [C++][FlightRPC] Consistently use FlightClientOptions::Defaults

2021-03-17 Thread GitBox
pitrou commented on pull request #9728: URL: https://github.com/apache/arrow/pull/9728#issuecomment-800968946 Can you fix the Python lint error? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] pitrou commented on pull request #9734: ARROW-11998: [C++] Make it easier to create vectors of move-only data types for tests

2021-03-17 Thread GitBox
pitrou commented on pull request #9734: URL: https://github.com/apache/arrow/pull/9734#issuecomment-800967868 Note: originally from https://github.com/westonpace/arrow/pull/6/commits/781dccacaea129668abf1aec87becfba910a266b

[GitHub] [arrow] alamb closed pull request #9695: ARROW-11955: [Rust][DataFusion] Support Union

2021-03-17 Thread GitBox
alamb closed pull request #9695: URL: https://github.com/apache/arrow/pull/9695 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] alamb commented on a change in pull request #9695: ARROW-11955: [Rust][DataFusion] Support Union

2021-03-17 Thread GitBox
alamb commented on a change in pull request #9695: URL: https://github.com/apache/arrow/pull/9695#discussion_r595886251 ## File path: rust/datafusion/src/logical_plan/builder.rs ## @@ -413,6 +436,32 @@ mod tests { Ok(()) } +#[test] +fn

[GitHub] [arrow] hunght3101 opened a new issue #9737: [Golang] How i create input format for IPC Reader?

2021-03-17 Thread GitBox
hunght3101 opened a new issue #9737: URL: https://github.com/apache/arrow/issues/9737 user@ In library ``` // NewReader returns a reader that reads records from an input stream. func NewReader(r io.Reader, opts ...Option) (*Reader, error) { return

[GitHub] [arrow] pitrou commented on a change in pull request #9720: ARROW-11976: [C++] Fix sporadic TSAN error with GatingTask

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9720: URL: https://github.com/apache/arrow/pull/9720#discussion_r595837653 ## File path: cpp/src/arrow/testing/gtest_util.cc ## @@ -719,43 +719,44 @@ ExtensionTypeGuard::~ExtensionTypeGuard() { } } -class GatingTask::Impl {

[GitHub] [arrow] pitrou commented on a change in pull request #9720: ARROW-11976: [C++] Fix sporadic TSAN error with GatingTask

2021-03-17 Thread GitBox
pitrou commented on a change in pull request #9720: URL: https://github.com/apache/arrow/pull/9720#discussion_r595837013 ## File path: cpp/src/arrow/testing/gtest_util.cc ## @@ -769,10 +770,8 @@ class GatingTask::Impl { } Status Unlock() { -{ - std::lock_guard

[GitHub] [arrow] hunght3101 commented on issue #9735: [Golang] Create ipc format for ipc Reader

2021-03-17 Thread GitBox
hunght3101 commented on issue #9735: URL: https://github.com/apache/arrow/issues/9735#issuecomment-800908056 @emkornfield Excuse me, i know `r = strings.NewReader("Object Apache Arrow")` is problem. But i dont know to input right to it. Please re-open this issues ?

[GitHub] [arrow] liyafan82 commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
liyafan82 commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-800898224 > please update the docs to match, something like. > "Slice the buffer to contain the uncompressed bytes" Updated. Thank you.

[GitHub] [arrow] liyafan82 commented on pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
liyafan82 commented on pull request #8949: URL: https://github.com/apache/arrow/pull/8949#issuecomment-800897864 > With the new enum, maybe we can make this an accessor that returns and enum instead? and then the byte can be extracted from there where necesssary? Sounds good. I have

[GitHub] [arrow] liyafan82 commented on a change in pull request #8949: ARROW-10880: [Java] Support compressing RecordBatch IPC buffers by LZ4

2021-03-17 Thread GitBox
liyafan82 commented on a change in pull request #8949: URL: https://github.com/apache/arrow/pull/8949#discussion_r595805905 ## File path: java/compression/src/main/java/org/apache/arrow/compression/Lz4CompressionCodec.java ## @@ -0,0 +1,158 @@ +/* + * Licensed to the Apache

  1   2   >