[GitHub] [arrow] jacques-n commented on a change in pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

2020-06-11 Thread GitBox
jacques-n commented on a change in pull request #7030: URL: https://github.com/apache/arrow/pull/7030#discussion_r439206598 ## File path: java/dataset/src/main/java/org/apache/arrow/memory/Ownerships.java ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [arrow] emkornfield commented on pull request #6156: ARROW-7539: [Java] FieldVector getFieldBuffers API should not set reader/writer indices

2020-06-11 Thread GitBox
emkornfield commented on pull request #6156: URL: https://github.com/apache/arrow/pull/6156#issuecomment-643059714 @jacques-n @rymurr do you know the progress of this internal to Dremio? It has been blocked a while on feedback, if we don't here back by Monday, I think we should rebase

[GitHub] [arrow] emkornfield commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

2020-06-11 Thread GitBox
emkornfield commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-643059101 @zhztheplayer it still looks like JNI is failing? This is an automated message from the Apache Git

[GitHub] [arrow] emkornfield commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

2020-06-11 Thread GitBox
emkornfield commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-643058410 @fsaintjacques do you have time to finish the review otherwise I can take a hopefully final look tomorrow.

[GitHub] [arrow] emkornfield closed pull request #7071: ARROW-7955: [Java] Support large buffer for file/stream IPC

2020-06-11 Thread GitBox
emkornfield closed pull request #7071: URL: https://github.com/apache/arrow/pull/7071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] emkornfield commented on pull request #7071: ARROW-7955: [Java] Support large buffer for file/stream IPC

2020-06-11 Thread GitBox
emkornfield commented on pull request #7071: URL: https://github.com/apache/arrow/pull/7071#issuecomment-643051092 Looks reasonable to me. Thanks @liyafan82 This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #7414: ARROW-6645: [Python] Use common boundschecking function for checking dictionary indices when converting to pandas

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7414: URL: https://github.com/apache/arrow/pull/7414#issuecomment-643041933 https://issues.apache.org/jira/browse/ARROW-6645 This is an automated message from the Apache Git

[GitHub] [arrow] wesm opened a new pull request #7414: ARROW-6645: [Python] Use common boundschecking function for checking dictionary indices when converting to pandas

2020-06-11 Thread GitBox
wesm opened a new pull request #7414: URL: https://github.com/apache/arrow/pull/7414 The new function is faster and this means less code to maintain. I had originally suggested doing away with the boundschecking but unless we demonstrate that it really is a performance issue this should

[GitHub] [arrow] scampi commented on pull request #6402: ARROW-7831: [Java] do not allocate a new offset buffer if the slice starts at 0 since the relative offset pointer would be unchanged

2020-06-11 Thread GitBox
scampi commented on pull request #6402: URL: https://github.com/apache/arrow/pull/6402#issuecomment-643033240 @wesm Sorry for the delay, I'll work on this in the next few days. Things have been hectic on my side This is an

[GitHub] [arrow] wesm commented on pull request #6806: ARROW-8312: [Java][Gandiva] support TreeNode in IN expression

2020-06-11 Thread GitBox
wesm commented on pull request #6806: URL: https://github.com/apache/arrow/pull/6806#issuecomment-643020747 ping again. this also needs to be rebased This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] wesm commented on pull request #7071: ARROW-7955: [Java] Support large buffer for file/stream IPC

2020-06-11 Thread GitBox
wesm commented on pull request #7071: URL: https://github.com/apache/arrow/pull/7071#issuecomment-643020925 ping @rymurr and/or @emkornfield This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] wesm commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Datasets Java API by JNI to C++

2020-06-11 Thread GitBox
wesm commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-643020617 @fsaintjacques @emkornfield ping on this, could we get this into the next release @zhztheplayer the CI failed, could you investigate?

[GitHub] [arrow] wesm commented on pull request #6156: ARROW-7539: [Java] FieldVector getFieldBuffers API should not set reader/writer indices

2020-06-11 Thread GitBox
wesm commented on pull request #6156: URL: https://github.com/apache/arrow/pull/6156#issuecomment-643020274 ping This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow] wesm commented on pull request #6402: ARROW-7831: [Java] do not allocate a new offset buffer if the slice starts at 0 since the relative offset pointer would be unchanged

2020-06-11 Thread GitBox
wesm commented on pull request #6402: URL: https://github.com/apache/arrow/pull/6402#issuecomment-643020217 @scampi are you able to address the feedback on this? Otherwise this might need someone to pick it up since 4 months have passed

[GitHub] [arrow] wesm commented on pull request #7378: ARROW-8866: [C++] Split UNION into SPARSE_UNION and DENSE_UNION

2020-06-11 Thread GitBox
wesm commented on pull request #7378: URL: https://github.com/apache/arrow/pull/7378#issuecomment-643004964 @bkietz can you rebase and address @pitrou's comments tomorrow? This is going to collide with ARROW-9075 so would prefer that this go in first and then I can rebase my patch on that

[GitHub] [arrow] mrkn commented on a change in pull request #7044: ARROW-6485: [Format][C++] Support the format of a COO sparse matrix that has separated row and column indices

2020-06-11 Thread GitBox
mrkn commented on a change in pull request #7044: URL: https://github.com/apache/arrow/pull/7044#discussion_r438924682 ## File path: cpp/src/arrow/tensor/util.h ## @@ -0,0 +1,38 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

[GitHub] [arrow] kou closed pull request #7397: ARROW-9090: [C++] Bump versions of bundled libraries

2020-06-11 Thread GitBox
kou closed pull request #7397: URL: https://github.com/apache/arrow/pull/7397 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] kou commented on pull request #7397: ARROW-9090: [C++] Bump versions of bundled libraries

2020-06-11 Thread GitBox
kou commented on pull request #7397: URL: https://github.com/apache/arrow/pull/7397#issuecomment-642989620 The command line that causes this link error has the following arguments in this order: * ... * `/usr/lib/s390x-linux-gnu/libssl.so` * ... *

[GitHub] [arrow] wesm commented on issue #7413: Installation issue with pyarrow and python 3.8

2020-06-11 Thread GitBox
wesm commented on issue #7413: URL: https://github.com/apache/arrow/issues/7413#issuecomment-642988373 Can you please open a JIRA issue? This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow] chaitanyav opened a new issue #7413: Installation issue with pyarrow and python 3.8

2020-06-11 Thread GitBox
chaitanyav opened a new issue #7413: URL: https://github.com/apache/arrow/issues/7413 Hello, I have compiled the arrow-cpp manually on my FreeBSD system with the following options to cmake ```cmake . -DARROW_PARQUET=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_ZLIB=ON

[GitHub] [arrow] wesm commented on a change in pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7382: URL: https://github.com/apache/arrow/pull/7382#discussion_r439116396 ## File path: cpp/src/arrow/compute/kernels/vector_take.cc ## @@ -15,67 +15,768 @@ // specific language governing permissions and limitations // under the

[GitHub] [arrow] wesm commented on a change in pull request #7410: ARROW-971: [C++][Compute] IsValid, IsNull kernels

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7410: URL: https://github.com/apache/arrow/pull/7410#discussion_r439108275 ## File path: cpp/src/arrow/compute/kernels/codegen_internal.h ## @@ -181,8 +180,7 @@ struct GetOutputType> { }; template -struct GetOutputType< -

[GitHub] [arrow] github-actions[bot] commented on pull request #7412: ARROW-9112: [R] Update autobrew script location

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7412: URL: https://github.com/apache/arrow/pull/7412#issuecomment-642963976 https://issues.apache.org/jira/browse/ARROW-9112 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson opened a new pull request #7412: ARROW-9112: [R] Update autobrew script location

2020-06-11 Thread GitBox
nealrichardson opened a new pull request #7412: URL: https://github.com/apache/arrow/pull/7412  @jeroen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] github-actions[bot] commented on pull request #7411: ARROW-8826: [Crossbow] remote URL should always have .git

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7411: URL: https://github.com/apache/arrow/pull/7411#issuecomment-642958643 https://issues.apache.org/jira/browse/ARROW-8826 This is an automated message from the Apache Git

[GitHub] [arrow] github-actions[bot] commented on pull request #7411: ARROW-8826: [Crossbow] remote URL should always have .git

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7411: URL: https://github.com/apache/arrow/pull/7411#issuecomment-642958214 Revision: 8d798e630b95022ad819cc82ec968234c5dcb43a Submitted crossbow builds: [ursa-labs/crossbow @

[GitHub] [arrow] nealrichardson opened a new pull request #7411: ARROW-8826: [Crossbow] remote URL should always have .git

2020-06-11 Thread GitBox
nealrichardson opened a new pull request #7411: URL: https://github.com/apache/arrow/pull/7411 Since crossbow doesn't have tests, I was afraid to poke at the python code for fear of breaking something else, so I've tried to work around the inconsistency in the Travis scripts. What's one

[GitHub] [arrow] nealrichardson commented on pull request #7411: ARROW-8826: [Crossbow] remote URL should always have .git

2020-06-11 Thread GitBox
nealrichardson commented on pull request #7411: URL: https://github.com/apache/arrow/pull/7411#issuecomment-642957691 @github-actions crossbow submit homebrew* This is an automated message from the Apache Git Service. To

[GitHub] [arrow] github-actions[bot] commented on pull request #7410: ARROW-971: [C++][Compute] IsValid, IsNull kernels

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7410: URL: https://github.com/apache/arrow/pull/7410#issuecomment-642932267 https://issues.apache.org/jira/browse/ARROW-971 This is an automated message from the Apache Git

[GitHub] [arrow] bkietz opened a new pull request #7410: ARROW-971: [C++][Compute] IsValid, IsNull kernels

2020-06-11 Thread GitBox
bkietz opened a new pull request #7410: URL: https://github.com/apache/arrow/pull/7410 Also adds a mixin for dead simple specification of randomized scalar kernel tests This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson commented on a change in pull request #7409: ARROW-8649: [Java] [Website] Java documentation on website is hidden

2020-06-11 Thread GitBox
nealrichardson commented on a change in pull request #7409: URL: https://github.com/apache/arrow/pull/7409#discussion_r439073471 ## File path: docs/source/index.rst ## @@ -52,11 +52,11 @@ such topics as: :caption: Arrow Libraries status - C++ - Python - Java

[GitHub] [arrow] pitrou commented on a change in pull request #7409: ARROW-8649: [Java] [Website] Java documentation on website is hidden

2020-06-11 Thread GitBox
pitrou commented on a change in pull request #7409: URL: https://github.com/apache/arrow/pull/7409#discussion_r439071554 ## File path: docs/source/index.rst ## @@ -52,11 +52,11 @@ such topics as: :caption: Arrow Libraries status - C++ - Python - Java

[GitHub] [arrow] github-actions[bot] commented on pull request #7409: ARROW-8649: [Java] [Website] Java documentation on website is hidden

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7409: URL: https://github.com/apache/arrow/pull/7409#issuecomment-642926946 https://issues.apache.org/jira/browse/ARROW-8649 This is an automated message from the Apache Git

[GitHub] [arrow] nealrichardson opened a new pull request #7409: ARROW-8649: [Java] [Website] Java documentation on website is hidden

2020-06-11 Thread GitBox
nealrichardson opened a new pull request #7409: URL: https://github.com/apache/arrow/pull/7409 This proposal moves the generated javadoc site from /docs/java/ to /docs/java/reference/. It appears that Sphinx is generating a /docs/java/index.html that has links to the Java prose docs, but

[GitHub] [arrow] kszucs commented on pull request #7408: ARROW-9110: [C++] Fix CPU cache size detection on macOS

2020-06-11 Thread GitBox
kszucs commented on pull request #7408: URL: https://github.com/apache/arrow/pull/7408#issuecomment-642894084 Will update to direct querying. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] pitrou commented on pull request #7408: ARROW-9110: [C++] Fix CPU cache size detection on macOS

2020-06-11 Thread GitBox
pitrou commented on pull request #7408: URL: https://github.com/apache/arrow/pull/7408#issuecomment-642892636 To avoid parsing, you could query those instead: ``` $ sysctl hw.l1dcachesize hw.l1dcachesize: 32768 $ sysctl hw.l2cachesize hw.l2cachesize: 262144 $ sysctl

[GitHub] [arrow] pitrou commented on pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
pitrou commented on pull request #7406: URL: https://github.com/apache/arrow/pull/7406#issuecomment-642890649 Well, thank you for improving and polishing Flight ;-) This is an automated message from the Apache Git Service.

[GitHub] [arrow] pitrou closed pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
pitrou closed pull request #7406: URL: https://github.com/apache/arrow/pull/7406 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] lidavidm commented on pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
lidavidm commented on pull request #7406: URL: https://github.com/apache/arrow/pull/7406#issuecomment-642890070 And rebased, thank you for all the reviews! :slightly_smiling_face: This is an automated message from the

[GitHub] [arrow] github-actions[bot] commented on pull request #7408: ARROW-9110: [C++] Fix CPU cache size detection on macOS

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7408: URL: https://github.com/apache/arrow/pull/7408#issuecomment-642885954 https://issues.apache.org/jira/browse/ARROW-9110 This is an automated message from the Apache Git

[GitHub] [arrow] kszucs opened a new pull request #7408: ARROW-9110: [C++] Fix CPU cache size detection on macOS

2020-06-11 Thread GitBox
kszucs opened a new pull request #7408: URL: https://github.com/apache/arrow/pull/7408 Querying locally shows my RAM size at the first position: ``` ❯ sysctl -a | grep hw\.cachesize hw.cachesize: 68719476736 32768 262144 16777216 0 0 0 0 0 0 ``` I'm unsure how to test

[GitHub] [arrow] jacques-n commented on a change in pull request #6402: ARROW-7831: [Java] do not allocate a new offset buffer if the slice starts at 0 since the relative offset pointer would be uncha

2020-06-11 Thread GitBox
jacques-n commented on a change in pull request #6402: URL: https://github.com/apache/arrow/pull/6402#discussion_r439003304 ## File path: java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthVector.java ## @@ -740,10 +740,16 @@ private void

[GitHub] [arrow] bkietz commented on a change in pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
bkietz commented on a change in pull request #7382: URL: https://github.com/apache/arrow/pull/7382#discussion_r438921113 ## File path: cpp/src/arrow/compute/kernels/vector_take.cc ## @@ -15,67 +15,768 @@ // specific language governing permissions and limitations // under the

[GitHub] [arrow] pitrou commented on pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
pitrou commented on pull request #7406: URL: https://github.com/apache/arrow/pull/7406#issuecomment-642857772 Ok, need to resolve conflicts now. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] pitrou closed pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
pitrou closed pull request #7398: URL: https://github.com/apache/arrow/pull/7398 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] pitrou commented on pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
pitrou commented on pull request #7398: URL: https://github.com/apache/arrow/pull/7398#issuecomment-642857268 The s390x infrastructure on Travis-CI seems unreliable. I'm going to merge anyway. This is an automated message

[GitHub] [arrow] wesm edited a comment on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm edited a comment on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642856297 Same issue as jemalloc I think. I really hope that the static lib issue will be addressed for 1.0. I can try to do it if no one else can work on it but I see that as a last

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642856297 Same issue as jemalloc I think. I really hope that the static lib issue will be addressed for 1.0. This is an

[GitHub] [arrow] nealrichardson commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
nealrichardson commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642852731 Does mimalloc have the same special build constraints like jemalloc such that we have to build it bundled? (Also, are we going to get that jemalloc static lib symbol

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642848987 BTW I ran benchmarks for this with MSVC 2017 with and without mimalloc and mimalloc has a pretty big impact, we should definitely endeavor to ship mimalloc in all of our Windows

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642848987 BTW I ran benchmarks for this with MSVC 2017 with and without mimalloc and mimalloc has a pretty big impact, we should definitely endeavor to ship mimalloc in all of our Windows

[GitHub] [arrow] nealrichardson commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
nealrichardson commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642852731 Does mimalloc have the same special build constraints like jemalloc such that we have to build it bundled? (Also, are we going to get that jemalloc static lib symbol

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642848987 BTW I ran benchmarks for this with MSVC 2017 with and without mimalloc and mimalloc has a pretty big impact, we should definitely endeavor to ship mimalloc in all of our Windows

[GitHub] [arrow] nealrichardson commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
nealrichardson commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642852731 Does mimalloc have the same special build constraints like jemalloc such that we have to build it bundled? (Also, are we going to get that jemalloc static lib symbol

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642848987 BTW I ran benchmarks for this with MSVC 2017 with and without mimalloc and mimalloc has a pretty big impact, we should definitely endeavor to ship mimalloc in all of our Windows

[GitHub] [arrow] itamarst commented on a change in pull request #7169: ARROW-5359: [Python] Support non-nanosecond out-of-range timestamps in conversion to pandas

2020-06-11 Thread GitBox
itamarst commented on a change in pull request #7169: URL: https://github.com/apache/arrow/pull/7169#discussion_r438971377 ## File path: python/pyarrow/tests/test_pandas.py ## @@ -3941,3 +3946,63 @@ def test_metadata_compat_missing_field_name(): result = table.to_pandas()

[GitHub] [arrow] pitrou commented on a change in pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
pitrou commented on a change in pull request #7406: URL: https://github.com/apache/arrow/pull/7406#discussion_r438970393 ## File path: python/pyarrow/tests/test_flight.py ## @@ -1428,3 +1428,25 @@ def test_middleware_multi_header(): for header, values in

[GitHub] [arrow] itamarst commented on a change in pull request #7169: ARROW-5359: [Python] Support non-nanosecond out-of-range timestamps in conversion to pandas

2020-06-11 Thread GitBox
itamarst commented on a change in pull request #7169: URL: https://github.com/apache/arrow/pull/7169#discussion_r438967894 ## File path: python/pyarrow/tests/test_pandas.py ## @@ -3941,3 +3946,63 @@ def test_metadata_compat_missing_field_name(): result = table.to_pandas()

[GitHub] [arrow] nealrichardson commented on pull request #7297: ARROW-6945: [Rust][Integration] Run rust integration tests

2020-06-11 Thread GitBox
nealrichardson commented on pull request #7297: URL: https://github.com/apache/arrow/pull/7297#issuecomment-642836792 @nevi-me @andygrove is it expected that everything except `primitive_no_batches` fails for Rust? If so, we can skip everything else, and y'all can unskip them iteratively

[GitHub] [arrow] lidavidm commented on a change in pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
lidavidm commented on a change in pull request #7406: URL: https://github.com/apache/arrow/pull/7406#discussion_r438958815 ## File path: python/pyarrow/tests/test_flight.py ## @@ -1428,3 +1428,25 @@ def test_middleware_multi_header(): for header, values in

[GitHub] [arrow] lidavidm commented on a change in pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
lidavidm commented on a change in pull request #7406: URL: https://github.com/apache/arrow/pull/7406#discussion_r438958440 ## File path: cpp/src/arrow/flight/flight_test.cc ## @@ -1601,6 +1620,26 @@ TEST_F(TestTls, OverrideHostname) { ASSERT_RAISES(IOError,

[GitHub] [arrow] lidavidm commented on a change in pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
lidavidm commented on a change in pull request #7398: URL: https://github.com/apache/arrow/pull/7398#discussion_r438945688 ## File path: python/pyarrow/_flight.pyx ## @@ -177,6 +185,16 @@ cdef class FlightUnavailableError(FlightError, ArrowException): return

[GitHub] [arrow] lidavidm commented on a change in pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
lidavidm commented on a change in pull request #7398: URL: https://github.com/apache/arrow/pull/7398#discussion_r438946065 ## File path: cpp/src/arrow/flight/client.h ## @@ -73,6 +91,16 @@ class ARROW_FLIGHT_EXPORT FlightClientOptions { std::string private_key; ///

[GitHub] [arrow] lidavidm commented on a change in pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
lidavidm commented on a change in pull request #7398: URL: https://github.com/apache/arrow/pull/7398#discussion_r438945868 ## File path: cpp/src/arrow/flight/client.h ## @@ -73,6 +91,16 @@ class ARROW_FLIGHT_EXPORT FlightClientOptions { std::string private_key; ///

[GitHub] [arrow] pitrou commented on a change in pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
pitrou commented on a change in pull request #7406: URL: https://github.com/apache/arrow/pull/7406#discussion_r438932983 ## File path: python/pyarrow/_flight.pyx ## @@ -1003,6 +1009,14 @@ cdef class FlightClient:

[GitHub] [arrow] pitrou commented on a change in pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
pitrou commented on a change in pull request #7398: URL: https://github.com/apache/arrow/pull/7398#discussion_r438929219 ## File path: python/pyarrow/_flight.pyx ## @@ -177,6 +185,16 @@ cdef class FlightUnavailableError(FlightError, ArrowException): return

[GitHub] [arrow] pitrou closed pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
pitrou closed pull request #7382: URL: https://github.com/apache/arrow/pull/7382 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] kszucs commented on pull request #7404: ARROW-9102: [Packaging] Upload built manylinux docker images

2020-06-11 Thread GitBox
kszucs commented on pull request #7404: URL: https://github.com/apache/arrow/pull/7404#issuecomment-642787409 I assume this is an approval :) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] kszucs closed pull request #7404: ARROW-9102: [Packaging] Upload built manylinux docker images

2020-06-11 Thread GitBox
kszucs closed pull request #7404: URL: https://github.com/apache/arrow/pull/7404 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] andygrove commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
andygrove commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438909250 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] andygrove commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
andygrove commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438905928 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] eyalleshem commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
eyalleshem commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438898154 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] eyalleshem commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
eyalleshem commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438898154 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] eyalleshem commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
eyalleshem commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438898154 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] eyalleshem commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
eyalleshem commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438898154 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] eyalleshem commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
eyalleshem commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438898154 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] wesm merged pull request #7403: [C++][MINOR][Docs] Fix typo

2020-06-11 Thread GitBox
wesm merged pull request #7403: URL: https://github.com/apache/arrow/pull/7403 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642742813 Here's an appveyor build https://ci.appveyor.com/project/wesm/arrow/builds/33463261. Will merge this shortly This

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642742047 @pitrou thanks for the fixes + improvements! This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] wesm closed pull request #7350: ARROW-6602: [Doc] Add a feature/implementation matrix

2020-06-11 Thread GitBox
wesm closed pull request #7350: URL: https://github.com/apache/arrow/pull/7350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] github-actions[bot] commented on pull request #7407: ARROW-9101: [Doc][C++] Document encoding expected for CSV data

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7407: URL: https://github.com/apache/arrow/pull/7407#issuecomment-642738699 https://issues.apache.org/jira/browse/ARROW-9101 This is an automated message from the Apache Git

[GitHub] [arrow] pitrou commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
pitrou commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642736534 Travis-CI: https://travis-ci.org/github/wesm/arrow/builds/697263135 This is an automated message from the Apache

[GitHub] [arrow] andygrove commented on a change in pull request #7384: ARROW-9082: [Rust] - Stream reader fail when steam not ended with (opt…

2020-06-11 Thread GitBox
andygrove commented on a change in pull request #7384: URL: https://github.com/apache/arrow/pull/7384#discussion_r438868762 ## File path: rust/arrow/src/ipc/reader.rs ## @@ -793,7 +793,21 @@ impl StreamReader { } // determine metadata length let mut

[GitHub] [arrow] pitrou opened a new pull request #7407: ARROW-9101: [Doc][C++] Document encoding expected for CSV data

2020-06-11 Thread GitBox
pitrou opened a new pull request #7407: URL: https://github.com/apache/arrow/pull/7407 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] github-actions[bot] commented on pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
github-actions[bot] commented on pull request #7406: URL: https://github.com/apache/arrow/pull/7406#issuecomment-642715859 https://issues.apache.org/jira/browse/ARROW-9093 This is an automated message from the Apache Git

[GitHub] [arrow] lidavidm commented on pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
lidavidm commented on pull request #7406: URL: https://github.com/apache/arrow/pull/7406#issuecomment-642707518 In particular the zerocopy flag may be interesting to enable by default: https://github.com/grpc/grpc/commit/48f026d90ece794eb718d7749e0b54b83ef76feb > For large RPCs (>=

[GitHub] [arrow] lidavidm opened a new pull request #7406: ARROW-9093: [FlightRPC][C++][Python] expose generic gRPC transport options

2020-06-11 Thread GitBox
lidavidm opened a new pull request #7406: URL: https://github.com/apache/arrow/pull/7406 This allows passing generic client options to the underlying gRPC client in C++/Python. The motivation is to expose these options: https://grpc.github.io/grpc/cpp/group__grpc__arg__keys.html

[GitHub] [arrow] wesm closed pull request #7405: ARROW-7676: [Packaging][Python] Ensure that the static libraries are not built in the wheel scripts

2020-06-11 Thread GitBox
wesm closed pull request #7405: URL: https://github.com/apache/arrow/pull/7405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] wesm commented on a change in pull request #7357: ARROW-9100: [C++] Add ascii_lower kernel

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7357: URL: https://github.com/apache/arrow/pull/7357#discussion_r438829269 ## File path: cpp/src/arrow/compute/kernels/scalar_string.cc ## @@ -48,6 +48,16 @@ struct AsciiUpper { } }; +struct AsciiLower { + template + static

[GitHub] [arrow] wesm commented on a change in pull request #7357: ARROW-9100: [C++] Add ascii_lower kernel

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7357: URL: https://github.com/apache/arrow/pull/7357#discussion_r438827531 ## File path: cpp/src/arrow/compute/kernels/scalar_string.cc ## @@ -48,6 +48,16 @@ struct AsciiUpper { } }; +struct AsciiLower { + template + static

[GitHub] [arrow] wesm commented on a change in pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7382: URL: https://github.com/apache/arrow/pull/7382#discussion_r438824681 ## File path: cpp/src/arrow/util/bit_block_counter.h ## @@ -17,14 +17,21 @@ #pragma once +#include #include +#include +#include +#include

[GitHub] [arrow] wesm commented on a change in pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on a change in pull request #7382: URL: https://github.com/apache/arrow/pull/7382#discussion_r438823668 ## File path: cpp/src/arrow/compute/kernels/vector_take_test.cc ## @@ -211,13 +306,63 @@ TYPED_TEST(TestTakeKernelWithString, TakeString) {

[GitHub] [arrow] wesm commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
wesm commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642690573 Please go ahead This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [arrow] pitrou commented on pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
pitrou commented on pull request #7382: URL: https://github.com/apache/arrow/pull/7382#issuecomment-642689244 I'm going to push changes on this PR, I think. Please hold on :-). This is an automated message from the Apache

[GitHub] [arrow] pitrou commented on a change in pull request #7382: ARROW-5760: [C++] New compute::Take implementation for better performance, faster dispatch, smaller code size / faster compilation

2020-06-11 Thread GitBox
pitrou commented on a change in pull request #7382: URL: https://github.com/apache/arrow/pull/7382#discussion_r438759834 ## File path: cpp/src/arrow/util/bit_block_counter.h ## @@ -17,14 +17,21 @@ #pragma once +#include #include +#include +#include +#include

[GitHub] [arrow] rymurr commented on pull request #7290: ARROW-1692: [Java] UnionArray round trip not working

2020-06-11 Thread GitBox
rymurr commented on pull request #7290: URL: https://github.com/apache/arrow/pull/7290#issuecomment-642647124 > It looks like the integration tests are still failing for union arrays between C++ and Java. yeah, something w/ Flight, though the IPC works fine. Checking what is

[GitHub] [arrow] lidavidm commented on pull request #7398: ARROW-8487: [FlightRPC] Provide a way to target a particular payload size

2020-06-11 Thread GitBox
lidavidm commented on pull request #7398: URL: https://github.com/apache/arrow/pull/7398#issuecomment-642640738 Rebased. This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow] pitrou closed pull request #7387: ARROW-5377: [C++] Make IpcPayload public and add GetPayloadSize

2020-06-11 Thread GitBox
pitrou closed pull request #7387: URL: https://github.com/apache/arrow/pull/7387 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] lidavidm commented on pull request #7290: ARROW-1692: [Java] UnionArray round trip not working

2020-06-11 Thread GitBox
lidavidm commented on pull request #7290: URL: https://github.com/apache/arrow/pull/7290#issuecomment-642630745 It looks like the integration tests are still failing for union arrays between C++ and Java. This is an

[GitHub] [arrow] lidavidm commented on pull request #7387: ARROW-5377: [C++] Make IpcPayload public and add GetPayloadSize

2020-06-11 Thread GitBox
lidavidm commented on pull request #7387: URL: https://github.com/apache/arrow/pull/7387#issuecomment-642628014 Yes, sounds good, thank you. This is an automated message from the Apache Git Service. To respond to the

  1   2   >