[GitHub] [arrow] praveenbingo commented on pull request #10517: ARROW-13050: [C++][Gandiva] Implement SPACE Hive function on Gandiva

2021-07-05 Thread GitBox
praveenbingo commented on pull request #10517: URL: https://github.com/apache/arrow/pull/10517#issuecomment-874507195 @projjal is appveyor failure expected? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow] praveenbingo closed pull request #10382: ARROW-12856: [C++][Gandiva] Implement castBIT and castBOOLEAN functions

2021-07-05 Thread GitBox
praveenbingo closed pull request #10382: URL: https://github.com/apache/arrow/pull/10382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-un

[GitHub] [arrow] praveenbingo closed pull request #10179: ARROW-12567: [C++][Gandiva] Implement ILIKE SQL function

2021-07-05 Thread GitBox
praveenbingo closed pull request #10179: URL: https://github.com/apache/arrow/pull/10179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-un

[GitHub] [arrow] praveenbingo closed pull request #10169: ARROW-12556: [C++][Gandiva] Implement BYTESUBSTRING function on Gandiva

2021-07-05 Thread GitBox
praveenbingo closed pull request #10169: URL: https://github.com/apache/arrow/pull/10169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-un

[GitHub] [arrow] github-actions[bot] commented on pull request #10660: ARROW-12759: [C++][Compute] Add ExecNode for group by

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10660: URL: https://github.com/apache/arrow/pull/10660#issuecomment-874495739 https://issues.apache.org/jira/browse/ARROW-12759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] michalursa opened a new pull request #10660: ARROW-12759: [C++][Compute] Add ExecNode for group by

2021-07-05 Thread GitBox
michalursa opened a new pull request #10660: URL: https://github.com/apache/arrow/pull/10660 Adding ExecNode for grouped aggregation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [arrow-datafusion] andygrove commented on pull request #688: run ballista integration test in CI

2021-07-05 Thread GitBox
andygrove commented on pull request #688: URL: https://github.com/apache/arrow-datafusion/pull/688#issuecomment-874453682 Thanks @houqp it is great to see this being worked on :rocket: -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [arrow-datafusion] andygrove commented on a change in pull request #683: Introduce (default) number of partitions option, use it in DataFusion/Ballista

2021-07-05 Thread GitBox
andygrove commented on a change in pull request #683: URL: https://github.com/apache/arrow-datafusion/pull/683#discussion_r664229932 ## File path: datafusion/src/execution/context.rs ## @@ -681,6 +684,14 @@ impl ExecutionConfig { self } +/// Customize defaul

[GitHub] [arrow-datafusion] andygrove commented on a change in pull request #683: Introduce (default) number of partitions option, use it in DataFusion/Ballista

2021-07-05 Thread GitBox
andygrove commented on a change in pull request #683: URL: https://github.com/apache/arrow-datafusion/pull/683#discussion_r664229657 ## File path: ballista/rust/core/src/utils.rs ## @@ -234,7 +234,9 @@ fn build_exec_plan_diagram( /// Create a DataFusion context that is compa

[GitHub] [arrow-datafusion] houqp commented on pull request #688: run ballista integration test in CI

2021-07-05 Thread GitBox
houqp commented on pull request #688: URL: https://github.com/apache/arrow-datafusion/pull/688#issuecomment-874450970 You are right, base image build took around 5 mins, then it's 20 mins for ballista build and 4 mins for running the integration tests. I will look into optimization for the

[GitHub] [arrow-datafusion] jorgecarleitao commented on pull request #688: run ballista integration test in CI

2021-07-05 Thread GitBox
jorgecarleitao commented on pull request #688: URL: https://github.com/apache/arrow-datafusion/pull/688#issuecomment-874429610 The build log is a bit unreadable in its current form. Could we trim the output? I went through the build log and I am not sure having a pre-built image hel

[GitHub] [arrow-datafusion] houqp commented on pull request #688: run ballista integration test in CI

2021-07-05 Thread GitBox
houqp commented on pull request #688: URL: https://github.com/apache/arrow-datafusion/pull/688#issuecomment-874423832 Integration test took 28m to run, compared to our main Rust test job which took 14m. I think we can cut down the build time by 10m if we avoid rebuilding the base image on

[GitHub] [arrow] emkornfield commented on pull request #10603: ARROW-13191: [Go] allow external schema in ipc readers

2021-07-05 Thread GitBox
emkornfield commented on pull request #10603: URL: https://github.com/apache/arrow/pull/10603#issuecomment-874423397 > Unless there's a context/use case I'm not thinking of (you'd know better than I @emkornfield), I don't like the idea of putting a check in the next function for this, but

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874418946 Revision: b9bef65b267f788f0b42ed187704575121c344c7 Submitted crossbow builds: [ursacomputing/crossbow @ actions-571](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874418682 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow-datafusion] andygrove closed pull request #685: Rename concurrency to partitions in ExecutionContext

2021-07-05 Thread GitBox
andygrove closed pull request #685: URL: https://github.com/apache/arrow-datafusion/pull/685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: githu

[GitHub] [arrow-datafusion] andygrove commented on pull request #685: Rename concurrency to partitions in ExecutionContext

2021-07-05 Thread GitBox
andygrove commented on pull request #685: URL: https://github.com/apache/arrow-datafusion/pull/685#issuecomment-874394111 Closed in favor of https://github.com/apache/arrow-datafusion/pull/683 -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [arrow] cyb70289 closed pull request #10602: ARROW-13173: [C++] TestAsyncUtil.ReadaheadFailed asserts occasionally

2021-07-05 Thread GitBox
cyb70289 closed pull request #10602: URL: https://github.com/apache/arrow/pull/10602 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664173698 ## File path: wheel.sh ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash + +arrow_dir=$1 +build_dir=$2 + +export ARROW_FLIGHT=OFF +export ARROW_JEMALLOC=OFF +export

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664173518 ## File path: wheel.sh ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash + +arrow_dir=$1 +build_dir=$2 + +export ARROW_FLIGHT=OFF +export ARROW_JEMALLOC=OFF +export

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664173327 ## File path: ci/scripts/python_wheel_macos_build.sh ## @@ -62,14 +62,18 @@ echo "=== (${PYTHON_VERSION}) Building Arrow C++ libraries ===" mkdir -p ${

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664172803 ## File path: wheel.sh ## @@ -0,0 +1,26 @@ +#!/usr/bin/env bash + +arrow_dir=$1 +build_dir=$2 + +export ARROW_FLIGHT=OFF +export ARROW_JEMALLOC=OFF +export

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664171185 ## File path: cpp/cmake_modules/FindSnappyAlt.cmake ## @@ -0,0 +1,117 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribut

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664170931 ## File path: ci/vcpkg/arm64-osx-static-release.cmake~ ## @@ -0,0 +1,25 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664170812 ## File path: ci/scripts/python_wheel_macos_build.sh ## @@ -62,14 +62,18 @@ echo "=== (${PYTHON_VERSION}) Building Arrow C++ libraries ===" mkdir -p ${

[GitHub] [arrow] github-actions[bot] commented on pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10659: URL: https://github.com/apache/arrow/pull/10659#issuecomment-874386746 https://issues.apache.org/jira/browse/ARROW-12122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] kszucs commented on a change in pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs commented on a change in pull request #10659: URL: https://github.com/apache/arrow/pull/10659#discussion_r664170706 ## File path: ci/scripts/python_wheel_macos_build.sh ## @@ -62,14 +62,18 @@ echo "=== (${PYTHON_VERSION}) Building Arrow C++ libraries ===" mkdir -p ${

[GitHub] [arrow] kszucs opened a new pull request #10659: ARROW-12122: [Python] Cannot install via pip. M1 mac [WIP]

2021-07-05 Thread GitBox
kszucs opened a new pull request #10659: URL: https://github.com/apache/arrow/pull/10659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-un

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874384308 Revision: b9bef65b267f788f0b42ed187704575121c344c7 Submitted crossbow builds: [ursacomputing/crossbow @ actions-570](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874384138 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow-datafusion] houqp opened a new pull request #688: run ballista integration test in CI

2021-07-05 Thread GitBox
houqp opened a new pull request #688: URL: https://github.com/apache/arrow-datafusion/pull/688 # Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/321 # Rationale for this change Avoid ballista integration test regressions

[GitHub] [arrow] kylepbit commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-07-05 Thread GitBox
kylepbit commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r664162550 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874375445 Revision: 81a3e5118e10c3bfe55e8ff484bc3d1580b50349 Submitted crossbow builds: [ursacomputing/crossbow @ actions-569](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874374893 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874373670 Revision: 96b4e219c45d12e1fe499d779cc8ae2ac97063c6 Submitted crossbow builds: [ursacomputing/crossbow @ actions-568](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874373471 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow-datafusion] westonpace commented on issue #686: Specific timezone support for `to_timetamp*()`

2021-07-05 Thread GitBox
westonpace commented on issue #686: URL: https://github.com/apache/arrow-datafusion/issues/686#issuecomment-874368411 > To be specific, if you do to_timestamp(date_string) then it can figure out the timezone from the string, sometimes. However, current architecture of Arrow/DF requires a

[GitHub] [arrow-datafusion] westonpace commented on issue #686: Specific timezone support for `to_timetamp*()`

2021-07-05 Thread GitBox
westonpace commented on issue #686: URL: https://github.com/apache/arrow-datafusion/issues/686#issuecomment-874367756 > I'm not sure I'd call it a "bug" per se, but all of the to_timestamp*() functions output TimestampTypes in Arrow that have a Timezone field of None. The issues here are:

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874360330 Revision: 96b4e219c45d12e1fe499d779cc8ae2ac97063c6 Submitted crossbow builds: [ursacomputing/crossbow @ actions-567](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874360075 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow-datafusion] jgoday commented on issue #554: implement lead and lag with 2nd and 3rd argument

2021-07-05 Thread GitBox
jgoday commented on issue #554: URL: https://github.com/apache/arrow-datafusion/issues/554#issuecomment-874356911 I have created a PR (#687) and commented some doubts about the implementation in it. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [arrow-datafusion] velvia commented on issue #686: Specific timezone support for `to_timetamp*()`

2021-07-05 Thread GitBox
velvia commented on issue #686: URL: https://github.com/apache/arrow-datafusion/issues/686#issuecomment-874356725 @alamb @westonpace your thoughts on this matter would be really welcome. :) To be specific, if you do `to_timestamp(date_string)` then it can figure out the tim

[GitHub] [arrow-datafusion] jgoday commented on pull request #687: #554: Lead/lag window function with offset and default value arguments

2021-07-05 Thread GitBox
jgoday commented on pull request #687: URL: https://github.com/apache/arrow-datafusion/pull/687#issuecomment-874356594 I have a few doubts: 1) **Lag/Lead** signatures has to be changed to support 2nd/3nd optional arguments, is this correct or am I misunderstanding the use of **Signa

[GitHub] [arrow-datafusion] jgoday opened a new pull request #687: #554: Lead/lag window function with offset and default value arguments

2021-07-05 Thread GitBox
jgoday opened a new pull request #687: URL: https://github.com/apache/arrow-datafusion/pull/687 # Which issue does this PR close? #Closes 554. # Rationale for this change Implement offset and default value optional arguments in lead/lag window functions. # What changes

[GitHub] [arrow-datafusion] velvia opened a new issue #686: Specific timezone support for `to_timetamp*()`

2021-07-05 Thread GitBox
velvia opened a new issue #686: URL: https://github.com/apache/arrow-datafusion/issues/686 **Describe the bug** I'm not sure I'd call it a "bug" per se, but all of the `to_timestamp*()` functions output `TimestampTypes` in Arrow that have a Timezone field of None. The issues here a

[GitHub] [arrow] lidavidm commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-07-05 Thread GitBox
lidavidm commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r664143005 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] lidavidm commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-07-05 Thread GitBox
lidavidm commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r664142839 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35 [WIP]

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874351395 Revision: c638329be4782536d6b0d2020b0e56f012f4400d Submitted crossbow builds: [ursacomputing/crossbow @ actions-566](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874351114 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow] kylepbit commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-07-05 Thread GitBox
kylepbit commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r664138603 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow] kylepbit commented on a change in pull request #9368: [WIP] [POC] Flight SQL

2021-07-05 Thread GitBox
kylepbit commented on a change in pull request #9368: URL: https://github.com/apache/arrow/pull/9368#discussion_r664136841 ## File path: format/FlightSQL.proto ## @@ -0,0 +1,296 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor licens

[GitHub] [arrow-datafusion] Dandandan commented on issue #679: Improved features and interoperability for SQLMetrics

2021-07-05 Thread GitBox
Dandandan commented on issue #679: URL: https://github.com/apache/arrow-datafusion/issues/679#issuecomment-874341354 This sounds great. Another use case of programmatically accessing the statistics is for re-optimization of the query (join reordering, removing empty results from the

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #10629: ARROW-13218: [Doc] Document/clarify conventions for timestamp storage

2021-07-05 Thread GitBox
jorisvandenbossche commented on a change in pull request #10629: URL: https://github.com/apache/arrow/pull/10629#discussion_r664130256 ## File path: format/Schema.fbs ## @@ -218,8 +218,33 @@ table Time { /// leap seconds, as a 64-bit integer. Note that UNIX time does not inclu

[GitHub] [arrow-datafusion] Dandandan commented on pull request #685: Rename concurrency to partitions in ExecutionContext

2021-07-05 Thread GitBox
Dandandan commented on pull request #685: URL: https://github.com/apache/arrow-datafusion/pull/685#issuecomment-874307544 Hehe :laughing: I had a slightly different idea / proposal in https://github.com/apache/arrow-datafusion/pull/683 I think we might want to still keep a co

[GitHub] [arrow] jonkeane closed pull request #10611: ARROW-13199: [R] add ubuntu 21.04 to nightly builds

2021-07-05 Thread GitBox
jonkeane closed pull request #10611: URL: https://github.com/apache/arrow/pull/10611 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] jonkeane commented on pull request #10611: ARROW-13199: [R] add ubuntu 21.04 to nightly builds

2021-07-05 Thread GitBox
jonkeane commented on pull request #10611: URL: https://github.com/apache/arrow/pull/10611#issuecomment-874304215 I've created https://issues.apache.org/jira/browse/ARROW-13261 to remove the additional service once GCC 11 building is fixed (either via https://issues.apache.org/jira/browse/

[GitHub] [arrow-datafusion] Dandandan commented on pull request #660: Allow non-equijoin filters in join condition

2021-07-05 Thread GitBox
Dandandan commented on pull request #660: URL: https://github.com/apache/arrow-datafusion/pull/660#issuecomment-874303330 @alamb do you mind to take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow-datafusion] andygrove opened a new pull request #685: Rename concurrency to partitions in ExecutionContext

2021-07-05 Thread GitBox
andygrove opened a new pull request #685: URL: https://github.com/apache/arrow-datafusion/pull/685 # Which issue does this PR close? Closes #684. # Rationale for this change See #684 # What changes are included in this PR? - Rename `with_c

[GitHub] [arrow-datafusion] andygrove opened a new issue #684: Rename ExecutionConfig with_concurrency to with_partitions

2021-07-05 Thread GitBox
andygrove opened a new issue #684: URL: https://github.com/apache/arrow-datafusion/issues/684 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Following on from the discussion in https://github.com/apache/arrow-datafusion/issues/6

[GitHub] [arrow-datafusion] Dandandan commented on pull request #683: Introduce (default) number of partitions option, use it in DataFusion

2021-07-05 Thread GitBox
Dandandan commented on pull request #683: URL: https://github.com/apache/arrow-datafusion/pull/683#issuecomment-874302210 FYI @andygrove -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [arrow-datafusion] Dandandan opened a new pull request #683: Introduce (default) number of partitions option, use it in DataFusion

2021-07-05 Thread GitBox
Dandandan opened a new pull request #683: URL: https://github.com/apache/arrow-datafusion/pull/683 # Which issue does this PR close? Closes #661 # Rationale for this change Currently we have `concurrency=partitions` which makes sense for good use of paralle

[GitHub] [arrow-datafusion] andygrove opened a new issue #682: Ballista: Implement configuration mechanism

2021-07-05 Thread GitBox
andygrove opened a new issue #682: URL: https://github.com/apache/arrow-datafusion/issues/682 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** There is currently no way to specify configuration parameters such as default partition

[GitHub] [arrow] keeratsingh commented on pull request #8780: [POC] ARROW-10671: [FlightRPC] Bearer Token refresh design with retry

2021-07-05 Thread GitBox
keeratsingh commented on pull request #8780: URL: https://github.com/apache/arrow/pull/8780#issuecomment-874285499 > Following up here - given 5.0.0 is targeting July, do you think this will be ready in time? Hey @lidavidm I am no longer working on this project. @tifflhl or @kylep-d

[GitHub] [arrow-datafusion] alamb commented on issue #679: Improved features and interoperability for SQLMetrics

2021-07-05 Thread GitBox
alamb commented on issue #679: URL: https://github.com/apache/arrow-datafusion/issues/679#issuecomment-874279780 > is that we need a way to get aggregate metrics for a query plan when the query execution is distributed Assigning a unique id (metrics id?) to each node in the plan (p

[GitHub] [arrow] andersonm-ibm commented on a change in pull request #10450: ARROW-9947: [Python] High-level Python API for Parquet encryption of files.

2021-07-05 Thread GitBox
andersonm-ibm commented on a change in pull request #10450: URL: https://github.com/apache/arrow/pull/10450#discussion_r664088018 ## File path: python/pyarrow/tests/parquet/test_parquet_encryption.py ## @@ -0,0 +1,301 @@ +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [arrow] pitrou closed pull request #10644: ARROW-13244: [C++] Add facility to get current thread id as uint64

2021-07-05 Thread GitBox
pitrou closed pull request #10644: URL: https://github.com/apache/arrow/pull/10644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow-datafusion] crepererum opened a new pull request #681: use `Weak` ptr to break catalog list <> info schema cyclic reference

2021-07-05 Thread GitBox
crepererum opened a new pull request #681: URL: https://github.com/apache/arrow-datafusion/pull/681 Fixes #680. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [arrow-datafusion] crepererum opened a new issue #680: `ExecutionContext` w/ information schema leaks catalogs

2021-07-05 Thread GitBox
crepererum opened a new issue #680: URL: https://github.com/apache/arrow-datafusion/issues/680 **Describe the bug** Catalog registered to `ExecutionContext` with an information schema are never freed due to a cyclic Arc chain. **To Reproduce** Steps to reproduce the behavior:

[GitHub] [arrow-experimental-rs-arrow2] abreis commented on pull request #1: Adds arrow2

2021-07-05 Thread GitBox
abreis commented on pull request #1: URL: https://github.com/apache/arrow-experimental-rs-arrow2/pull/1#issuecomment-874228534 I support this donation and I've sent a signed CLA. Best of luck with the project! -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow] thisisnic commented on a change in pull request #10624: ARROW-12992: [R] bindings for substr(), substring(), str_sub()

2021-07-05 Thread GitBox
thisisnic commented on a change in pull request #10624: URL: https://github.com/apache/arrow/pull/10624#discussion_r664040870 ## File path: r/tests/testthat/test-dplyr-string-functions.R ## @@ -866,3 +866,57 @@ test_that("str_like", { df ) }) + +test_that("substrings",

[GitHub] [arrow] thisisnic commented on a change in pull request #10624: ARROW-12992: [R] bindings for substr(), substring(), str_sub()

2021-07-05 Thread GitBox
thisisnic commented on a change in pull request #10624: URL: https://github.com/apache/arrow/pull/10624#discussion_r663049290 ## File path: r/tests/testthat/test-dplyr-string-functions.R ## @@ -866,3 +866,57 @@ test_that("str_like", { df ) }) + +test_that("substrings",

[GitHub] [arrow] pitrou closed pull request #10591: ARROW-13158: [Python] Fix StructScalar contains and repr with duplicate field names

2021-07-05 Thread GitBox
pitrou closed pull request #10591: URL: https://github.com/apache/arrow/pull/10591 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

[GitHub] [arrow] rok commented on pull request #10598: ARROW-13054: [C++] Add option to specify the first day of the week for the "day_of_week" temporal kernel

2021-07-05 Thread GitBox
rok commented on pull request #10598: URL: https://github.com/apache/arrow/pull/10598#issuecomment-874223797 Thanks for the review @pitrou! I've implemented the suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [arrow] rok commented on a change in pull request #10598: ARROW-13054: [C++] Add option to specify the first day of the week for the "day_of_week" temporal kernel

2021-07-05 Thread GitBox
rok commented on a change in pull request #10598: URL: https://github.com/apache/arrow/pull/10598#discussion_r664038313 ## File path: cpp/src/arrow/compute/kernels/scalar_temporal.cc ## @@ -118,16 +141,31 @@ struct Day { // ---

[GitHub] [arrow] pitrou commented on pull request #10202: ARROW-12673: [C++] Add parser handler for incorrect column counts

2021-07-05 Thread GitBox
pitrou commented on pull request #10202: URL: https://github.com/apache/arrow/pull/10202#issuecomment-874220564 > Is adding this overhead acceptable for error reporting? If the overhead is paid just in case of error then it would be ok. I don't think it would be a great idea to add

[GitHub] [arrow] lidavidm closed pull request #10654: ARROW-13258: [Python] Improve the repr of ParquetFileFragment

2021-07-05 Thread GitBox
lidavidm closed pull request #10654: URL: https://github.com/apache/arrow/pull/10654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubs

[GitHub] [arrow] pitrou commented on a change in pull request #10598: ARROW-13054: [C++] Add option to specify the first day of the week for the "day_of_week" temporal kernel

2021-07-05 Thread GitBox
pitrou commented on a change in pull request #10598: URL: https://github.com/apache/arrow/pull/10598#discussion_r664031227 ## File path: cpp/src/arrow/compute/kernels/scalar_temporal.cc ## @@ -118,16 +141,31 @@ struct Day { //

[GitHub] [arrow] pitrou commented on pull request #10602: ARROW-13173: [C++] TestAsyncUtil.ReadaheadFailed asserts occasionally

2021-07-05 Thread GitBox
pitrou commented on pull request #10602: URL: https://github.com/apache/arrow/pull/10602#issuecomment-874207579 No particular comment, no. I'll let you merge if you're ok with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [arrow] github-actions[bot] commented on pull request #10658: ARROW-13230: [Docs][Python] Add CSV writer docs

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10658: URL: https://github.com/apache/arrow/pull/10658#issuecomment-874202935 https://issues.apache.org/jira/browse/ARROW-13230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [arrow] lidavidm opened a new pull request #10658: ARROW-13230: [Docs][Python] Add CSV writer docs

2021-07-05 Thread GitBox
lidavidm opened a new pull request #10658: URL: https://github.com/apache/arrow/pull/10658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow] pitrou commented on a change in pull request #10585: ARROW-12016 [C++] Implement array_sort_indices and sort_indices for BOOL type

2021-07-05 Thread GitBox
pitrou commented on a change in pull request #10585: URL: https://github.com/apache/arrow/pull/10585#discussion_r663999228 ## File path: cpp/src/arrow/compute/kernels/vector_sort.cc ## @@ -477,6 +491,87 @@ class ArrayCountSorter { } }; +using ::arrow::internal::Bitmap; +

[GitHub] [arrow-datafusion] andygrove commented on issue #679: Improved features and interoperability for SQLMetrics

2021-07-05 Thread GitBox
andygrove commented on issue #679: URL: https://github.com/apache/arrow-datafusion/issues/679#issuecomment-874187741 This looks great. One thing I would like to add to this is that we need a way to get aggregate metrics for a query plan when the query execution is distributed. My thought

[GitHub] [arrow] thisisnic commented on a change in pull request #10624: ARROW-12992: [R] bindings for substr(), substring(), str_sub()

2021-07-05 Thread GitBox
thisisnic commented on a change in pull request #10624: URL: https://github.com/apache/arrow/pull/10624#discussion_r662899545 ## File path: r/R/expression.R ## @@ -32,6 +32,8 @@ "str_to_upper" = "utf8_upper", "str_reverse" = "utf8_reverse", # str_trim is defined in dpl

[GitHub] [arrow] pitrou commented on a change in pull request #10567: ARROW-13096: [C++] Implement logarithm compute functions

2021-07-05 Thread GitBox
pitrou commented on a change in pull request #10567: URL: https://github.com/apache/arrow/pull/10567#discussion_r663994305 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -686,6 +686,118 @@ struct Atan2 { } }; +struct LogNatural { + template + st

[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #10628: ARROW-12364: [Python] [Dataset] Add metadata_collector option to ds.write_dataset()

2021-07-05 Thread GitBox
jorisvandenbossche commented on a change in pull request #10628: URL: https://github.com/apache/arrow/pull/10628#discussion_r663986392 ## File path: python/pyarrow/parquet.py ## @@ -1958,8 +1958,11 @@ def write_to_dataset(table, root_path, partition_cols=None, "im

[GitHub] [arrow] pitrou commented on a change in pull request #10567: ARROW-13096: [C++] Implement logarithm compute functions

2021-07-05 Thread GitBox
pitrou commented on a change in pull request #10567: URL: https://github.com/apache/arrow/pull/10567#discussion_r663993579 ## File path: cpp/src/arrow/compute/kernels/scalar_arithmetic.cc ## @@ -686,6 +686,118 @@ struct Atan2 { } }; +struct LogNatural { + template + st

[GitHub] [arrow] wesm closed issue #10634: pq.write_to_dataset() and ds.write_dataset() both throw InvalidLocationConstraint when using S3FileSystem

2021-07-05 Thread GitBox
wesm closed issue #10634: URL: https://github.com/apache/arrow/issues/10634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] wesm closed issue #10564: Failed to install pyarrow module by using 'pip3.7 install pyarrow' in a docker container

2021-07-05 Thread GitBox
wesm closed issue #10564: URL: https://github.com/apache/arrow/issues/10564 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] wesm closed issue #10531: How to resolve `pyarrow.deserialize` FutureWarning

2021-07-05 Thread GitBox
wesm closed issue #10531: URL: https://github.com/apache/arrow/issues/10531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] wesm commented on issue #10531: How to resolve `pyarrow.deserialize` FutureWarning

2021-07-05 Thread GitBox
wesm commented on issue #10531: URL: https://github.com/apache/arrow/issues/10531#issuecomment-874172472 Plasma isn't being maintained, so I guess this issue has to do with the fact that Plasma is using deprecated APIs and not being updated. It might be time to propose its removal from the

[GitHub] [arrow] wesm closed issue #10492: Doc update ? For Reading and Writing the Apache Parquet Format

2021-07-05 Thread GitBox
wesm closed issue #10492: URL: https://github.com/apache/arrow/issues/10492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] wesm closed issue #10394: There is a vulnerability in guava 23.0 ,upgrade recommended

2021-07-05 Thread GitBox
wesm closed issue #10394: URL: https://github.com/apache/arrow/issues/10394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] wesm closed issue #10393: There is a vulnerability in guava 20.0 ,upgrade recommended

2021-07-05 Thread GitBox
wesm closed issue #10393: URL: https://github.com/apache/arrow/issues/10393 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arr

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874159290 Revision: 1711d4f782893bfd6b908c750b6c0e1b60d4f84d Submitted crossbow builds: [ursacomputing/crossbow @ actions-565](https://github.com/ursacomputing/crossbow/

[GitHub] [arrow] github-actions[bot] removed a comment on pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
github-actions[bot] removed a comment on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874158269 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could

[GitHub] [arrow] ianmcook commented on pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
ianmcook commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874158743 @github-actions crossbow submit test-r-rtools-35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [arrow] github-actions[bot] commented on pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
github-actions[bot] commented on pull request #10657: URL: https://github.com/apache/arrow/pull/10657#issuecomment-874158269 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you ope

[GitHub] [arrow] ianmcook opened a new pull request #10657: [R] Try fix test-r-rtools-35

2021-07-05 Thread GitBox
ianmcook opened a new pull request #10657: URL: https://github.com/apache/arrow/pull/10657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-

[GitHub] [arrow] pitrou commented on pull request #10591: ARROW-13158: [Python] Fix StructScalar contains and repr with duplicate field names

2021-07-05 Thread GitBox
pitrou commented on pull request #10591: URL: https://github.com/apache/arrow/pull/10591#issuecomment-874153925 +1 for merging as is. Can you rebase first? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [arrow] pitrou commented on pull request #10644: ARROW-13244: [C++] Add facility to get current thread id as uint64

2021-07-05 Thread GitBox
pitrou commented on pull request #10644: URL: https://github.com/apache/arrow/pull/10644#issuecomment-874153432 Rebased, will merge if green. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   >