[GitHub] [arrow] Dandandan commented on pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

2021-01-06 Thread GitBox
Dandandan commented on pull request #9116: URL: https://github.com/apache/arrow/pull/9116#issuecomment-755945236 @jorgecarleitao yes, definitely! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [arrow] Dandandan commented on pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

2021-01-06 Thread GitBox
Dandandan commented on pull request #9116: URL: https://github.com/apache/arrow/pull/9116#issuecomment-755945052 So what remains is the collision check. This is an automated message from the Apache Git Service. To respond to

[GitHub] [arrow] Dandandan commented on pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

2021-01-06 Thread GitBox
Dandandan commented on pull request #9116: URL: https://github.com/apache/arrow/pull/9116#issuecomment-755944532 The unit tests are succeeding now, had to do with the `RandomState` being reset on every `execute` 🎉 Th

[GitHub] [arrow] jorgecarleitao commented on pull request #9114: ARROW-11149: [Rust] DF Support List/LargeList in create_batch_empty

2021-01-06 Thread GitBox
jorgecarleitao commented on pull request #9114: URL: https://github.com/apache/arrow/pull/9114#issuecomment-755931020 I believe that we can make this with generics, now that we have `GenericListBuilder::()`. It would make the code simpler to read, and a consistent function signature. ---

[GitHub] [arrow] zhztheplayer commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Dataset Java API by JNI to C++

2021-01-06 Thread GitBox
zhztheplayer commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-755902629 @emkornfield And would you mind to take another look on latest code? Thanks a lot. This is an automated mess

[GitHub] [arrow] zhztheplayer commented on pull request #7030: ARROW-7808: [Java][Dataset] Implement Dataset Java API by JNI to C++

2021-01-06 Thread GitBox
zhztheplayer commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-755898896 @pitrou I've moved the pool class out to jni_wrapper.cpp. I saw your comments in https://issues.apache.org/jira/browse/ARROW-11143, thanks for that. Regarding the stati

[GitHub] [arrow] github-actions[bot] commented on pull request #9121: ARROW-11158: [Julia] Implement Decimal256 support for Julia

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9121: URL: https://github.com/apache/arrow/pull/9121#issuecomment-755889561 https://issues.apache.org/jira/browse/ARROW-11158 This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #9121: [ARROW-11158] [Julia] Implement Decimal256 support for Julia

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9121: URL: https://github.com/apache/arrow/pull/9121#issuecomment-755888625 Thanks for opening a pull request! Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Then could

[GitHub] [arrow] quinnj opened a new pull request #9121: [ARROW-11158] [Julia] Implement Decimal256 support for Julia

2021-01-06 Thread GitBox
quinnj opened a new pull request #9121: URL: https://github.com/apache/arrow/pull/9121 This PR also includes a few other bugfixes since the original Julia code donation. The details of the release process of the code here and transition from the JuliaData/Arrow.jl repository are still

[GitHub] [arrow] github-actions[bot] commented on pull request #9120: ARROW-11159: [Developer] Consolidate pull request related jobs

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9120: URL: https://github.com/apache/arrow/pull/9120#issuecomment-755885833 https://issues.apache.org/jira/browse/ARROW-11159 This is an automated message from the Apache Git Ser

[GitHub] [arrow] kou opened a new pull request #9120: ARROW-11159: [Developer] Consolidate pull request related jobs

2021-01-06 Thread GitBox
kou opened a new pull request #9120: URL: https://github.com/apache/arrow/pull/9120 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [arrow] kou commented on pull request #9120: ARROW-11159: [Developer] Consolidate pull request related jobs

2021-01-06 Thread GitBox
kou commented on pull request #9120: URL: https://github.com/apache/arrow/pull/9120#issuecomment-755884354 Demo: https://github.com/kou/arrow/pull/10 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [arrow] arw2019 commented on pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
arw2019 commented on pull request #9115: URL: https://github.com/apache/arrow/pull/9115#issuecomment-755872744 @github-actions crossbow submit test-conda-python-3.7-spark-branch-3.0 This is an automated message from the Apach

[GitHub] [arrow] arw2019 commented on a change in pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
arw2019 commented on a change in pull request #9115: URL: https://github.com/apache/arrow/pull/9115#discussion_r553099694 ## File path: ci/docker/conda-python-spark.dockerfile ## @@ -20,21 +20,15 @@ ARG arch=amd64 ARG python=3.6 FROM ${repo}:${arch}-conda-python-${python} -

[GitHub] [arrow] arw2019 commented on pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
arw2019 commented on pull request #9115: URL: https://github.com/apache/arrow/pull/9115#issuecomment-755871620 > My intention was to test pyarrow against spark branch-3.0 in addition to the current tests that also include arrow Java for the master branch. We still want to maintain that to

[GitHub] [arrow] arw2019 commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
arw2019 commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755869838 @github-actions crossbow submit test-conda-python-3.8-dask-master This is an automated message from the Apache Git

[GitHub] [arrow] jonkeane commented on pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
jonkeane commented on pull request #9118: URL: https://github.com/apache/arrow/pull/9118#issuecomment-755837763 @github-actions crossbow submit test-r-version-compatibility This is an automated message from the Apache Git

[GitHub] [arrow] kou closed pull request #9119: ARROW-11152: [CI][C++] Fix Homebrew numpy installation on macOS builds

2021-01-06 Thread GitBox
kou closed pull request #9119: URL: https://github.com/apache/arrow/pull/9119 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [arrow] jonkeane commented on pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
jonkeane commented on pull request #9118: URL: https://github.com/apache/arrow/pull/9118#issuecomment-755823111 @github-actions crossbow submit test-r-version-compatibility This is an automated message from the Apache Git Ser

[GitHub] [arrow] github-actions[bot] commented on pull request #9119: ARROW-11152: [CI][C++] Fix Homebrew numpy installation on macOS builds

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9119: URL: https://github.com/apache/arrow/pull/9119#issuecomment-755820912 https://issues.apache.org/jira/browse/ARROW-11152 This is an automated message from the Apache Git Ser

[GitHub] [arrow] BryanCutler commented on a change in pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
BryanCutler commented on a change in pull request #9115: URL: https://github.com/apache/arrow/pull/9115#discussion_r553045953 ## File path: ci/docker/conda-python-spark.dockerfile ## @@ -20,21 +20,15 @@ ARG arch=amd64 ARG python=3.6 FROM ${repo}:${arch}-conda-python-${python}

[GitHub] [arrow] github-actions[bot] commented on pull request #9117: ARROW-9187: [R] Add bindings for arithmetic kernels

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9117: URL: https://github.com/apache/arrow/pull/9117#issuecomment-755810799 https://issues.apache.org/jira/browse/ARROW-9187 This is an automated message from the Apache Git Serv

[GitHub] [arrow] seddonm1 commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
seddonm1 commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755792827 > I agree with @seddonm1 that ideally we would follow Postgres when possible so we can just re-use all their documentation and semantics and so we don't end up inadvertently crea

[GitHub] [arrow] kszucs edited a comment on pull request #8881: ARROW-9400: [Python] Do not depend on conda-forge static libraries in Windows wheel builds

2021-01-06 Thread GitBox
kszucs edited a comment on pull request #8881: URL: https://github.com/apache/arrow/pull/8881#issuecomment-755791646 INFRA needs to set a new secret for the bot https://issues.apache.org/jira/browse/INFRA-21280 Until that please use crossbow locally: ```bash # have arrow

[GitHub] [arrow] kszucs commented on pull request #8881: ARROW-9400: [Python] Do not depend on conda-forge static libraries in Windows wheel builds

2021-01-06 Thread GitBox
kszucs commented on pull request #8881: URL: https://github.com/apache/arrow/pull/8881#issuecomment-755791646 INFRA needs to set a new secret for the bot https://issues.apache.org/jira/browse/INFRA-21280 Until that please use crossbow locally: ```bash python dev/tasks/cro

[GitHub] [arrow] andygrove commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
andygrove commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755791235 I am also +1 for choosing Postgres dialect as the default. This is an automated message from the Apache Git Ser

[GitHub] [arrow] nealrichardson commented on pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
nealrichardson commented on pull request #9115: URL: https://github.com/apache/arrow/pull/9115#issuecomment-755784738 @github-actions crossbow submit test-conda-python-3.7-spark-branch-3.0 This is an automated message from th

[GitHub] [arrow] nealrichardson opened a new pull request #9119: ARROW-11152: [CI][C++] Fix Homebrew numpy installation on macOS builds

2021-01-06 Thread GitBox
nealrichardson opened a new pull request #9119: URL: https://github.com/apache/arrow/pull/9119 numpy was failing because installing it triggered an update of gcc, and that failed. This is an automated message from the Apache

[GitHub] [arrow] jonkeane commented on a change in pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
jonkeane commented on a change in pull request #9118: URL: https://github.com/apache/arrow/pull/9118#discussion_r553018847 ## File path: r/tests/testthat/test-parquet-compatibility.R ## @@ -0,0 +1,45 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more co

[GitHub] [arrow] nealrichardson commented on a change in pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
nealrichardson commented on a change in pull request #9118: URL: https://github.com/apache/arrow/pull/9118#discussion_r553016317 ## File path: dev/tasks/r/azure.linux.yml ## @@ -37,6 +37,7 @@ jobs: export R_IMAGE={{ r_image }} export R_TAG={{ r_tag }}

[GitHub] [arrow] alamb commented on a change in pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
alamb commented on a change in pull request #9038: URL: https://github.com/apache/arrow/pull/9038#discussion_r553017434 ## File path: rust/benchmarks/src/bin/tpch.rs ## @@ -656,7 +656,7 @@ fn create_logical_plan(ctx: &mut ExecutionContext, query: usize) -> Result

[GitHub] [arrow] alamb commented on pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
alamb commented on pull request #9038: URL: https://github.com/apache/arrow/pull/9038#issuecomment-755778331 I'll plan to merge this in as soon as the CI passes This is an automated message from the Apache Git Service. To res

[GitHub] [arrow] alamb commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
alamb commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755777874 I agree with @seddonm1 that ideally we would follow Postgres when possible so we can just re-use all their documentation and semantics and so we don't end up inadvertently creating

[GitHub] [arrow] codecov-io edited a comment on pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
codecov-io edited a comment on pull request #9038: URL: https://github.com/apache/arrow/pull/9038#issuecomment-753241531 # [Codecov](https://codecov.io/gh/apache/arrow/pull/9038?src=pr&el=h1) Report > Merging [#9038](https://codecov.io/gh/apache/arrow/pull/9038?src=pr&el=desc) (e4f69e6)

[GitHub] [arrow] jonkeane commented on pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
jonkeane commented on pull request #9118: URL: https://github.com/apache/arrow/pull/9118#issuecomment-755773625 @github-actions crossbow submit test-r-version-compatibility This is an automated message from the Apache Git Ser

[GitHub] [arrow] jonkeane opened a new pull request #9118: [WIP] Arrow 10623: [R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

2021-01-06 Thread GitBox
jonkeane opened a new pull request #9118: URL: https://github.com/apache/arrow/pull/9118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] jorgecarleitao commented on pull request #9111: ARROW-11140: [Rust] [CI] Experimenting with Buildkite

2021-01-06 Thread GitBox
jorgecarleitao commented on pull request #9111: URL: https://github.com/apache/arrow/pull/9111#issuecomment-755766760 Ok, I think that this is now ready for a spin. It took a bit more than I antecipated, but the core work is ready. This basically deprecates all our rust jobs on github, rep

[GitHub] [arrow] jorgecarleitao commented on pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

2021-01-06 Thread GitBox
jorgecarleitao commented on pull request #9116: URL: https://github.com/apache/arrow/pull/9116#issuecomment-755765653 👍 I think that this could be also beneficial for the hash aggregate! This is an automated message from the

[GitHub] [arrow] nealrichardson commented on pull request #8947: ARROW-9187: [R] Add bindings for arithmetic kernels

2021-01-06 Thread GitBox
nealrichardson commented on pull request #8947: URL: https://github.com/apache/arrow/pull/8947#issuecomment-755762668 Rebase wasn't clean so I opened #9117 with the commits cherry-picked. This is an automated message from the

[GitHub] [arrow] nealrichardson closed pull request #8947: ARROW-9187: [R] Add bindings for arithmetic kernels

2021-01-06 Thread GitBox
nealrichardson closed pull request #8947: URL: https://github.com/apache/arrow/pull/8947 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] github-actions[bot] commented on pull request #9116: ARROW-11156: [Rust][DataFusion] Create hashes vectorized in hash join [WIP]

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9116: URL: https://github.com/apache/arrow/pull/9116#issuecomment-755761637 https://issues.apache.org/jira/browse/ARROW-11156 This is an automated message from the Apache Git Ser

[GitHub] [arrow] nealrichardson opened a new pull request #9117: ARROW-9187: [R] Add bindings for arithmetic kernels

2021-01-06 Thread GitBox
nealrichardson opened a new pull request #9117: URL: https://github.com/apache/arrow/pull/9117 Replaces #8947 @jonkeane This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [arrow] nealrichardson closed pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
nealrichardson closed pull request #8894: URL: https://github.com/apache/arrow/pull/8894 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [arrow] bkietz commented on pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on pull request #8894: URL: https://github.com/apache/arrow/pull/8894#issuecomment-755754260 non Apache CI: https://github.com/bkietz/arrow/runs/1659121616 This is an automated message from the Apache Git Ser

[GitHub] [arrow] kou closed pull request #8805: ARROW-10725: [Python][Compute] Expose sort options in Python bindings

2021-01-06 Thread GitBox
kou closed pull request #8805: URL: https://github.com/apache/arrow/pull/8805 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [arrow] terencehonles commented on pull request #8915: ARROW-10904: [Python][CI][Packaging] Add support for Python 3.9 macOS wheels

2021-01-06 Thread GitBox
terencehonles commented on pull request #8915: URL: https://github.com/apache/arrow/pull/8915#issuecomment-755723513 > FWIW I have a mac with macOS 10.14 and I confirmed that I can install the high sierra wheel that this produced. Thanks, hopefully this made the cut for the next rele

[GitHub] [arrow] ovr commented on a change in pull request #9114: ARROW-11149: [Rust] DF Support List/LargeList in create_batch_empty

2021-01-06 Thread GitBox
ovr commented on a change in pull request #9114: URL: https://github.com/apache/arrow/pull/9114#discussion_r552842524 ## File path: rust/arrow/src/array/array_list.rs ## @@ -295,6 +298,185 @@ impl fmt::Debug for FixedSizeListArray { } } +macro_rules! build_empty_list_ar

[GitHub] [arrow] kszucs commented on pull request #9096: [Python][Packaging] Refactor manylinux and windows wheel building [WIP]

2021-01-06 Thread GitBox
kszucs commented on pull request #9096: URL: https://github.com/apache/arrow/pull/9096#issuecomment-755716186 @xhochy we have a [running build](https://github.com/ursa-labs/crossbow/runs/1658737075) for windows on github actions now. I'll do a couple of things to try to improve the build

[GitHub] [arrow] xhochy commented on pull request #8881: ARROW-9400: [Python] Do not depend on conda-forge static libraries in Windows wheel builds

2021-01-06 Thread GitBox
xhochy commented on pull request #8881: URL: https://github.com/apache/arrow/pull/8881#issuecomment-755713376 @github-actions crossbow submit wheel-win-cp38 This is an automated message from the Apache Git Service. To

[GitHub] [arrow] seberg commented on pull request #8834: ARROW-10833: [Python] Allow pyarrow to be compiled on NumPy <1.16.6 and work on 1.20+

2021-01-06 Thread GitBox
seberg commented on pull request #8834: URL: https://github.com/apache/arrow/pull/8834#issuecomment-755697447 We talked about this again, and unfortunately the best we can do is to say that we will release NumPy 1.20 by the end of the month. Assuming your release process goes smoothly, th

[GitHub] [arrow] xhochy commented on pull request #8881: ARROW-9400: [Python] Do not depend on conda-forge static libraries in Windows wheel builds

2021-01-06 Thread GitBox
xhochy commented on pull request #8881: URL: https://github.com/apache/arrow/pull/8881#issuecomment-755689064 I can rebase and see whether there is progress but it might still have issues and I had hopes for #9096 to land in time. --

[GitHub] [arrow] jorisvandenbossche commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
jorisvandenbossche commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755684431 @github-actions crossbow submit test-conda-python-3.8-dask-master This is an automated message from

[GitHub] [arrow] seddonm1 commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
seddonm1 commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755682309 Thanks @jorgecarleitao and @ovr My view would be to elect an initial dialect (my preference being Postgres - https://www.postgresql.org/docs/13/functions-string.html which

[GitHub] [arrow] github-actions[bot] commented on pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9115: URL: https://github.com/apache/arrow/pull/9115#issuecomment-755672614 https://issues.apache.org/jira/browse/ARROW-10457 This is an automated message from the Apache Git Ser

[GitHub] [arrow] nealrichardson commented on pull request #8915: ARROW-10904: [Python][CI][Packaging] Add support for Python 3.9 macOS wheels

2021-01-06 Thread GitBox
nealrichardson commented on pull request #8915: URL: https://github.com/apache/arrow/pull/8915#issuecomment-755665794 FWIW I have a mac with macOS 10.14 and I confirmed that I can install the high sierra wheel that this produced. ---

[GitHub] [arrow] seddonm1 commented on pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
seddonm1 commented on pull request #9038: URL: https://github.com/apache/arrow/pull/9038#issuecomment-755664456 @alamb thanks for taking the time to review this as I know it ended up as quite a large PR 👍 . I have updated based on your comment. @yordan-pavlov yes this is basically as

[GitHub] [arrow] seddonm1 commented on a change in pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
seddonm1 commented on a change in pull request #9038: URL: https://github.com/apache/arrow/pull/9038#discussion_r552950902 ## File path: rust/datafusion/src/optimizer/utils.rs ## @@ -305,6 +312,7 @@ pub fn expr_sub_expressions(expr: &Expr) -> Result> { low.as_ref(

[GitHub] [arrow] seddonm1 commented on a change in pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
seddonm1 commented on a change in pull request #9038: URL: https://github.com/apache/arrow/pull/9038#discussion_r552950551 ## File path: rust/datafusion/src/physical_plan/expressions.rs ## @@ -3769,4 +4002,166 @@ mod tests { let batch = RecordBatch::try_new(Arc::new(sc

[GitHub] [arrow] kou commented on pull request #8915: ARROW-10904: [Python][CI][Packaging] Add support for Python 3.9 macOS wheels

2021-01-06 Thread GitBox
kou commented on pull request #8915: URL: https://github.com/apache/arrow/pull/8915#issuecomment-755656441 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [arrow] kou closed pull request #8915: ARROW-10904: [Python][CI][Packaging] Add support for Python 3.9 macOS wheels

2021-01-06 Thread GitBox
kou closed pull request #8915: URL: https://github.com/apache/arrow/pull/8915 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [arrow] seddonm1 commented on a change in pull request #9038: ARROW-10356: [Rust][DataFusion] Add support for is_in (WIP)

2021-01-06 Thread GitBox
seddonm1 commented on a change in pull request #9038: URL: https://github.com/apache/arrow/pull/9038#discussion_r552947771 ## File path: rust/datafusion/src/optimizer/utils.rs ## @@ -416,6 +424,7 @@ pub fn rewrite_expression(expr: &Expr, expressions: &Vec) -> Result

[GitHub] [arrow] github-actions[bot] commented on pull request #9114: ARROW-11149: [Rust] DF Support List/LargeList in create_batch_empty

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9114: URL: https://github.com/apache/arrow/pull/9114#issuecomment-755643977 https://issues.apache.org/jira/browse/ARROW-11149 This is an automated message from the Apache Git Ser

[GitHub] [arrow] Dandandan opened a new pull request #9116: ARROW-11156: [Rust][DataFusion] Vectorized hashing [WIP

2021-01-06 Thread GitBox
Dandandan opened a new pull request #9116: URL: https://github.com/apache/arrow/pull/9116 Create hashes vectorized in hash join This is based on the open PR https://github.com/apache/arrow/pull/9070 The idea is as follows: * We still use the `HashMap` but rather than usi

[GitHub] [arrow] nealrichardson commented on pull request #8881: ARROW-9400: [Python] Do not depend on conda-forge static libraries in Windows wheel builds

2021-01-06 Thread GitBox
nealrichardson commented on pull request #8881: URL: https://github.com/apache/arrow/pull/8881#issuecomment-755588630 @xhochy what's the status of this? Fixing windows wheels is one of our (several) release blockers. Is rebase all that is needed to be able to merge this?

[GitHub] [arrow] balancap commented on pull request #8977: ARROW-10675: [C++][Python] Support AWS S3 web identity credentials.

2021-01-06 Thread GitBox
balancap commented on pull request #8977: URL: https://github.com/apache/arrow/pull/8977#issuecomment-755570648 @pitrou Thanks for checking the MR! I'll push some modifications following your comments, and look more closely at the CI failures in the next couple of days. -

[GitHub] [arrow] github-actions[bot] commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
github-actions[bot] commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755471367 https://issues.apache.org/jira/browse/ARROW-11147 This is an automated message from the Apache Git Ser

[GitHub] [arrow] Dandandan edited a comment on pull request #8697: ARROW-10633: [Rust][DataFusion] Dependency version updates

2021-01-06 Thread GitBox
Dandandan edited a comment on pull request #8697: URL: https://github.com/apache/arrow/pull/8697#issuecomment-755468767 @rdettai rusoto just released a new version with 1.0 , tonic has an open PR for the upgrade. Any other dependencies we should look out for when upgrading to 1.0? --

[GitHub] [arrow] Dandandan commented on pull request #8697: ARROW-10633: [Rust][DataFusion] Dependency version updates

2021-01-06 Thread GitBox
Dandandan commented on pull request #8697: URL: https://github.com/apache/arrow/pull/8697#issuecomment-755468767 @rdettai rustoto just released a new version with 1.0 , tonic has an open PR for the upgrade. Any other dependencies we should look out for when upgrading to 1.0?

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552875692 ## File path: cpp/src/arrow/compute/kernels/scalar_cast_internal.h ## @@ -62,6 +63,13 @@ void AddSimpleCast(InputType in_ty, OutputType out_ty, CastFunction

[GitHub] [arrow] arw2019 commented on pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
arw2019 commented on pull request #9115: URL: https://github.com/apache/arrow/pull/9115#issuecomment-755461385 @github-actions crossbow submit test-conda-python-3.7-spark-branch-3.0 This is an automated message from the Apac

[GitHub] [arrow] arw2019 opened a new pull request #9115: ARROW-10457: [CI] Fix Spark branch-3.0 integration tests

2021-01-06 Thread GitBox
arw2019 opened a new pull request #9115: URL: https://github.com/apache/arrow/pull/9115 cc @BryanCutler This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552856136 ## File path: cpp/src/arrow/type.h ## @@ -1517,6 +1517,12 @@ class ARROW_EXPORT FieldRef { std::string ToString() const; size_t hash() const; + stru

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552849670 ## File path: cpp/src/arrow/compute/cast.cc ## @@ -118,8 +118,86 @@ class CastMetaFunction : public MetaFunction { } // namespace +const FunctionDoc pr

[GitHub] [arrow] jorgecarleitao commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
jorgecarleitao commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755451875 @andygrove @alamb , opinions? My (weak) opinion is that as long as we do not conflict with ANSI SQL (in that we offer a conflicting or incomplete spec), I do not see

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552846350 ## File path: cpp/src/arrow/compute/kernels/scalar_cast_test.cc ## @@ -1801,6 +1801,10 @@ TYPED_TEST(TestDictionaryCast, Basic) { // TODO: Should castin

[GitHub] [arrow] ovr commented on pull request #9108: ARROW-11138: [Rust] Add ltrim, rtrim to built-in functions

2021-01-06 Thread GitBox
ovr commented on pull request #9108: URL: https://github.com/apache/arrow/pull/9108#issuecomment-755449354 @seddonm1 > Hopefully this can drive a decision on what dialect to support. Should I create an issue to discuss it or mailing list? Is there any channel for communicatio

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552843114 ## File path: cpp/src/arrow/dataset/partition.h ## @@ -294,5 +285,22 @@ class ARROW_DS_EXPORT PartitioningOrFactory { std::shared_ptr partitioning_; };

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552842762 ## File path: cpp/src/arrow/dataset/partition.cc ## @@ -573,5 +530,192 @@ Result> PartitioningOrFactory::GetOrInferSchema( return factory()->Inspect(path

[GitHub] [arrow] ovr commented on a change in pull request #9114: ARROW-11149: [Rust] DF Support List/LargeList in create_batch_empty

2021-01-06 Thread GitBox
ovr commented on a change in pull request #9114: URL: https://github.com/apache/arrow/pull/9114#discussion_r552842524 ## File path: rust/arrow/src/array/array_list.rs ## @@ -295,6 +298,185 @@ impl fmt::Debug for FixedSizeListArray { } } +macro_rules! build_empty_list_ar

[GitHub] [arrow] ovr opened a new pull request #9114: ARROW-11149: [Rust] DF Support List/LargeList in create_batch_empty

2021-01-06 Thread GitBox
ovr opened a new pull request #9114: URL: https://github.com/apache/arrow/pull/9114 Hello! Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552840772 ## File path: cpp/src/arrow/dataset/expression.h ## @@ -0,0 +1,245 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

[GitHub] [arrow] jbrockmendel commented on a change in pull request #8957: ARROW-10768: [Python] pass ndim to pandas make_block

2021-01-06 Thread GitBox
jbrockmendel commented on a change in pull request #8957: URL: https://github.com/apache/arrow/pull/8957#discussion_r552840598 ## File path: python/pyarrow/pandas_compat.py ## @@ -730,15 +733,18 @@ def _reconstruct_block(item, columns=None, extension_columns=None):

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552837218 ## File path: cpp/src/arrow/compute/kernels/util_internal.h ## @@ -50,6 +54,12 @@ int GetBitWidth(const DataType& type); // rather than duplicating compiled

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552837041 ## File path: cpp/src/arrow/compute/kernels/util_internal.h ## @@ -18,8 +18,12 @@ #pragma once #include +#include Review comment: No, will remo

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552836459 ## File path: cpp/src/arrow/compute/kernels/scalar_project_test.cc ## @@ -0,0 +1,122 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552835857 ## File path: cpp/src/arrow/compute/kernels/scalar_cast_temporal.cc ## @@ -256,14 +251,47 @@ struct CastFunctor { template <> struct CastFunctor { stati

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552834257 ## File path: cpp/src/arrow/dataset/partition.cc ## @@ -573,5 +530,192 @@ Result> PartitioningOrFactory::GetOrInferSchema( return factory()->Inspect(path

[GitHub] [arrow] pitrou commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
pitrou commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552790445 ## File path: cpp/src/arrow/compute/kernels/scalar_cast_internal.h ## @@ -62,6 +63,13 @@ void AddSimpleCast(InputType in_ty, OutputType out_ty, CastFunction

[GitHub] [arrow] terencehonles commented on a change in pull request #8915: ARROW-10904: [Python][CI][Packaging] Add support for Python 3.9 macOS wheels

2021-01-06 Thread GitBox
terencehonles commented on a change in pull request #8915: URL: https://github.com/apache/arrow/pull/8915#discussion_r552826019 ## File path: dev/tasks/python-wheels/github.osx.yml ## @@ -0,0 +1,143 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more con

[GitHub] [arrow] jorisvandenbossche commented on pull request #8957: ARROW-10768: [Python] pass ndim to pandas make_block

2021-01-06 Thread GitBox
jorisvandenbossche commented on pull request #8957: URL: https://github.com/apache/arrow/pull/8957#issuecomment-755432871 @jbrockmendel Do you want to update this PR? This is an automated message from the Apache Git Service.

[GitHub] [arrow] jorisvandenbossche commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
jorisvandenbossche commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755430432 @github-actions crossbow submit test-conda-python-3.8-dask-master test-conda-python-3.7-dask-latest Th

[GitHub] [arrow] jorisvandenbossche commented on pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
jorisvandenbossche commented on pull request #8894: URL: https://github.com/apache/arrow/pull/8894#issuecomment-755427844 (I reviewed the minimal python changes, which look good, and looked at part of the C++ dataset changes, but no need to wait on further review from my side) ---

[GitHub] [arrow] bkietz commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
bkietz commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552797920 ## File path: python/pyarrow/tests/parquet/test_dataset.py ## @@ -509,7 +509,7 @@ def test_filters_invalid_column(tempdir, use_legacy_dataset): _gene

[GitHub] [arrow] jorisvandenbossche commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
jorisvandenbossche commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755413336 @github-actions crossbow submit test-conda-python-3.7-dask-master test-conda-python-3.7-dask-latest ---

[GitHub] [arrow] arw2019 commented on pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
arw2019 commented on pull request #9113: URL: https://github.com/apache/arrow/pull/9113#issuecomment-755411788 cc @jorisvandenbossche This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [arrow] arw2019 opened a new pull request #9113: ARROW-11147: [CI][Python] pin pandas=0.25.3 for dask-master

2021-01-06 Thread GitBox
arw2019 opened a new pull request #9113: URL: https://github.com/apache/arrow/pull/9113 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [arrow] pitrou commented on a change in pull request #8894: ARROW-10322: [C++][Dataset] Minimize Expression

2021-01-06 Thread GitBox
pitrou commented on a change in pull request #8894: URL: https://github.com/apache/arrow/pull/8894#discussion_r552773181 ## File path: python/pyarrow/tests/parquet/test_dataset.py ## @@ -509,7 +509,7 @@ def test_filters_invalid_column(tempdir, use_legacy_dataset): _gene

[GitHub] [arrow] nealrichardson edited a comment on pull request #8650: ARROW-10530: [R] Use Converter API to convert SEXP to Array/ChunkedArray

2021-01-06 Thread GitBox
nealrichardson edited a comment on pull request #8650: URL: https://github.com/apache/arrow/pull/8650#issuecomment-754955102 Since the latest commits aren't compiling, I did some benchmarking on https://github.com/apache/arrow/pull/8650/commits/bcb1be733697b0e7ca86534a6700b5816e0dad46. Sum

[GitHub] [arrow] nealrichardson commented on pull request #8650: ARROW-10530: [R] Use Converter API to convert SEXP to Array/ChunkedArray

2021-01-06 Thread GitBox
nealrichardson commented on pull request #8650: URL: https://github.com/apache/arrow/pull/8650#issuecomment-755396635 > the logic behind having a `start` was to facilitate one R vector contributing to two different chunks in the chunker ... Right, we want to be able to chunk the inpu

[GitHub] [arrow] pitrou closed pull request #9091: ARROW-11024: [Python] Add test for List data Parquet roundtrip

2021-01-06 Thread GitBox
pitrou closed pull request #9091: URL: https://github.com/apache/arrow/pull/9091 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   >