[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13204: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (--doctest-cython)

2022-05-23 Thread GitBox
jorisvandenbossche commented on code in PR #13204: URL: https://github.com/apache/arrow/pull/13204#discussion_r880112592 ## python/pyarrow/_compute.pyx: ## @@ -2036,15 +2036,16 @@ cdef class Expression(_Weakrefable): >>> import pyarrow.compute as pc >>> (pc.field("a"

[GitHub] [arrow-datafusion] waynexia commented on pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#issuecomment-1135469545 Thanks for all the reviews and information! >I think it is important to understand what cranelift is, and what it isn't. Cranelift is a code generator originally intended

[GitHub] [arrow] jorisvandenbossche commented on a diff in pull request #13199: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (--doctest-modules)

2022-05-23 Thread GitBox
jorisvandenbossche commented on code in PR #13199: URL: https://github.com/apache/arrow/pull/13199#discussion_r880100079 ## python/pyarrow/dataset.py: ## @@ -622,62 +631,74 @@ def dataset(source, schema=None, format=None, filesystem=None, Examples +Crea

[GitHub] [arrow] AlenkaF commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)

2022-05-23 Thread GitBox
AlenkaF commented on PR #13216: URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135460968 Planning to correct the CI errors asap. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow-rs] Ted-Jiang commented on issue #1705: Parquet: Read/write support for PageIndex (ColumnIndex / OffsetIndex)

2022-05-23 Thread GitBox
Ted-Jiang commented on issue #1705: URL: https://github.com/apache/arrow-rs/issues/1705#issuecomment-1135454503 > Hi @Ted-Jiang, I think the first step will be to support reading and writing the page index. Once that is done, we can look to include support in #1605. I'm currently actively w

[GitHub] [arrow] jorisvandenbossche commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)

2022-05-23 Thread GitBox
jorisvandenbossche commented on PR #13216: URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135454063 I am wondering if we can include the doctests in the existing sphinx doc build as a separate step, to avoid adding yet another build in the regular CI (also, setting up the doc

[GitHub] [arrow] jorisvandenbossche commented on pull request #13216: ARROW-16018: [Doc][Python] Run doctests on Python docstring examples (CI job)

2022-05-23 Thread GitBox
jorisvandenbossche commented on PR #13216: URL: https://github.com/apache/arrow/pull/13216#issuecomment-1135452250 CI shows: ``` ValueError: Found errors with docker-compose: - Service `conda-python-doctest` is defined in `services` but not in `x-hierarchy` ``` -- This i

[GitHub] [arrow-rs] tustvold commented on issue #1705: Parquet: Read/write support for PageIndex (ColumnIndex / OffsetIndex)

2022-05-23 Thread GitBox
tustvold commented on issue #1705: URL: https://github.com/apache/arrow-rs/issues/1705#issuecomment-1135450037 Hi @Ted-Jiang, I think the first step will be to support reading and writing the page index. Once that is done, we can look to include support in #1605. I'm currently actively work

[GitHub] [arrow-datafusion] waynexia commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r880084522 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

[GitHub] [arrow-rs] viirya opened a new issue, #1734: Latest nightly fails to build with feature simd

2022-05-23 Thread GitBox
viirya opened a new issue, #1734: URL: https://github.com/apache/arrow-rs/issues/1734 **Describe the bug** After upgrading to latest nightly toolchain, `cargo build` failed. **To Reproduce** `cargo +nightly build --all --features simd` ``` error: expected one of `!`

[GitHub] [arrow] cyb70289 commented on pull request #13222: MINOR: [C++] Move static declaration to non-static declaration to improve performance

2022-05-23 Thread GitBox
cyb70289 commented on PR #13222: URL: https://github.com/apache/arrow/pull/13222#issuecomment-1135425796 appveyor test failure may deserve deeper investigation, though unlikely related to this pr: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/43639301/job/n3qanqaq

[GitHub] [arrow] pravindra closed pull request #13124: ARROW-16634: [Gandiva][C++] Add udfdegrees alias

2022-05-23 Thread GitBox
pravindra closed pull request #13124: ARROW-16634: [Gandiva][C++] Add udfdegrees alias URL: https://github.com/apache/arrow/pull/13124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [arrow-datafusion] waynexia commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r880063760 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

[GitHub] [arrow-rs] Ted-Jiang commented on issue #1705: Parquet: Read/write support for PageIndex (ColumnIndex / OffsetIndex)

2022-05-23 Thread GitBox
Ted-Jiang commented on issue #1705: URL: https://github.com/apache/arrow-rs/issues/1705#issuecomment-1135415886 @alamb @tustvold Our team is building a query engine which is sensitive to time-to-first-result latency(bottleneck is scan and agg). So we want to support PageIndex in arrow-r

[GitHub] [arrow-datafusion] waynexia commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r880054517 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

[GitHub] [arrow-datafusion] waynexia commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r880053266 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

[GitHub] [arrow-datafusion] waynexia commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
waynexia commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r880053159 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreement

[GitHub] [arrow-rs] Ted-Jiang commented on issue #1715: Why `Parquet` is a part of `Arrow`?

2022-05-23 Thread GitBox
Ted-Jiang commented on issue #1715: URL: https://github.com/apache/arrow-rs/issues/1715#issuecomment-1135400045 >Also, I think fast conversion between arrow <--> parquet is important and having them in the same repo may help with that. +1 👍 @alamb And i have a question, if parquet

[GitHub] [arrow] ursabot commented on pull request #13217: MINOR: [Docs] Update phrasing of intro sentence in the new contributors guide

2022-05-23 Thread GitBox
ursabot commented on PR #13217: URL: https://github.com/apache/arrow/pull/13217#issuecomment-1135394722 Benchmark runs are scheduled for baseline = 119ead4346d8749f793476a182f1cc55c7d6401b and contender = 43a604de6b44394f99900a3ae9e869d3c11670eb. 43a604de6b44394f99900a3ae9e869d3c11670eb is

[GitHub] [arrow-ballista] andygrove merged pull request #34: MINOR: Improve the examples

2022-05-23 Thread GitBox
andygrove merged PR #34: URL: https://github.com/apache/arrow-ballista/pull/34 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.

[GitHub] [arrow] cyb70289 commented on a diff in pull request #13222: MINOR: [C++] Move static declaration to non-static declaration to improve performance

2022-05-23 Thread GitBox
cyb70289 commented on code in PR #13222: URL: https://github.com/apache/arrow/pull/13222#discussion_r880034314 ## cpp/src/arrow/compute/exec/expression.cc: ## @@ -107,9 +107,10 @@ ValueDescr Expression::descr() const { return CallNotNull(*this)->descr; } +const std::shared

[GitHub] [arrow] cyb70289 opened a new pull request, #13223: MINOR: [C++] Mark ChunkResolver move constructor noexcept

2022-05-23 Thread GitBox
cyb70289 opened a new pull request, #13223: URL: https://github.com/apache/arrow/pull/13223 It also makes ChunkedArray nothrow move constructible as it contains a ChunkResolver data member. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [arrow-datafusion] waynexia commented on pull request #2604: chore(doc) remove ballista from datafusion-cli readme

2022-05-23 Thread GitBox
waynexia commented on PR #2604: URL: https://github.com/apache/arrow-datafusion/pull/2604#issuecomment-1135345163 Thanks for this improvement. Could you please remove this paragraph https://github.com/apache/arrow-datafusion/blob/master/datafusion-cli/README.md#ballista as well. And I supp

[GitHub] [arrow-datafusion] ming535 commented on pull request #2521: Support for OFFSET in LogicalPlan

2022-05-23 Thread GitBox
ming535 commented on PR #2521: URL: https://github.com/apache/arrow-datafusion/pull/2521#issuecomment-1135341256 Hi, why `limit` be an input of `offset`? From my understanding, the `offset` is the starting point for where data is coming from, so I think `offset` should be an input of `limi

[GitHub] [arrow-datafusion] ming535 commented on pull request #2521: Support for OFFSET in LogicalPlan

2022-05-23 Thread GitBox
ming535 commented on PR #2521: URL: https://github.com/apache/arrow-datafusion/pull/2521#issuecomment-1135341257 Hi, why `limit` be an input of `offset`? From my understanding, the `offset` is the starting point for where data is coming from, so I think `offset` should be an input of `limi

[GitHub] [arrow-datafusion] ming535 opened a new pull request, #2604: chore(doc) remove ballista from datafusion-cli readme

2022-05-23 Thread GitBox
ming535 opened a new pull request, #2604: URL: https://github.com/apache/arrow-datafusion/pull/2604 # Which issue does this PR close? NONE # Rationale for this change improve documentation # What changes are included in this PR? - remove ballista from datafusion-cli re

[GitHub] [arrow-datafusion] alanpaulkwan opened a new issue, #2603: R package

2022-05-23 Thread GitBox
alanpaulkwan opened a new issue, #2603: URL: https://github.com/apache/arrow-datafusion/issues/2603 Dear Arrow DataFusion If I understand what you're doing, it would be phenomenal to extend this to R. the R community is significant in data science and arrow is definitely a game chang

[GitHub] [arrow] ursabot commented on pull request #13178: ARROW-16509: [R][Docs] Make corrections to datasets vignette

2022-05-23 Thread GitBox
ursabot commented on PR #13178: URL: https://github.com/apache/arrow/pull/13178#issuecomment-1135322321 Benchmark runs are scheduled for baseline = 9719b374408cfd37087f481c8e3f3a98fc89a3a8 and contender = 119ead4346d8749f793476a182f1cc55c7d6401b. 119ead4346d8749f793476a182f1cc55c7d6401b is

[GitHub] [arrow] ursabot commented on pull request #13222: MINOR: [C++] Move static declaration to non-static declaration to improve performance

2022-05-23 Thread GitBox
ursabot commented on PR #13222: URL: https://github.com/apache/arrow/pull/13222#issuecomment-1135319489 Benchmark runs are scheduled for baseline = 43a604de6b44394f99900a3ae9e869d3c11670eb and contender = 9ab685a2735e8b874aec866075ce9054ab1fe6f2. Results will be available as each benchmark

[GitHub] [arrow] westonpace commented on pull request #13222: MINOR: [C++] Move static declaration to non-static declaration to improve performance

2022-05-23 Thread GitBox
westonpace commented on PR #13222: URL: https://github.com/apache/arrow/pull/13222#issuecomment-1135319448 @ursabot please benchmark lang=R -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [arrow] westonpace opened a new pull request, #13222: MINOR: [C++] Move static declaration to non-static declaration to improve performance

2022-05-23 Thread GitBox
westonpace opened a new pull request, #13222: URL: https://github.com/apache/arrow/pull/13222 According to conbench there was a slight regression on #12957 . Poking around a bit it seems that a static local variable is implemented using some kind of global lock (__cxa_guard_acquire / __cxa

[GitHub] [arrow-rs] kazuk opened a new issue, #1733: prebuilt binary package for release.

2022-05-23 Thread GitBox
kazuk opened a new issue, #1733: URL: https://github.com/apache/arrow-rs/issues/1733 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

[GitHub] [arrow-rs] kazuk opened a new pull request, #1732: `cargo install` installs not globally

2022-05-23 Thread GitBox
kazuk opened a new pull request, #1732: URL: https://github.com/apache/arrow-rs/pull/1732 # Which issue does this PR close? Closes #1710. # Rationale for this change # What changes are included in this PR? remove `globally` word. # Are ther

[GitHub] [arrow] github-actions[bot] commented on pull request #13221: ARROW-16638: [Go][Parquet] Fix boolean column skip

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13221: URL: https://github.com/apache/arrow/pull/13221#issuecomment-1135249208 :warning: Ticket **has not been started in JIRA**, please click 'Start Progress'. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow] github-actions[bot] commented on pull request #13221: ARROW-16638: [Go][Parquet] Fix boolean column skip

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13221: URL: https://github.com/apache/arrow/pull/13221#issuecomment-1135249189 https://issues.apache.org/jira/browse/ARROW-16638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] mdepero opened a new pull request, #13221: ARROW-16638: [Go][Parquet] Fix boolean column skip

2022-05-23 Thread GitBox
mdepero opened a new pull request, #13221: URL: https://github.com/apache/arrow/pull/13221 Uses `nil` for `defLvls` and `repLvls` when skipping boolean values, since the scratch buffer allocated for n boolean values when skipping is not large enough to hold n def and rep levels, resulting i

[GitHub] [arrow] ursabot commented on pull request #13137: ARROW-16557: [Go] Enable Slicing memory.Buffer objects

2022-05-23 Thread GitBox
ursabot commented on PR #13137: URL: https://github.com/apache/arrow/pull/13137#issuecomment-1135244675 Benchmark runs are scheduled for baseline = d829ab1d5fca7b38517b54983464b595eb0d64d2 and contender = 9719b374408cfd37087f481c8e3f3a98fc89a3a8. 9719b374408cfd37087f481c8e3f3a98fc89a3a8 is

[GitHub] [arrow] viirya commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
viirya commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1135232824 Hmm, I ran `archery integration --with-cpp=true --with-rust=true` locally without any issue. I'll look into the difference. -- This is an automated message from the Apache Git Service. T

[GitHub] [arrow] assignUser commented on pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on PR #13149: URL: https://github.com/apache/arrow/pull/13149#issuecomment-1135210352 @nealrichardson @jonkeane @kszucs This is now ready for re-review (the [run above](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-f042596390-github-r-nightly-pa

[GitHub] [arrow] assignUser commented on a diff in pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on code in PR #13149: URL: https://github.com/apache/arrow/pull/13149#discussion_r879916387 ## dev/tasks/r/github.nightly.yml: ## @@ -0,0 +1,381 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: To expand on this: I think i

[GitHub] [arrow] assignUser commented on a diff in pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on code in PR #13149: URL: https://github.com/apache/arrow/pull/13149#discussion_r879915341 ## dev/tasks/r/github.nightly.yml: ## @@ -0,0 +1,381 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

[GitHub] [arrow] westonpace commented on pull request #13218: ARROW-16590: [C++] Consolidate files dealing with row-major storage

2022-05-23 Thread GitBox
westonpace commented on PR #13218: URL: https://github.com/apache/arrow/pull/13218#issuecomment-1135203213 CI failures appear unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [arrow] westonpace closed pull request #13172: ARROW-16590: [C++] Consolidate files dealing with row-major storage, add some helper methods

2022-05-23 Thread GitBox
westonpace closed pull request #13172: ARROW-16590: [C++] Consolidate files dealing with row-major storage, add some helper methods URL: https://github.com/apache/arrow/pull/13172 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [arrow] assignUser commented on a diff in pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on code in PR #13149: URL: https://github.com/apache/arrow/pull/13149#discussion_r879914668 ## dev/tasks/r/github.nightly.yml: ## @@ -0,0 +1,424 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

[GitHub] [arrow] westonpace commented on pull request #13172: ARROW-16590: [C++] Consolidate files dealing with row-major storage, add some helper methods

2022-05-23 Thread GitBox
westonpace commented on PR #13172: URL: https://github.com/apache/arrow/pull/13172#issuecomment-1135202233 The second half of this PR has been moved to #13320 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [arrow] github-actions[bot] commented on pull request #13220: ARROW-16637: [C++] Add row-based utilities for encoding a batch and merging row tables

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13220: URL: https://github.com/apache/arrow/pull/13220#issuecomment-1135201517 https://issues.apache.org/jira/browse/ARROW-16637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] westonpace opened a new pull request, #13220: ARROW-16637: [C++] Add row-based utilities for encoding a batch and merging row tables

2022-05-23 Thread GitBox
westonpace opened a new pull request, #13220: URL: https://github.com/apache/arrow/pull/13220 This PR incorporates a part of https://github.com/apache/arrow/pull/12326 , breaking up the larger PR and making it easier to merge. -- This is an automated message from the Apache Git Servic

[GitHub] [arrow] assignUser commented on a diff in pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on code in PR #13149: URL: https://github.com/apache/arrow/pull/13149#discussion_r879913646 ## dev/tasks/r/github.nightly.yml: ## @@ -0,0 +1,467 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See

[GitHub] [arrow] viirya commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
viirya commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1135175310 cc @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[GitHub] [arrow] github-actions[bot] commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1135165525 :warning: Ticket **has not been started in JIRA**, please click 'Start Progress'. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow] github-actions[bot] commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1135165516 :warning: Ticket **has no components in JIRA**, make sure you assign one. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [arrow] github-actions[bot] commented on pull request #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13219: URL: https://github.com/apache/arrow/pull/13219#issuecomment-1135165497 https://issues.apache.org/jira/browse/ARROW-16636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] viirya opened a new pull request, #13219: ARROW-16636: [Rust] Activate several IPC integration tests for rust

2022-05-23 Thread GitBox
viirya opened a new pull request, #13219: URL: https://github.com/apache/arrow/pull/13219 arrow-rs has fixed severals integration test failures: generate_decimal128_case generate_interval_case generate_map_case generate_non_canonical_map_case generate_nested_large_offsets_c

[GitHub] [arrow-rs] viirya commented on pull request #1731: Fix schema comparison for non_canonical_map when running flight test

2022-05-23 Thread GitBox
viirya commented on PR #1731: URL: https://github.com/apache/arrow-rs/pull/1731#issuecomment-1135162614 Thanks @tustvold. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [arrow-rs] viirya merged pull request #1731: Fix schema comparison for non_canonical_map when running flight test

2022-05-23 Thread GitBox
viirya merged PR #1731: URL: https://github.com/apache/arrow-rs/pull/1731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apach

[GitHub] [arrow-rs] viirya closed issue #1730: Fix schema comparison for non_canonical_map when running flight test

2022-05-23 Thread GitBox
viirya closed issue #1730: Fix schema comparison for non_canonical_map when running flight test URL: https://github.com/apache/arrow-rs/issues/1730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow-datafusion] andygrove opened a new pull request, #2602: Remove `ExecutionProps` from `OptimizerRule` trait

2022-05-23 Thread GitBox
andygrove opened a new pull request, #2602: URL: https://github.com/apache/arrow-datafusion/pull/2602 # Which issue does this PR close? Part of https://github.com/apache/arrow-datafusion/issues/2599 and https://github.com/apache/arrow-datafusion/issues/2535 # Rationale

[GitHub] [arrow] ursabot commented on pull request #13132: ARROW-16551: [Go] Improve Temporal Types

2022-05-23 Thread GitBox
ursabot commented on PR #13132: URL: https://github.com/apache/arrow/pull/13132#issuecomment-1135149447 Benchmark runs are scheduled for baseline = 442b24b0b9cf11d45245564fa7448f1ca4931ae7 and contender = d829ab1d5fca7b38517b54983464b595eb0d64d2. d829ab1d5fca7b38517b54983464b595eb0d64d2 is

[GitHub] [arrow] thisisnic closed pull request #13217: MINOR: [Docs] Update phrasing of intro sentence in the new contributors guide

2022-05-23 Thread GitBox
thisisnic closed pull request #13217: MINOR: [Docs] Update phrasing of intro sentence in the new contributors guide URL: https://github.com/apache/arrow/pull/13217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [arrow] thisisnic closed pull request #13178: ARROW-16509: [R][Docs] Make corrections to datasets vignette

2022-05-23 Thread GitBox
thisisnic closed pull request #13178: ARROW-16509: [R][Docs] Make corrections to datasets vignette URL: https://github.com/apache/arrow/pull/13178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] lidavidm commented on pull request #13109: ARROW-15365: [Python] Expose full cast options in the pyarrow.compute.cast function

2022-05-23 Thread GitBox
lidavidm commented on PR #13109: URL: https://github.com/apache/arrow/pull/13109#issuecomment-1135141706 I would still like to see a test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [arrow] westonpace commented on pull request #13172: ARROW-16590: [C++] Consolidate files dealing with row-major storage, add some helper methods

2022-05-23 Thread GitBox
westonpace commented on PR #13172: URL: https://github.com/apache/arrow/pull/13172#issuecomment-1135136130 Per an offline request from @bkietz I've split this PR in two. The first PR is #13218 and contains only the refactoring. The changes & new utilities will be put into a future PR. -

[GitHub] [arrow] github-actions[bot] commented on pull request #13218: ARROW-16590: [C++] Consolidate files dealing with row-major storage

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13218: URL: https://github.com/apache/arrow/pull/13218#issuecomment-1135135704 Thanks for opening a pull request! If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue

[GitHub] [arrow] westonpace opened a new pull request, #13218: ARROW-16590: [C++] Consolidate files dealing with row-major storage

2022-05-23 Thread GitBox
westonpace opened a new pull request, #13218: URL: https://github.com/apache/arrow/pull/13218 The primary goal of this refactor of old code was to improve the readability and clarity of the code base. I did not make any functional changes to the code and if any functional changes are sugges

[GitHub] [arrow-datafusion] viirya commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
viirya commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r879852643 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

[GitHub] [arrow-datafusion] viirya commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
viirya commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r879851451 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

[GitHub] [arrow-datafusion] korowa commented on a diff in pull request #2591: Optional filter in `JOIN ON` clause

2022-05-23 Thread GitBox
korowa commented on code in PR #2591: URL: https://github.com/apache/arrow-datafusion/pull/2591#discussion_r879834683 ## datafusion/core/src/physical_plan/hash_join.rs: ## @@ -791,6 +826,109 @@ fn build_join_indexes( } } +fn apply_join_filter( +left: &RecordBatch, +

[GitHub] [arrow-datafusion] korowa commented on pull request #2591: Optional filter in `JOIN ON` clause

2022-05-23 Thread GitBox
korowa commented on PR #2591: URL: https://github.com/apache/arrow-datafusion/pull/2591#issuecomment-1135091891 > @korowa Great to see this happening! How do you think to support filter in `SortMergeJoinExec` as well? > > Cc @richox, you might be interested in this as well. I'

[GitHub] [arrow-datafusion] korowa commented on pull request #2591: Optional filter in `JOIN ON` clause

2022-05-23 Thread GitBox
korowa commented on PR #2591: URL: https://github.com/apache/arrow-datafusion/pull/2591#issuecomment-1135089747 > Thank you @korowa for being the first to test out the new process around building against Ballista with changes there as well. Please let me know if you have any feedback on th

[GitHub] [arrow] westonpace commented on pull request #13215: ARROW-16633: [C++] Fix incorrect Decimal literal conversion in `ir_consumer.cc`

2022-05-23 Thread GitBox
westonpace commented on PR #13215: URL: https://github.com/apache/arrow/pull/13215#issuecomment-1135086173 I have no problem with the execution engine having multiple serialization formats. However, I don't personally have any time or resources to spend on this IR so my vote would be to dr

[GitHub] [arrow] westonpace commented on a diff in pull request #13078: ARROW-15590: [C++] Add support for joins to the Substrait consumer

2022-05-23 Thread GitBox
westonpace commented on code in PR #13078: URL: https://github.com/apache/arrow/pull/13078#discussion_r879814219 ## cpp/src/arrow/engine/substrait/relation_internal.cc: ## @@ -225,6 +225,76 @@ Result FromProto(const substrait::Rel& rel, }); } +case substrait::

[GitHub] [arrow] nealrichardson commented on a diff in pull request #13090: ARROW-15622: [R] Implement union_all and union for arrow_dplyr_query

2022-05-23 Thread GitBox
nealrichardson commented on code in PR #13090: URL: https://github.com/apache/arrow/pull/13090#discussion_r879809450 ## r/tests/testthat/test-dplyr-union.R: ## @@ -0,0 +1,74 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreement

[GitHub] [arrow] westonpace commented on a diff in pull request #13078: ARROW-15590: [C++] Add support for joins to the Substrait consumer

2022-05-23 Thread GitBox
westonpace commented on code in PR #13078: URL: https://github.com/apache/arrow/pull/13078#discussion_r879808662 ## cpp/src/arrow/engine/substrait/relation_internal.cc: ## @@ -225,6 +225,76 @@ Result FromProto(const substrait::Rel& rel, }); } +case substrait::

[GitHub] [arrow] lidavidm commented on pull request #13215: ARROW-16633: [C++] Fix incorrect Decimal literal conversion in `ir_consumer.cc`

2022-05-23 Thread GitBox
lidavidm commented on PR #13215: URL: https://github.com/apache/arrow/pull/13215#issuecomment-1135064717 Either we should add a test case for this, if we're keeping the compute IR; or we need to decide that we're just going to remove all this code entirely and not bother fixing it. -- Th

[GitHub] [arrow] nealrichardson commented on pull request #13090: ARROW-15622: [R] Implement union_all and union for arrow_dplyr_query

2022-05-23 Thread GitBox
nealrichardson commented on PR #13090: URL: https://github.com/apache/arrow/pull/13090#issuecomment-1135060385 > Not sure why, but I find I must use the fully qualified name `dplyr::union` or else will get an error. Probably namespace collision: ``` > union function (x,

[GitHub] [arrow] jonkeane commented on a diff in pull request #13196: ARROW-16407: [R] Extend `parse_date_time` to cover hour, dates, and minutes components

2022-05-23 Thread GitBox
jonkeane commented on code in PR #13196: URL: https://github.com/apache/arrow/pull/13196#discussion_r879766587 ## r/R/dplyr-datetime-helpers.R: ## @@ -179,7 +179,24 @@ build_formats <- function(orders) { orders <- unique(c(orders1, orders2)) } - supported_orders <- c(

[GitHub] [arrow] glin commented on issue #13211: [R] Can't binary install arrow 8.0.0 from RStudio Public Package Maneger

2022-05-23 Thread GitBox
glin commented on issue #13211: URL: https://github.com/apache/arrow/issues/13211#issuecomment-1135052355 @nealrichardson Looking into this now - I think the arrow 8.0.0 builds for Ubuntu 20 and 22 are timing out in our build system. We added a 1 hour timeout recently to deal with hanging b

[GitHub] [arrow] wjones127 commented on pull request #13090: ARROW-15622: [R] Implement union_all and union for arrow_dplyr_query

2022-05-23 Thread GitBox
wjones127 commented on PR #13090: URL: https://github.com/apache/arrow/pull/13090#issuecomment-1135049390 The timeouts on RTools mingw seem to be random. I ran locally on mingw64 and was able to get it to build and pass all R tests. -- This is an automated message from the Apache Git Serv

[GitHub] [arrow] nealrichardson commented on a diff in pull request #13150: ARROW-16549: [C++] Simplify AggregateNodeOptions aggregates/targets

2022-05-23 Thread GitBox
nealrichardson commented on code in PR #13150: URL: https://github.com/apache/arrow/pull/13150#discussion_r879793073 ## r/R/query-engine.R: ## @@ -121,11 +119,13 @@ ExecPlan <- R6Class("ExecPlan", x }) } +target_names <- names(.data$aggre

[GitHub] [arrow] rzeyde-vastdata commented on pull request #13215: ARROW-16633: [C++] Fix incorrect Decimal literal conversion in `ir_consumer.cc`

2022-05-23 Thread GitBox
rzeyde-vastdata commented on PR #13215: URL: https://github.com/apache/arrow/pull/13215#issuecomment-1135038826 Many thanks! I've created a Jira user: https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rzeyde-vastdata @lidavidm What should be the next step(s)? -- This i

[GitHub] [arrow-datafusion] alamb commented on pull request #2591: Optional filter in `JOIN ON` clause

2022-05-23 Thread GitBox
alamb commented on PR #2591: URL: https://github.com/apache/arrow-datafusion/pull/2591#issuecomment-1135034574 I have this on my queue to review carefully tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
alamb commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r879772780 ## datafusion/jit/src/api.rs: ## @@ -604,6 +606,15 @@ impl<'a> CodeBlock<'a> { internal_err!("No func with the name {} exist", fn_name) }

[GitHub] [arrow] zeroshade closed pull request #13137: ARROW-16557: [Go] Enable Slicing memory.Buffer objects

2022-05-23 Thread GitBox
zeroshade closed pull request #13137: ARROW-16557: [Go] Enable Slicing memory.Buffer objects URL: https://github.com/apache/arrow/pull/13137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [arrow] westonpace commented on a diff in pull request #13150: ARROW-16549: [C++] Simplify AggregateNodeOptions aggregates/targets

2022-05-23 Thread GitBox
westonpace commented on code in PR #13150: URL: https://github.com/apache/arrow/pull/13150#discussion_r879775029 ## r/R/query-engine.R: ## @@ -121,11 +119,13 @@ ExecPlan <- R6Class("ExecPlan", x }) } +target_names <- names(.data$aggregati

[GitHub] [arrow-datafusion] tustvold commented on pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
tustvold commented on PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#issuecomment-1135022377 I think it is important to understand what cranelift is, and what it isn't. Cranelift is a code generator originally intended to take optimised WASM and convert it to native co

[GitHub] [arrow] zeroshade closed pull request #13132: ARROW-16551: [Go] Improve Temporal Types

2022-05-23 Thread GitBox
zeroshade closed pull request #13132: ARROW-16551: [Go] Improve Temporal Types URL: https://github.com/apache/arrow/pull/13132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
Dandandan commented on code in PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#discussion_r879773733 ## datafusion/jit/src/compile.rs: ## @@ -0,0 +1,184 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreemen

[GitHub] [arrow] github-actions[bot] commented on pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13149: URL: https://github.com/apache/arrow/pull/13149#issuecomment-1135020630 Revision: f47e6136c1d551862ed52a147b03b933c84ea5e1 Submitted crossbow builds: [ursacomputing/crossbow @ actions-f042596390](https://github.com/ursacomputing/crossbow/bra

[GitHub] [arrow] github-actions[bot] commented on pull request #13215: ARROW-16633: [C++] Fix incorrect Decimal literal conversion in `ir_consumer.cc`

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13215: URL: https://github.com/apache/arrow/pull/13215#issuecomment-1135020512 https://issues.apache.org/jira/browse/ARROW-16633 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow] github-actions[bot] commented on pull request #13215: ARROW-16633: [C++] Fix incorrect Decimal literal conversion in `ir_consumer.cc`

2022-05-23 Thread GitBox
github-actions[bot] commented on PR #13215: URL: https://github.com/apache/arrow/pull/13215#issuecomment-1135020533 :warning: Ticket **has not been started in JIRA**, please click 'Start Progress'. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [arrow] assignUser commented on pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
assignUser commented on PR #13149: URL: https://github.com/apache/arrow/pull/13149#issuecomment-1135019593 @github-actions crossbow submit r-nightly-packages -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [arrow-datafusion] alamb commented on pull request #2587: Evaluate JIT'd expression over arrays

2022-05-23 Thread GitBox
alamb commented on PR #2587: URL: https://github.com/apache/arrow-datafusion/pull/2587#issuecomment-1135017030 > It just looks like a bare translation of what we build. So I suspect the vectorization is not done here (after translation). Further, I find this [I64X8 type](https://docs.rs/cr

[GitHub] [arrow-datafusion] alamb commented on pull request #2580: Support limit pushdown through left right outer join

2022-05-23 Thread GitBox
alamb commented on PR #2580: URL: https://github.com/apache/arrow-datafusion/pull/2580#issuecomment-1135014883 https://github.com/apache/arrow-datafusion/pull/2596 is now merged 👏 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [arrow-datafusion] alamb merged pull request #2596: Support limit pushdown through left right outer join

2022-05-23 Thread GitBox
alamb merged PR #2596: URL: https://github.com/apache/arrow-datafusion/pull/2596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-datafusion] alamb closed issue #2579: Push Limit through outer Join

2022-05-23 Thread GitBox
alamb closed issue #2579: Push Limit through outer Join URL: https://github.com/apache/arrow-datafusion/issues/2579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

[GitHub] [arrow-datafusion] alamb commented on pull request #2580: Support limit pushdown through left right outer join

2022-05-23 Thread GitBox
alamb commented on PR #2580: URL: https://github.com/apache/arrow-datafusion/pull/2580#issuecomment-1135013241 ![Screen Shot 2022-05-23 at 2 38 36 PM](https://user-images.githubusercontent.com/490673/169885111-87efdc66-6f1d-4b15-aca6-6d7049c516a0.png) Strangely github won't let this

[GitHub] [arrow-datafusion] alamb commented on pull request #2580: Support limit pushdown through left right outer join

2022-05-23 Thread GitBox
alamb commented on PR #2580: URL: https://github.com/apache/arrow-datafusion/pull/2580#issuecomment-1135012881 reopened as part of https://github.com/apache/arrow-datafusion/pull/2596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [arrow-datafusion] alamb commented on pull request #2588: Move SQL Query Planner into new `datafusion-sql` crate

2022-05-23 Thread GitBox
alamb commented on PR #2588: URL: https://github.com/apache/arrow-datafusion/pull/2588#issuecomment-1135012260 I didn't review the code of this PR but I like where it is heading -- thank you @andygrove -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [arrow-datafusion] andygrove commented on a diff in pull request #2601: WIP: Move most logical plan optimizer rules to new `datafusion-optimizer` crate

2022-05-23 Thread GitBox
andygrove commented on code in PR #2601: URL: https://github.com/apache/arrow-datafusion/pull/2601#discussion_r879765978 ## datafusion/core/src/execution/context.rs: ## @@ -33,13 +33,18 @@ use crate::{ MemTable, ViewTable, }, logical_plan::{PlanType, ToStringi

[GitHub] [arrow] jonkeane commented on a diff in pull request #13149: ARROW-16403:[R][CI] Create Crossbow task for R nightly builds

2022-05-23 Thread GitBox
jonkeane commented on code in PR #13149: URL: https://github.com/apache/arrow/pull/13149#discussion_r879761798 ## dev/tasks/r/github.nightly.yml: ## @@ -0,0 +1,381 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See th

  1   2   3   >