[GitHub] [arrow] siddhantrao23 commented on a diff in pull request #13446: ARROW-16917: [C++][Gandiva] Add a Secondary Cache to cache gandiva object code

2022-09-11 Thread GitBox
siddhantrao23 commented on code in PR #13446: URL: https://github.com/apache/arrow/pull/13446#discussion_r968035541 ## cpp/src/gandiva/jni/jni_common.cc: ## @@ -663,7 +665,17 @@ std::shared_ptr JavaSecondaryCache::Get( auto data = std::shared_ptr( new arrow::Buffer(rei

[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #3422: support cast/try_cast in prune with signed integer and decimal

2022-09-11 Thread GitBox
Dandandan commented on code in PR #3422: URL: https://github.com/apache/arrow-datafusion/pull/3422#discussion_r968035105 ## datafusion/core/src/physical_optimizer/pruning.rs: ## @@ -495,22 +500,30 @@ fn rewrite_expr_to_prunable( } match column_expr { -// `col

[GitHub] [arrow-datafusion] Ted-Jiang opened a new pull request, #3450: Add `InList` support for timestamp type. (#3449)

2022-09-11 Thread GitBox
Ted-Jiang opened a new pull request, #3450: URL: https://github.com/apache/arrow-datafusion/pull/3450 # Which issue does this PR close? Closes #3449 . # Rationale for this change After this change we support ``` ❯ \d test +---+-

[GitHub] [arrow] cyb70289 commented on a diff in pull request #14088: ARROW-17670: [Go] Implement Filter function for Primitive and FixedSize types

2022-09-11 Thread GitBox
cyb70289 commented on code in PR #14088: URL: https://github.com/apache/arrow/pull/14088#discussion_r968027079 ## go/arrow/compute/internal/exec/span.go: ## @@ -85,16 +86,20 @@ type ArraySpan struct { Children []ArraySpan } +func (a *ArraySpan) MayHaveNulls() bool { +

[GitHub] [arrow-rs] jorgecarleitao commented on issue #2594: Split up Arrow Crate

2022-09-11 Thread GitBox
jorgecarleitao commented on issue #2594: URL: https://github.com/apache/arrow-rs/issues/2594#issuecomment-1243250513 Hey, Thanks for the ping! I think it would not benefit arrow2 directly right now as it has different declarations for `Field` (e.g. we do not have dict_id on it). Arrow

[GitHub] [arrow] ursabot commented on pull request #14076: ARROW-17616: [CI][Java] Solving regex to support last Arrow Java versions >= 10.0.0

2022-09-11 Thread GitBox
ursabot commented on PR #14076: URL: https://github.com/apache/arrow/pull/14076#issuecomment-1243241356 Benchmark runs are scheduled for baseline = a63e60bad89b41266d155bc496eb383765702492 and contender = f42f3df080bee157a4de4912a29a918082d03e7e. f42f3df080bee157a4de4912a29a918082d03e7e is

[GitHub] [arrow-rs] liukun4515 opened a new pull request, #2703: Support bitwise and operation in the kernel

2022-09-11 Thread GitBox
liukun4515 opened a new pull request, #2703: URL: https://github.com/apache/arrow-rs/pull/2703 # Which issue does this PR close? part of #2702 # Rationale for this change # What changes are included in this PR? # Are there any user-facing c

[GitHub] [arrow] kou merged pull request #14059: ARROW-17632: [Python][C++] Add details of where libarrow is being found during build

2022-09-11 Thread GitBox
kou merged PR #14059: URL: https://github.com/apache/arrow/pull/14059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
Dandandan commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967992083 ## datafusion/core/tests/sql/aggregates.rs: ## @@ -1834,11 +1834,11 @@ async fn aggregate_avg_add() -> Result<()> { assert_eq!(results.len(), 1);

[GitHub] [arrow-ballista] dependabot[bot] commented on pull request #206: Update object_store requirement from 0.4.0 to 0.5.0

2022-09-11 Thread GitBox
dependabot[bot] commented on PR #206: URL: https://github.com/apache/arrow-ballista/pull/206#issuecomment-1243212441 The following labels could not be found: `auto-dependencies`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [arrow-ballista] dependabot[bot] opened a new pull request, #206: Update object_store requirement from 0.4.0 to 0.5.0

2022-09-11 Thread GitBox
dependabot[bot] opened a new pull request, #206: URL: https://github.com/apache/arrow-ballista/pull/206 Updates the requirements on [object_store](https://github.com/apache/arrow-rs) to permit the latest version. Commits https://github.com/apache/arrow-rs/commit/dd58805b1c46691

[GitHub] [arrow-rs] liukun4515 opened a new issue, #2702: Support bitwise op in arrow-rs

2022-09-11 Thread GitBox
liukun4515 opened a new issue, #2702: URL: https://github.com/apache/arrow-rs/issues/2702 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** from the issue about bitwise op in the datafusion, we should migration the bitwise o

[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
codecov-commenter commented on PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#issuecomment-1243188185 # [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3398?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_

[GitHub] [arrow-datafusion] HaoYang670 commented on a diff in pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
HaoYang670 commented on code in PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#discussion_r967962498 ## datafusion/common/src/scalar.rs: ## @@ -2575,7 +2575,7 @@ mod tests { // The alignment requirements differ across architectures and // t

[GitHub] [arrow-datafusion] HaoYang670 commented on a diff in pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
HaoYang670 commented on code in PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#discussion_r967958710 ## datafusion/common/src/scalar.rs: ## @@ -2575,7 +2575,7 @@ mod tests { // The alignment requirements differ across architectures and // t

[GitHub] [arrow] kou merged pull request #14076: ARROW-17616: [CI][Java] Solving regex to support last Arrow Java versions >= 10.0.0

2022-09-11 Thread GitBox
kou merged PR #14076: URL: https://github.com/apache/arrow/pull/14076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.or

[GitHub] [arrow-datafusion] Ted-Jiang opened a new issue, #3449: Support timestamp data type in In_list node

2022-09-11 Thread GitBox
Ted-Jiang opened a new issue, #3449: URL: https://github.com/apache/arrow-datafusion/issues/3449 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated whe

[GitHub] [arrow-ballista] Ted-Jiang commented on pull request #198: [WIP]Enable code coverage in pull request.

2022-09-11 Thread GitBox
Ted-Jiang commented on PR #198: URL: https://github.com/apache/arrow-ballista/pull/198#issuecomment-1243156549 @Ted-Jiang I think we should create this project in https://app.codecov.io/ , but i am not the owner of the repo 😂 ``` https://codecov.io/upload/v4?package=bash-1.0.6&token=

[GitHub] [arrow-datafusion] HaoYang670 commented on a diff in pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
HaoYang670 commented on code in PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#discussion_r967949079 ## datafusion/common/src/scalar.rs: ## @@ -2575,7 +2575,7 @@ mod tests { // The alignment requirements differ across architectures and // t

[GitHub] [arrow-ballista] Ted-Jiang commented on pull request #198: [WIP]Enable code coverage in pull request.

2022-09-11 Thread GitBox
Ted-Jiang commented on PR #198: URL: https://github.com/apache/arrow-ballista/pull/198#issuecomment-1243154963 @andygrove Sorry there is still something missing. I will fix it let the report appear in UI. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [arrow-datafusion] HaoYang670 commented on a diff in pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
HaoYang670 commented on code in PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#discussion_r967939957 ## datafusion/common/src/scalar.rs: ## @@ -2575,7 +2575,7 @@ mod tests { // The alignment requirements differ across architectures and // t

[GitHub] [arrow-ballista] andygrove commented on pull request #199: bump datafusion version

2022-09-11 Thread GitBox
andygrove commented on PR #199: URL: https://github.com/apache/arrow-ballista/pull/199#issuecomment-1243130823 This PR uses DataFusion 8b59b207aaadd6f2c19c28d1f1431a0cb8d110ae which already uses Arrow 22 -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [arrow-ballista] andygrove commented on pull request #199: bump datafusion version

2022-09-11 Thread GitBox
andygrove commented on PR #199: URL: https://github.com/apache/arrow-ballista/pull/199#issuecomment-1243124961 > This looks like it bumps Ballista to point at `312a7dd3b9642e64555bd0cfa90f4e10145ea103` which is a commit from 2022-08-31 (just after the arrow 21 upgrade)? > > I have a

[GitHub] [arrow] kou commented on issue #14068: parquet.dll : libcrypto-3-x64.dll missing module

2022-09-11 Thread GitBox
kou commented on issue #14068: URL: https://github.com/apache/arrow/issues/14068#issuecomment-1243111662 Could you attach the log when you install Apache Arrow via vcpkg? I want to see all outputs from CMake. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [arrow] oraix closed issue #13198: Fail to build pyarrow on Windows 10 from source

2022-09-11 Thread GitBox
oraix closed issue #13198: Fail to build pyarrow on Windows 10 from source URL: https://github.com/apache/arrow/issues/13198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [arrow-datafusion] liukun4515 commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
liukun4515 commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967911283 ## datafusion/core/tests/sql/aggregates.rs: ## @@ -1834,11 +1834,11 @@ async fn aggregate_avg_add() -> Result<()> { assert_eq!(results.len(), 1);

[GitHub] [arrow-datafusion] liukun4515 commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
liukun4515 commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967911124 ## datafusion/core/tests/sql/aggregates.rs: ## @@ -1834,11 +1834,11 @@ async fn aggregate_avg_add() -> Result<()> { assert_eq!(results.len(), 1);

[GitHub] [arrow-rs] liukun4515 commented on pull request #2661: optimize the `numeric_cast_with_error`

2022-09-11 Thread GitBox
liukun4515 commented on PR #2661: URL: https://github.com/apache/arrow-rs/pull/2661#issuecomment-1243073883 @viirya @tustvold PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [arrow-ballista] andygrove opened a new issue, #205: Add documentation on extending Ballista

2022-09-11 Thread GitBox
andygrove opened a new issue, #205: URL: https://github.com/apache/arrow-ballista/issues/205 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** It would be good to have some documentation explaining how to extend Ballista to add cust

[GitHub] [arrow-datafusion] andygrove commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
andygrove commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967907752 ## datafusion/optimizer/src/type_coercion.rs: ## @@ -64,7 +66,15 @@ impl OptimizerRule for TypeCoercion { _ => DFSchemaRef::new(DFSchema::empty()

[GitHub] [arrow-ballista] andygrove opened a new pull request, #204: [Python] Allow some config options to be set when creating context

2022-09-11 Thread GitBox
andygrove opened a new pull request, #204: URL: https://github.com/apache/arrow-ballista/pull/204 # Which issue does this PR close? Part of https://github.com/apache/arrow-ballista/issues/203 # Rationale for this change Allow jobs to be tuned from Python

[GitHub] [arrow-rs] alamb closed issue #2620: Release `object_store` `0.5.0`

2022-09-11 Thread GitBox
alamb closed issue #2620: Release `object_store` `0.5.0` URL: https://github.com/apache/arrow-rs/issues/2620 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [arrow-rs] alamb commented on issue #2620: Release `object_store` `0.5.0`

2022-09-11 Thread GitBox
alamb commented on issue #2620: URL: https://github.com/apache/arrow-rs/issues/2620#issuecomment-1243066869 Thanks all@ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [arrow-rs] alamb commented on issue #2620: Release `object_store` `0.5.0`

2022-09-11 Thread GitBox
alamb commented on issue #2620: URL: https://github.com/apache/arrow-rs/issues/2620#issuecomment-1243066825 The release is available here: https://dist.apache.org/repos/dist/release/arrow/arrow-object-store-rs-0.5.0 It has also been published to crates.io: https://crates.i

[GitHub] [arrow-ballista] andygrove opened a new issue, #203: [Python] Allow configuration options to be set when creating BallistaContext

2022-09-11 Thread GitBox
andygrove opened a new issue, #203: URL: https://github.com/apache/arrow-ballista/issues/203 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** The Python bindings for BallistaContext have hard-coded configuration settings that canno

[GitHub] [arrow-ballista] andygrove commented on issue #15: Implement Python bindings for BallistaContext

2022-09-11 Thread GitBox
andygrove commented on issue #15: URL: https://github.com/apache/arrow-ballista/issues/15#issuecomment-1243063350 This has now been implemented. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [arrow-ballista] andygrove closed issue #15: Implement Python bindings for BallistaContext

2022-09-11 Thread GitBox
andygrove closed issue #15: Implement Python bindings for BallistaContext URL: https://github.com/apache/arrow-ballista/issues/15 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [arrow] lidavidm commented on issue #14089: Reading every batch while being written to the server

2022-09-11 Thread GitBox
lidavidm commented on issue #14089: URL: https://github.com/apache/arrow/issues/14089#issuecomment-1243056686 The APIs are not different. This is up to the application. You are synchronizing two threads, that is nothing specific to Arrow or Flight. -- This is an automated message from the

[GitHub] [arrow] iamsmkr commented on issue #14089: Reading every batch while being written to the server

2022-09-11 Thread GitBox
iamsmkr commented on issue #14089: URL: https://github.com/apache/arrow/issues/14089#issuecomment-1243052489 Thanks @lidavidm. This clarifies! But I do wonder what APIs to use to achieve this. Is there any example code that I could refer? Any help is much appreciated! -- This is an autom

[GitHub] [arrow-datafusion] ursabot commented on pull request #3445: Use hash repartitioning for aggregates on dictionaries

2022-09-11 Thread GitBox
ursabot commented on PR #3445: URL: https://github.com/apache/arrow-datafusion/pull/3445#issuecomment-1243043787 Benchmark runs are scheduled for baseline = 8df5496ce77638e57bde8468801e78d61e423d92 and contender = 81addf78c6b110808eaa23e93d4b148b2b6fecc5. 81addf78c6b110808eaa23e93d4b148b2

[GitHub] [arrow-datafusion] alamb merged pull request #3445: Use hash repartitioning for aggregates on dictionaries

2022-09-11 Thread GitBox
alamb merged PR #3445: URL: https://github.com/apache/arrow-datafusion/pull/3445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-datafusion] alamb closed issue #331: Implement vectorized hashing for dictionary types

2022-09-11 Thread GitBox
alamb closed issue #331: Implement vectorized hashing for dictionary types URL: https://github.com/apache/arrow-datafusion/issues/331 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [arrow-datafusion] alamb commented on pull request #3380: Pushdown `RowFilter` in `ParquetExec`

2022-09-11 Thread GitBox
alamb commented on PR #3380: URL: https://github.com/apache/arrow-datafusion/pull/3380#issuecomment-1243042836 Will review carefully tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow-rs] viirya commented on a diff in pull request #2693: Split out arrow-buffer crate (#2594)

2022-09-11 Thread GitBox
viirya commented on code in PR #2693: URL: https://github.com/apache/arrow-rs/pull/2693#discussion_r967885014 ## arrow-buffer/src/bytes.rs: ## @@ -0,0 +1,144 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NO

[GitHub] [arrow-rs] viirya commented on a diff in pull request #2693: Split out arrow-buffer crate (#2594)

2022-09-11 Thread GitBox
viirya commented on code in PR #2693: URL: https://github.com/apache/arrow-rs/pull/2693#discussion_r967883686 ## arrow-buffer/src/alloc/mod.rs: ## @@ -20,34 +20,31 @@ use std::alloc::{handle_alloc_error, Layout}; use std::fmt::{Debug, Formatter}; -use std::mem::size_of; use

[GitHub] [arrow-rs] viirya commented on pull request #2650: Overflow-checking variant of arithmetic scalar kernels

2022-09-11 Thread GitBox
viirya commented on PR #2650: URL: https://github.com/apache/arrow-rs/pull/2650#issuecomment-1243035462 @tustvold Updated to use `try_unary`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [arrow] lidavidm commented on issue #14089: Reading every batch while being written to the server

2022-09-11 Thread GitBox
lidavidm commented on issue #14089: URL: https://github.com/apache/arrow/issues/14089#issuecomment-1243033923 Well, yes of course: the server implementation isn't actually making the data available until the call finishes: ``` scala override def acceptPut(context: CallContext, fl

[GitHub] [arrow] iamsmkr commented on issue #14089: Reading every batch while being written to the server

2022-09-11 Thread GitBox
iamsmkr commented on issue #14089: URL: https://github.com/apache/arrow/issues/14089#issuecomment-1243031219 @lidavidm Thanks for your reply. I have written a small working example (in Scala) for you to have a look [here](https://github.com/iamsmkr/arrow-flight-scala). In the `Main.

[GitHub] [arrow] aucahuasi commented on pull request #14077: ARROW-16870: [C++] Fix link issues with ldd and clang for flight examples

2022-09-11 Thread GitBox
aucahuasi commented on PR #14077: URL: https://github.com/apache/arrow/pull/14077#issuecomment-1243023744 It seems the CI job that is failing is also affecting other PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [arrow] zeroshade commented on pull request #14026: ARROW-17584: [Go] Use unsafe.Slice from Go 1.17

2022-09-11 Thread GitBox
zeroshade commented on PR #14026: URL: https://github.com/apache/arrow/pull/14026#issuecomment-1243023009 @tschaub any particular reason you're using TinyGo over just using go's `GOOS=js GOARCH=wasm` and building a wasm binary that way? I was able to build a simple wasm binary using Arrow G

[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
codecov-commenter commented on PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#issuecomment-1243015572 # [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3444?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_

[GitHub] [arrow] zeroshade commented on a diff in pull request #14079: RFC: [Format] Add ADBC

2022-09-11 Thread GitBox
zeroshade commented on code in PR #14079: URL: https://github.com/apache/arrow/pull/14079#discussion_r967866679 ## docs/source/format/ADBC.rst: ## @@ -0,0 +1,310 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See th

[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #3398: Fix the test failure of `size_of_scalar` failure on M1/M2 Mac

2022-09-11 Thread GitBox
Dandandan commented on code in PR #3398: URL: https://github.com/apache/arrow-datafusion/pull/3398#discussion_r967865729 ## datafusion/common/src/scalar.rs: ## @@ -2575,7 +2575,7 @@ mod tests { // The alignment requirements differ across architectures and // th

[GitHub] [arrow-datafusion] iajoiner commented on issue #3097: DataFusion 12.0.0 Release

2022-09-11 Thread GitBox
iajoiner commented on issue #3097: URL: https://github.com/apache/arrow-datafusion/issues/3097#issuecomment-1243011816 @andygrove @alamb https://github.com/apache/arrow-datafusion/issues/3383 Shall this one also be resolved before 12? -- This is an automated message from the Apache

[GitHub] [arrow-datafusion] Dandandan commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
Dandandan commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967863267 ## datafusion/optimizer/src/type_coercion.rs: ## @@ -64,7 +66,15 @@ impl OptimizerRule for TypeCoercion { _ => DFSchemaRef::new(DFSchema::empty()

[GitHub] [arrow-datafusion] kmitchener opened a new issue, #3448: support scientific notation for SQL literals

2022-09-11 Thread GitBox
kmitchener opened a new issue, #3448: URL: https://github.com/apache/arrow-datafusion/issues/3448 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated wh

[GitHub] [arrow-ballista] andygrove commented on pull request #202: MINOR: Add tuning guide to user guide

2022-09-11 Thread GitBox
andygrove commented on PR #202: URL: https://github.com/apache/arrow-ballista/pull/202#issuecomment-1243005903 cc @thinkharderdev @yahoNanJing @avantgardnerio -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [arrow-ballista] andygrove opened a new pull request, #202: MINOR: Add tuning guide to user guide

2022-09-11 Thread GitBox
andygrove opened a new pull request, #202: URL: https://github.com/apache/arrow-ballista/pull/202 # Which issue does this PR close? N/A # Rationale for this change We need to each people how to get the best performance and scalability out of Ballista

[GitHub] [arrow-datafusion] iajoiner commented on issue #3414: pruning support cast/try_cast expr

2022-09-11 Thread GitBox
iajoiner commented on issue #3414: URL: https://github.com/apache/arrow-datafusion/issues/3414#issuecomment-1243003763 On it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [arrow-ballista] andygrove commented on pull request #200: Add support for DateTimeIntervalExpr and more binary operators

2022-09-11 Thread GitBox
andygrove commented on PR #200: URL: https://github.com/apache/arrow-ballista/pull/200#issuecomment-1243001649 @yahoNanJing @thinkharderdev @avantgardnerio This is now ready for review and allows Ballista to support a wider range of queries. -- This is an automated message from the Apache

[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3447: add `cast` support inside `values`

2022-09-11 Thread GitBox
codecov-commenter commented on PR #3447: URL: https://github.com/apache/arrow-datafusion/pull/3447#issuecomment-1243000262 # [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3447?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_

[GitHub] [arrow-ballista] avantgardnerio commented on pull request #199: bump datafusion version

2022-09-11 Thread GitBox
avantgardnerio commented on PR #199: URL: https://github.com/apache/arrow-ballista/pull/199#issuecomment-1242996680 This looks like it bumps Ballista to point at `312a7dd3b9642e64555bd0cfa90f4e10145ea103` which is a commit from 2022-08-31 (just after the arrow 21 upgrade)? I have a f

[GitHub] [arrow-datafusion] kmitchener opened a new pull request, #3447: add `cast` support inside `values`

2022-09-11 Thread GitBox
kmitchener opened a new pull request, #3447: URL: https://github.com/apache/arrow-datafusion/pull/3447 # Which issue does this PR close? Closes #3446 . # Rationale for this change # What changes are included in this PR? # Are there any user-facing

[GitHub] [arrow-datafusion] kmitchener opened a new issue, #3446: support `cast` inside `values`

2022-09-11 Thread GitBox
kmitchener opened a new issue, #3446: URL: https://github.com/apache/arrow-datafusion/issues/3446 **Is your feature request related to a problem or challenge? Please describe what you are trying to do.** A clear and concise description of what the problem is. Ex. I'm always frustrated wh

[GitHub] [arrow] github-actions[bot] commented on pull request #14093: ARROW-17416: [R] Implement lubridate::with_tz and lubridate::force_tz

2022-09-11 Thread GitBox
github-actions[bot] commented on PR #14093: URL: https://github.com/apache/arrow/pull/14093#issuecomment-1242991113 https://issues.apache.org/jira/browse/ARROW-17416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3445: Use hash repartitioning for aggregates on dictionaries

2022-09-11 Thread GitBox
codecov-commenter commented on PR #3445: URL: https://github.com/apache/arrow-datafusion/pull/3445#issuecomment-1242987256 # [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3445?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_

[GitHub] [arrow-datafusion] andygrove commented on pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
andygrove commented on PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#issuecomment-1242981447 Thanks @Dandandan. This looks like a great improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [arrow-datafusion] andygrove commented on a diff in pull request #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
andygrove commented on code in PR #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444#discussion_r967840953 ## datafusion/optimizer/src/type_coercion.rs: ## @@ -64,7 +66,15 @@ impl OptimizerRule for TypeCoercion { _ => DFSchemaRef::new(DFSchema::empty()

[GitHub] [arrow-datafusion] isidentical opened a new pull request, #3445: Use hash repartitioning for aggregates on dictionaries

2022-09-11 Thread GitBox
isidentical opened a new pull request, #3445: URL: https://github.com/apache/arrow-datafusion/pull/3445 # Which issue does this PR close? Closes #331. # Rationale for this change Hash repartitioning for aggregates on dictionaries was not available when it was in

[GitHub] [arrow-datafusion] thinkharderdev commented on pull request #3380: Pushdown `RowFilter` in `ParquetExec`

2022-09-11 Thread GitBox
thinkharderdev commented on PR #3380: URL: https://github.com/apache/arrow-datafusion/pull/3380#issuecomment-1242975750 > @thinkharderdev please let me know when you think this PR is ready for final review / merge (it still says "RFC" in the title so I am not quite sure) I think it's

[GitHub] [arrow] dhruv9vats commented on a diff in pull request #14059: ARROW-17632: [Python][C++] Add details of where libarrow is being found during build

2022-09-11 Thread GitBox
dhruv9vats commented on code in PR #14059: URL: https://github.com/apache/arrow/pull/14059#discussion_r967832304 ## cpp/src/arrow/ArrowConfig.cmake.in: ## @@ -172,3 +172,16 @@ endmacro() arrow_keep_backward_compatibility(Arrow arrow) check_required_components(Arrow) + +funct

[GitHub] [arrow] dhruv9vats commented on a diff in pull request #14059: ARROW-17632: [Python][C++] Add details of where libarrow is being found during build

2022-09-11 Thread GitBox
dhruv9vats commented on code in PR #14059: URL: https://github.com/apache/arrow/pull/14059#discussion_r967832304 ## cpp/src/arrow/ArrowConfig.cmake.in: ## @@ -172,3 +172,16 @@ endmacro() arrow_keep_backward_compatibility(Arrow arrow) check_required_components(Arrow) + +funct

[GitHub] [arrow-datafusion] codecov-commenter commented on pull request #3428: Pass `return_type` to `AccumulatorFunctionImplementation ` for user defined aggregates

2022-09-11 Thread GitBox
codecov-commenter commented on PR #3428: URL: https://github.com/apache/arrow-datafusion/pull/3428#issuecomment-1242955479 # [Codecov](https://codecov.io/gh/apache/arrow-datafusion/pull/3428?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_

[GitHub] [arrow-rs] v1gnesh commented on issue #2677: Arrow Row Format

2022-09-11 Thread GitBox
v1gnesh commented on issue #2677: URL: https://github.com/apache/arrow-rs/issues/2677#issuecomment-1242954235 Thanks for clarifying. As you say, I'll have to make a separate stream for each row-type. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [arrow-rs] ursabot commented on pull request #2699: Clarify docs of binary and string builders

2022-09-11 Thread GitBox
ursabot commented on PR #2699: URL: https://github.com/apache/arrow-rs/pull/2699#issuecomment-1242953111 Benchmark runs are scheduled for baseline = 8206f013363addad1bf37b2ff96aa44003d70f22 and contender = e646ae86b345906c9ac76a61afcdc0aa71fded48. e646ae86b345906c9ac76a61afcdc0aa71fded48 i

[GitHub] [arrow-rs] tustvold merged pull request #2699: Clarify docs of binary and string builders

2022-09-11 Thread GitBox
tustvold merged PR #2699: URL: https://github.com/apache/arrow-rs/pull/2699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apa

[GitHub] [arrow-datafusion] ursabot commented on pull request #3430: Implement bitwise XOR operator (`#`)

2022-09-11 Thread GitBox
ursabot commented on PR #3430: URL: https://github.com/apache/arrow-datafusion/pull/3430#issuecomment-1242949464 Benchmark runs are scheduled for baseline = deea8c3e8436614c7333025d7c44917487ef439a and contender = 8df5496ce77638e57bde8468801e78d61e423d92. 8df5496ce77638e57bde8468801e78d61

[GitHub] [arrow-datafusion] alamb commented on pull request #3430: Implement bitwise XOR operator (`#`)

2022-09-11 Thread GitBox
alamb commented on PR #3430: URL: https://github.com/apache/arrow-datafusion/pull/3430#issuecomment-1242948600 Thanks again @askoa -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [arrow-datafusion] alamb merged pull request #3430: Implement bitwise XOR operator (`#`)

2022-09-11 Thread GitBox
alamb merged PR #3430: URL: https://github.com/apache/arrow-datafusion/pull/3430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arro

[GitHub] [arrow-datafusion] alamb closed issue #3420: Support bitwise XOR operator (`#`)

2022-09-11 Thread GitBox
alamb closed issue #3420: Support bitwise XOR operator (`#`) URL: https://github.com/apache/arrow-datafusion/issues/3420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [arrow-rs] alamb commented on issue #2620: Release `object_store` `0.5.0`

2022-09-11 Thread GitBox
alamb commented on issue #2620: URL: https://github.com/apache/arrow-rs/issues/2620#issuecomment-1242946237 Update here is that https://crates.io/crates/quick-xml/0.24.1 has been released so I think we are still good -- This is an automated message from the Apache Git Service. To respond

[GitHub] [arrow-rs] alamb commented on pull request #2626: Update thrift v0.16 and vendor parquet-format (#2502)

2022-09-11 Thread GitBox
alamb commented on PR #2626: URL: https://github.com/apache/arrow-rs/pull/2626#issuecomment-1242946096 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[GitHub] [arrow-datafusion] alamb commented on pull request #3380: RFC: Integrate `RowFilter` into `ParquetExec`

2022-09-11 Thread GitBox
alamb commented on PR #3380: URL: https://github.com/apache/arrow-datafusion/pull/3380#issuecomment-1242946029 @thinkharderdev please let me know when you think this PR is ready for final review / merge (it still says "RFC") -- This is an automated message from the Apache Git Service. T

[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #3380: RFC: Integrate `RowFilter` into `ParquetExec`

2022-09-11 Thread GitBox
alamb commented on code in PR #3380: URL: https://github.com/apache/arrow-datafusion/pull/3380#discussion_r967811884 ## datafusion/core/src/physical_plan/file_format/row_filter.rs: ## @@ -0,0 +1,359 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more co

[GitHub] [arrow-rs] datapythonista commented on issue #2696: Implement collect for int values

2022-09-11 Thread GitBox
datapythonista commented on issue #2696: URL: https://github.com/apache/arrow-rs/issues/2696#issuecomment-1242938676 Ah, I see. Thanks a lot for the help. I may give it a try and see if I can implement it. Closing this issue. -- This is an automated message from the Apache Git Service. To

[GitHub] [arrow-rs] datapythonista closed issue #2696: Implement collect for int values

2022-09-11 Thread GitBox
datapythonista closed issue #2696: Implement collect for int values URL: https://github.com/apache/arrow-rs/issues/2696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[GitHub] [arrow-rs] datapythonista commented on a diff in pull request #2699: Clarify docs of binary and string builders

2022-09-11 Thread GitBox
datapythonista commented on code in PR #2699: URL: https://github.com/apache/arrow-rs/pull/2699#discussion_r967799851 ## arrow/src/array/builder/generic_string_builder.rs: ## @@ -28,16 +28,17 @@ pub struct GenericStringBuilder { } impl GenericStringBuilder { -/// Creates

[GitHub] [arrow-datafusion] Dandandan opened a new pull request, #3444: Evaluate expressions after type coercion

2022-09-11 Thread GitBox
Dandandan opened a new pull request, #3444: URL: https://github.com/apache/arrow-datafusion/pull/3444 # Which issue does this PR close? Closes #3431 # Rationale for this change # What changes are included in this PR? # Are there any user-facing ch

[GitHub] [arrow-rs] tustvold commented on issue #2696: Implement collect for int values

2022-09-11 Thread GitBox
tustvold commented on issue #2696: URL: https://github.com/apache/arrow-rs/issues/2696#issuecomment-1242903442 We've not implemented a NativeAdapter for StringArray yet, so you can only collect an iterator of `Option<&str>` So ``` ["A',"b"].into_iter().map(Some).collect::()

[GitHub] [arrow-rs] tustvold commented on issue #2677: Arrow Row Format

2022-09-11 Thread GitBox
tustvold commented on issue #2677: URL: https://github.com/apache/arrow-rs/issues/2677#issuecomment-1242902278 This will not allow operations on data with multiple schema, same as with RecordBatch. That being said, in the case of rows with different variants, nulls will be inserted b