Re: [PR] feat: metadata columns [datafusion]

2025-01-09 Thread via GitHub
chenkovsky commented on code in PR #14057: URL: https://github.com/apache/datafusion/pull/14057#discussion_r1909953573 ## datafusion/common/src/dfschema.rs: ## @@ -319,52 +465,56 @@ impl DFSchema { qualifiers.push(qualifier.cloned()); } } -

Re: [PR] Improve performance of `find_in_set` function [datafusion]

2025-01-09 Thread via GitHub
tlm365 commented on PR #14020: URL: https://github.com/apache/datafusion/pull/14020#issuecomment-2581930161 > Oh with literal support I think the code becomes much more tricky. > Wondering if the performance benefit still worthy such complications. @tlm365 can we a criterion to check `fin

Re: [PR] test: Add plan execution during tests for bounded source [datafusion]

2025-01-09 Thread via GitHub
berkaysynnada commented on PR #14013: URL: https://github.com/apache/datafusion/pull/14013#issuecomment-2581930069 > Is this PR ready to merge @berkaysynnada ? Looks good to me Yeap, once the main is resolved, I'll merge it -- This is an automated message from the Apache Git Service

Re: [PR] Refactor into `LexOrdering::collapse`, `LexRequirement::collapse` avoid clone [datafusion]

2025-01-09 Thread via GitHub
berkaysynnada commented on PR #14038: URL: https://github.com/apache/datafusion/pull/14038#issuecomment-2581927796 I will merge this once main is updated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Document how to add new docs are dealt with to HOWTOs [datafusion]

2025-01-09 Thread via GitHub
xarus01 commented on issue #13368: URL: https://github.com/apache/datafusion/issues/13368#issuecomment-2581808311 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] 'array_repeat' if the repeat count value is 0, return NULL instead of empty array [datafusion]

2025-01-09 Thread via GitHub
jatin510 commented on PR #14046: URL: https://github.com/apache/datafusion/pull/14046#issuecomment-2581741481 Created a different PR as this is related to specific array udf. Introducing default cli formatter for sqllogic test and df-cli : https://github.com/apache/datafusion/pull/14052

Re: [PR] 'array_repeat' if the repeat count value is 0, return NULL instead of empty array [datafusion]

2025-01-09 Thread via GitHub
jatin510 closed pull request #14046: 'array_repeat' if the repeat count value is 0, return NULL instead of empty array URL: https://github.com/apache/datafusion/pull/14046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] build: bump spark version to 3.3.4, 3.4.4, 3.5.4 for spark-3.3, spark-3.4 and spark-3.5 [datafusion-comet]

2025-01-09 Thread via GitHub
SteNicholas commented on PR #1243: URL: https://github.com/apache/datafusion-comet/pull/1243#issuecomment-2581668258 @andygrove, shoud this pull request introduce the diff files for bumping to the latest version? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] fix: speed up ConcurrentHashMap#computeIfAbsent of JDK8 [datafusion-comet]

2025-01-09 Thread via GitHub
SteNicholas commented on PR #1245: URL: https://github.com/apache/datafusion-comet/pull/1245#issuecomment-2581666737 @andygrove, you could refers to https://github.com/apache/shardingsphere/pull/13275. BTW, the JMH testing is as follows: ``` import org.openjdk.jmh.annotations.Benchma

Re: [PR] chore: deprecate `ValuesExec` in favour of `MemoryExec` [datafusion]

2025-01-09 Thread via GitHub
jonathanc-n commented on code in PR #14032: URL: https://github.com/apache/datafusion/pull/14032#discussion_r1909749745 ## datafusion/physical-plan/src/memory.rs: ## @@ -293,6 +383,15 @@ impl MemoryExec { Boundedness::Bounded, ) } + +fn compute_pro

Re: [I] Fix build issues on latest stable Rust toolchain (1.84) [datafusion]

2025-01-09 Thread via GitHub
niebayes commented on issue #14061: URL: https://github.com/apache/datafusion/issues/14061#issuecomment-2581658416 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] fix: make get_valid_types handle TypeSignature::Numeric correctly [datafusion]

2025-01-09 Thread via GitHub
niebayes commented on PR #14060: URL: https://github.com/apache/datafusion/pull/14060#issuecomment-2581654390 I need to bump Rust to 1.84 to pass CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [I] Clarify logic in `nth_value` function [datafusion]

2025-01-09 Thread via GitHub
zjregee commented on issue #14063: URL: https://github.com/apache/datafusion/issues/14063#issuecomment-2581645662 take -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] feat: support `INSERT INTO [TABLE] FUNCTION` of Clickhouse [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
byte-sourcerer commented on PR #1633: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1633#issuecomment-2581640329 > @byte-sourcerer could you take a look when you get the chance to resolve the conflicts on the branch Resolved. -- This is an automated message from the

Re: [I] Fix build issues on latest stable Rust toolchain (1.84) [datafusion]

2025-01-09 Thread via GitHub
niebayes commented on issue #14061: URL: https://github.com/apache/datafusion/issues/14061#issuecomment-2581636120 I've also noticed that this implicit upgrade causes CI to fail, particularly for rust clippy check. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] fix: incorrect error message of function_length_check [datafusion]

2025-01-09 Thread via GitHub
niebayes commented on code in PR #14056: URL: https://github.com/apache/datafusion/pull/14056#discussion_r1909698909 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -440,13 +440,13 @@ fn get_valid_types( fn function_length_check(length: usize, expected_length: usiz

[PR] Minor: Make `group_schema` as `PhysicalGroupBy` method [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 opened a new pull request, #14064: URL: https://github.com/apache/datafusion/pull/14064 ## Which issue does this PR close? Closes #. ## Rationale for this change Whenever we have `PhysicalGroupBy`, we can call this function instead of the util like one

Re: [PR] Do not push down filter through distinct on [datafusion]

2025-01-09 Thread via GitHub
github-actions[bot] closed pull request #12943: Do not push down filter through distinct on URL: https://github.com/apache/datafusion/pull/12943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] feat(datafusion-functions-aggregate): add support for lists and other nested types in `min` and `max` [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 commented on code in PR #13991: URL: https://github.com/apache/datafusion/pull/13991#discussion_r1909655204 ## datafusion/functions-aggregate/src/min_max/min_max_generic.rs: ## @@ -0,0 +1,491 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or mor

Re: [PR] feat(datafusion-functions-aggregate): add support for lists and other nested types in `min` and `max` [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 commented on code in PR #13991: URL: https://github.com/apache/datafusion/pull/13991#discussion_r1909649089 ## datafusion/functions-aggregate/src/min_max/min_max_generic.rs: ## @@ -0,0 +1,491 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or mor

Re: [I] Doc attribution: make `user_doc` to work with predefined consts. [datafusion]

2025-01-09 Thread via GitHub
Chen-Yuan-Lai commented on issue #14001: URL: https://github.com/apache/datafusion/issues/14001#issuecomment-2581552590 Agree, thanks @comphead @ding-young -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] fix: incorrect error message of function_length_check [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 commented on code in PR #14056: URL: https://github.com/apache/datafusion/pull/14056#discussion_r1909644459 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -440,13 +440,13 @@ fn get_valid_types( fn function_length_check(length: usize, expected_length: us

[PR] docs: Update TPC-H benchmark results [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove opened a new pull request, #1257: URL: https://github.com/apache/datafusion-comet/pull/1257 ## Which issue does this PR close? N/A ## Rationale for this change There have been a number of performance improvements since 0.4.0 so we should update

Re: [PR] feat: metadata columns [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 commented on code in PR #14057: URL: https://github.com/apache/datafusion/pull/14057#discussion_r1909641034 ## datafusion/common/src/dfschema.rs: ## @@ -319,52 +465,56 @@ impl DFSchema { qualifiers.push(qualifier.cloned()); } } -

Re: [PR] feat: Implement custom RecordBatch serde for shuffle for improved performance [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1190: URL: https://github.com/apache/datafusion-comet/pull/1190#discussion_r1909634336 ## native/core/benches/shuffle_writer.rs: ## @@ -31,67 +31,52 @@ use std::sync::Arc; use tokio::runtime::Runtime; fn criterion_benchmark(c: &mut Criterio

Re: [PR] fix: make get_valid_types handle TypeSignature::Numeric correctly [datafusion]

2025-01-09 Thread via GitHub
jayzhan211 commented on code in PR #14060: URL: https://github.com/apache/datafusion/pull/14060#discussion_r1909635152 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -551,6 +551,10 @@ fn get_valid_types( // and their default type is double precision

Re: [PR] feat: Implement custom RecordBatch serde for shuffle for improved performance [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on PR #1190: URL: https://github.com/apache/datafusion-comet/pull/1190#issuecomment-2581533639 > Thanks @andygrove thats amazing, I suppose it is extensible to support nested types in future? Yes, I don't foresee any issues with that. -- This is an automated me

Re: [PR] ALTER TABLE DROP {COLUMN|CONSTRAINT} RESTRICT [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio merged PR #1651: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] From trailing commas [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio commented on code in PR #1645: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1645#discussion_r1909605886 ## tests/sqlparser_snowflake.rs: ## @@ -2483,6 +2483,37 @@ fn test_sf_trailing_commas() { snowflake().verified_only_select_with_canonical("SELECT 1

Re: [PR] perf: Improve query planning to more reliably fall back to columnar shuffle when native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on PR #1209: URL: https://github.com/apache/datafusion-comet/pull/1209#issuecomment-2581472592 Thanks for the reviews @viirya and @kazuyukitanimura -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] feat: Implement custom RecordBatch serde for shuffle for improved performance [datafusion-comet]

2025-01-09 Thread via GitHub
comphead commented on PR #1190: URL: https://github.com/apache/datafusion-comet/pull/1190#issuecomment-2581491053 Thanks @andygrove thats amazing, I suppose it is extensible to support nested types in future? -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Try using Comet columnar shuffle when Comet native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove closed issue #1248: Try using Comet columnar shuffle when Comet native shuffle is not supported URL: https://github.com/apache/datafusion-comet/issues/1248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] perf: Improve query planning to more reliably fall back to columnar shuffle when native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1209: URL: https://github.com/apache/datafusion-comet/pull/1209#discussion_r1909598881 ## spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala: ## @@ -78,6 +78,7 @@ abstract class CometTestBase conf.set(CometConf.COMET_ENABLED.key

Re: [I] Optimize TPC-H q16 to use columnar shuffle rather than native shuffle [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove closed issue #1208: Optimize TPC-H q16 to use columnar shuffle rather than native shuffle URL: https://github.com/apache/datafusion-comet/issues/1208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] chore: Improve shuffle configuration [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove closed pull request #1207: chore: Improve shuffle configuration URL: https://github.com/apache/datafusion-comet/pull/1207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] perf: Improve query planning to more reliably fall back to columnar shuffle when native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove merged PR #1209: URL: https://github.com/apache/datafusion-comet/pull/1209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] fix: Fall back to Spark for unsupported partition or sort expressions in window aggregates [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove merged PR #1253: URL: https://github.com/apache/datafusion-comet/pull/1253 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [I] Spark SQL test "window function: partition and order expressions" fails with scala.MatchError [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove closed issue #1249: Spark SQL test "window function: partition and order expressions" fails with scala.MatchError URL: https://github.com/apache/datafusion-comet/issues/1249 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] fix: Fall back to Spark for unsupported partition or sort expressions in window aggregates [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on PR #1253: URL: https://github.com/apache/datafusion-comet/pull/1253#issuecomment-2581468945 Thanks for the review @kazuyukitanimura -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Add support for the Snowflake MINUS set operator [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio merged PR #1652: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1652 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] Add support for ClickHouse `FORMAT` on `INSERT` [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio commented on code in PR #1628: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1628#discussion_r1909594021 ## src/ast/dml.rs: ## @@ -547,7 +561,15 @@ impl Display for Insert { write!(f, "{source}")?; } -if self.source.is_none()

Re: [I] Jan 1, 2025: This week(s) in DataFusion [datafusion]

2025-01-09 Thread via GitHub
alamb commented on issue #13970: URL: https://github.com/apache/datafusion/issues/13970#issuecomment-2581464312 Blog on DuckDB vs Datafusion: https://performancede.substack.com/p/duckdb-vs-datafusion -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] Add support for ClickHouse `FORMAT` on `INSERT` [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio commented on code in PR #1628: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1628#discussion_r1909593827 ## src/ast/query.rs: ## @@ -2465,14 +2465,25 @@ impl fmt::Display for GroupByExpr { #[cfg_attr(feature = "serde", derive(Serialize, Deserialize))] #[c

Re: [PR] perf: Improve query planning to more reliably fall back to columnar shuffle when native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
kazuyukitanimura commented on code in PR #1209: URL: https://github.com/apache/datafusion-comet/pull/1209#discussion_r1909581449 ## spark/src/test/scala/org/apache/spark/sql/CometTestBase.scala: ## @@ -78,6 +78,7 @@ abstract class CometTestBase conf.set(CometConf.COMET_ENAB

Re: [PR] feat: support `INSERT INTO [TABLE] FUNCTION` of Clickhouse [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio commented on PR #1633: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1633#issuecomment-2581449238 @byte-sourcerer could you take a look when you get the chance to resolve the conflicts on the branch -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Fix MySQL parsing of GRANT, REVOKE, and CREATE VIEW [datafusion-sqlparser-rs]

2025-01-09 Thread via GitHub
iffyio merged PR #1538: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr

Re: [PR] feat: add support for array_remove expression [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1179: URL: https://github.com/apache/datafusion-comet/pull/1179#discussion_r1909509839 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2529,4 +2529,21 @@ class CometExpressionSuite extends CometTestBase with Adaptiv

Re: [PR] feat: add support for array_remove expression [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1179: URL: https://github.com/apache/datafusion-comet/pull/1179#discussion_r1909508060 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2529,4 +2529,21 @@ class CometExpressionSuite extends CometTestBase with Adaptiv

Re: [PR] feat: add support for array_remove expression [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1179: URL: https://github.com/apache/datafusion-comet/pull/1179#discussion_r1909503394 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2529,4 +2529,21 @@ class CometExpressionSuite extends CometTestBase with Adaptiv

Re: [PR] chore: See which Spark SQL tests fail if we set default shuffle mode to jvm [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove closed pull request #1247: chore: See which Spark SQL tests fail if we set default shuffle mode to jvm URL: https://github.com/apache/datafusion-comet/pull/1247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] chore: See which Spark SQL tests fail if we set default shuffle mode to jvm [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on PR #1247: URL: https://github.com/apache/datafusion-comet/pull/1247#issuecomment-2581331512 Filed https://github.com/apache/datafusion-comet/issues/1256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[I] [EPIC] Fix Spark SQL failures when shuffle mode is jvm [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove opened a new issue, #1256: URL: https://github.com/apache/datafusion-comet/issues/1256 ### Describe the bug When we run Spark SQL tests with shuffle mode of jvm we see some test failures. - https://github.com/apache/datafusion-comet/issues/1238 - https://github.com

Re: [PR] minor: re-export TypeSignatureClass from the datafusion-expr package [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #14051: URL: https://github.com/apache/datafusion/pull/14051 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Refactor into `LexOrdering::collapse`, `LexRequirement::collapse` avoid clone [datafusion]

2025-01-09 Thread via GitHub
alamb commented on code in PR #14038: URL: https://github.com/apache/datafusion/pull/14038#discussion_r1909473831 ## datafusion/physical-expr-common/src/sort_expr.rs: ## @@ -540,6 +557,33 @@ impl LexRequirement { .collect(), ) } + +/// Construc

Re: [PR] Fix bug in `nth_value` when `ignoreNulls` is true and no nulls in values [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #14042: URL: https://github.com/apache/datafusion/pull/14042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

[I] Clarify logic in `nth_value` function [datafusion]

2025-01-09 Thread via GitHub
alamb opened a new issue, #14063: URL: https://github.com/apache/datafusion/issues/14063 took me a while to figure out what's happening here; maybe instead check slice.nulls() explicitly and return None if it doesn't exist (i.e. there's no nulls)? also the pre-existing

Re: [PR] Fix bug in `nth_value` when `ignoreNulls` is true and no nulls in values [datafusion]

2025-01-09 Thread via GitHub
alamb commented on code in PR #14042: URL: https://github.com/apache/datafusion/pull/14042#discussion_r1909471428 ## datafusion/functions-window/src/nth_value.rs: ## @@ -360,9 +360,10 @@ impl PartitionEvaluator for NthValueEvaluator { })

Re: [I] `ignoreNulls` in `nth_value` doesn't work when no nulls are in input [datafusion]

2025-01-09 Thread via GitHub
alamb closed issue #14041: `ignoreNulls` in `nth_value` doesn't work when no nulls are in input URL: https://github.com/apache/datafusion/issues/14041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Clarify logic in `nth_value` function [datafusion]

2025-01-09 Thread via GitHub
alamb commented on issue #14063: URL: https://github.com/apache/datafusion/issues/14063#issuecomment-2581309897 I think this is a good initial issue for someone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] docs(ci): use up-to-date protoc with docs.rs [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #14048: URL: https://github.com/apache/datafusion/pull/14048#issuecomment-2581306041 Thanks again @wackywendell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [I] datafusion-substrait API docs on docs.rs are broken [datafusion]

2025-01-09 Thread via GitHub
alamb closed issue #13853: datafusion-substrait API docs on docs.rs are broken URL: https://github.com/apache/datafusion/issues/13853 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] docs(ci): use up-to-date protoc with docs.rs [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #14048: URL: https://github.com/apache/datafusion/pull/14048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Improve perfomance of `reverse` function [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #14025: URL: https://github.com/apache/datafusion/pull/14025#issuecomment-2581304653 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Improve perfomance of `reverse` function [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #14025: URL: https://github.com/apache/datafusion/pull/14025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] test: Add plan execution during tests for bounded source [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #14013: URL: https://github.com/apache/datafusion/pull/14013#issuecomment-2581304264 Is this PR ready to merge @berkaysynnada ? Looks good to me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Custom scalar to sql overrides support for DuckDB Unparser dialect [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #13915: URL: https://github.com/apache/datafusion/pull/13915#issuecomment-2581302761 Thanks for the contribution @sgrebnov and for the review @goldmedal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] Custom scalar to sql overrides support for DuckDB Unparser dialect [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #13915: URL: https://github.com/apache/datafusion/pull/13915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] feat: add `AsyncCatalogProvider` helpers for asynchronous catalogs [datafusion]

2025-01-09 Thread via GitHub
alamb commented on code in PR #13800: URL: https://github.com/apache/datafusion/pull/13800#discussion_r1909466549 ## datafusion/catalog/src/async.rs: ## @@ -0,0 +1,747 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

[PR] minor: Add link to example for `AsyncSchemaProvider` [datafusion]

2025-01-09 Thread via GitHub
alamb opened a new pull request, #14062: URL: https://github.com/apache/datafusion/pull/14062 ## Which issue does this PR close? - Follow on to https://github.com/apache/datafusion/pull/13800 ## Rationale for this change Add a link to the relevant example to the documenta

Re: [PR] feat: add `AsyncCatalogProvider` helpers for asynchronous catalogs [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #13800: URL: https://github.com/apache/datafusion/pull/13800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [I] Make all SchemaProvider trait APIs async [datafusion]

2025-01-09 Thread via GitHub
alamb closed issue #10339: Make all SchemaProvider trait APIs async URL: https://github.com/apache/datafusion/issues/10339 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] feat: add `AsyncCatalogProvider` helpers for asynchronous catalogs [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #13800: URL: https://github.com/apache/datafusion/pull/13800#issuecomment-2581296524 Thanks again @westonpace , @comphead and @findepi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] feat: add `AsyncCatalogProvider` helpers for asynchronous catalogs [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #13800: URL: https://github.com/apache/datafusion/pull/13800#issuecomment-2581296277 🚀 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

Re: [PR] Simplify error handling in case.rs (#13990) [datafusion]

2025-01-09 Thread via GitHub
alamb commented on PR #14033: URL: https://github.com/apache/datafusion/pull/14033#issuecomment-2581281922 Thanks again @cj-zhukov -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Simplify error handling in case.rs (#13990) [datafusion]

2025-01-09 Thread via GitHub
alamb merged PR #14033: URL: https://github.com/apache/datafusion/pull/14033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [I] Simplify error handling in case.rs [datafusion]

2025-01-09 Thread via GitHub
alamb closed issue #13990: Simplify error handling in case.rs URL: https://github.com/apache/datafusion/issues/13990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] fix: Fall back to Spark for unsupported partition or sort expressions in window aggregates [datafusion-comet]

2025-01-09 Thread via GitHub
codecov-commenter commented on PR #1253: URL: https://github.com/apache/datafusion-comet/pull/1253#issuecomment-2581266227 ## [Codecov](https://app.codecov.io/gh/apache/datafusion-comet/pull/1253?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_ca

[I] Implement cast from Long to Timestamp [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove opened a new issue, #1255: URL: https://github.com/apache/datafusion-comet/issues/1255 ### What is the problem the feature request solves? We do not currently support casting from long to timestamp. The relevant test in CometCastSuite is ignored: ```scala ignore(

Re: [PR] fix: incorrect error message of function_length_check [datafusion]

2025-01-09 Thread via GitHub
comphead commented on code in PR #14056: URL: https://github.com/apache/datafusion/pull/14056#discussion_r1909371158 ## datafusion/expr/src/type_coercion/functions.rs: ## @@ -440,13 +440,13 @@ fn get_valid_types( fn function_length_check(length: usize, expected_length: usiz

Re: [PR] perf: Improve query planning to more reliably fall back to columnar shuffle when native shuffle is not supported [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on PR #1209: URL: https://github.com/apache/datafusion-comet/pull/1209#issuecomment-2581116828 Thanks for the review @viirya. @kazuyukitanimura @parthchandra did you also want to review? -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] build(deps): bump datafusion from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #985: URL: https://github.com/apache/datafusion-python/pull/985#issuecomment-2581116707 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump datafusion from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer commented on PR #985: URL: https://github.com/apache/datafusion-python/pull/985#issuecomment-2581116621 Closing due to update already applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump datafusion from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer closed pull request #985: build(deps): bump datafusion from 43.0.0 to 44.0.0 URL: https://github.com/apache/datafusion-python/pull/985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] build(deps): bump datafusion-substrait from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #986: URL: https://github.com/apache/datafusion-python/pull/986#issuecomment-2581116356 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump datafusion-substrait from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer closed pull request #986: build(deps): bump datafusion-substrait from 43.0.0 to 44.0.0 URL: https://github.com/apache/datafusion-python/pull/986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] build(deps): bump datafusion-substrait from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer commented on PR #986: URL: https://github.com/apache/datafusion-python/pull/986#issuecomment-2581116293 Closing due to update already applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump datafusion-ffi from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer closed pull request #988: build(deps): bump datafusion-ffi from 43.0.0 to 44.0.0 URL: https://github.com/apache/datafusion-python/pull/988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] build(deps): bump datafusion-ffi from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #988: URL: https://github.com/apache/datafusion-python/pull/988#issuecomment-2581116067 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump datafusion-ffi from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer commented on PR #988: URL: https://github.com/apache/datafusion-python/pull/988#issuecomment-2581115992 Closing due to update already applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump datafusion-functions-window-common from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #989: URL: https://github.com/apache/datafusion-python/pull/989#issuecomment-2581115706 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump datafusion-functions-window-common from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer commented on PR #989: URL: https://github.com/apache/datafusion-python/pull/989#issuecomment-2581115622 Closing due to update already applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] build(deps): bump datafusion-functions-window-common from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer closed pull request #989: build(deps): bump datafusion-functions-window-common from 43.0.0 to 44.0.0 URL: https://github.com/apache/datafusion-python/pull/989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] build(deps): bump datafusion-proto from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer closed pull request #990: build(deps): bump datafusion-proto from 43.0.0 to 44.0.0 URL: https://github.com/apache/datafusion-python/pull/990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] build(deps): bump datafusion-proto from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
dependabot[bot] commented on PR #990: URL: https://github.com/apache/datafusion-python/pull/990#issuecomment-2581115257 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump datafusion-proto from 43.0.0 to 44.0.0 [datafusion-python]

2025-01-09 Thread via GitHub
timsaucer commented on PR #990: URL: https://github.com/apache/datafusion-python/pull/990#issuecomment-2581115159 Closing due to update already applied. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] Fix build issues on latest stable Rust toolchain (1.84) [datafusion]

2025-01-09 Thread via GitHub
Omega359 opened a new issue, #14061: URL: https://github.com/apache/datafusion/issues/14061 ### Describe the bug 1.84 was released today and /dev/rust_lint.sh does not currently pass under this release. ### To Reproduce _No response_ ### Expected behavior _

Re: [PR] chore: extract hash_funcs expressions to folders based on spark grouping [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove merged PR #1221: URL: https://github.com/apache/datafusion-comet/pull/1221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@

Re: [PR] feat: add support for array_remove expression [datafusion-comet]

2025-01-09 Thread via GitHub
jatin510 commented on code in PR #1179: URL: https://github.com/apache/datafusion-comet/pull/1179#discussion_r1909287084 ## spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala: ## @@ -2529,4 +2529,21 @@ class CometExpressionSuite extends CometTestBase with Adaptive

Re: [PR] fix: Fall back to Spark for unsupported partition or sort expressions in window aggregates [datafusion-comet]

2025-01-09 Thread via GitHub
andygrove commented on code in PR #1253: URL: https://github.com/apache/datafusion-comet/pull/1253#discussion_r1909275211 ## spark/src/test/scala/org/apache/comet/exec/CometAggregateSuite.scala: ## @@ -89,6 +89,24 @@ class CometAggregateSuite extends CometTestBase with Adaptive

Re: [PR] fix: making is_nullable ansi aware for sum_decimal and avg_decimal [datafusion-comet]

2025-01-09 Thread via GitHub
vaibhawvipul closed pull request #981: fix: making is_nullable ansi aware for sum_decimal and avg_decimal URL: https://github.com/apache/datafusion-comet/pull/981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

  1   2   >