Re: [I] Better timezone functionalities [datafusion]

2024-05-14 Thread via GitHub
mhilton commented on issue #10368: URL: https://github.com/apache/datafusion/issues/10368#issuecomment-2111684373 In postgresql the `AT TIME ZONE` expression changes a `TIMESTAMP WITH TIME ZONE` value to `TIMESTAMP WITHOUT TIME ZONE` value at the wall-clock time in the requested time zone (

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
vidyasankarv commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600987926 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -1444,13 +1483,136 @@ fn parse_str_to_time_only_timestamp(value: &str) -> CometResult> { O

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on PR #10386: URL: https://github.com/apache/datafusion/pull/10386#issuecomment-2111641749 I removed the dynamic dispatch API for now because I think there is a better way to write it, but I can't really do that until we get to a point where we have a `TreeNodeRewr

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
vidyasankarv commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600979987 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,8 +565,54 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHel

Re: [I] Use `min_value` and `max_value` on statistics to avoid `ExecutionPlan.execute` [datafusion]

2024-05-14 Thread via GitHub
samuelcolvin commented on issue #10400: URL: https://github.com/apache/datafusion/issues/10400#issuecomment-2111636911 No code yet, if you'd like to work on this, feel free. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
vidyasankarv commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600978368 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,8 +565,54 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHel

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
vidyasankarv commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600977960 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -635,6 +642,38 @@ impl Cast { Ok(cast_array) } +fn cast_string_to_date( +

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on PR #10386: URL: https://github.com/apache/datafusion/pull/10386#issuecomment-2111619964 Alright I think this is in a good state now. I added a dynamic dispatch API which I think could be useful if we ever add `TreeNodeRewriter` impl for `OptimizerRule`, then we

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600965226 ## datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs: ## @@ -673,7 +659,7 @@ impl<'a> ConstEvaluator<'a> { /// * `false = true` and `true =

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600964946 ## datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs: ## @@ -472,9 +459,8 @@ struct ConstEvaluator<'a> { input_batch: RecordBatch, }

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600964827 ## datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs: ## @@ -447,12 +435,11 @@ impl TreeNodeRewriter for Canonicalizer { } } -#[allo

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600964505 ## datafusion/optimizer/src/lib.rs: ## @@ -48,6 +48,7 @@ pub mod propagate_empty_relation; pub mod push_down_filter; pub mod push_down_limit; pub mod re

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600964245 ## datafusion/common/src/tree_node.rs: ## @@ -172,7 +172,7 @@ pub trait TreeNode: Sized { /// TreeNodeRewriter::f_up(ChildNode2) /// TreeNodeRewr

Re: [PR] feat: RewriteCycle API for short-circuiting optimizer loops [datafusion]

2024-05-14 Thread via GitHub
erratic-pattern commented on code in PR #10386: URL: https://github.com/apache/datafusion/pull/10386#discussion_r1600964048 ## datafusion/common/src/tree_node.rs: ## @@ -503,7 +503,7 @@ pub trait TreeNodeVisitor: Sized { /// /// # See Also: /// * [`TreeNode::visit`] to inspec

Re: [I] Use `min_value` and `max_value` on statistics to avoid `ExecutionPlan.execute` [datafusion]

2024-05-14 Thread via GitHub
edmondop commented on issue #10400: URL: https://github.com/apache/datafusion/issues/10400#issuecomment-2111592876 @samuelcolvin have you already started to write code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] Support UDAF to align Builtin aggregate function [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on PR #10493: URL: https://github.com/apache/datafusion/pull/10493#issuecomment-2111520905 > However, given that the point of this exercise is to port built in functions to UDAF I think all the new features will be properly tested once that porting is complete Do

Re: [PR] Support UDAF to align Builtin aggregate function [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 merged PR #10493: URL: https://github.com/apache/datafusion/pull/10493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [PR] Support UDAF to align Builtin aggregate function [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on PR #10493: URL: https://github.com/apache/datafusion/pull/10493#issuecomment-2111521693 Thanks @alamb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Support UDAF to align Builtin aggregate function [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on code in PR #10493: URL: https://github.com/apache/datafusion/pull/10493#discussion_r1600900768 ## datafusion/core/src/physical_optimizer/aggregate_statistics.rs: ## @@ -137,43 +134,48 @@ fn take_optimizable(node: &dyn ExecutionPlan) -> Option> None

Re: [PR] Support UDAF to align Builtin aggregate function [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on code in PR #10493: URL: https://github.com/apache/datafusion/pull/10493#discussion_r1600899609 ## datafusion/optimizer/src/analyzer/count_wildcard_rule.rs: ## @@ -54,23 +56,37 @@ fn is_wildcard(expr: &Expr) -> bool { } fn is_count_star_aggregate(aggre

Re: [PR] feat: Support spark base64 function [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on PR #420: URL: https://github.com/apache/datafusion-comet/pull/420#issuecomment-2111517172 Could we find another library supporting that? We can implement custom base64 function in Comet. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600898381 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef,

Re: [PR] Fix window expr deserialization [datafusion]

2024-05-14 Thread via GitHub
lewiszlw commented on code in PR #10506: URL: https://github.com/apache/datafusion/pull/10506#discussion_r1600897119 ## datafusion/proto/tests/cases/roundtrip_physical_plan.rs: ## @@ -253,8 +253,7 @@ fn roundtrip_nested_loop_join() -> Result<()> { fn roundtrip_window() -> Resul

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600896291 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef,

Re: [PR] Fix window expr deserialization [datafusion]

2024-05-14 Thread via GitHub
lewiszlw commented on PR #10506: URL: https://github.com/apache/datafusion/pull/10506#issuecomment-2111511903 > Thank you for this PR @lewiszlw -- I can't quite understand what it is fixing. Is there any way you can add a unit test to the round trip serialization tests that fails prior to t

[PR] feat: Expose Parquet Schema Adapter [datafusion]

2024-05-14 Thread via GitHub
HawaiianSpork opened a new pull request, #10515: URL: https://github.com/apache/datafusion/pull/10515 Allow users of the Datafusion Parquet physical executor to define how to map parquet schema to the table schema. This can be useful as there can be layers on top of parquet like Delta

Re: [I] feat: enable optional UDF arguments in `regexp_*` functions [datafusion]

2024-05-14 Thread via GitHub
jonahgao closed issue #10512: feat: enable optional UDF arguments in `regexp_*` functions URL: https://github.com/apache/datafusion/issues/10512 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
jonahgao merged PR #10514: URL: https://github.com/apache/datafusion/pull/10514 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
advancedxy commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600892577 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef

Re: [PR] feat: Support spark base64 function [datafusion-comet]

2024-05-14 Thread via GitHub
leoluan2009 commented on PR #420: URL: https://github.com/apache/datafusion-comet/pull/420#issuecomment-2111490861 > Looks like there are incompatible results compared with Spark. datafusion use Basic type of base64 scheme while spark use MIME type. After search the source of https:/

Re: [PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on code in PR #10514: URL: https://github.com/apache/datafusion/pull/10514#discussion_r1600873617 ## datafusion/functions/src/regex/mod.rs: ## @@ -28,12 +28,53 @@ make_udf_function!( REGEXP_REPLACE, regexp_replace ); -export_functions!(( -

Re: [PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
jonahgao commented on code in PR #10514: URL: https://github.com/apache/datafusion/pull/10514#discussion_r1600872435 ## datafusion/functions/src/regex/mod.rs: ## @@ -28,12 +28,53 @@ make_udf_function!( REGEXP_REPLACE, regexp_replace ); -export_functions!(( -regexp

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600867066 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef,

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600867066 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef,

[PR] feat: expose `named_struct` in python [datafusion-python]

2024-05-14 Thread via GitHub
Michael-J-Ward opened a new pull request, #700: URL: https://github.com/apache/datafusion-python/pull/700 Ref #692 # Which issue does this PR close? Closes #692 # Rationale for this change This UDF has not been ported over and was requested. # What changes are

Re: [PR] feat: add python bindings for ends_with function [datafusion-python]

2024-05-14 Thread via GitHub
andygrove merged PR #693: URL: https://github.com/apache/datafusion-python/pull/693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@d

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600866070 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -1444,13 +1483,136 @@ fn parse_str_to_time_only_timestamp(value: &str) -> CometResult> { Ok(S

Re: [PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
jonahgao commented on code in PR #10514: URL: https://github.com/apache/datafusion/pull/10514#discussion_r1600865497 ## datafusion/functions/src/regex/mod.rs: ## @@ -28,12 +28,53 @@ make_udf_function!( REGEXP_REPLACE, regexp_replace ); -export_functions!(( -regexp

Re: [PR] tests: Move random data generation methods from CometCastSuite to new DataGenerator class [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #426: URL: https://github.com/apache/datafusion-comet/pull/426#discussion_r1600860138 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -35,7 +35,11 @@ import org.apache.comet.expressions.{CometCast, Compatible} class CometCa

Re: [I] Update the DataFusion in Python website [datafusion-python]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on issue #687: URL: https://github.com/apache/datafusion-python/issues/687#issuecomment-2111455295 Thanks, @Weijun-H And I see the new logo on https://datafusion.apache.org/ ![new-logo](https://datafusion.apache.org/_images/2x_bgwhite_original.png) -

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
advancedxy commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600856424 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef

Re: [I] Update the DataFusion in Python website [datafusion-python]

2024-05-14 Thread via GitHub
Weijun-H commented on issue #687: URL: https://github.com/apache/datafusion-python/issues/687#issuecomment-2111442760 @Michael-J-Ward, I have updated the issue. I hope this makes it clearer. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Safeguard round function decimal places overflow [datafusion]

2024-05-14 Thread via GitHub
github-actions[bot] closed pull request #6232: Safeguard round function decimal places overflow URL: https://github.com/apache/datafusion/pull/6232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] [built-in function] add `greatest(T,...)` and `least(T,...)` SQL functions [datafusion]

2024-05-14 Thread via GitHub
github-actions[bot] closed pull request #6527: [built-in function] add `greatest(T,...)` and `least(T,...)` SQL functions URL: https://github.com/apache/datafusion/pull/6527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Minor: remove unused source file `udf.rs` [datafusion]

2024-05-14 Thread via GitHub
jonahgao commented on PR #10497: URL: https://github.com/apache/datafusion/pull/10497#issuecomment-2111431245 Thank you for double-checking @alamb . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Minor: remove unused source file `udf.rs` [datafusion]

2024-05-14 Thread via GitHub
jonahgao merged PR #10497: URL: https://github.com/apache/datafusion/pull/10497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600836820 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef,

Re: [PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on PR #10514: URL: https://github.com/apache/datafusion/pull/10514#issuecomment-2111418206 Requesting review from @jayzhan211 and @jonahgao -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] doc: Add Plan Stability Testing to development guide [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on PR #432: URL: https://github.com/apache/datafusion-comet/pull/432#issuecomment-2111417474 Merged. Thanks @kazuyukitanimura @andygrove -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] doc: Add Plan Stability Testing to development guide [datafusion-comet]

2024-05-14 Thread via GitHub
viirya merged PR #432: URL: https://github.com/apache/datafusion-comet/pull/432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dataf

[PR] feat: optional args for regexp_* UDFs [datafusion]

2024-05-14 Thread via GitHub
Michael-J-Ward opened a new pull request, #10514: URL: https://github.com/apache/datafusion/pull/10514 ## Which issue does this PR close? Closes #10512 ## Rationale for this change The inner implementations of these UDFs depend on the number of arguments passed. This exp

Re: [I] cannot import datafusion-37.1.0 in python [datafusion]

2024-05-14 Thread via GitHub
l1t1 commented on issue #10513: URL: https://github.com/apache/datafusion/issues/10513#issuecomment-2111415672 BTW, the above issue occurs in windows 7 python 3.8. version 37.1.0 can be imported in windows 10 python 3.12, but the datafusion-cli is still missing. ```python C:\Users\L

Re: [PR] doc: Add Plan Stability Testing to development guide [datafusion-comet]

2024-05-14 Thread via GitHub
viirya commented on code in PR #432: URL: https://github.com/apache/datafusion-comet/pull/432#discussion_r1600831651 ## docs/source/contributor-guide/development.md: ## @@ -85,6 +85,32 @@ mvn test -Dsuites="org.apache.comet.CometCastSuite valid" -Dskip.surefire.tests= Other

[I] cannot import datafusion-37.1.0 in python [datafusion]

2024-05-14 Thread via GitHub
l1t1 opened a new issue, #10513: URL: https://github.com/apache/datafusion/issues/10513 ### Describe the bug I upgrade to the version 37.1.0 and cannot import it in python. ### To Reproduce ``` D:\>pip install datafusion -U Looking in indexes: https://pypi.tuna.tsi

Re: [PR] tests: Move random data generation methods from CometCastSuite to new DataGenerator class [datafusion-comet]

2024-05-14 Thread via GitHub
advancedxy commented on code in PR #426: URL: https://github.com/apache/datafusion-comet/pull/426#discussion_r1600826060 ## spark/src/test/scala/org/apache/comet/DataGenerator.scala: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or mo

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
advancedxy commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600824499 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -104,66 +108,110 @@ fn test_murmur3() { let _expected = vec![ 142593372, 1485273170, -9705

Re: [PR] feat: Add xxhash64 function support [datafusion-comet]

2024-05-14 Thread via GitHub
advancedxy commented on code in PR #424: URL: https://github.com/apache/datafusion-comet/pull/424#discussion_r1600823779 ## core/src/execution/datafusion/spark_hash.rs: ## @@ -193,27 +241,67 @@ macro_rules! hash_array_decimal { fn create_hashes_dictionary( array: &ArrayRef

Re: [I] datafusion-cli not installed [datafusion]

2024-05-14 Thread via GitHub
l1t1 commented on issue #9294: URL: https://github.com/apache/datafusion/issues/9294#issuecomment-2111399270 version 37.1.0 still has the issue ``` D:\>pip install datafusion -U Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Requirement already satisfied: datafusion

Re: [I] feat: enable optional UDF arguments in `regexp_*` functions [datafusion]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on issue #10512: URL: https://github.com/apache/datafusion/issues/10512#issuecomment-2111393585 take. will have a PR up shortly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[I] feat: enable optional UDF arguments in `regexp_*` functions [datafusion]

2024-05-14 Thread via GitHub
Michael-J-Ward opened a new issue, #10512: URL: https://github.com/apache/datafusion/issues/10512 ### Is your feature request related to a problem or challenge? The same discussion / reasoning as in #10424. ### Describe the solution you'd like The same solution as impleme

Re: [PR] build(deps): bump mimalloc from 0.1.39 to 0.1.41 [datafusion-python]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on PR #657: URL: https://github.com/apache/datafusion-python/pull/657#issuecomment-2111384249 ## Blocked on - [ ] [upstream tracking issue](https://github.com/purpleprotocol/mimalloc_rust/issues/111) in `mimalloc` for these build failures. -- This is

Re: [PR] build(deps): bump mimalloc from 0.1.39 to 0.1.41 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #657: URL: https://github.com/apache/datafusion-python/pull/657#issuecomment-2111381100 Sorry, only users with push access can use that command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] build(deps): bump mimalloc from 0.1.39 to 0.1.41 [datafusion-python]

2024-05-14 Thread via GitHub
Michael-J-Ward commented on PR #657: URL: https://github.com/apache/datafusion-python/pull/657#issuecomment-2111381074 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] chore: Create initial release process scripts for official ASF source release [datafusion-comet]

2024-05-14 Thread via GitHub
comphead commented on code in PR #429: URL: https://github.com/apache/datafusion-comet/pull/429#discussion_r1600810257 ## core/Cargo.toml: ## @@ -16,17 +16,21 @@ # under the License. [package] -name = "comet" +name = "datafusion-comet" version = "0.1.0" +homepage = "https:/

Re: [PR] fix: parsing timestamp with date format [datafusion]

2024-05-14 Thread via GitHub
shanretoo commented on PR #10476: URL: https://github.com/apache/datafusion/pull/10476#issuecomment-2111369835 It seems I don't have write access to this repo. Please merge this pr for me, thanks! @alamb -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Implement conversion from ColumnStatistics to NullableInterval [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on code in PR #10510: URL: https://github.com/apache/datafusion/pull/10510#discussion_r1600797390 ## datafusion/expr/src/interval_arithmetic.rs: ## @@ -1640,9 +1693,12 @@ impl NullableInterval { datatype: DataType::Boolean,

Re: [PR] Minor: Extend more style of udaf `expr_fn`, Remove order args for`covar_samp` and `covar_pop` [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 merged PR #10492: URL: https://github.com/apache/datafusion/pull/10492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [PR] Minor: Extend more style of udaf `expr_fn`, Remove order args for`covar_samp` and `covar_pop` [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on code in PR #10492: URL: https://github.com/apache/datafusion/pull/10492#discussion_r1600793234 ## datafusion/proto/tests/cases/roundtrip_logical_plan.rs: ## @@ -622,8 +622,8 @@ async fn roundtrip_expr_api() -> Result<()> { ), array_repla

Re: [PR] tests: Move random data generation methods from CometCastSuite to new DataGenerator class [datafusion-comet]

2024-05-14 Thread via GitHub
kazuyukitanimura commented on code in PR #426: URL: https://github.com/apache/datafusion-comet/pull/426#discussion_r1600787943 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -35,7 +35,11 @@ import org.apache.comet.expressions.{CometCast, Compatible} class

Re: [PR] chore: use fullpath in macro to avoid declaring in other module [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 merged PR #10503: URL: https://github.com/apache/datafusion/pull/10503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@dat

Re: [I] Remove `Expr::GetIndexedField` and `GetFieldAccess` and always use function `get_field` for indexing [datafusion]

2024-05-14 Thread via GitHub
jayzhan211 commented on issue #10374: URL: https://github.com/apache/datafusion/issues/10374#issuecomment-2111340945 > @alamb Do you think we should also rewrite the array operator to function in parser step? It is currently in optimizer step. I think the downside of moving array rewrite in

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
kazuyukitanimura commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600592149 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,8 +565,54 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPla

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600746929 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,9 +563,33 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600746050 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,9 +563,33 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600745021 ## spark/src/test/scala/org/apache/comet/CometCastSuite.scala: ## @@ -563,8 +565,54 @@ class CometCastSuite extends CometTestBase with AdaptiveSparkPlanHelper

Re: [PR] feat: Implement Spark-compatible CAST from String to Date [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #383: URL: https://github.com/apache/datafusion-comet/pull/383#discussion_r1600729203 ## core/src/execution/datafusion/expressions/cast.rs: ## @@ -635,6 +642,38 @@ impl Cast { Ok(cast_array) } +fn cast_string_to_date( +

Re: [PR] doc: Add Plan Stability Testing to development guide [datafusion-comet]

2024-05-14 Thread via GitHub
andygrove commented on code in PR #432: URL: https://github.com/apache/datafusion-comet/pull/432#discussion_r1600714950 ## docs/source/contributor-guide/development.md: ## @@ -85,6 +85,32 @@ mvn test -Dsuites="org.apache.comet.CometCastSuite valid" -Dskip.surefire.tests= Oth

Re: [PR] build(deps): bump async-trait from 0.1.77 to 0.1.80 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #637: URL: https://github.com/apache/datafusion-python/pull/637#issuecomment-2111218990 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump mio from 0.8.10 to 0.8.11 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #608: URL: https://github.com/apache/datafusion-python/pull/608#issuecomment-2111218957 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump parking_lot from 0.12.1 to 0.12.2 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #653: URL: https://github.com/apache/datafusion-python/pull/653#issuecomment-2111218923 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump mio from 0.8.10 to 0.8.11 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #608: URL: https://github.com/apache/datafusion-python/pull/608#issuecomment-2111218953 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #642: URL: https://github.com/apache/datafusion-python/pull/642#issuecomment-2111218967 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump tokio from 1.36.0 to 1.37.0 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #623: URL: https://github.com/apache/datafusion-python/pull/623#issuecomment-2111218943 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump prost from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #627: build(deps): bump prost from 0.12.3 to 0.12.4 URL: https://github.com/apache/datafusion-python/pull/627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #642: URL: https://github.com/apache/datafusion-python/pull/642#issuecomment-2111218973 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump prost-types from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #634: build(deps): bump prost-types from 0.12.3 to 0.12.4 URL: https://github.com/apache/datafusion-python/pull/634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #642: build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 URL: https://github.com/apache/datafusion-python/pull/642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] build(deps): bump pyo3 from 0.20.2 to 0.20.3 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #597: URL: https://github.com/apache/datafusion-python/pull/597#issuecomment-2111218961 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump prost-types from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #634: URL: https://github.com/apache/datafusion-python/pull/634#issuecomment-2111218982 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] chore: Update cargo lock [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #697: chore: Update cargo lock URL: https://github.com/apache/datafusion-python/pull/697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] build(deps): bump pyo3 from 0.20.2 to 0.20.3 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #597: build(deps): bump pyo3 from 0.20.2 to 0.20.3 URL: https://github.com/apache/datafusion-python/pull/597 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] build(deps): bump prost from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #627: URL: https://github.com/apache/datafusion-python/pull/627#issuecomment-2111218976 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump object_store from 0.9.1 to 0.10.0 [datafusion-python]

2024-05-14 Thread via GitHub
dependabot[bot] commented on PR #648: URL: https://github.com/apache/datafusion-python/pull/648#issuecomment-2111218927 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor ver

Re: [PR] build(deps): bump async-trait from 0.1.77 to 0.1.80 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #637: build(deps): bump async-trait from 0.1.77 to 0.1.80 URL: https://github.com/apache/datafusion-python/pull/637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] build(deps): bump prost-types from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #634: build(deps): bump prost-types from 0.12.3 to 0.12.4 URL: https://github.com/apache/datafusion-python/pull/634 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #642: build(deps): bump pyo3-build-config from 0.20.2 to 0.21.2 URL: https://github.com/apache/datafusion-python/pull/642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] build(deps): bump h2 from 0.3.24 to 0.3.26 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #625: build(deps): bump h2 from 0.3.24 to 0.3.26 URL: https://github.com/apache/datafusion-python/pull/625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] build(deps): bump async-trait from 0.1.77 to 0.1.80 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #637: build(deps): bump async-trait from 0.1.77 to 0.1.80 URL: https://github.com/apache/datafusion-python/pull/637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] build(deps): bump tokio from 1.36.0 to 1.37.0 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #623: build(deps): bump tokio from 1.36.0 to 1.37.0 URL: https://github.com/apache/datafusion-python/pull/623 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] build(deps): bump prost from 0.12.3 to 0.12.4 [datafusion-python]

2024-05-14 Thread via GitHub
andygrove closed pull request #627: build(deps): bump prost from 0.12.3 to 0.12.4 URL: https://github.com/apache/datafusion-python/pull/627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

  1   2   3   4   >