Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
alamb commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608748477 ## datafusion/functions/src/math/monotonicity.rs: ## @@ -0,0 +1,241 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
alamb commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608116040 ## datafusion/expr/src/udf.rs: ## @@ -426,6 +467,59 @@ pub trait ScalarUDFImpl: Debug + Send + Sync { false } +/// Computes the output interval f

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
berkaysynnada commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608112791 ## datafusion/functions/src/math/monotonicity.rs: ## @@ -0,0 +1,241 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
berkaysynnada commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608102197 ## datafusion/expr/src/udf.rs: ## @@ -426,6 +467,59 @@ pub trait ScalarUDFImpl: Debug + Send + Sync { false } +/// Computes the output in

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
alamb commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608079456 ## datafusion/functions/src/math/monotonicity.rs: ## @@ -0,0 +1,241 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-21 Thread via GitHub
alamb commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1608069310 ## datafusion/expr/src/udf.rs: ## @@ -426,6 +467,59 @@ pub trait ScalarUDFImpl: Debug + Send + Sync { false } +/// Computes the output interval f

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-17 Thread via GitHub
ozankabak merged PR #10504: URL: https://github.com/apache/datafusion/pull/10504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@data

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-17 Thread via GitHub
ozankabak commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2117335725 I will go ahead and merge this and we will fix any issues with a quick follow-on PR in case @alamb discovers any when he has time to take a look. -- This is an automated message

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-17 Thread via GitHub
berkaysynnada commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2117059842 > 3\. If all the inputs of a function are `Singleton`, should the output `SortProperties` also be `Singleton`? If so, maybe we can do this check once before calling individual

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-16 Thread via GitHub
alamb commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2116425657 > @alamb, it'd be good if you could take a quick look before we merge (but not necessary as I reviewed carefully). FYI, defining an "all-increasing" function becomes a one-liner in thi

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-15 Thread via GitHub
alamb commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2113113341 > Having range information would help ScalarFunctionExpr's order calculations since many of them have monotonicity pattern on some defined intervals. I have given an example of it for

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-15 Thread via GitHub
alamb commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2113111066 > @berkaysynnada kindly reminded me that the type undergoing the refactor disappears in this extended formulation. In that case this PR may supersede the refactor one. Indeed, t

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-15 Thread via GitHub
berkaysynnada commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1601707815 ## datafusion/core/src/physical_optimizer/enforce_distribution.rs: ## @@ -3572,7 +3572,11 @@ pub(crate) mod tests { expr: col("c", &schema).unwr

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-15 Thread via GitHub
ozankabak commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2112592630 That PR and this are orthogonal. The work in this PR will simply inherit/benefit from the refactor in the other PR. I will review this one in detail tomorrow. -- This is a

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-15 Thread via GitHub
berkaysynnada commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2111732264 > I haven't had a chance to review this PR yet @berkaysynnada -- I wonder if you have seen the API in #10117 from @tinfoil-knight Yes, I have. It is becoming a nicer and

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-14 Thread via GitHub
alamb commented on code in PR #10504: URL: https://github.com/apache/datafusion/pull/10504#discussion_r1600576360 ## datafusion/expr/src/udf.rs: ## @@ -202,18 +203,27 @@ impl ScalarUDF { Arc::new(move |args| captured.invoke(args)) } -/// This function specifi

Re: [PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-14 Thread via GitHub
alamb commented on PR #10504: URL: https://github.com/apache/datafusion/pull/10504#issuecomment-2111036745 I haven't had a chance to review this PR yet @berkaysynnada -- I wonder if you have seen the API in https://github.com/apache/datafusion/pull/10117 from @tinfoil-knight -- This is

[PR] PhysicalExpr Orderings with Range Information [datafusion]

2024-05-14 Thread via GitHub
berkaysynnada opened a new pull request, #10504: URL: https://github.com/apache/datafusion/pull/10504 ## Which issue does this PR close? Closes #9832. ## Rationale for this change There exists a bug in `CastExpr` orderings, as detailed in the [issue](http