kevinwilfong opened a new pull request, #13614:
URL: https://github.com/apache/gluten/pull/13614

   ## What changes are proposed in this pull request?
   
   There is currently a rewrite rule that converts an IN expression with a 
non-foldable expression in the list from, e.g.
   
   `x IN (1, y, z)`
   
   to
   
   `x IN (1) OR x = y OR x = z`
   
   pulling the non-foldable expressions out of the IN list into equals 
expressions.
   
   This is only supported in filter expressions and filters pushed down to DSV1 
readers.
   
   This change extends support to expressions more generally (e.g. in 
projections and filter pushed down to DSV2 readers) by implementing the rewrite 
in the ExpressionConverter itself, where we currently check and fallback if we 
find an IN expression with non-foldable expressions in the list.
   
   It also fixes a bug, where if the value (i.e. the needle) is 
non-deterministic. In this case introducing the equals expressions changes the 
semantics as it may have a different value in each check.
   
   e.g.
   
   `rand() in (...)`
   
   is not the same as
   
   `rand() IN (...) OR rand() = ... OR rand() = ... OR ...`
   
   ## How was this patch tested?
   
   Tested with GlutenExpressionMappingSuite and MiscOperatorSuite, including 
the new coverage added for projections and DSV2 filters, and the 
non-deterministic case that was fixed.
   
   ## Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: OpenAI Codex GPT-6
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to