rkrishn7 commented on PR #17546: URL: https://github.com/apache/datafusion/pull/17546#issuecomment-3293238849
> My understanding of the issue which I would like to clarify + document in the PR (maybe in the code) if it is correct is that this applies to a situation like: > > ``` > SELECT a > FROM t > WHERE b = 2 > ``` > > So it makes sense that we have a filter that references the column `b` which is not projected. The filter referencing a non-projected column is not an issue, and we _can_ compute equivalence properties for a non-projected column but things downstream (not touched in this PR) will fail / error if they get equivalence properties for a column that is not in the projection because equivalence properties are to communicate information to the parent operator and as far as the parent operator is concerned there is no column `b`. Yes, exactly! The actual errors were happening due to usage downstream when the filter column (not in the output schema) still existed in the equivalence info and and there was an attempt to do something with it. Specifically in the case I saw, `physical_expr::add_offset_to_expr` was failing because the column index was set to `usize::MAX` (I guess to indicate that it wasn't valid anymore). But downstream wasn't aware so the add was still occurring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
