adriangb commented on code in PR #24574: URL: https://github.com/apache/datafusion/pull/24574#discussion_r3837979808
########## datafusion/sqllogictest/test_files/projection_pushdown.slt: ########## @@ -2183,3 +2183,74 @@ physical_plan # Reset the config changed above (the SLT runner expects target_partitions = 4). statement ok SET datafusion.execution.target_partitions = 4; + +##################### +# Section: extraction aliases inside a subquery advance the alias generator +# +# Regression test for `advance_generator_past_existing` in +# `extract_leaf_expressions.rs`. +# +# `ExtractLeafExpressions` names the columns it extracts +# `__datafusion_extracted_N`, handing out N from a shared `AliasGenerator`. That +# prefix is reserved for the optimizer, but nothing stops a user from writing it, +# so before extracting the rule scans the plan for existing +# `__datafusion_extracted_N` aliases and bumps the generator past the highest one. +# +# The bug: that scan used `apply`, which walks plan nodes but does *not* descend +# into subquery plans held inside expressions, while the extraction itself uses +# `transform_down_with_subqueries` and *does* rewrite inside subqueries. So an +# alias living only inside a subquery was invisible to the scan, and extraction +# then minted the very same name next to it. +# +# Each ingredient below is load-bearing: +# +# * The `IN (<subquery>)` sits in the SELECT list, not in a WHERE clause, so +# `decorrelate_predicate_subquery` (which runs earlier) leaves it alone and it +# is still a subquery expression by the time extraction runs. A subquery in +# WHERE would be flattened into the main plan, where the old scan could see it. +# * The alias inside the subquery is literally `__datafusion_extracted_1`. Rename +# it to anything outside the reserved prefix and there is nothing to collide +# with -- the query then passes with or without the fix and guards nothing. +# * The inner `WHERE s['value'] > 120` is what forces extraction to *generate* an +# alias inside that same subquery. Without a leaf expression there, the +# generator is never called where the collision would happen. +# +# Without the fix, extraction reuses `__datafusion_extracted_1` and planning +# aborts with: Optimizer rule 'push_down_leaf_projections' failed Schema error: +# Schema contains duplicate unqualified field name __datafusion_extracted_1. +# With the fix, the generator starts at 2, as the plan below shows. +# +# This is `EXPLAIN` under `logical_plan_only` rather than an executed query +# because a surviving `InSubquery` expression has no physical plan; the logical +# plan is both the observable result and exactly what regressed. +##################### + +statement ok +set datafusion.explain.logical_plan_only = true; + +query TT +EXPLAIN +SELECT + id, + id IN ( + SELECT id + FROM ( + SELECT id, s['label'] AS __datafusion_extracted_1 Review Comment: I assume there's some way to hit this behavior without literally naming a column `__datafusion_extracted_1`. But since the literal use of `__datafusion_extracted_1` causes an error (and it shouldn't) this is a valid regression test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
