[
https://issues.apache.org/jira/browse/SPARK-38529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L. C. Hsieh updated SPARK-38529:
--------------------------------
Summary: Prevent GeneratorNestedColumnAliasing to be applied to non-Explode
generators (was: GeneratorNestedColumnAliasing works incorrectly for
non-Explode generators)
> Prevent GeneratorNestedColumnAliasing to be applied to non-Explode generators
> -----------------------------------------------------------------------------
>
> Key: SPARK-38529
> URL: https://issues.apache.org/jira/browse/SPARK-38529
> Project: Spark
> Issue Type: Bug
> Components: Optimizer
> Affects Versions: 3.2.1
> Reporter: Min Yang
> Priority: Minor
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The Project(_, g: Generate) branch in GeneratorNestedColumnAliasing is only
> supposed to work for ExplodeBase generators but we do not explicitly return
> for other types like Inline. Currently the bug is not trigger because there
> is another bug in the "prune unrequired child" branch in the ColumnPruning
> which makes other generators like Inline always go to that branch even if it
> is not applicable.
>
> An easy example to show the bug:
> Input: <col1: int, col2: array<struct<field1 struct<field1: int, field2:
> int>, field2 int>>>
> Project(field1.field1 as ...)
> - Generate(Inline(col2), ..., field1, field2)
>
> We will try to incorrectly push the .field1 on field1 into the input of the
> Inline (col2).
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
