[ https://issues.apache.org/jira/browse/SPARK-38529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan reassigned SPARK-38529: ----------------------------------- Assignee: Min Yang > Prevent GeneratorNestedColumnAliasing to be applied to non-Explode generators > ----------------------------------------------------------------------------- > > Key: SPARK-38529 > URL: https://issues.apache.org/jira/browse/SPARK-38529 > Project: Spark > Issue Type: Improvement > Components: Optimizer > Affects Versions: 3.2.1 > Reporter: Min Yang > Assignee: Min Yang > Priority: Minor > Fix For: 3.3.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > The Project(_, g: Generate) branch in GeneratorNestedColumnAliasing is only > supposed to work for ExplodeBase generators but we do not explicitly return > for other types like Inline. Currently the bug is not trigger because there > is another bug in the "prune unrequired child" branch in the ColumnPruning > which makes other generators like Inline always go to that branch even if it > is not applicable. > > An easy example to show the bug: > Input: <col1: int, col2: array<struct<field1 struct<field1: int, field2: > int>, field2 int>>> > Project(field1.field1 as ...) > - Generate(Inline(col2), ..., field1, field2) > > We will try to incorrectly push the .field1 on field1 into the input of the > Inline (col2). > > -- This message was sent by Atlassian Jira (v8.20.7#820007) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org