[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727721#comment-17727721 ] Dongjoon Hyun commented on SPARK-39854: --- I verified with 3.4.0 and added `3.4.0` to the Affected

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606023#comment-17606023 ] Dongjoon Hyun commented on SPARK-39854: --- It turns out SPARK-35194 caused this regression. I tested

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606016#comment-17606016 ] Dongjoon Hyun commented on SPARK-39854: --- Thank you, [~jiajiwu] . The recommended workaround would

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573286#comment-17573286 ] Apache Spark commented on SPARK-39854: -- User 'jiaji-wu' has created a pull request for this issue:

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570540#comment-17570540 ] Apache Spark commented on SPARK-39854: -- User 'jiaji-wu' has created a pull request for this issue:

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570541#comment-17570541 ] Apache Spark commented on SPARK-39854: -- User 'jiaji-wu' has created a pull request for this issue:

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570535#comment-17570535 ] Jiaji Wu commented on SPARK-39854: -- One workaround is to exclude *ColumnPruning* by set spark config: