[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434515#comment-17434515
]
Shardul Mahadik commented on SPARK-36877:
-
Was able to get around this by re-using the RDD for
[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431069#comment-17431069
]
Wenchen Fan commented on SPARK-36877:
-
You were calling `df3.repartition(5).write`, and
[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427825#comment-17427825
]
Shardul Mahadik commented on SPARK-36877:
-
{quote} Getting RDD means the physical plan is
[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427089#comment-17427089
]
Wenchen Fan commented on SPARK-36877:
-
> shouldn't it reuse the result from previous stages?
One
[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427087#comment-17427087
]
Wenchen Fan commented on SPARK-36877:
-
> Should calling df.rdd trigger actual job execution when AQE
[
https://issues.apache.org/jira/browse/SPARK-36877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421939#comment-17421939
]
Hyukjin Kwon commented on SPARK-36877:
--
cc [~maryannxue] too FYI
> Calling ds.rdd with AQE enabled
