[ https://issues.apache.org/jira/browse/SPARK-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259653#comment-15259653 ]
Saurabh Santhosh commented on SPARK-11072: ------------------------------------------ Will this resolve https://issues.apache.org/jira/browse/SPARK-14948 ? Can you please add a test case covering this scenario for future releases > simplify self join handling > --------------------------- > > Key: SPARK-11072 > URL: https://issues.apache.org/jira/browse/SPARK-11072 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Wenchen Fan > > self-join is a diamond problem that confuse our analyzer. Our current > solution is creating new instances of leaf nodes in the right tree of join > node, and update all attribute reference there. Thus there is no diamond > anymore and problem fixed. > However, our execution engine can handle diamond plan and we only need to > distinguish the output between left and right. So we can simplify the > self-join handling by introducing a new Plan `NewOutput` to give different > output attributes. > The extra `NewOutput` layer is quite cheap and can be completely removed when > we have local nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org