[ https://issues.apache.org/jira/browse/HIVE-9424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286124#comment-14286124 ]
Chao commented on HIVE-9424: ---------------------------- [~xuefuz] I think the issue will not happen with the current impl - it could only occur when the "main" SparkTask is not in the root task set, due to the post order traversing of the TaskGraphWalker. In my dynamic partition implementation this case could happen, since the "main" task could be child of pruning task. Yes, I think it's OK not to include it in 0.15. It will not affect anything that we have right now. > SparkMapJoinResolver shouldn't process newly generated SparkTask [Spark > Branch] > ------------------------------------------------------------------------------- > > Key: HIVE-9424 > URL: https://issues.apache.org/jira/browse/HIVE-9424 > Project: Hive > Issue Type: Bug > Affects Versions: spark-branch > Reporter: Chao > Assignee: Chao > Priority: Blocker > Attachments: HIVE-9424.1-spark.patch > > > Sometimes, in SparkMapJoinResolver, after a new SparkTask is generated and > added to context, it may be processed by the {{dispatch}} method in this > class. This could introduce wrong plan since {{generateLocalWork}} will > overwrite existing local work in the SparkTask. -- This message was sent by Atlassian JIRA (v6.3.4#6332)