[
https://issues.apache.org/jira/browse/HIVE-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505442#comment-14505442
]
Chinna Rao Lalam commented on HIVE-8858:
----------------------------------------
Hi [~jxiang],
Thanks for the review..
Can this be part of explain extended?
bq.At the time of sparktask execution, sparkpaln was available so it is
difficult to do at explain extended.
If we have to log into the log file, should we put in a buffer and log it in
one log.info call?
bq.We are doing the log in one log.info call
Another thing is that in assigning those numbers, can they match with the
corresponding works/operators? For example, MapInput 1 is corresponding to Map
1 while MapInput 2 is corresponding to Map 2?
bq.It may not match with the corresponding works/operators. To identify each
tran uniquely in the plan just we have added a sequence numbers for all the
trans.
> Visualize generated Spark plan [Spark Branch]
> ---------------------------------------------
>
> Key: HIVE-8858
> URL: https://issues.apache.org/jira/browse/HIVE-8858
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Xuefu Zhang
> Assignee: Chinna Rao Lalam
> Attachments: HIVE-8858-spark.patch, HIVE-8858.1-spark.patch,
> HIVE-8858.2-spark.patch, HIVE-8858.3-spark.patch, HIVE-8858.4-spark.patch
>
>
> The spark plan generated by SparkPlanGenerator contains info which isn't
> available in Hive's explain plan, such as RDD caching. Also, the graph is
> slight different from orignal SparkWork. Thus, it would be nice to visualize
> the plan as is done for SparkWork.
> Preferrably, the visualization can happen as part of Hive explain extended.
> If not feasible, we at least can log this at info level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
