Liang Lee created SPARK-9498: -------------------------------- Summary: Some statistical information missed when the driver is out of the cluster Key: SPARK-9498 URL: https://issues.apache.org/jira/browse/SPARK-9498 Project: Spark Issue Type: Improvement Components: Web UI Affects Versions: 1.4.0, 1.3.1 Reporter: Liang Lee
When an application is submited and the driver is out of the Spark cluster, Some statistical information missed sometimes. In stage detail inforamtion page, it will display following info when the driver is in the spark cluster: Details for Stage 7 Total task time across all tasks: 37 min Input Size / Records: 55.8 GB / 60488 Shuffle write: 26.6 GB / 585242962 But when the dreive is out of the spark cluster, it will sometimes display above info, while sometimes not, just like this: Details for Stage 7 Total task time across all tasks: 37 min That is the Input Size and Shuffle data does not display. I have check the code and find that when the input size is zero then it will not display. And the input size is sent by each Executors and collected by Driver. The problem is that the data that should be repored by Executors, is missed. But I don't know why. Could anyone help to solve this problem? -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org