Hi wxhsdp, These times are computed from Java's System.currentTimeMillis(), which is "the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC." Thus, this quantity doesn't mean much by itself, but is only meaningful when you subtract it from another System.currentTimeMillis() to find the time elapsed. For instance, in your case (Finish Time - Launch Time) = 1862, which means it took 1862 ms for the task to complete (but the actual execution only took 1781 ms, the rest being overhead).
Correct, (Shuffle Finish Time - Launch Time) is the total time it took for this task to fetch blocks, and (Finish Time - Shuffle Finish Time) is the actual execution after fetching all blocks. "Fetch Wait Time" on the other hand is the time spent blocking on the thread to wait for shuffle blocks while not doing anything else. For instance, the example given in the code comments is: "if block B is being fetched while the task is not finished with processing block A, it is not considered to be blocking on block B." "Shuffle Write Time" is the time written to write the shuffle files (only for ShuffleMapTask). It is in nanoseconds, which is slightly inconsistent with other values in these metrics. By the way, all of this information is available in the code comments: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala On Tue, May 6, 2014 at 11:10 PM, wxhsdp <wxh...@gmail.com> wrote: > any ideas? thanks! > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/details-about-event-log-tp5411p5476.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
