Github user LantaoJin commented on the issue:
https://github.com/apache/spark/pull/22411
Most of the information we wanted could be analyzed out from event log
except some metrics in Executor side which doesn't heartbeat to Driver, e.g RPC
count with NameNode. Another case is #21221, before that we had to hack code to
get the similar metrics. Event log as a structured, unified, overall,
replay-able log, it offers a possibility to analysis offline, even realtime. We
prefer to use it since the history UI exposes less information than user
expected, further more not smart and hard to customize. We are on going on this
based on event log. Thanks @cloud-fan, I suggest to add this interface to
```DataWritingCommand```. Pattern matching each implementations looks trick. It
looks common, maybe it could be used in physical plan optimization in future.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
