Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r224865402
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -341,7 +341,9 @@ private class LiveExecutorStageSummary(
metrics.shuffleWriteMetrics.recordsWritten,
metrics.memoryBytesSpilled,
metrics.diskBytesSpilled,
- isBlacklisted)
+ isBlacklisted,
--- End diff --
> you are sending the entire ExecutorSummary for all executors when you
really just need 2 fields out of it of some executors
The current scala code avoids that by just fetching executor summaries it
needs; the problem I can see from JS is that it would cause potentially many
requests to the driver (which are more expensive than a hash table or level db
lookup), but maybe it's not so bad if it's restricted to what is being shown in
a single page.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
