[
https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174053#comment-14174053
]
Nan Zhu commented on SPARK-3957:
--------------------------------
I agree with [~andrewor14], I was also thinking about piggyback the information
in the heartbeat between heartbeatReceiver and the executor
...not sure about the current Hadoop implementation, in 1.x version, TaskStatus
was piggyback in the heartbeat between TaskTracker and JobTracker...to me, it's
a very natural way to do this....
I accepted it this morning and have started some work, so, [~devlakhani],
please let me finish this, thanks
> Broadcast variable memory usage not reflected in UI
> ---------------------------------------------------
>
> Key: SPARK-3957
> URL: https://issues.apache.org/jira/browse/SPARK-3957
> Project: Spark
> Issue Type: Bug
> Components: Block Manager, Web UI
> Affects Versions: 1.0.2, 1.1.0
> Reporter: Shivaram Venkataraman
> Assignee: Nan Zhu
>
> Memory used by broadcast variables are not reflected in the memory usage
> reported in the WebUI. For example, the executors tab shows memory used in
> each executor but this number doesn't include memory used by broadcast
> variables. Similarly the storage tab only shows list of rdds cached and how
> much memory they use.
> We should add a separate column / tab for broadcast variables to make it
> easier to debug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
