[
https://issues.apache.org/jira/browse/SPARK-18364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649506#comment-15649506
]
Steven Rand commented on SPARK-18364:
-------------------------------------
The solution I'd initially had in mind, simply creating a {{MetricsSystem}} in
{{YarnShuffleService}}, isn't ideal, since then {{network-yarn}} has to depend
on {{core}}.
I tried splitting the metrics code currently in {{core}} into its own module,
but this is pretty tough given how much other code from {{core}} the metrics
code depends on. And the {{metrics}} module can't depend on {{core}}, since
that creates a circular dependency (core has to depend on metrics).
Another idea might be to try to get metrics out of the executors instead of
from ExternalShuffleBlockHandler, though this also seems tricky.
Open to ideas on how to proceed here if anyone has them.
> expose metrics for YarnShuffleService
> -------------------------------------
>
> Key: SPARK-18364
> URL: https://issues.apache.org/jira/browse/SPARK-18364
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 2.0.1
> Reporter: Steven Rand
> Priority: Minor
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> ExternalShuffleService exposes metrics as of SPARK-16405. However,
> YarnShuffleService does not.
> The work of instrumenting ExternalShuffleBlockHandler was already done in
> SPARK-1645, so this JIRA is for creating a MetricsSystem in
> YarnShuffleService similarly to how ExternalShuffleService already does it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
