Looks like metrics are not a hot topic to discuss - yet so important to
sleep well when jobs are running in production.
I've created Spark-4537 https://issues.apache.org/jira/browse/SPARK-4537
to track this issue.
-kr, Gerard.
On Thu, Nov 20, 2014 at 9:25 PM, Gerard Maas gerard.m...@gmail.com
As the Spark Streaming tuning guide indicates, the key indicators of a
healthy streaming job are:
- Processing Time
- Total Delay
The Spark UI page for the Streaming job [1] shows these two indicators but
the metrics source for Spark Streaming (StreamingSource.scala) [2] does
not.
Any reasons
