Hi,
My impression is to use StreamingListener to track metrics and react
appropriately.
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.scheduler.StreamingListener
Pozdrawiam,
Jacek Laskowski
https://medium.com/@jaceklaskowski/
Mastering Apache Spark 2.0
Hi, there
From the Spark UI, we can monitor the following two metrics:
• Processing Time - The time to process each batch of data.
• Scheduling Delay - the time a batch waits in a queue for the
processing of previous batches to finish.
However, what is the best way to monitor
