You would want to add a listener to your Spark Streaming context. Have a look at the StatsReportListener [1].
[1] http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.scheduler.StatsReportListener 2016-05-17 7:18 GMT+10:00 Samuel Zhou <zhou...@gmail.com>: > Hi, > > Does anyone know how to get the batch information(like batch time, input > size, processing time, status) from Streaming UI by using Scala/Java API ? > Because I want to put the information in log files and the streaming jobs > are managed by YARN. > > Thanks, > Samuel >
