Currently there is no built-in support for persisting streaming statistics
for applications that are over.
However, you can access all the same metrics through the StreamingListener
<http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.scheduler.StreamingListener>
interface.
You can create your own StreamingListener object with overridden methods,
and the system will call the methods with the metric data. Then you can
yourself save the metric data to some storage yourself.
class MyListener extends StreamingListener {
override def batchStarted(...) { // save the metric data to some
storage }
...
}
ssc.addStreamingListener(new MyListener())
TD
On Wed, Sep 9, 2015 at 10:51 PM, b.bhavesh <[email protected]> wrote:
> Hello,
>
> How can I keep history of streaming statistics for completed applications.
> Where in Spark, the information presented on UI is stored?
>
> Thanks,
> b.bhavesh
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-keep-history-of-streaming-statistics-tp24635.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
