You can also use the REST api introduced in 1.4 – although it’s harder to parse:
* jobs from the same batch are not grouped together * You only get total delay, not scheduling delay From: Hari Shreedharan Date: Tuesday, September 29, 2015 at 5:27 AM To: Shixiong Zhu Cc: Siva, "user@spark.apache.org<mailto:user@spark.apache.org>" Subject: Re: Monitoring tools for spark streaming +1. The Streaming UI should give you more than enough information. Thanks, Hari On Mon, Sep 28, 2015 at 9:55 PM, Shixiong Zhu <zsxw...@gmail.com<mailto:zsxw...@gmail.com>> wrote: Which version are you using? Could you take a look at the new Streaming UI in 1.4.0? Best Regards, Shixiong Zhu 2015-09-29 7:52 GMT+08:00 Siva <sbhavan...@gmail.com<mailto:sbhavan...@gmail.com>>: Hi, Could someone recommend the monitoring tools for spark streaming? By extending StreamingListener we can dump the delay in processing of batches and some alert messages. But are there any Web UI tools where we can monitor failures, see delays in processing, error messages and setup alerts etc. Thanks
