Hi all, I am facing and issue for long running spark job on yarn. If there occures some bottle neck on hdfs and/or kafka, active batch count increases immidiately.
I am plannning to check the active batch count with java client and create alarms for the operations group. So, is it possible to retrieve active batch count with any java api as we can see on monitoring page below ? /proxy/application_1534314004365_0001/streaming/ Regards, Serkan ENERJİSA serkan....@enerjisa.com<mailto:serkan....@enerjisa.com> www.enerjisa.com.tr [Description: Description: Açıklama: Tick-Tock-Boom-Facebook]<http://www.facebook.com/enerjisa> [Description: Description: Açıklama: Tick-Tock-Boom-GooglePlus] <https://plus.google.com/u/0/116640294170184539826/posts> [Description: Description: Açıklama: Tick-Tock-Boom-Youtube] <http://www.youtube.com/enerjisacorp> [cid:image5d27ab.JPG@08344e31.439da30d] ________________________________ Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene telefonla ya da e-posta ile bildirmeli ve bilgisayarından silmelidir. Bu iletinin içeriğinden yalnızca iletiyi gönderen kişi sorumludur. This communication may contain information that is legally privileged, confidential or exempt from disclosure. If you are not the intended recipient, please note that any dissemination, distribution, or copying of this communication is strictly prohibited. Anyone who receives this message in error should notify the sender immediately by telephone or by return communication and delete it from his or her computer. Only the person who has sent this message is responsible for its content.
