looks like nobody knows the answer on this question ;) On Sat, Mar 31, 2018 at 1:59 PM, Igor Makhlin <igor.makh...@gmail.com> wrote:
> Hi All, > > I'm looking for a way to distinguish between normal and abnormal > termination of a spark streaming application with (checkpointing enabled). > > Adding application listener doesn't really help because onApplicationEnd > event has no information regarding the cause of the termination. > > ssc_.sc.addSparkListener(new SparkListener { > override def onApplicationEnd(applicationEnd: SparkListenerApplicationEnd): > Unit = { > > I need to manage an internal metadata if streaming application has been > terminated and that termination is not recoverable I have to delete the > metadata (state of stream in this particular application). > > > > -- > Sincerely, > > Igor Makhlin > -- Sincerely, Igor Makhlin