[Spark Streaming] - Killing application from within code

[Sidney Feiner]

Hey, I'm using connections to Elasticsearch from within my Spark Streaming 
application.
I'm using Futures to maximize performance when it sends network requests to the 
ES cluster.
Basically, I want my app to crash if any one of the executors fails to connect 
to ES.

The exception gets catched and returned in my Future as a Failure(ex: 
NoNodeAvailableException) but when I handle it, I can't seem to kill my app.
I tried using:

fut andThen {
  case Failure(ex: NoNodeAvailableException) =>
    throw ex
}

fut andThen {
  case Failure(ex: NoNodeAvailableException) =>
    System.exit(-1)
}

fut onFailure {
  case ex: NoNodeAvailableException =>
    throw ex
}

fut onFailure {
  case ex: NoNodeAvailableException =>
    System.exit(-1)
}


But none of them seem to be killing my app. The System.exit(-1) kills my 
executor but that doesn't seem like the correct way to do it.
And no matter what way I try, the driver stays alive.

Is there a way to programmatically kill the application from within one of the 
workers?

Thanks a lot :)


Sidney Feiner / SW Developer
M: +972.528197720 / Skype: sidney.feiner.startapp

[emailsignature]