You can cause a failure by throwing an exception in the code running on the executors. The task will be retried (if spark.task.maxFailures > 1), and then the stage is failed. No further tasks are processed after that, and an exception is thrown on the driver. You could catch the exception and see if it was caused by your own special exception.
On Mon, Jan 4, 2016 at 1:05 PM, domibd <d...@lipn.univ-paris13.fr> wrote: > Hello, > > Is there a way to stop under a condition a process (like map-reduce) using > an RDD ? > > (this could be use if the process does not always need to > explore all the RDD) > > thanks > > Dominique > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/stopping-a-process-usgin-an-RDD-tp25870.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
