Update: I am using --supervise flag for fault tolerance.
On Mon, Mar 21, 2016 at 4:16 PM, Hao Ren <inv...@gmail.com> wrote: > Using spark 1.6.1 > Spark Streaming Jobs are submitted via spark-submit (cluster mode) > > I tried to kill drivers via webUI, it does not work. These drivers are > still running. > I also tried: > 1. spark-submit --master <master-url> --kill <driver-id> > 2. ./bin/spark-class org.apache.spark.deploy.Client kill <master url> > <driver ID> > > Neither works. The workaround is to ssh to the driver node, then kill -9 > ... > jsp shows the same classname DriverWrapper, so need to pick carefully... > > Any idea why this happens ? > BTW, my streaming job's batch duration is one hour. So do we need to wait > for job processing to kill kill driver ? > > -- > Hao Ren > > Data Engineer @ leboncoin > > Paris, France > -- Hao Ren Data Engineer @ leboncoin Paris, France
