I’d suggest something like Apache YARN, or Apache Mesos with Marathon or something similar to allow for management, in particular restart on failure.
mn On Aug 13, 2014, at 7:15 PM, Tobias Pfeiffer <t...@preferred.jp> wrote: > Hi, > > On Thu, Aug 14, 2014 at 5:49 AM, salemi <alireza.sal...@udo.edu> wrote: > what is the best way to make a spark streaming driver highly available. > > I would also be interested in that. In particular for Streaming applications > where the Spark driver is running for a long time, this might be important, I > think. > > Thanks > Tobias >
