To make it clear, Spark Standalone is similar to Yarn as a simple cluster
management system.
Spark Master --- Yarn Resource Manager
Spark Worker --- Yarn Node Manager
On Mon, Aug 17, 2015 at 4:59 AM, Ruslan Dautkhanov dautkha...@gmail.com
wrote:
There is no Spark master in YARN mode.
There is no Spark master in YARN mode. It's standalone mode terminology.
In YARN cluster mode, Spark's Application Master (Spark Driver runs in it)
will be restarted
automatically by RM up to yarn.resourcemanager.am.max-retries
times (default is 2).
--
Ruslan Dautkhanov
On Fri, Jul 17, 2015 at
Hi,
Is Spark master high availability supported on YARN (yarn-client mode)
analogous to
https://spark.apache.org/docs/1.4.0/spark-standalone.html#high-availability?
Thanks
Bhaskie
Thanks Akhil,
I'm using Spark in standalone mode so i guess Mesos is not an option here.
On Tue, May 12, 2015 at 1:27 PM, Akhil Das ak...@sigmoidanalytics.com
wrote:
Mesos has a HA option (of course it includes zookeeper)
Thanks
Best Regards
On Tue, May 12, 2015 at 4:53 PM, James King
Mesos has a HA option (of course it includes zookeeper)
Thanks
Best Regards
On Tue, May 12, 2015 at 4:53 PM, James King jakwebin...@gmail.com wrote:
I know that it is possible to use Zookeeper and File System (not for
production use) to achieve HA.
Are there any other options now or in the
I know that it is possible to use Zookeeper and File System (not for
production use) to achieve HA.
Are there any other options now or in the near future?
