Hi Welly, you will need Zookeeper if you want to setup the standalone cluster in HA mode. http://spark.apache.org/docs/latest/spark-standalone.html#high-availability
In the YARN case you probably have already Zookeeper in place if you are running YARN in HA mode. Regards, Andreas On Wed, Nov 25, 2015 at 10:02 AM, Welly Tambunan <if05...@gmail.com> wrote: > Hi Ufuk > > >In failure cases I find YARN more convenient, because it takes care of > restarting failed task manager processes/containers for you. > > So this mean that we don't need zookeeper ? > > > Cheers > > On Wed, Nov 25, 2015 at 3:46 PM, Ufuk Celebi <u...@apache.org> wrote: > >> > On 25 Nov 2015, at 02:35, Welly Tambunan <if05...@gmail.com> wrote: >> > >> > Hi All, >> > >> > I would like to know if there any feature differences between using >> Standalone Cluster vs YARN ? >> > >> > Until now we are using Standalone cluster for our jobs. >> > Is there any added value for using YARN ? >> > >> > We don't have any hadoop infrastructure in place right now but we can >> provide that if there's some value to that. >> >> There are no features, which only work on YARN or in standalone clusters. >> YARN mode is essentially starting a standalone cluster in YARN containers. >> >> In failure cases I find YARN more convenient, because it takes care of >> restarting failed task manager processes/containers for you. >> >> – Ufuk >> >> > > > -- > Welly Tambunan > Triplelands > > http://weltam.wordpress.com > http://www.triplelands.com <http://www.triplelands.com/blog/> >
