Re: Spark application on yarn cluster clarification

[Fawze Abujaber]

Hi Soheil,

Resource manager and NodeManager are enough, of your you need the roles of
DataNode and NameNode to be able accessing the Data.

On Thu, 18 Jan 2018 at 10:12 Soheil Pourbafrani 
wrote:

> I am setting up a Yarn cluster to run Spark applications on that, but I'm
> confused a bit!
>
> Consider I have a 4-node yarn cluster including one resource manager and 3
> node manager and spark are installed in all 4 nodes.
>
> Now my question is when I want to submit spark application to yarn
> cluster, is it needed spark daemons (both master and slaves) to be running,
> or not, running just resource and node managers suffice?
>
> Thanks
>

Spark application on yarn cluster clarification

[Soheil Pourbafrani]

I am setting up a Yarn cluster to run Spark applications on that, but I'm
confused a bit!

Consider I have a 4-node yarn cluster including one resource manager and 3
node manager and spark are installed in all 4 nodes.

Now my question is when I want to submit spark application to yarn cluster,
is it needed spark daemons (both master and slaves) to be running, or not,
running just resource and node managers suffice?

Thanks