Re: Do we really need mesos or yarn? or is standalone sufficent?

[vaquar khan]

Hi Kant,

Hope following information will help .

1)Cluster
https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-standalone.html
http://spark.apache.org/docs/latest/hardware-provisioning.html


2) Yarn vs Mesos
https://www.linkedin.com/pulse/mesos-compare-yarn-vaquar-
khan-?articleId=7126978319066345849

Yarn for Hadoop and Mesos for enterprise so if use together will be more
beneficial. In short Yarn on Mesos.

  http://mesosphere.github.io/presentations/myriad-strata/#/


Regards,
Vaquar khan

On 16 Dec 2016 18:46, "kant kodali"  wrote:

Hi Saif,

What do you mean by small cluster? Any specific size?

Also can you shine some light on how YARN takes a win over mesos?




Thanks,
kant

On Fri, Dec 16, 2016 at 10:45 AM,  wrote:

> In my experience, Standalone works very well in small cluster where there
> isn’t anything else running.
>
>
>
> Bigger cluster or shared resources, YARN takes a win, surpassing the
> overhead of spawning containers as opposed to a background running worker.
>
>
>
> Best is if you try both, if standalone is good enough keep it till you
> need more. Otherwise, try YARN or MESOS depending on the rest of your
> components.
>
>
>
> 2cents
>
>
>
> Saif
>
>
>
> *From:* kant kodali [mailto:kanth...@gmail.com]
> *Sent:* Friday, December 16, 2016 3:14 AM
> *To:* user @spark
> *Subject:* Do we really need mesos or yarn? or is standalone sufficent?
>
>
>
> Do we really need mesos or yarn? or is standalone sufficient for
> production systems? I understand the difference but I don't know the
> capabilities of standalone cluster. does anyone have experience deploying
> standalone in the production?
>
>
>
>
>

Re: Do we really need mesos or yarn? or is standalone sufficent?

[kant kodali]

Hi Saif,

What do you mean by small cluster? Any specific size?

Also can you shine some light on how YARN takes a win over mesos?

Thanks,
kant

On Fri, Dec 16, 2016 at 10:45 AM,  wrote:

> In my experience, Standalone works very well in small cluster where there
> isn’t anything else running.
>
>
>
> Bigger cluster or shared resources, YARN takes a win, surpassing the
> overhead of spawning containers as opposed to a background running worker.
>
>
>
> Best is if you try both, if standalone is good enough keep it till you
> need more. Otherwise, try YARN or MESOS depending on the rest of your
> components.
>
>
>
> 2cents
>
>
>
> Saif
>
>
>
> *From:* kant kodali [mailto:kanth...@gmail.com]
> *Sent:* Friday, December 16, 2016 3:14 AM
> *To:* user @spark
> *Subject:* Do we really need mesos or yarn? or is standalone sufficent?
>
>
>
> Do we really need mesos or yarn? or is standalone sufficient for
> production systems? I understand the difference but I don't know the
> capabilities of standalone cluster. does anyone have experience deploying
> standalone in the production?
>
>
>
>
>

Re: Do we really need mesos or yarn? or is standalone sufficent?

[Anant Chintamaneni]

+1

Sent from my iPhone

> On Dec 16, 2016, at 10:45 AM,  
>  wrote:
> 
> In my experience, Standalone works very well in small cluster where there 
> isn’t anything else running.
>  
> Bigger cluster or shared resources, YARN takes a win, surpassing the overhead 
> of spawning containers as opposed to a background running worker.
>  
> Best is if you try both, if standalone is good enough keep it till you need 
> more. Otherwise, try YARN or MESOS depending on the rest of your components.
>  
> 2cents
>  
> Saif
>  
> From: kant kodali [mailto:kanth...@gmail.com] 
> Sent: Friday, December 16, 2016 3:14 AM
> To: user @spark
> Subject: Do we really need mesos or yarn? or is standalone sufficent?
>  
> Do we really need mesos or yarn? or is standalone sufficient for production 
> systems? I understand the difference but I don't know the capabilities of 
> standalone cluster. does anyone have experience deploying standalone in the 
> production?
>  
>  

RE: Do we really need mesos or yarn? or is standalone sufficent?

[Saif.A.Ellafi]

In my experience, Standalone works very well in small cluster where there isn’t 
anything else running.

Bigger cluster or shared resources, YARN takes a win, surpassing the overhead 
of spawning containers as opposed to a background running worker.

Best is if you try both, if standalone is good enough keep it till you need 
more. Otherwise, try YARN or MESOS depending on the rest of your components.

2cents

Saif

From: kant kodali [mailto:kanth...@gmail.com]
Sent: Friday, December 16, 2016 3:14 AM
To: user @spark
Subject: Do we really need mesos or yarn? or is standalone sufficent?

Do we really need mesos or yarn? or is standalone sufficient for production 
systems? I understand the difference but I don't know the capabilities of 
standalone cluster. does anyone have experience deploying standalone in the 
production?