I am looking to decide what is best for my production grade spark
application(s).
YARN
=
1. YARN supports security. When Spark is run over YARN the communication
between processes can use secure authentication through Kerberos.
2. Spark standalone cluster can only run Spark jobs and
What are ideas around Spark cluster for streaming purposes ?
What is better standalone / Mesos / YARN ?
Please share cluster details and size of data and type of processing.
(multiple processing points) (architecture or similar)
I see folks using YARN cluster for streaming purposes.
Regards,
Do we have any comparisons in terms of resource utilization, scheduling of
running Spark in the below three modes
1) Standalone
2) over YARN
3) over Mesos
Can some one share resources (thoughts/URLs) on this area.
--
Deepak
I'm not sure what you're looking for, since you can't really compare
Standalone with YARN or Mesos, as Standalone is assuming the Spark
workers/master owns the cluster, and YARN/Mesos is trying to share the
cluster among different applications/frameworks.
And when you refer to resource
or standalone? Which is the more
stable option currently? Which is the more future-proof option?
Thanks,
Ishaaq
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/standalone-vs-YARN-tp4271.html
Sent from the Apache Spark User List mailing list archive
:
http://apache-spark-user-list.1001560.n3.nabble.com/standalone-vs-YARN-tp4271.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
--
SUREN HIRAMAN, VP TECHNOLOGY
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR
NEW YORK, NY 10001
O: (917) 525-2466
