...@spark.incubator.apache.org
Subject: Re: major Spark performance problem
Hi,
There is also an option to run spark applications on top of mesos in fine
grained mode, then it is possible for fair scheduling (applications will run in
parallel and mesos is responsible for scheduling all tasks) so
-
From: elyast [mailto:lukasz.jastrzeb...@gmail.com]
Sent: Friday, March 07, 2014 20:01
To: u...@spark.incubator.apache.org
Subject: Re: major Spark performance problem
Hi,
There is also an option to run spark applications on top of mesos in fine
grained mode, then it is possible for fair
in may not be
faster however the benefit is the fair scheduling (small jobs will not be
stuck by the big ones).
Best regards
Lukasz Jastrzebski
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/major-Spark-performance-problem-tp2364p2403.html
Sent from
Dana,
When you run multiple applications under Spark, and if each application
takes up the entire cluster resources, it is expected that one will block
the other completely, thus you're seeing that the wall time add together
sequentially. In addition there is some overhead associated with
