Re: Spark executors resources. Blocking?

2015-01-14 Thread Brett Meyer
buttl...@llnl.gov, user user@spark.apache.org Subject: Re: Spark executors resources. Blocking? Thanks for your answer David, It is as I thought then. When you write that there could be some approaches to solve this using Yarn or Mesos, can you give any idea about this? Or better yet, is there any

Re: Spark executors resources. Blocking?

2015-01-13 Thread Luis Guerra
or Mesos to schedule the system. The same issues will come up, but they have a much broader range of approaches that you can take to solve the problem. Dave *From:* Luis Guerra [mailto:luispelay...@gmail.com] *Sent:* Monday, January 12, 2015 8:36 AM *To:* user *Subject:* Spark executors

Spark executors resources. Blocking?

2015-01-12 Thread Luis Guerra
Hello all, I have a naive question regarding how spark uses the executors in a cluster of machines. Imagine the scenario in which I do not know the input size of my data in execution A, so I set Spark to use 20 (out of my 25 nodes, for instance). At the same time, I also launch a second execution