Hi Suman, I'll assume that you are using spark submit to run your application. You can pass the --num-executors flag to ask for more containers. If you want to allocate more memory for each executor, you may also pass in the --executor-memory flag (this accepts a string in the format 1g, 512m etc.).
-Andrew 2014-12-18 14:37 GMT-08:00 Suman Somasundar <suman.somasun...@oracle.com>: > > Hi, > > > > I am using Spark 1.1.1 on Yarn. When I try to run K-Means, I see from the > Yarn dashboard that only 3 containers are being used. How do I increase the > number of containers used? > > > > P.S: When I run K-Means on Mahout with the same settings, I see that there > are 25-30 containers being used. > > > > Thanks, > Suman. >
