Keep in mind that your executors will be able to run some fixed number
of tasks in parallel, given your configuration. You should not
necessarily expect that arbitrarily many RDDs and tasks would schedule
simultaneously.
On Mon, Jan 19, 2015 at 5:34 PM, critikaled <isasmani....@gmail.com> wrote:
> Hi, john and david
> I tried this to run them concurrently List(RDD1,RDD2,.....).par.foreach{
> rdd=>
> rdd.collect().foreach(println)
> }
> this was able to successfully register the task but the parallelism of the
> stages is limited it was able run 4 of them some time and only one of them
> some time which was not consistent.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
