If you're running on one machine with 2 cores, I believe all you can get out of it are 2 concurrent tasks at any one time. So setting your default parallelism to 20 won't help.
On Fri, Apr 4, 2014 at 11:41 AM, Eduardo Costa Alfaia < [email protected]> wrote: > Hi all, > > I have put this line in my spark-env.sh: > -Dspark.default.parallelism=20 > > this parallelism level, is it correct? > The machine's processor is a dual core. > > Thanks > > -- > Informativa sulla Privacy: http://www.unibs.it/node/8155 >
