Re: default parallelism and mesos executors

Thanks Iulian, I'll retest with 1.6.x once it's released (probably won't have enough spare time to test with the RC). On 11/12/2015 15:00, Iulian Dragoș wrote: On Wed, Dec 9, 2015 at 4:29 PM, Adrian Bridgett > wrote: (resending,

Re: default parallelism and mesos executors

On Wed, Dec 9, 2015 at 4:29 PM, Adrian Bridgett wrote: > (resending, text only as first post on 2nd never seemed to make it) > > Using parallelize() on a dataset I'm only seeing two tasks rather than the > number of cores in the Mesos cluster. This is with spark 1.5.1 and

default parallelism and mesos executors

(resending, text only as first post on 2nd never seemed to make it) Using parallelize() on a dataset I'm only seeing two tasks rather than the number of cores in the Mesos cluster. This is with spark 1.5.1 and using the mesos coarse grained scheduler. Running pyspark in a console seems to

default parallelism and mesos executors

Using parallelize() on a dataset I'm only seeing two tasks rather than the number of cores in the Mesos cluster. This is with spark 1.5.1 and using the mesos coarse grained scheduler. Running pyspark in a console seems to show that it's taking a while before the mesos executors come online