It happens only when StorageLevel is used with 1 replica ( StorageLevel.
MEMORY_ONLY_2,StorageLevel.MEMORY_AND_DISK_2) , StorageLevel.MEMORY_ONLY ,
StorageLevel.MEMORY_AND_DISK works - the problems must be clearly somewhere
between mesos-spark . From console I see that spark is trying to replicate
More info
when using *spark.mesos.coarse* everything works as expected. I think this
must be a bug in spark-mesos integration.
2015-03-27 9:23 GMT+01:00 Ondrej Smola ondrej.sm...@gmail.com:
It happens only when StorageLevel is used with 1 replica ( StorageLevel.
Yes, only when using fine grained mode and replication
(StorageLevel.MEMORY_ONLY_2
etc).
2015-03-27 19:06 GMT+01:00 Tathagata Das t...@databricks.com:
Does it fail with just Spark jobs (using storage levels) on non-coarse
mode?
TD
On Fri, Mar 27, 2015 at 4:39 AM, Ondrej Smola
Does it fail with just Spark jobs (using storage levels) on non-coarse mode?
TD
On Fri, Mar 27, 2015 at 4:39 AM, Ondrej Smola ondrej.sm...@gmail.com
wrote:
More info
when using *spark.mesos.coarse* everything works as expected. I think
this must be a bug in spark-mesos integration.
Seems like a bug, could you file a JIRA?
@Tim: Patrick said you take a look at Mesos related issues. Could you take
a look at this. Thanks!
TD
On Fri, Mar 27, 2015 at 1:25 PM, Ondrej Smola ondrej.sm...@gmail.com
wrote:
Yes, only when using fine grained mode and replication
