MetadataFetchFailedException if executorLost when spark.speculation enabled ?

2016-05-01 Thread Renyi Xiong
Hi, We observed MetadataFetchFailedException during executorLost if spark.speculation enabled. looks like something out of sync between original task on failed executor and its speculative counterpart task? is it a known issue? please let me know if you need more details. Thanks, Renyi.

[ANNOUNCE] Spark branch-2.0

2016-05-01 Thread Reynold Xin
Hi devs, Three weeks ago I mentioned on the dev list creating branch-2.0 (effectively "feature freeze") in 2 - 3 weeks. I've just created Spark's branch-2.0 to form the basis of the 2.0 release. We have closed ~ 1700 issues. That's huge progress, and we should celebrate that. Compared with past r