Re: Potential racing condition in DAGScheduler when Spark 1.5 caching

Where do you see a race in the DAGScheduler? On a quick look at your stack trace, this just looks to me like a Job where a Stage failed and then the DAGScheduler aborted the failed Job. On Thu, Sep 24, 2015 at 12:00 PM, robin_up wrote: > Hi > > After upgrade to 1.5, we

Re: Potential racing condition in DAGScheduler when Spark 1.5 caching

I believe that this is an instance of https://issues.apache.org/jira/browse/SPARK-10422, which should be fixed in upcoming 1.5.1 release. On Thu, Sep 24, 2015 at 12:52 PM, Mark Hamstra wrote: > Where do you see a race in the DAGScheduler? On a quick look at your >

Re: Potential racing condition in DAGScheduler when Spark 1.5 caching

Josh Looked closer, I think you are correct, not a racing condition. This only shows up on persisting string, other data format looks fine. Also whe we reverted to 1.4 the issue's gone. Thanks On Thursday, 24 September 2015, Josh Rosen wrote: > I believe that this is an