[ https://issues.apache.org/jira/browse/SPARK-1602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Davidson resolved SPARK-1602. ----------------------------------- Resolution: Fixed > Cancelled jobs can lead to corrupted cached partitions > ------------------------------------------------------ > > Key: SPARK-1602 > URL: https://issues.apache.org/jira/browse/SPARK-1602 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.0.0, 0.9.1 > Reporter: Aaron Davidson > Assignee: Reynold Xin > Priority: Critical > Fix For: 1.0.0 > > > When jobs are cancelled, the InterruptibleIterator simply returns hasNext = > false, which can confuse the CacheManager into thinking that the RDD was > fully computed, causing it to store the incomplete result into the > BlockManager. > This unfortunately will lead to incorrect results being returned on all > future operations containing this RDD while it's still cached. -- This message was sent by Atlassian JIRA (v6.2#6252)