Aaron Davidson created SPARK-1602:
-------------------------------------
Summary: Cancelled jobs can lead to corrupted cached partitions
Key: SPARK-1602
URL: https://issues.apache.org/jira/browse/SPARK-1602
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.0, 0.9.1
Reporter: Aaron Davidson
Assignee: Reynold Xin
Priority: Critical
Fix For: 1.0.0
When jobs are cancelled, the InterruptibleIterator simply returns hasNext =
false, which can confuse the CacheManager into thinking that the RDD was fully
computed, causing it to store the incomplete result into the BlockManager.
This unfortunately will lead to incorrect results being returned on all future
operations containing this RDD while it's still cached.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
