Kousuke Saruta created SPARK-2670:
-------------------------------------
Summary: FetchFailedException should be thrown when local fetch
has failed
Key: SPARK-2670
URL: https://issues.apache.org/jira/browse/SPARK-2670
Project: Spark
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Kousuke Saruta
In BasicBlockFetchIterator, when remote fetch has failed, then FetchResult
which size is -1 is set to results.
{code}
case None => {
logError("Could not get block(s) from " + cmId)
for ((blockId, size) <- req.blocks) {
results.put(new FetchResult(blockId, -1, null))
}
{code}
The size -1 means fetch fail and BlockStoreShuffleFetcher#unpackBlock throws
FetchFailedException so that we can retry.
But, when local fetch has failed, the failed FetchResult is not set.
So, we cannot retry for the FetchResult.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
