[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan Blue updated MAPREDUCE-6639:
---------------------------------
    Attachment: MAPREDUCE-6639.2.patch

Adding a better patch. The problem was in the error handling. The calling 
thread waits for the operation to succeed or until signaled that unknownError 
is set. When an error is passed to signal that thread, there's a check for 
whether another error has already been set, but the check is wrong. It only 
sets the error if another error is already set, rather than if no other error 
has been set. The result is that if there is an error, the caller is never 
signaled and waits indefinitely for all of the tasks to complete successfully.

> Process hangs in LocatedFileStatusFetcher if FileSystem.get throws
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6639
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6639
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 2.7.2
>            Reporter: Ryan Blue
>            Assignee: Ryan Blue
>         Attachments: MAPREDUCE-6639.2.patch
>
>
> ListLocatedFileStatusFetcher uses a thread pool, but one of the Callable 
> thread functions, 
> [{{ProcessInitialInputPathCallable}}|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/LocatedFileStatusFetcher.java#L306],
>  doesn't catch exceptions (the callbacks do). When an exception is thrown, 
> the thread exists and doesn't signal the error to the calling thread, which 
> continues waiting to be signaled. This can happen when a FS implementation 
> cannot be found.
> The solution is to use a try-catch around the implementation of 
> {{ProcessInitialInputPathCallable#call}} that calls {{registerError}} when 
> any exception is caught.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to