[ https://issues.apache.org/jira/browse/MAPREDUCE-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166218#comment-13166218 ]
Robert Joseph Evans commented on MAPREDUCE-3519: ------------------------------------------------ I like the patch. I also traced down from where the synchronized call is happening looking for a potential race condition between reads and writes. The only place I found that uses conf from LocalDirsHandlerService.getLocalPathForWrite is LocalDirAllocator.confChanged. Inside that code we access conf three times, once to get the local file system, so I don't see that as being an issue. But we call it twice to get the value of the contextConfigItemName. The first to see if the value has changed, and the second to turn that value into an array. It would be better to change the code so that it only calls conf.get once for that value, and then to turn it into a trimmed array of Strings call StringUtils.getTrimmedStrings, which is what the second method does internally. > Deadlock in LocalDirsHandlerService and ShuffleHandler > ------------------------------------------------------ > > Key: MAPREDUCE-3519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3519 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager > Affects Versions: 0.23.1, 0.24.0 > Reporter: Ravi Gummadi > Assignee: Ravi Gummadi > Fix For: 0.23.1 > > Attachments: 3519.patch, 3519.v1.patch, deadlock.txt > > > MAPREDUCE-3121 cloned Configuration object in LocalDirsHandlerService.init() > to avoid others to access that configuration object. But since it is used in > local FileSystem object creation in LocalDirAllocator.AllocatorPerContext and > the same FileSystem object is used in > ShuffleHandler.Shuffle.localDirAllocator, this is causing a deadlock when > accessing this configuration object from LocalDirsHandlerService and > ShuffleHandler along with AllocatorPerContext object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira