Michael Harris created HADOOP-12412:
---------------------------------------
Summary: Concurrency in FileSystem$Cache is very broken
Key: HADOOP-12412
URL: https://issues.apache.org/jira/browse/HADOOP-12412
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.7.0
Reporter: Michael Harris
Assignee: Michael Harris
Priority: Critical
The FileSystem cache uses a mild amount of concurrency to protect the cache
itself, but does nothing to prevent multiple of the same filesystem from being
constructed and initialized simultaneously. At best, this leads to potentially
expensive wasted work. At worst, as is the case for Spark, it can lead to
deadlocks/livelocks, especially when the same configuration object is passed
into both calls. This should be refactored to use a results cache approach
(reference Java Concurrency in Practice chapter 5 section 6 for an example of
how to do this correctly), which will be both higher-performance and safer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
