MaWenJin created HDFS-4309: ------------------------------ Summary: Multithreaded get through the Cache FileSystem Object to lead LeaseChecker memory leak Key: HDFS-4309 URL: https://issues.apache.org/jira/browse/HDFS-4309 Project: Hadoop HDFS Issue Type: Bug Components: hdfs-client Affects Versions: 2.0.2-alpha, 2.0.1-alpha, 0.23.4, 0.23.1, 0.20.205.0 Reporter: MaWenJin
If multiple threads concurrently execute the following methods will result in the thread fs = createFileSystem (uri, conf) method is called.And create multiple DFSClient, start at the same time LeaseChecker daemon thread, may not be able to use shutdownhook close it after the process, resulting in a memory leak. private FileSystem getInternal(URI uri, Configuration conf, Key key) throws IOException{ FileSystem fs = null; synchronized (this) { fs = map.get(key); } if (fs != null) { return fs; } // this is fs = createFileSystem(uri, conf); synchronized (this) { // refetch the lock again FileSystem oldfs = map.get(key); if (oldfs != null) { // a file system is created while lock is releasing fs.close(); // close the new file system return oldfs; // return the old file system } // now insert the new file system into the map if (map.isEmpty() && !clientFinalizer.isAlive()) { Runtime.getRuntime().addShutdownHook(clientFinalizer); } fs.key = key; map.put(key, fs); if (conf.getBoolean("fs.automatic.close", true)) { toAutoClose.add(key); } return fs; } } -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira