Kihwal Lee created HDFS-5010: -------------------------------- Summary: Reduce the frequency of getCurrentUser() calls from namenode Key: HDFS-5010 URL: https://issues.apache.org/jira/browse/HDFS-5010 Project: Hadoop HDFS Issue Type: Improvement Reporter: Kihwal Lee
According to the "worst" jstack of a busy namenode I took, there were 29 BLOCKED handler threads. All of them were blocked on the UGI class lock. Here is the breakdown: -2 ensureInitialized() - from non static synchronized methods. This Jira will unblock these. -27 getCurrentUser() Among the 27 threads that were blocked at getCurrentUser(), - 18 FSPermissionChecker() - from FSNamesystem#getPermissionChecker() in most namenode RPC methods - 8 BlockTokenSecretManager#generateToken() - getBlockLocations() - 1 NameNodeRpcServer.mkdirs I think FSPermissionChecker can be modified to be created with a passed in UGI. FSNamesystem can the one already stored in RPC server by calling getRemoteUser(). This will eliminate a bulk of getCurrentUser() calls from namenode RPC handlers. A similar change can be made to mkdirs. Block token generation is not as straightforward. Even without it we can eliminate majority of the calls. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira