[
https://issues.apache.org/jira/browse/HADOOP-7456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063685#comment-13063685
]
Luke Lu commented on HADOOP-7456:
---------------------------------
Is it still a problem in 0.20.203? If so, we need to fix it in 20x as well.
> Connection with RemoteException is not removed from cached HashTable and
> cause memory leak
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-7456
> URL: https://issues.apache.org/jira/browse/HADOOP-7456
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.2
> Reporter: Angelo K. Huang
>
> In a long running system like Oozie, we use hadoop client APIs, such as
> FileSystem.exists(), to check files exist on hdfs or not to kick off a user
> job. But in a production environment, user sometimes gives wrong or invalid
> format of file/directory paths. In that case, after server was up for couple
> days, we found around 80% of memory were taken away by hadoop ipc client
> connections. In one of the connections, there was a hashtable contains 200k
> entries. We cross-checked Hadoop code and found out that in
> org.apache.hadoop.ipc.Client.receiveResponse(), if state if fatal, the call
> object does not remove from the hashtable (calls) and keeps in the memory
> until system throws OutOfMemory error or crash. The code in question is here :
> * org.apache.hadoop.ipc.Client.receiveResponse()
> } else if (state == Status.FATAL.state) {
> // Close the connection
> markClosed(new RemoteException(WritableUtils.readString(in),
> WritableUtils.readString(in)));
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
