Rajesh Balamohan created HDFS-9520:
--------------------------------------
Summary: PeerCache evicts too frequently causing connection
restablishments
Key: HDFS-9520
URL: https://issues.apache.org/jira/browse/HDFS-9520
Project: Hadoop HDFS
Issue Type: Bug
Reporter: Rajesh Balamohan
Env: 20 node setup
dfs.client.socketcache.capacity = 16
Issue:
======
Monitored PeerCache and it was evicting lots of connections during close. Set
"dfs.client.socketcache.capacity=20" and tested again. Evictions still
happened. Ideally it should not be evicting the connections since
"dfs.client.socketcache.capacity" is set to 20 (no. of DN). Screenshot of
profiler is attached in the JIRA.
Workaround:
===========
Temp fix was to set "dfs.client.socketcache.capacity=1000" to prevent eviction.
Added more debug logs revealed that multimap.size() was 40 instead of 20.
LinkedListMultimap returns the total values instead of key size causing lots of
evictions.
{code}
if (capacity == multimap.size()) {
evictOldest();
}
{code}
Should this be (capacity == multimap.keySet().size()) ?
\cc [~gopalv], [~sseth]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
