support for persistent connections to improve pread() performance.
------------------------------------------------------------------
Key: HADOOP-3672
URL: https://issues.apache.org/jira/browse/HADOOP-3672
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Affects Versions: 0.17.0
Environment: Linux 2.6.9-55 , Dual Core Opteron 280 2.4Ghz , 4GB
memory
Reporter: George Wu
preads() establish new connections per request. yourkit java profiles show that
this connection overhead is pretty significant on the DataNode.
I wrote a simple microbenchmark program which does many iterations of pread()
from different offsets of a large file. I hacked DFSClient/DataNode code to
re-use the same connection/DataNode request handler thread. The performance
improvement was 7% when the data is served from disk and 80% when the data is
served from the OS page cache.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
