[ https://issues.apache.org/jira/browse/HDFS-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HDFS-2533: ------------------------------ Attachment: hdfs-2533.txt Slightly improved version. I found it was pretty trivial to fix contention in two other places: in these places we were doing a lock around an file.exists() call unnecessarily, since we were about to open the file for read right afterwards. Given that, the exists check is unnecessary - we'll get FileNotFoundException when we try to read the file. With this patch the numbers improve to: | Threads | Trunk | HDFS-2533v2 | | 4 | 226556 KB/s | 237805 KB/sec (1.05x) | | 16 | 377474 KB/s | 499399 KB/sec (1.32x) | | 8 | 410114 KB/s | 474560 KB/sec (1.15x) | > Remove needless synchronization on FSDataSet.getBlockFile > --------------------------------------------------------- > > Key: HDFS-2533 > URL: https://issues.apache.org/jira/browse/HDFS-2533 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node > Affects Versions: 0.23.0 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Priority: Minor > Attachments: hdfs-2533.txt, hdfs-2533.txt > > > HDFS-1148 discusses lock contention issues in FSDataset. It provides a more > comprehensive fix, converting it all to RWLocks, etc. This JIRA is for one > very specific fix which gives a decent performance improvement for > TestParallelRead: getBlockFile() currently holds the lock which is completely > unnecessary. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira