Hello, I am trying to understand the lock in FsVolumeImpl#getDfsUsed(). As many of you know, a busy datanode can occasionally block on the call to FsVolumeImpl#getDfsUsed(). This method is used in a number of places, especially used in storage reports in heartbeat messages.
There have been some attempt to make this situation better, for example HDFS-7999 stops a writer if it blocks FsDatasetImpl for too long, preventing heartbeat messages. However, when I looked at the code closer, I started to wonder if FsVolumeImpl#getDfsUsed() needs FsDatasetImpl lock at all. Because the BlockPoolSlice hash table bpSlices is a ConcurrentHashMap so iteration over this object doesn’t need a lock. It also seems to be an overkill because the operation does not modify BlockPoolSlice object at all. I do not see extra benefits locking on FsDatasetImpl object. Thoughts? Wei-Chiu Chuang -- A very happy Hadoop contributor
