[
https://issues.apache.org/jira/browse/HDFS-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253497#comment-13253497
]
amith commented on HDFS-3194:
-----------------------------
Hi Suja,
I have done analysis on the reported defect and came up with a patch.
DataBlockScanner thread will scan for the new blocks for every 5 seconds(new
period start), but in the scan method of BlockPoolSliceScanner finally we are
rolling the logs(irrespective of entries in logs).
When a new period starts we are loading the number of entries from the logs and
non scanned blocks are scanned.
Since there is log roll for every 5 sec after 10 seconds the previously scanned
entries are lost!!!!
causing the block scan for the next period. Thus we can see the time difference
is 15 sec for every consecutive scan.
In the provided patch I have introduced log roll only after certain number of
entries in the logs are reached(which is similar to 20.x version HDFS
behaviour), care is taken not to break log interface which already exist.
> Continuous block scanning at DN side
> ------------------------------------
>
> Key: HDFS-3194
> URL: https://issues.apache.org/jira/browse/HDFS-3194
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: suja s
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HDFS-3194.patch
>
>
> Block scanning interval by default should be taken as 21 days(3 weeks) and
> each block scanning should happen once in 21 days.
> Here the block is being scanned continuosly.
> 2012-04-03 10:44:47,056 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:02,064 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:17,071 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:32,079 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
