[
https://issues.apache.org/jira/browse/HDFS-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399136#comment-13399136
]
Uma Maheswara Rao G commented on HDFS-3194:
-------------------------------------------
{quote}
scanBlockPoolSlice gets called every 5 seconds by DataBlockScanner.run().
{quote}
verification logs will track the information about prvious scan time. So, while
scanning, it will load this info from this logs and will ensure that one block
scanned only after 3*weeks time.
HDFS-3194_7.patch addresses one potential problem. For every 5 sec run, logs
are simply rolling. for every 3 slices, all verification logs are getting
cleaned as max rolled index is 2 I remember and created new ones. So, the same
block will get scanned for every 15secs again and again. I remember, we have
verification log size limit in 20.x versions. So, it will not be rolled untill
it reaches that size. That has been broken here with some other JIRAs.
Let me check your point why startNewPeriod(); is required for every 5 sec slice.
> DataNode block scanner is running too frequently
> ------------------------------------------------
>
> Key: HDFS-3194
> URL: https://issues.apache.org/jira/browse/HDFS-3194
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 2.0.0-alpha
> Reporter: suja s
> Assignee: amith
> Attachments: HDFS-3194.patch, HDFS-3194_1.patch, HDFS-3194_2.patch,
> HDFS-3194_4.patch, HDFS-3194_6.patch, HDFS-3194_7.patch
>
>
> Block scanning interval by default should be taken as 21 days(3 weeks) and
> each block scanning should happen once in 21 days.
> Here the block is being scanned continuosly.
> 2012-04-03 10:44:47,056 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:02,064 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:17,071 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for
> BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473_1003
> 2012-04-03 10:45:32,079 INFO
> org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner: Verification
> succeeded for BP-241703115-xx.xx.xx.55-1333086229434:blk_-2666054955039014473
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
