[
https://issues.apache.org/jira/browse/HDFS-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737324#action_12737324
]
Suresh Srinivas commented on HDFS-492:
--------------------------------------
Comments on the previous version of the patch. I will review the next version
that works with trunk:
# {{BlockManager.numBlocksWithCorruptReplicas()}}
#* Currently this can be obtained by by looking
{{BlockManager.corruptReplicaBlocksCount}} (unsynchronized). However this count
is updated every {{dfs.replication.interva}}. If that approximate number is
sufficent, then there is no need to introduce this method. Also existing method
{{FSNamesystem.getCorruptReplicaBlocks()}} instead of new method added
{{FSNamesystem.numBlocksWithCorruptReplicas()}}, can be used.
#* If the number needs to be accurate, then remove
{{BlockManager.corruptReplicaBlocksCount}} and synchronize the method
{{FSNamesystem.getCorruptReplicaBlocks()}} and use
{{BlockManager.numBlocksWithCorruptReplicas()}} to get the count.
# {{CorruptReplicasMap.numBlocksWithCorruptReplicas()}} - instead of adding
this, use existing method {{CorruptReplicasMap.size()}}
# Comments in {{CorruptReplicasMap.java}} can be shorted. No need to explain
how the code works - a brief comment will be sufficient . Also remove the
comment with XXXBZ
# {{CorruptReplicasMap.getCorruptReplicaBlockIds}} - it is better to take
{{long}} as second param instead of object {{Long}}. Also returning {{long[]}}
is better than {{List<Long>}}
# TestCorruptReplicaInfo.java - use assertEquals() instead of assertTrue().
When the test fails it gives information about expected Vs actual value.
# TestCorruptReplicaInfo.block_map - can just be a HashMap
Nits:
# Generic - instead of {{n}}, use {{numExpectedBlocks}}
# CorruptReplicasMap.java - better to change the type of corruptReplicasMap
from TreeMap to SortedMap
> Expose corrupt replica/block information
> ----------------------------------------
>
> Key: HDFS-492
> URL: https://issues.apache.org/jira/browse/HDFS-492
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, name-node
> Affects Versions: 0.21.0
> Reporter: Bill Zeller
> Assignee: Bill Zeller
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hdfs-492-4.patch, hdfs-492-5.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> This adds two additional functions to FSNamesystem to provide more
> information about corrupt replicas. It also adds two servlets to the namenode
> that provide information (in JSON) about all blocks with corrupt replicas as
> well as information about a specific block. It also changes the file browsing
> servlet by adding a link from block ids to the above mentioned block
> information page.
> These JSON pages are designed to be used by client side tools which wish to
> analyze corrupt block/replicas. The only change to an existing (non-servlet)
> class is described below.
> Currently, CorruptReplicasMap stores a map of corrupt replica information and
> allows insertion and deletion. It also gives information about the corrupt
> replicas for a specific block. It does not allow iteration over all corrupt
> blocks. Two additional functions will be added to FSNamesystem (which will
> call BlockManager which will call CorruptReplicasMap). The first will return
> the size of the corrupt replicas map, which represents the number of blocks
> that have corrupt replicas (but less than the number of corrupt replicas if a
> block has multiple corrupt replicas). The second will allow "paging" through
> a list of block ids that contain corrupt replicas:
> {{public synchronized List<Long> getCorruptReplicaBlockIds(int n, Long
> startingBlockId)}}
> {{n}} is the number of block ids to return and {{startingBlockId}} is the
> block id offset. To prevent a large number of items being returned at one
> time, n is constrained to 0 <= {{n}} <= 100. If {{startingBlockId}} is null,
> up to {{n}} items are returned starting at the beginning of the list.
> Ordering is enforced through the internal use of TreeMap in
> CorruptReplicasMap.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
