[
https://issues.apache.org/jira/browse/HDFS-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Zeller updated HDFS-492:
-----------------------------
Description:
This adds two additional functions to FSNamesystem to provide more information
about corrupt replicas. It also adds two servlets to the namenode that provide
information (in JSON) about all blocks with corrupt replicas as well as
information about a specific block. It also changes the file browsing servlet
by adding a link from block ids to the above mentioned block information page.
These JSON pages are designed to be used by client side tools which wish to
analyze corrupt block/replicas. The only change to an existing (non-servlet)
class is described below).
Currently, CorruptReplicasMap stores a map of corrupt replica information and
allows insertion and deletion. It also gives information about the corrupt
replicas for a specific block. It does not allow iteration over all corrupt
blocks. Two additional functions will be added to FSNamesystem (which will call
BlockManager which will call CorruptReplicasMap). The first will return the
size of the corrupt replicas map, which represents the number of blocks that
have corrupt replicas (but less than the number of corrupt replicas if a block
has multiple corrupt replicas). The second will allow "paging" through a list
of block ids that contain corrupt replicas:
{{public synchronized List<Long> getCorruptReplicaBlockIds(int n, Long
startingBlockId)}}
{{n}} is the number of block ids to return and {{startingBlockId}} is the block
id offset. To prevent a large number of items being returned at one time, n is
constrained to 0 <= {{n}} <= 100. If {{startingBlockId}} is null, up to {{n}}
items are returned starting at the beginning of the list. Ordering is enforced
through the internal use of TreeMap in CorruptReplicasMap.
was:
This adds two additional functions to FSNamesystem to provide more information
about corrupt replicas. It also adds two servlets to the namenode that provide
information (in JSON) about all blocks with corrupt replicas as well as
information about a specific block. It also changes the file browsing servlet
by adding a link from block ids to the above mentioned block information page.
These JSON pages are designed to be used by client side tools which wish to
analyze corrupt block/replicas. The only change to an existing (non-servlet)
class is described below).
Currently, CorruptReplicasMap stores a map of corrupt replica information and
allows insertion and deletion. It also gives information about the corrupt
replicas for a specific block. It does not allow iteration over all corrupt
blocks. Two additional functions will be added to FSNamesystem (which will call
BlockManager which will call CorruptReplicasMap). The first will return the
size of the corrupt replicas map, which represents the number of blocks that
have corrupt replicas (but less than the number of corrupt replicas if a block
has multiple corrupt replicas). The second will allow "paging" through a list
of block ids that contain corrupt replicas:
{{ public synchronized List<Long> getCorruptReplicaBlockIds(int n, Long
startingBlockId) }}
{{n}} is the number of block ids to return and {{startingBlockId}} is the block
id offset. To prevent a large number of items being returned at one time, n is
constrained to 0 <= {{n}} <= 100. If {{startingBlockId}} is null, up to {{n}}
items are returned starting at the beginning of the list. Ordering is enforced
through the internal use of TreeMap in CorruptReplicasMap.
> Expose corrupt replica/block information
> ----------------------------------------
>
> Key: HDFS-492
> URL: https://issues.apache.org/jira/browse/HDFS-492
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: data-node, name-node
> Affects Versions: 0.21.0
> Reporter: Bill Zeller
> Priority: Minor
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> This adds two additional functions to FSNamesystem to provide more
> information about corrupt replicas. It also adds two servlets to the namenode
> that provide information (in JSON) about all blocks with corrupt replicas as
> well as information about a specific block. It also changes the file browsing
> servlet by adding a link from block ids to the above mentioned block
> information page.
> These JSON pages are designed to be used by client side tools which wish to
> analyze corrupt block/replicas. The only change to an existing (non-servlet)
> class is described below).
> Currently, CorruptReplicasMap stores a map of corrupt replica information and
> allows insertion and deletion. It also gives information about the corrupt
> replicas for a specific block. It does not allow iteration over all corrupt
> blocks. Two additional functions will be added to FSNamesystem (which will
> call BlockManager which will call CorruptReplicasMap). The first will return
> the size of the corrupt replicas map, which represents the number of blocks
> that have corrupt replicas (but less than the number of corrupt replicas if a
> block has multiple corrupt replicas). The second will allow "paging" through
> a list of block ids that contain corrupt replicas:
> {{public synchronized List<Long> getCorruptReplicaBlockIds(int n, Long
> startingBlockId)}}
> {{n}} is the number of block ids to return and {{startingBlockId}} is the
> block id offset. To prevent a large number of items being returned at one
> time, n is constrained to 0 <= {{n}} <= 100. If {{startingBlockId}} is null,
> up to {{n}} items are returned starting at the beginning of the list.
> Ordering is enforced through the internal use of TreeMap in
> CorruptReplicasMap.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
