[
https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15380342#comment-15380342
]
Virajith Jalaparti commented on HDFS-10636:
-------------------------------------------
The patch contains the following changes (copied here from comments in
HDFS-9809).
# Moving the {{java.io.File}} related APIs in {{ReplicaInfo}}
({{getBlockFile}}, {{getMetaFile}}) to a subclass of {{ReplicaInfo}} called
{{LocalReplica}}. The classes {{FinalizedReplica}}, {{ReplicaInPipeline}},
{{ReplicaUnderRecovery}}, and {{ReplicaWaitingToBeRecovered}} are changed to be
subclasses of {{LocalReplica}} instead of {{ReplicaInfo}}. The motivation
behind this change is that we can have {{ReplicaInfo}} s that point to blocks
located in remote stores and as a result don’t have associated {{java.io.File}}
s.
We added various functions to {{ReplicaInfo}} in order to replace the calls to
{{ReplicaInfo#getBlockFile}}, and {{ReplicaInfo#getMetaFile}} in the rest of
the code.
# Using {{ReplicaInfo.getState()}} to get the state of a {{ReplicaInfo}}
instead of using {{instanceof}}. A related change is to use the class
{{ReplicaInfo}} to refer to the replica objects instead of the particular
subclass (this required adding additional abstract functions to the
{{ReplicaInfo}} class).
# Addition of a {{ReplicaBuilder}} and replacing calls to the constructors of
different {{ReplicaInfo}} subclasses ({{ReplicaInPipeline}},
{{ReplicaBeingWritten}}, etc.) with calls to the {{ReplicaBuilder}} with the
appropriate parameters ({{ReplicaState}}, {{blockId}} etc.) set.
# Changes related to {{ReplicaInPipeline}}
* Change the {{ReplicaInPipeline}} to {{LocalReplicaInPipeline}}, and change
{{ReplicaInPipelineInterface}} to {{ReplicaInPipeline}}.
* Add a {{getReplicaInfo}} function to the (new) {{ReplicaInPipeline}}
interface.
* Move the functions related to writer threads ({{stopWriter}},
{{attemptToSetWriter}}, {{interruptThread}} and {{setWriter}}) to the new
{{ReplicaInPipeline}} interface (i.e., the old {{ReplicaInPipelineInterface}}),
as only {{ReplicaInPipeline}} objects will be associated with writer threads.
The idea behind the changes above is to add a new {{ProvidedReplica}} class (an
implementation of {{ReplicaInfo}}) which can be:
(a) used to represent replicas stored in a provided storage (described in more
detail in the design documentation of HDFS-9806).
(b) treated as any other {{ReplicaInfo}} in the rest of the code. This would
avoid changes to the rest of the Datanode as part of HDFS-9806.
(c) written to using the existing replication pipeline, without implementing a
separate write pipeline for HDFS-9806.
> Modify ReplicaInfo to remove the assumption that replica metadata and data
> are stored in java.io.File.
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10636
> URL: https://issues.apache.org/jira/browse/HDFS-10636
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, fs
> Reporter: Virajith Jalaparti
>
> Replace java.io.File related APIs from {{ReplicaInfo}}, and enable the
> definition of new {{ReplicaInfo}} sub-classes whose metadata and data can be
> present on external storages (HDFS-9806).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
