[
https://issues.apache.org/jira/browse/HDFS-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hairong Kuang updated HDFS-624:
-------------------------------
Attachment: pipelineRecovery.patch
This patch implemented the new pipeline recovery algorithm described in the
design doc. With this patch, a client does not go through a primary data to
perform pipeline recovery.
Major changes include:
1. if setting up a pipeline for append, data streamer calls
setupPipelineForAppendOrRecovery(); If setting up a pipeline for error
recovery, data streamer calls processDatanodeError() which in turn calls
setupPipelineForAppendOrRecovery().
2. When send a opWrite request to datanode to set up a pipeline, a client sends
the reason, i.e, construction stage (create, append, append recovery, or
streaming recovery), along with newGS, minBytesRcvd, and maxBytesRcvd etc.
In above two case, a client gets a new generation stamp and an access token
from NameNode before setting up a pipeline and sends the pipeline update to
NameNode after a successful pipeline setup.
3. Depending on the reason for setting up a pipeline, Datanode calls
createTemporary, createRbw, append, recoverAppend, or recoverRbw to handle the
request respectively.
> Client support pipeline recovery
> --------------------------------
>
> Key: HDFS-624
> URL: https://issues.apache.org/jira/browse/HDFS-624
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: Append Branch
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: Append Branch
>
> Attachments: pipelineRecovery.patch
>
>
> This jira aims to
> 1. set up initial pipeline for append;
> 2. recover failed pipeline setup for append;
> 2. set up pipeline to recover failed data streaming.
> The algorithm is described in the design document in the pipeline recovery
> and pipeline set up sections. Pipeline close and failed pipeline close are
> not included in this jira.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
