[
https://issues.apache.org/jira/browse/HDFS-16485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18038503#comment-18038503
]
ASF GitHub Bot commented on HDFS-16485:
---------------------------------------
github-actions[bot] commented on PR #4033:
URL: https://github.com/apache/hadoop/pull/4033#issuecomment-3535145588
We're closing this stale PR because it has been open for 100 days with no
activity. This isn't a judgement on the merit of the PR in any way. It's just a
way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working
on it, please feel free to re-open it and ask for a committer to remove the
stale tag and review again.
Thanks all for your contribution.
> [SPS]: allow re-satisfy path after restarting sps process
> ---------------------------------------------------------
>
> Key: HDFS-16485
> URL: https://issues.apache.org/jira/browse/HDFS-16485
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: qinyuren
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When SPSPathIdProcessor thread call getNextSPSPath(), it get the pathId from
> namenode and namenode will also remove this pathId from pathsToBeTraveresed
> queue.
> {code:java}
> public Long getNextPathId() {
> synchronized (pathsToBeTraveresed) {
> return pathsToBeTraveresed.poll();
> }
> } {code}
> If SPS process restart, this path will not continue the move operation until
> namenode restart.
> So we want to provide a way for the SPS to continue performing the move
> operation after SPS restart.
> First solution:
> 1) When SPSPathIdProcessor thread call getNextSPSPath(), namenode return
> pathId and then move this pathId to a pathsBeingTraveresed queue;
> 2) After SPS finish a path movement operation, it call a rpc to namenode to
> remove this pathId from pathsBeingTraveresed queue;
> 3) If SPS restart, SPSPathIdProcessor thread should call a rpc to namenode to
> get all pathId from pathsBeingTraveresed queue;
> Second solution:
> We added timeout detection in the application layer, if a path does not
> complete the movement within the specified time, we can re-satisfy this path
> even though it has "hdfs.sps" xattr already.
> We choose the second solution because the first solution will add more rpc
> operation and may affect namenode performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
