[
https://issues.apache.org/jira/browse/HDFS-17509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844943#comment-17844943
]
ASF GitHub Bot commented on HDFS-17509:
---------------------------------------
ZanderXu commented on code in PR #6784:
URL: https://github.com/apache/hadoop/pull/6784#discussion_r1595242159
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientProtocol.java:
##########
@@ -667,39 +667,28 @@ public void rename2(final String src, final String dst,
public void concat(String trg, String[] src) throws IOException {
rpcServer.checkOperation(NameNode.OperationCategory.WRITE);
- // See if the src and target files are all in the same namespace
- LocatedBlocks targetBlocks = getBlockLocations(trg, 0, 1);
- if (targetBlocks == null) {
- throw new IOException("Cannot locate blocks for target file - " + trg);
- }
- LocatedBlock lastLocatedBlock = targetBlocks.getLastLocatedBlock();
- String targetBlockPoolId = lastLocatedBlock.getBlock().getBlockPoolId();
- for (String source : src) {
- LocatedBlocks sourceBlocks = getBlockLocations(source, 0, 1);
- if (sourceBlocks == null) {
- throw new IOException(
- "Cannot located blocks for source file " + source);
- }
- String sourceBlockPoolId =
- sourceBlocks.getLastLocatedBlock().getBlock().getBlockPoolId();
- if (!sourceBlockPoolId.equals(targetBlockPoolId)) {
- throw new IOException("Cannot concatenate source file " + source
- + " because it is located in a different namespace"
- + " with block pool id " + sourceBlockPoolId
- + " from the target file with block pool id "
- + targetBlockPoolId);
- }
- }
+ // Concat only effects when all files in same namespace.
+ // And in router view, a file only exists in one RemoteLocation.
Review Comment:
Normally a file only exits in one Namespace. But maybe multiple namespaces
contain this file and RBF returns the file in the first namespace to the client.
This first namespace is got by the `OrderedResolver`.
you can refer to this case:
- /path mounts to NS1, NS2 and NS3
- NS2 and NS3 contains /path
- `OrderedResolver` returns NS1, NS2 and NS3
for this case, we should proxy this `concat` to NS2 instead of NS1, right?
> RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file.
> ----------------------------------------------------------------------
>
> Key: HDFS-17509
> URL: https://issues.apache.org/jira/browse/HDFS-17509
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: liuguanghua
> Priority: Minor
> Labels: pull-request-available
>
> hdfs dfs -concat /tmp/merge /tmp/t1 /tmp/t2
> When /tmp/merge is a empty file, this command will throw NPE via DFSRouter.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
