steveloughran commented on code in PR #5488:
URL: https://github.com/apache/hadoop/pull/5488#discussion_r1144771186
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -519,11 +531,38 @@ public AbfsClientRenameResult renamePath(
final String destination,
final String continuation,
final TracingContext tracingContext,
- final String sourceEtag,
+ String sourceEtag,
boolean isMetadataIncompleteState)
Review Comment:
aa proposed, add a new `boolean isNamespaceEnabled` parameter
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##########
@@ -519,11 +531,38 @@ public AbfsClientRenameResult renamePath(
final String destination,
final String continuation,
final TracingContext tracingContext,
- final String sourceEtag,
+ String sourceEtag,
boolean isMetadataIncompleteState)
throws AzureBlobFileSystemException {
final List<AbfsHttpHeader> requestHeaders = createDefaultHeaders();
+ // etag passed in, so source is a file
+ final boolean hasEtag = !isEmpty(sourceEtag);
+ boolean isDir = !hasEtag;
+ if (!hasEtag && renameResilience) {
Review Comment:
and add `&& isNamespaceEnabled` to the condition
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java:
##########
@@ -441,11 +441,19 @@ public boolean rename(final Path src, final Path dst)
throws IOException {
return dstFileStatus.isDirectory() ? false : true;
}
+ boolean isNamespaceEnabled =
abfsStore.getIsNamespaceEnabled(tracingContext);
+
// Non-HNS account need to check dst status on driver side.
- if (!abfsStore.getIsNamespaceEnabled(tracingContext) && dstFileStatus ==
null) {
+ if (!isNamespaceEnabled && dstFileStatus == null) {
dstFileStatus = tryGetFileStatus(qualifiedDstPath, tracingContext);
}
+ // for Non-HNS accounts, rename resiliency cannot be maintained
+ // as eTags are not preserved in rename
Review Comment:
don't do it this way. AzureBlobFileSystemStore.getIsNamespaceEnabled()
provides the information, so add a new isNamespaceEnabled parameter to
abfsclient.renamePath() and use that in the decision making
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
