[
https://issues.apache.org/jira/browse/HADOOP-19572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17953069#comment-17953069
]
ASF GitHub Bot commented on HADOOP-19572:
-----------------------------------------
anmolanmol1234 commented on code in PR #7698:
URL: https://github.com/apache/hadoop/pull/7698#discussion_r2099461766
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:
##########
@@ -2053,26 +2033,15 @@ public boolean isNonEmptyDirectory(String path,
TracingContext tracingContext) throws AzureBlobFileSystemException {
// This method is only called internally to determine state of a path
// and hence don't need identity transformation to happen.
- ListResponseData listResponseData = listPath(path, false, 1, null,
tracingContext, null, false);
- return !isEmptyListResults(listResponseData);
- }
-
- /**
- * Check if the list call returned empty results without any continuation
token.
- * @param listResponseData The response of listing API from the server.
- * @return True if empty results without continuation token.
- */
- private boolean isEmptyListResults(ListResponseData listResponseData) {
- AbfsHttpOperation result = listResponseData.getOp().getResult();
- boolean isEmptyList = result != null && result.getStatusCode() == HTTP_OK
&& // List Call was successful
- result.getListResultSchema() != null && // Parsing of list response
was successful
- listResponseData.getFileStatusList().isEmpty() &&
listResponseData.getRenamePendingJsonPaths().isEmpty() &&// No paths were
returned
- StringUtils.isEmpty(listResponseData.getContinuationToken()); // No
continuation token was returned
- if (isEmptyList) {
- LOG.debug("List call returned empty results without any continuation
token.");
- return true;
- }
- return false;
+ String continuationToken = null;
+ List<FileStatus> fileStatusList = new ArrayList<>();
+ // We need to loop on continuation token until we get an entry or
continuation token becomes null.
+ do {
+ ListResponseData listResponseData = listPath(path, false, 1, null,
tracingContext, null);
+ fileStatusList.addAll(listResponseData.getFileStatusList());
Review Comment:
continuationToken = listResponseData.getContinuationToken();
if (StringUtils.isNotEmpty(continuationToken) && fileStatusList != null
&& !fileStatusList.isEmpty()) {
return true;
}
} while (StringUtils.isNotEmpty(continuationToken));
we should just return true from here as soon as we receive one entry in the
list right ?
> ABFS: [FnsOverBlob] Empty Page Issue on Subsequent ListBlob call
> ----------------------------------------------------------------
>
> Key: HADOOP-19572
> URL: https://issues.apache.org/jira/browse/HADOOP-19572
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.5.0, 3.4.1
> Reporter: Anuj Modi
> Priority: Major
> Labels: pull-request-available
>
> We came across a new behavior from server where ListBlob call can return
> empty list even after returning a next marker(continuation token) from
> previous list call.
> This is to handle that case and do not infer listing to be incomplete.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
