[
https://issues.apache.org/jira/browse/HADOOP-19157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840122#comment-17840122
]
Steve Loughran commented on HADOOP-19157:
-----------------------------------------
note: this is not a problem with abfs -it just has the most ambitious test
runner.
{code}
[ERROR]
testMkdirsPopulatingAllNonexistentAncestors(org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir)
Time elapsed: 0.475 s <<< ERROR!
java.io.FileNotFoundException:
abfs://stevel-test...@stevelukwest.dfs.core.windows.net/fork-0002/test/testMkdirsPopulatingAllNonexistentAncestors/a/b/c/d/e/f/g/h/i/j/k/L
nested dir should exist: not found
abfs://stevel-test...@stevelukwest.dfs.core.windows.net/fork-0002/test/testMkdirsPopulatingAllNonexistentAncestors/a/b/c/d/e/f/g/h/i/j/k/L
in
abfs://stevel-test...@stevelukwest.dfs.core.windows.net/fork-0002/test/testMkdirsPopulatingAllNonexistentAncestors/a/b/c/d/e/f/g/h/i/j/k
at
org.apache.hadoop.fs.contract.ContractTestUtils.verifyPathExists(ContractTestUtils.java:985)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertPathExists(ContractTestUtils.java:963)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.assertPathExists(AbstractFSContractTestBase.java:319)
at
org.apache.hadoop.fs.contract.AbstractContractMkdirTest.testMkdirsPopulatingAllNonexistentAncestors(AbstractContractMkdirTest.java:150)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.FileNotFoundException: Operation failed: "The specified path
does not exist.", 404, HEAD,
https://stevelukwest.dfs.core.windows.net/stevel-testing/fork-0002/test/testMkdirsPopulatingAllNonexistentAncestors/a/b/c/d/e/f/g/h/i/j/k/L?upn=false&action=getStatus&timeout=90s,
rId: 50a0ad90-f01f-0065-688c-950836000000
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1503)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:736)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:724)
at
org.apache.hadoop.fs.contract.ContractTestUtils.verifyPathExists(ContractTestUtils.java:979)
... 18 more
Caused by: Operation failed: "The specified path does not exist.", 404, HEAD,
https://stevelukwest.dfs.core.windows.net/stevel-testing/fork-0002/test/testMkdirsPopulatingAllNonexistentAncestors/a/b/c/d/e/f/g/h/i/j/k/L?upn=false&action=getStatus&timeout=90s,
rId: 50a0ad90-f01f-0065-688c-950836000000
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.completeExecute(AbfsRestOperation.java:270)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.lambda$execute$0(AbfsRestOperation.java:216)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.measureDurationOfInvocation(IOStatisticsBinding.java:494)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfInvocation(IOStatisticsBinding.java:465)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:214)
at
org.apache.hadoop.fs.azurebfs.services.AbfsClient.getPathStatus(AbfsClient.java:1083)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.getFileStatus(AzureBlobFileSystemStore.java:1115)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:734)
... 20 more
[ERROR]
testNoMkdirOverFile(org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractMkdir)
Time elapsed: 0.437 s <<< ERROR!
java.io.FileNotFoundException: Operation failed: "The specified path does not
exist.", 404, HEAD,
https://stevelukwest.dfs.core.windows.net/stevel-testing/fork-0002/test/testNoMkdirOverFile?upn=false&action=getStatus&timeout=90s,
rId: 50a0ada5-f01f-0065-7d8c-950836000000
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1503)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.open(AzureBlobFileSystem.java:286)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.open(AzureBlobFileSystem.java:271)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:995)
at
org.apache.hadoop.fs.contract.ContractTestUtils.readDataset(ContractTestUtils.java:232)
at
org.apache.hadoop.fs.contract.AbstractContractMkdirTest.testNoMkdirOverFile(AbstractContractMkdirTest.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Caused by: Operation failed: "The specified path does not exist.", 404, HEAD,
https://stevelukwest.dfs.core.windows.net/stevel-testing/fork-0002/test/testNoMkdirOverFile?upn=false&action=getStatus&timeout=90s,
rId: 50a0ada5-f01f-0065-7d8c-950836000000
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.completeExecute(AbfsRestOperation.java:270)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.lambda$execute$0(AbfsRestOperation.java:216)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.measureDurationOfInvocation(IOStatisticsBinding.java:494)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDurationOfInvocation(IOStatisticsBinding.java:465)
at
org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:214)
at
org.apache.hadoop.fs.azurebfs.services.AbfsClient.getPathStatus(AbfsClient.java:1083)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.openFileForRead(AzureBlobFileSystemStore.java:851)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.open(AzureBlobFileSystem.java:283)
... 19 more
[INFO] Running org.apache.hadoop.fs.azurebfs.services.ITestReadBufferManager
[ERROR] Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 13.199
s <<< FAILURE! - in
org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory
[ERROR]
testRecursiveRootListing(org.apache.hadoop.fs.azurebfs.contract.ITestAbfsFileSystemContractRootDirectory)
Time elapsed: 8.849 s <<< ERROR!
java.io.FileNotFoundException: Operation failed: "The specified path does not
exist.", 404, GET,
https://stevelukwest.dfs.core.windows.net/stevel-testing?upn=false&resource=filesystem&maxResults=5000&directory=fork-0005/test&timeout=90&recursive=false,
rId: 2f1b5a39-a01f-005f-718c-954b95000000, PathNotFound, "The specified path
does not exist. RequestId:2f1b5a39-a01f-005f-718c-954b95000000
Time:2024-04-23T14:42:01.4791674Z"
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.checkException(AzureBlobFileSystem.java:1503)
at
org.apache.hadoop.fs.azurebfs.services.AbfsListStatusRemoteIterator.addNextBatchIteratorToQueue(AbfsListStatusRemoteIterator.java:153)
at
org.apache.hadoop.fs.azurebfs.services.AbfsListStatusRemoteIterator.<init>(AbfsListStatusRemoteIterator.java:69)
at
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.listStatusIterator(AzureBlobFileSystem.java:1313)
at
org.apache.hadoop.fs.contract.ContractTestUtils.treeWalk(ContractTestUtils.java:1539)
at
org.apache.hadoop.fs.contract.ContractTestUtils.lambda$treeWalk$0(ContractTestUtils.java:1542)
{code}
> [ABFS] Filesystem contract tests to use methodPath for robust parallel test
> runs
> --------------------------------------------------------------------------------
>
> Key: HADOOP-19157
> URL: https://issues.apache.org/jira/browse/HADOOP-19157
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure, test
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> hadoop-azure supports parallel test runs, but unlike hadoop-aws, the azure
> ones are parallelised across methods in the same test suites.
> this can fail badly where contract tests have hard coded filenames and assume
> that they can use this across all test cases. Shows up when you are testing
> on a store with reduced IO capacity triggering retries and making some test
> cases slower
> Fix: hadoop-common contract tests to use methodPath() names
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
