[ https://issues.apache.org/jira/browse/HADOOP-14266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967989#comment-15967989 ]
Steve Loughran commented on HADOOP-14266: ----------------------------------------- Let's wait for Aaron & his review. * Never noticed the code style {{for (; statusIterator.hasNext();)}}. before. Nice - I'll use it myself * added a patch with some javadoc tweaks, and made the empty FileStatus entry returned on all non-s3guard listings a constant. * testing, s3 ireland. * got one test failure on the 004 patch, with args {{-Dparallel-tests -DtestsThreadCount=8 -Ddynamo}}: {code} Tests run: 44, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 86.082 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract testRenameEmptyToDirWithSamePrefixAllowed(org.apache.hadoop.fs.s3a.ITestS3AFileSystemContract) Time elapsed: 3.221 sec <<< FAILURE! junit.framework.AssertionFailedError: Rename result expected:<true> but was:<false> at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:174) at junit.framework.TestCase.assertEquals(TestCase.java:333) at org.apache.hadoop.fs.FileSystemContractBaseTest.rename(FileSystemContractBaseTest.java:567) at org.apache.hadoop.fs.FileSystemContractBaseTest.testRenameEmptyToDirWithSamePrefixAllowed(FileSystemContractBaseTest.java:715) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at junit.framework.TestCase.runBare(TestCase.java:141) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) Tests run: 61, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 164.347 sec - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextMainOperations Results : Failed tests: ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameEmptyToDirWithSamePrefixAllowed:715->FileSystemContractBaseTest.rename:567 Rename result expected:<true> but was:<false> {code} This failure implies that attempting to rename an empty dir to itself failed, when it should be a no-op returning true. Maybe the dir wasn't considered empty. I could not create this when running I've managed to get this, with my changes, and after rolling back, with/without dynamo. {code} Tests run: 4, Failures: 0, Errors: 1, Skipped: 3, Time elapsed: 12.364 sec <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency testConsistentListFiles(org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency) Time elapsed: 8.944 sec <<< ERROR! java.io.FileNotFoundException: No such file or directory: s3a://hwdev-steve-ireland-new/fork-6/test/doTestListFiles-0-0-1-true at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:1852) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:1748) at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1710) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerListFiles(S3AFileSystem.java:2435) at org.apache.hadoop.fs.s3a.S3AFileSystem.listFiles(S3AFileSystem.java:2418) at org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.doTestListFiles(ITestS3GuardListConsistency.java:231) at org.apache.hadoop.fs.s3a.ITestS3GuardListConsistency.testConsistentListFiles(ITestS3GuardListConsistency.java:170) {code} > S3Guard: S3AFileSystem::listFiles() to employ MetadataStore > ----------------------------------------------------------- > > Key: HADOOP-14266 > URL: https://issues.apache.org/jira/browse/HADOOP-14266 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: HADOOP-13345 > Reporter: Mingliang Liu > Assignee: Mingliang Liu > Attachments: HADOOP-14266-HADOOP-13345.000.patch, > HADOOP-14266-HADOOP-13345.001.patch, HADOOP-14266-HADOOP-13345.002.patch, > HADOOP-14266-HADOOP-13345.003.patch, HADOOP-14266-HADOOP-13345.003.patch, > HADOOP-14266-HADOOP-13345.004.patch, HADOOP-14266-HADOOP-13345-005.patch, > HADOOP-14266-HADOOP-13345.005.patch > > > Similar to [HADOOP-13926], this is to track the effort of employing > MetadataStore in {{S3AFileSystem::listFiles()}}. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org