steveloughran edited a comment on issue #1810: HADOOP-16746 mkdirs and s3guard auth mode URL: https://github.com/apache/hadoop/pull/1810#issuecomment-577828928 Move from to intermingled dirListingUnion algorithms in the same method, the auth and nonauth merge/update operations have been split into their own methods. There is a bit of duplication -but at least now the different operations are isolated enough it's possible to understand them. TestS3Guard has been extended to help test of some of this. We can't verify that existing entries don't get overwritten, but the unit tests are at least checking both algorithms. Tested: S3 Ireland. One odd test failure which has me worried -but I've seen it happen on another branch *without any of this change, or its predecessor in* ``` java.lang.AssertionError: Expected a java.io.FileNotFoundException to be thrown, but got the result: : 16 at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499) at org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384) at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.expectExceptionWhenReading(ITestS3GuardOutOfBandOperations.java:975) at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.deleteFileInListing(ITestS3GuardOutOfBandOperations.java:959) at org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations.testListingDelete(ITestS3GuardOutOfBandOperations.java:309) ``` what is happening is we've deleted a file, "spun" for it to go way from the unguarded FS, then when opening in the guarded FS we see data back. Need to look more -it could be because my store is versioned -but if so, why fail so fast?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
