virajjasani commented on PR #6539: URL: https://github.com/apache/hadoop/pull/6539#issuecomment-1992509237
Something seems odd. This test overrides endpoint/region configs so setting any endpoint/region should have made no difference: ``` @Test public void testCentralEndpointAndNullRegionFipsWithCRUD() throws Throwable { describe("Access the test bucket using central endpoint and" + " null region and fips enabled, perform file system CRUD operations"); final Configuration conf = getConfiguration(); final Configuration newConf = new Configuration(conf); removeBaseAndBucketOverrides( newConf, ENDPOINT, AWS_REGION, FIPS_ENDPOINT); newConf.set(ENDPOINT, CENTRAL_ENDPOINT); newConf.setBoolean(FIPS_ENDPOINT, true); newFS = new S3AFileSystem(); newFS.initialize(getFileSystem().getUri(), newConf); assertOpsUsingNewFs(); } ``` I tested using these settings and there is no difference in behaviour because the test overrides base and bucket configs for endpoint/region. I tried: 1. endpoint: us-west-2, region: unset 2. endpoint: central, region: unset 3. endpoint: unset, region: unset From the stacktrace from Jira: ``` [ERROR] Tests run: 18, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 56.26 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion [ERROR] testCentralEndpointAndNullRegionFipsWithCRUD(org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion) Time elapsed: 4.821 s <<< ERROR! java.net.UnknownHostException: getFileStatus on s3a://stevel-london/test/testCentralEndpointAndNullRegionFipsWithCRUD/srcdir: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.: software.amazon.awssdk.core.exception.SdkClientException: Received an UnknownHostException when attempting to interact with a service. See cause for the exact endpoint that is failing to resolve. If this is happening on an endpoint that previously worked, there may be a network connectivity issue or your DNS cache could be storing endpoints for too long.: stevel-london.s3-fips.eu-west-2.amazonaws.com at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.fs.s3a.impl.ErrorTranslation.wrapWithInnerIOE(ErrorTranslation.java:182) at org.apache.hadoop.fs.s3a.impl.ErrorTranslation.maybeExtractIOException(ErrorTranslation.java:152) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:207) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:155) at org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:4066) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:3922) at org.apache.hadoop.fs.s3a.S3AFileSystem$MkdirOperationCallbacksImpl.probePathStatus(S3AFileSystem.java:3794) at org.apache.hadoop.fs.s3a.impl.MkdirOperation.probePathStatusOrNull(MkdirOperation.java:173) at org.apache.hadoop.fs.s3a.impl.MkdirOperation.getPathStatusExpectingDir(MkdirOperation.java:194) at org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:108) at org.apache.hadoop.fs.s3a.impl.MkdirOperation.execute(MkdirOperation.java:57) at org.apache.hadoop.fs.s3a.impl.ExecutingStoreOperation.apply(ExecutingStoreOperation.java:76) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528) at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449) at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2707) at org.apache.hadoop.fs.s3a.S3AFileSystem.trackDurationAndSpan(S3AFileSystem.java:2726) at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:3766) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2494) at org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion.assertOpsUsingNewFs(ITestS3AEndpointRegion.java:461) at org.apache.hadoop.fs.s3a.ITestS3AEndpointRegion.testCentralEndpointAndNullRegionFipsWithCRUD(ITestS3AEndpointRegion.java:454) ``` Here, we set: ``` removeBaseAndBucketOverrides( newConf, ENDPOINT, AWS_REGION, FIPS_ENDPOINT); newConf.set(ENDPOINT, CENTRAL_ENDPOINT); newConf.setBoolean(FIPS_ENDPOINT, true); newFS = new S3AFileSystem(); newFS.initialize(getFileSystem().getUri(), newConf); ``` How could stacktrace show different region than us-east-2 when the test overrides endpoint to central and removes region? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org