[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-23 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866696567 + maybe I could test this by setting the sysprop `aws.region` to something invalid. If the region resolution is going through the chain then this would get picked up ahead

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-23 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866694906 One thought here: would you ever want the s3a connector to fall back to that bundled region lookup sequence? I'm wondering in particular if it makes a difference in

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866183472 tested with `-Dparallel-tests -DtestsThreadCount=7 -Dmarkers=keep -Dscale -Ds3guard -Ddynamo` transient read buffer underflow failure; one extra S3Guard write than

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865965433 Test run with `-Dparallel-tests -DtestsThreadCount=7 -Dmarkers=keep` -no failures! rerunning with dynamo & scale -- This is an automated message from the Apache

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865943337 + No new tests here. The only way to test would be to destroy ~/.aws/config and & skip if the env vars/syssprops resolved or running in EC2. If we could set the resolve

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865941502 ## `cloudstore regions` This verifies that the local system is set up to fail without the patch. ``` > bin/hadoop jar $CLOUDSTORE regions

[GitHub] [hadoop] steveloughran commented on pull request #3133: HADOOP-17771. S3AFS creation fails without region set in ~/.aws/config.

2021-06-22 Thread GitBox
steveloughran commented on pull request #3133: URL: https://github.com/apache/hadoop/pull/3133#issuecomment-865940542 Testing in progress, in a setup where * the failure mode is replicated * fs.s3a.endpoint is not set globally or for the test bucket. -- This is an automated