[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614578&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614578
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 15:40
Start Date: 24/Jun/21 15:40
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867742004
merged into trunk; building and testing for 3.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614578)
Time Spent: 3h 50m (was: 3h 40m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614576
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 15:37
Start Date: 24/Jun/21 15:37
Worklog Time Spent: 10m
Work Description: steveloughran merged pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614576)
Time Spent: 3h 40m (was: 3.5h)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614569&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614569
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 15:29
Start Date: 24/Jun/21 15:29
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867732526
thanks, merging!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614569)
Time Spent: 3.5h (was: 3h 20m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614527&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614527
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 14:07
Start Date: 24/Jun/21 14:07
Worklog Time Spent: 10m
Work Description: mukund-thakur commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867665936
@steveloughran Thanks. Tests run fine after setting the sts region and
endpoints. So we are good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614527)
Time Spent: 3h 20m (was: 3h 10m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614517&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614517
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 13:47
Start Date: 24/Jun/21 13:47
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867650724
@mukund-thakur see HADOOP-16565 for the STS behaviour; no regressions there
& that patch provides some diagnostics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614517)
Time Spent: 3h 10m (was: 3h)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614490&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614490
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 13:00
Start Date: 24/Jun/21 13:00
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867617024
The STS stuff has always needed it; look in testing.md and elsewhere for
references to the option. Maybe the SDK provided the fallback if the default
endpoint was used. That is not a regression, and all the docs discuss it.
I don't want to fix that here as its clearly not an issue in actual
deployments; this patch can focus on the regression
Set up your region for testing and it will go away
```
fs.s3a.assumed.role.sts.endpoint
${sts.london.endpoint}
fs.s3a.assumed.role.sts.endpoint.region
${sts.london.region}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614490)
Time Spent: 3h (was: 2h 50m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614476&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614476
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 12:43
Start Date: 24/Jun/21 12:43
Worklog Time Spent: 10m
Work Description: mukund-thakur commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867605687
After taking a glance. Not debugged in detail.
@steveloughran I think STS tests still require the region to be set in
~/.aws/config as the stack trace is different.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614476)
Time Spent: 2h 50m (was: 2h 40m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent problem on test clusters outside AWS infra: it really depended
> on the VM/docker image whether things worked or not.
> h2. Quick Fix: set {{fs.s3a.endpoint}} to {{s3.amazonaws.com}}
> If you have found this JIRA because you are encountering this problem, you
> can fix it in by explicitly declaring the endpoint in {{core-site.xml}}
> {code}
>
> fs.s3a.endpoint
> s3.amazonaws.com
>
> {code}
> For Apache Spark, this can be done in {{spark-defaults.conf}}
> {code}
> spark.hadoop.fs.s3a.endpoint s3.amazonaws.com
> {code}
> If you know the exact AWS region your data lives in, set the endpoint to be
> that region's endpoint, and so save an HTTPS request to s3.amazonaws.com
> every time an S3A Filesystem instance is created.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614474&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614474
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 12:39
Start Date: 24/Jun/21 12:39
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867504339
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 10s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/5/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 71m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3133 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux cd5626ccf857 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2cdceaa45ce4311a55d9af628c855b8dde314212 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/5/testReport/ |
| Max. process+thread count | 545 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614473&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614473
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 12:39
Start Date: 24/Jun/21 12:39
Worklog Time Spent: 10m
Work Description: mukund-thakur commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867603192
Test run in progress. I have commented the region field in ~/.aws/config
I see failures like
`[ERROR]
testCreateWithRenewer(org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens)
Time elapsed: 0.827 s <<< ERROR!
com.amazonaws.SdkClientException: Unable to find a region via the region
provider chain. Must provide an explicit region in the builder or setup
environment to supply a region.
at
com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:462)
at
com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
at
com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
at
org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding.maybeInitSTS(SessionTokenBinding.java:312)
at
org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding.prepareSTSClient(SessionTokenBinding.java:341)
at
org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding.createTokenIdentifier(SessionTokenBinding.java:362)
at
org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding.createTokenIdentifier(SessionTokenBinding.java:63)
at
org.apache.hadoop.fs.s3a.auth.delegation.AbstractDelegationTokenBinding.createDelegationToken(AbstractDelegationTokenBinding.java:142)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.lambda$createDelegationToken$0(S3ADelegationTokens.java:435)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:499)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:444)
at
org.apache.hadoop.fs.s3a.auth.delegation.S3ADelegationTokens.createDelegationToken(S3ADelegationTokens.java:433)
at
org.apache.hadoop.fs.s3a.auth.delegation.ITestSessionDelegationTokens.testCreateWithRenewer(ITestSessionDelegationTokens.java:230)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 614473)
Time Spent: 2.5h (was: 2h 20m)
> S3AFS creation fails "Unable to find a region via the region provider chain."
> -
>
> Key: HADOOP-17771
> URL: https://issues.apache.org/jira/browse/HADOOP-17771
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.3.1
> Environment: * fs.s3a.endpoint is unset
> * Host outside EC2
> * without the file ~/.aws/config or without a region set in it
> * without the system property aws.region declaring a region
> * without the environment variable AWS_REGION declaring a region.
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> If you don't have {{fs.s3a.endpoint}} set and lack a region set in
> env var {{AWS_REGION_ENV_VAR}}, system property {{aws.region}} or the file
> ~/.aws/config
> then S3A FS creation fails with the message
> "Unable to find a region via the region provider chain."
> This is caused by the move to the AWS S3 client builder API in HADOOP-13551
> This is pretty dramatic and no doubt everyone will be asking "why didn't you
> notice this?",
> But in fact there are some reasons.
> # when running in EC2, all is well. Meaning our big test runs were all happy.
> # if a developer has fs.s3a.endpoint set for the test bucket, all is well.
>Those of us who work with buckets in the "regions tend to do this, not
> least because it can save a HEAD request every time an FS is created.
> # if you have a region set in ~/.aws/config then all is well
> reason #3 is the real surprise and the one which has really caught us out.
> Even my tests against buckets in usw-2 through central didn't fail because of
> course I, like my colleagues, have the AWS cli client installed locally. This
> was sufficient to make the problem go away. It is also why this has been an
> intermittent
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614472&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614472
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 12:38
Start Date: 24/Jun/21 12:38
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867525084
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 35s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 71m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3133 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 08fb3df3a830 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6decb38ff0bce0d82ccc9357cd268a1f8767fc32 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/testReport/ |
| Max. process+thread count | 543 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoo
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614463&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614463
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 12:25
Start Date: 24/Jun/21 12:25
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867595206
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 43s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 5s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 71m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3133 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 7a0c55ffbf3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 993eea471208a0a65d7e3c12ca3c8813942d0883 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/7/testReport/ |
| Max. process+thread count | 656 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Power
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614437&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614437
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 24/Jun/21 10:27
Start Date: 24/Jun/21 10:27
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-867525084
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 29m 35s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 21s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 71m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3133 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 08fb3df3a830 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6decb38ff0bce0d82ccc9357cd268a1f8767fc32 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/6/testReport/ |
| Max. process+thread count | 543 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U:
[jira] [Work logged] (HADOOP-17771) S3AFS creation fails "Unable to find a region via the region provider chain."
[
https://issues.apache.org/jira/browse/HADOOP-17771?focusedWorklogId=614071&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614071
]
ASF GitHub Bot logged work on HADOOP-17771:
---
Author: ASF GitHub Bot
Created on: 23/Jun/21 15:30
Start Date: 23/Jun/21 15:30
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3133:
URL: https://github.com/apache/hadoop/pull/3133#issuecomment-866939056
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/2/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 0m 21s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 2 unchanged - 0 fixed
= 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 59s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 5s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 75m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3133/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3133 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 2e43df8d71e4 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 88a6c0f100b372b3eda775c2b23c0e0bb0a29cf5 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Bui
