bogthe commented on a change in pull request #3260: URL: https://github.com/apache/hadoop/pull/3260#discussion_r717605122
########## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md ########## @@ -1580,6 +1580,68 @@ Why explicitly declare a bucket bound to the central endpoint? It ensures that if the default endpoint is changed to a new region, data store in US-east is still reachable. +## <a name="accesspoints"></a>Configuring S3 AccessPoints usage with S3A +S3a now supports [S3 Access Point](https://aws.amazon.com/s3/features/access-points/) usage which +improves VPC integration with S3 and simplifies your data's permission model because different +policies can be applied now on the Access Point level. For more information about why to use and +how to create them make sure to read the official documentation. + +Accessing data through an access point, is done by using its ARN, as opposed to just the bucket name. +You can set the Access Point ARN property using the following per bucket configuration property: +```xml +<property> + <name>fs.s3a.sample-bucket.accesspoint.arn</name> + <value> {ACCESSPOINT_ARN_HERE} </value> + <description>Configure S3a traffic to use this AccessPoint</description> +</property> +``` + +This configures access to the `sample-bucket` bucket for S3A, to go through the +new Access Point ARN. So, for example `s3a://sample-bucket/key` will now use your +configured ARN when getting data from S3 instead of your bucket. + +You can also use an Access Point name as a path URI such as `s3a://finance-team-access/key`, by +configuring the `.accesspoint.arn` property as a per-bucket override: +```xml +<property> + <name>fs.s3a.finance-team-access.accesspoint.arn</name> + <value> {ACCESSPOINT_ARN_HERE} </value> + <description>Configure S3a traffic to use this AccessPoint</description> +</property> +``` + +The `fs.s3a.accesspoint.required` property can also require all access to S3 to go through Access +Points. This has the advantage of increasing security inside a VPN / VPC as you only allow access +to known sources of data defined through Access Points. In case there is a need to access a bucket +directly (without Access Points) then you can use per bucket overrides to disable this setting on a +bucket by bucket basis i.e. `fs.s3a.{YOUR-BUCKET}.accesspoint.required`. + +```xml +<!-- Require access point only access --> +<property> + <name>fs.s3a.accesspoint.required</name> + <value>true</value> +</property> +<!-- Disable it on a per-bucket basis if needed --> +<property> + <name>fs.s3a.example-bucket.accesspoint.required</name> + <value>false</value> +</property> +``` + +Before using Access Points make sure you're not impacted by the following: +- `ListObjectsV1` is not supported, this is also deprecated on AWS S3 for performance reasons; +- The endpoint for S3 requests will automatically change from `s3.amazonaws.com` to use +`s3-accesspoint.REGION.amazonaws.{com | com.cn}` depending on the Access Point ARN. This **only** +happens if the `fs.s3a.endpoint` property isn't set. The endpoint property overwrites any changes, Review comment: nope, removing good catch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
