This is an automated email from the ASF dual-hosted git repository.
fokko pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-python.git
The following commit(s) were added to refs/heads/main by this push:
new 627a5eb7 Documentation `s3.profile-name` and `client.profile-name`
(#3085)
627a5eb7 is described below
commit 627a5eb72764dcb16e263d50b0414df3da4851e9
Author: Thomas Pfeiffer <[email protected]>
AuthorDate: Tue Mar 3 07:16:56 2026 +0100
Documentation `s3.profile-name` and `client.profile-name` (#3085)
Closes #3066
# Rationale for this change
This PR addes a small remark in the documentation to `s3.profile-name`
and `client.profile-name`, since they are currently only supported when
using `FsspecFileIO`. Support for AWS profile for `PyArrowFileIO` is not
yet implemented, see #3065
## Are these changes tested?
n/a, purely changes to documentation.
## Are there any user-facing changes?
Yes, minor change log adjustment needed compared to #2948.
- `client.profile-name`: Sets the AWS profile for both the catalog
client and FileIO (unified configuration). (only supported by
`FsspecFileIO` currently)
- `s3.profile-name`: Sets the AWS profile specifically for S3 FileIO.
(only supported by `FsspecFileIO` currently)
<!-- In the case of user-facing changes, please add the changelog label.
-->
---------
Co-authored-by: Thomas Pfeiffer (ext.)
<[email protected]>
---
mkdocs/docs/configuration.md | 54 ++++++++++++++++++++++----------------------
1 file changed, 27 insertions(+), 27 deletions(-)
diff --git a/mkdocs/docs/configuration.md b/mkdocs/docs/configuration.md
index 391cca78..3f82f788 100644
--- a/mkdocs/docs/configuration.md
+++ b/mkdocs/docs/configuration.md
@@ -110,25 +110,25 @@ For the FileIO there are several configuration options
available:
<!-- markdown-link-check-disable -->
| Key | Example | Description
|
-|-----------------------------|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| s3.endpoint | <https://10.0.19.25/> | Configure an
alternative endpoint of the S3 service for the FileIO to access. This could be
used to use S3FileIO with any s3-compatible object storage service that has a
different endpoint, or access a private S3 endpoint in a virtual private cloud.
|
-| s3.access-key-id | admin | Configure the
static access key id used to access the FileIO.
|
-| s3.secret-access-key | password | Configure the
static secret access key used to access the FileIO.
|
-| s3.session-token | AQoDYXdzEJr... | Configure the
static session token used to access the FileIO.
|
-| s3.profile-name | default | Configure the AWS
profile used to access the S3 FileIO.
|
-| s3.role-session-name | session | An optional
identifier for the assumed role session.
|
-| s3.role-arn | arn:aws:... | AWS Role ARN. If
provided instead of access_key and secret_key, temporary credentials will be
fetched by assuming this role.
|
-| s3.signer | bearer | Configure the
signature version of the FileIO.
|
-| s3.signer.uri | <http://my.signer:8080/s3> | Configure the
remote signing uri if it differs from the catalog uri. Remote signing is only
implemented for `FsspecFileIO`. The final request is sent to
`<s3.signer.uri>/<s3.signer.endpoint>`.
|
-| s3.signer.endpoint | v1/main/s3-sign | Configure the
remote signing endpoint. Remote signing is only implemented for `FsspecFileIO`.
The final request is sent to `<s3.signer.uri>/<s3.signer.endpoint>`. (default :
v1/aws/s3/sign). |
-| s3.region | us-west-2 | Configure the
default region used to initialize an `S3FileSystem`. `PyArrowFileIO` attempts
to automatically tries to resolve the region if this isn't set (only supported
for AWS S3 Buckets).
|
-| s3.resolve-region | False | Only supported
for `PyArrowFileIO`, when enabled, it will always try to resolve the location
of the bucket (only supported for AWS S3 Buckets).
|
-| s3.proxy-uri | <http://my.proxy.com:8080> | Configure the
proxy server to be used by the FileIO.
|
-| s3.connect-timeout | 60.0 | Configure socket
connection timeout, in seconds.
|
-| s3.request-timeout | 60.0 | Configure socket
read timeouts on Windows and macOS, in seconds.
|
+|-----------------------------|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| s3.endpoint | <https://10.0.19.25/> | Configure an
alternative endpoint of the S3 service for the FileIO to access. This could be
used to use S3FileIO with any s3-compatible object storage service that has a
different endpoint, or access a private S3 endpoint in a virtual private cloud.
|
+| s3.access-key-id | admin | Configure the
static access key id used to access the FileIO.
|
+| s3.secret-access-key | password | Configure the
static secret access key used to access the FileIO.
|
+| s3.session-token | AQoDYXdzEJr... | Configure the
static session token used to access the FileIO.
|
+| s3.profile-name | default | Configure the AWS
profile used to access the S3 FileIO (only supported by `FsspecFileIO`
currently).
|
+| s3.role-session-name | session | An optional
identifier for the assumed role session.
|
+| s3.role-arn | arn:aws:... | AWS Role ARN. If
provided instead of access_key and secret_key, temporary credentials will be
fetched by assuming this role.
|
+| s3.signer | bearer | Configure the
signature version of the FileIO.
|
+| s3.signer.uri | <http://my.signer:8080/s3> | Configure the
remote signing uri if it differs from the catalog uri. Remote signing is only
implemented for `FsspecFileIO`. The final request is sent to
`<s3.signer.uri>/<s3.signer.endpoint>`.
|
+| s3.signer.endpoint | v1/main/s3-sign | Configure the
remote signing endpoint. Remote signing is only implemented for `FsspecFileIO`.
The final request is sent to `<s3.signer.uri>/<s3.signer.endpoint>`. (default :
v1/aws/s3/sign).
|
+| s3.region | us-west-2 | Configure the
default region used to initialize an `S3FileSystem`. `PyArrowFileIO` attempts
to automatically tries to resolve the region if this isn't set (only supported
for AWS S3 Buckets).
|
+| s3.resolve-region | False | Only supported
for `PyArrowFileIO`, when enabled, it will always try to resolve the location
of the bucket (only supported for AWS S3 Buckets).
|
+| s3.proxy-uri | <http://my.proxy.com:8080> | Configure the
proxy server to be used by the FileIO.
|
+| s3.connect-timeout | 60.0 | Configure socket
connection timeout, in seconds.
|
+| s3.request-timeout | 60.0 | Configure socket
read timeouts on Windows and macOS, in seconds.
|
| s3.force-virtual-addressing | False | Whether to use
virtual addressing of buckets. If true, then virtual addressing is always
enabled. If false, then virtual addressing is only enabled if endpoint_override
is empty. This can be used for non-AWS backends that only support virtual
hosted-style access. |
-| s3.retry-strategy-impl | None | Ability to set a
custom S3 retry strategy. A full path to a class needs to be given that extends
the
[S3RetryStrategy](https://github.com/apache/arrow/blob/639201bfa412db26ce45e73851432018af6c945e/python/pyarrow/_s3fs.pyx#L110)
base class. |
-| s3.anonymous | True | Configure whether
to use anonymous connection. If False (default), uses key/secret if configured
or boto's credential resolver. |
+| s3.retry-strategy-impl | None | Ability to set a
custom S3 retry strategy. A full path to a class needs to be given that extends
the
[S3RetryStrategy](https://github.com/apache/arrow/blob/639201bfa412db26ce45e73851432018af6c945e/python/pyarrow/_s3fs.pyx#L110)
base class. |
+| s3.anonymous | True | Configure whether
to use anonymous connection. If False (default), uses key/secret if configured
or boto's credential resolver.
|
<!-- markdown-link-check-enable-->
@@ -839,15 +839,15 @@ catalog:
configures the AWS credentials for both Glue Catalog and S3 FileIO.
-| Key | Example | Description
|
-| ------------------------ | -------------- |
------------------------------------------------------------------------------------------------------
|
-| client.region | us-east-1 | Set the region of both the
Glue/DynamoDB Catalog and the S3 FileIO |
-| client.access-key-id | admin | Configure the static access key
id used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
-| client.secret-access-key | password | Configure the static secret
access key used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
-| client.session-token | AQoDYXdzEJr... | Configure the static session
token used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
-| client.profile-name | default | Configure the AWS profile used
to access both the Glue/DynamoDB Catalog and the S3 FileIO |
-| client.role-session-name | session | An optional
identifier for the assumed role session.
|
-| client.role-arn | arn:aws:... | AWS Role ARN. If
provided instead of access_key and secret_key, temporary credentials will be
fetched by assuming this role.
|
+| Key | Example | Description
|
+| ------------------------ | -------------- |
--------------------------------------------------------------------------------------------------------------------------------------
|
+| client.region | us-east-1 | Set the region of both the
Glue/DynamoDB Catalog and the S3 FileIO
|
+| client.access-key-id | admin | Configure the static access key
id used to access both the Glue/DynamoDB Catalog and the S3 FileIO
|
+| client.secret-access-key | password | Configure the static secret
access key used to access both the Glue/DynamoDB Catalog and the S3 FileIO
|
+| client.session-token | AQoDYXdzEJr... | Configure the static session
token used to access both the Glue/DynamoDB Catalog and the S3 FileIO
|
+| client.profile-name | default | Configure the AWS profile used
to access both the Glue/DynamoDB Catalog and the S3 FileIO (only supported by
`FsspecFileIO` currently) |
+| client.role-session-name | session | An optional identifier for the
assumed role session.
|
+| client.role-arn | arn:aws:... | AWS Role ARN. If provided
instead of access_key and secret_key, temporary credentials will be fetched by
assuming this role. |
<!-- prettier-ignore-start -->
