This is an automated email from the ASF dual-hosted git repository.
FelixYBW pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 198e347083 [Velox] Add S3 limitation (#12713)
198e347083 is described below
commit 198e347083d78b26d527eacd13651bc3a3f9961b
Author: BInwei Yang <[email protected]>
AuthorDate: Thu Aug 6 01:01:47 2026 -0700
[Velox] Add S3 limitation (#12713)
* Document known limitations for S3 access and SDK
Added known limitations regarding cross-region S3 access and AWS C++ SDK
behavior.
* Remove known limitation section from VeloxS3.md
Removed known limitation section from VeloxS3 documentation.
---
docs/get-started/VeloxS3.md | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/docs/get-started/VeloxS3.md b/docs/get-started/VeloxS3.md
index d61ddc4dd9..237000da6e 100644
--- a/docs/get-started/VeloxS3.md
+++ b/docs/get-started/VeloxS3.md
@@ -71,6 +71,23 @@ These settings apply to all buckets by default. To override
a single bucket, use
Gluten's static backend pass-through prefix, for example
`spark.gluten.velox.hive.s3.bucket.my-bucket.part-upload-async`.
+# Known Limitation: Cross-Region S3 Access
+
+If your S3 data and your configured endpoint are in different AWS regions —
for example, your bucket is in `us-west-2` but you have set the endpoint to
`us-east-1` (or left it unconfigured, causing requests to default to
`us-east-1`) — you will encounter a runtime error.
+
+Unlike the Java S3 SDK used by `s3a://` (which often handles 301 cross-region
redirects automatically), the AWS C++ SDK (`aws-sdk-cpp`) does **not**
automatically follow 301 `PermanentRedirect` responses for S3 payload requests.
It treats the redirect as a non-retriable error:
+
+```
+Error Type: SDK Error (100)
+Error: PermanentRedirect — The bucket you are attempting to access must be
addressed using the specified endpoint.
+```
+
+**Resolution:** Set the endpoint explicitly to the region where your bucket
resides, for example:
+
+```properties
+spark.hadoop.fs.s3a.endpoint=s3.us-west-2.amazonaws.com
+```
+
# Local Caching support
Velox supports a local cache when reading data from S3 but not strictly tested
and there are several limitations. Please refer [Velox Local
Cache](VeloxLocalCache.md) part for more detailed configurations.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
