alamb opened a new issue, #402: URL: https://github.com/apache/arrow-rs-object-store/issues/402
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** - related to https://github.com/apache/datafusion/issues/16306 I would like to be able to retrieve data stored in S3 without having to explicitly specify a region (see https://github.com/apache/datafusion/issues/16306 for the DataFusion usecase) Here is an example program. ```rust use object_store::aws::AmazonS3Builder; use object_store::{ObjectStore}; use object_store::path::Path; #[tokio::main] async fn main() { // Goal is to read a parquet file from S3 without having to specify the region // s3://clickhouse-public-datasets/hits_compatible/athena_partitioned/hits_1.parquet let path = "hits_compatible/athena_partitioned/hits_1.parquet"; let store = AmazonS3Builder::new() .with_bucket_name("clickhouse-public-datasets") .with_skip_signature(true) // NOTE the region is not specified. If it is set to `eu-central-1` // this example works great, but if it is not set, it fails with: // ``` // Error getting object from S3: Generic S3 error: Error performing in 120.898541ms - // Received redirect without LOCATION, this normally indicates an incorrectly configured region // ``` //.with_region("eu-central-1") .build() .expect("Failed to create S3 object store"); let path = Path::from(path); let result = match store. get(&path) .await { Ok(result) => result, Err(e) => { eprintln!("Error getting object from S3: {e}"); return; } }; println!("Successfully retrieved object from S3"); println!("Result meta: {:#?} ", result.meta); } ``` **Describe the solution you'd like** I want the request to succeed without having to explicitly specify the region. **Describe alternatives you've considered** Using `curl` you can see the response from AWS actually includes the correct region in the `x-amz-bucket-region` header > < x-amz-bucket-region: eu-central-1 ```shell curl -v https://s3.us-east-1.amazonaws.com/clickhouse-public-datasets/hits_compatible/athena_partitioned/hits_1.parquet ... * using HTTP/1.x > GET /clickhouse-public-datasets/hits_compatible/athena_partitioned/hits_1.parquet HTTP/1.1 > Host: s3.us-east-1.amazonaws.com > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off < HTTP/1.1 301 Moved Permanently < x-amz-bucket-region: eu-central-1 < x-amz-request-id: SG3WAD43HD3NADYW < x-amz-id-2: qKq7zFq+trkB04Ie5CrBIZmYBMS5Lk4gVYH6Pq5NrO1rC+pmTPG8bxelOM3y3vRcweQiS1pBDwU= < Content-Type: application/xml < Transfer-Encoding: chunked < Date: Fri, 06 Jun 2025 14:35:38 GMT < Server: AmazonS3 < <?xml version="1.0" encoding="UTF-8"?> * Connection #0 to host s3.us-east-1.amazonaws.com left intact <Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><Endpoint>clickhouse-public-datasets.s3.eu-central-1.amazonaws.com</Endpoint><Bucket>clickhouse-public-datasets</Bucket><RequestId>SG3WAD43HD3NADYW</RequestId><HostId>qKq7zFq+trkB04Ie5CrBIZmYBMS5Lk4gVYH6Pq5NrO1rC+pmTPG8bxelOM3y3vRcweQiS1pBDwU=</HostId></Error> **Additional context** <!-- Add any other context or screenshots about the feature request here. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
