geruh commented on issue #515:
URL: https://github.com/apache/iceberg-python/issues/515#issuecomment-2029207168
Your glue calls look, fine but your S3 calls are the problem. I was able to
reproduce the issue by having the incorrect region for my AWS profile at
`./aws/config` and passing in the region config upon initializing the catalog.
aws_config
```
[test]
region = us-east-1
```
catalog init
```
catalog = pyiceberg.catalog.load_catalog(
catalog_name, **{"type": "glue", "profile_name": "test", "region_name":
"us-west-2"}
)
```
Which leads to this exception
```
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
OSError: When reading information for key
'test/metadata/00000-c0fc4e45-d79d-41a1-ba92-a4122c09171c.metadata.json' in
bucket 'test_bucket': AWS Error UNKNOWN (HTTP status 301) during HeadObject
operation: No response body.
```
It looks like when we infer the correct FileIO the aws profile region config
takes precedence over the catalogs region config. For now you should be able to
work around this by ensuring the profiles region is in sync with the config
passed into the catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
