hussein-awala commented on code in PR #1709:
URL: https://github.com/apache/iceberg-python/pull/1709#discussion_r1966816889
##########
pyiceberg/io/pyarrow.py:
##########
@@ -414,19 +421,19 @@ def _initialize_oss_fs(self) -> FileSystem:
return S3FileSystem(**client_kwargs)
def _initialize_s3_fs(self, netloc: Optional[str]) -> FileSystem:
- from pyarrow.fs import S3FileSystem, resolve_s3_region
+ from pyarrow.fs import S3FileSystem
# Resolve region from netloc(bucket), fallback to user-provided region
provided_region = get_first_property_value(self.properties, S3_REGION,
AWS_REGION)
try:
- bucket_region = resolve_s3_region(bucket=netloc)
+ bucket_region = _cached_resolve_s3_region(bucket=netloc)
except (OSError, TypeError):
bucket_region = None
logger.warning(f"Unable to resolve region for bucket {netloc},
using default region {provided_region}")
Review Comment:
How about moving try/except to `_cached_resolve_s3_region` and caching even
the None value? Does the result change when we retry to resolve the same
netloc? Or do we need to warn the user every time it fails to resolve the
netloc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
