kevinjqliu commented on issue #515: URL: https://github.com/apache/iceberg-python/issues/515#issuecomment-1999853333
If it works in PySpark, it's probably not the Glue configuration but in pyiceberg. Can you double-check the AWS settings? Your AWS profile looks like it can access the Glue catalog and read its content. Does it have permission to read the underlying s3 file? Secondly, ``` OSError: When reading information for key 'path/to/s3/table/location/metadata/100000-458c8ffc-de06-4eb5-bc4a-b94c3034a548.metadata.json' in bucket 's3_bucket_name': AWS Error UNKNOWN (HTTP status 400) during HeadObject operation: No response body. ``` That S3 path looks fishy to me. Esp the prefix `path/to/s3/table/location/metadata/` and no `s3://`. We can also check if PyArrow FS is parsing the `metadata_location` correctly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
