petartushev commented on issue #11997:
URL: https://github.com/apache/iceberg/issues/11997#issuecomment-2676269191
I resolved the issue. I cannot quite explain the first issue
`software.amazon.awssdk.services.s3.model.S3Exception: The bucket you are
attempting to access must be addressed using the specified endpoint. Please
send all future requests to this endpoint.`. I solved this one when I added
```
.config("spark.sql.catalog.rest.s3.cross-region-access-enabled", "true")
```
After that, I got the issue
`software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id
you provided does not exist in our records.`. This one I got because my `spark`
application was actually making API calls to `aws` proper, not my `minio`
service locally. This was resolved once I added the `minio` endpoint, and
credentials to the catalog configuration, as I had to my `hadoop` configuration:
```
.config("spark.sql.catalog.rest.s3.endpoint", Variables.MINIO_API_ENDPOINT)
.config("spark.sql.catalog.rest.s3.access-key-id", AWS_ACCESS_KEY_ID)
.config("spark.sql.catalog.rest.s3.secret-access-key", AWS_SECRET_ACCESS_KEY)
.config("spark.sql.catalog.rest.s3.region", AWS_REGION)
```
As for why I could create and delete a iceberg table, and update the schema
of an existing iceberg table, but not insert, I don't know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
