sean-lynch commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2367259667
@clamar14's suggestion of
```
.config("spark.driver.extraJavaOptions", "-Daws.region=us-west-1")
```
was the only way I could find to provide a region as part of the
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2362808067
based on above details it working fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
AwasthiSomesh closed issue #10078: Spark configuration for amazon access key
and secret key with glue catalog for apache Iceberg is not honoring
URL: https://github.com/apache/iceberg/issues/10078
--
This is an automated message from the Apache Git Service.
To respond to the message, please
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2352335756
@andythsu @clamar14 no need extra jars and other configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2352331487
@andythsu @clamar14
**Please try with aws custom credential provider follow the below approach
if needed, i resolved this issue using below implementation**
***
clamar14 commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2345880274
thank you all, I finally solved it this way:
`
.config("spark.sql.catalog.AwsDataCatalog.s3.access-key-id", "xxx")
.config("spark.sql.catalog.AwsDataCatalog.s3.secret-a
HemantMarve commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2344456352
@clamar14 , @nastra Try after adding below property:
`.config("spark.sql.catalog.AwsDataCatalog.client.region","us-south")`
`.config("spark.sql.catalog.your_catalog
andythsu commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2338251099
@clamar14 @nastra I ended up using different jars and putting everything
together instead of using `iceberg-aws-bundle`
```
def get_builder() -> SparkSession.Builder:
nastra commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2293078478
@andythsu how does your Spark config look like?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abov
andythsu commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2292613192
facing the same issue as well. I'm on `iceberg-aws-bundle-1.6.0` but it
still complains region not set
--
This is an automated message from the Apache Git Service.
To respond to
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2041494668
@lxs360
do you have any solution for above query ?.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2041494046
@nastra I checked everything .. I hope spark configuration for amazon s3
configuration is not working for iceberg but
System.setProperty("aws.region", "us-west-2");
nastra commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2036229721
@AwasthiSomesh you might want to go through the link that I posted (and also
through the Glue docs on how to connect to Iceberg). The settings you need
could depend on your local AW
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2036225160
@lxs360 were you able resolve this issue ? I'm also facing the same issue
using same spark configuration?
I see you already reported this issue long back
-https://gith
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2036193452
@nastra Thanks a lot for your reply.
I am facing credentials load issue from spark .. could you please help how
to add access-key and secret key with region on spark
nastra commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2036175205
I see you have
```
.config("spark.sql.catalog.AwsDataCatalog","org.apache.iceberg.spark.SparkSessionCatalog")
.config("spark.sql.catalog.AwsDataCatalog",
"org.apache.icebe
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2035029132
@nastra Even I see we are getting error for region unable to load.
Error:-
Exception in thread "main"
software.amazon.awssdk.core.exception.SdkClientException: Una
AwasthiSomesh commented on issue #10078:
URL: https://github.com/apache/iceberg/issues/10078#issuecomment-2034985982
@nastra Thanks for your reply
After adding this we see the same error.
Could you please check is there any mistake from my end
.config("spark.sql.catalog.Aw
AwasthiSomesh opened a new issue, #10078:
URL: https://github.com/apache/iceberg/issues/10078
Hi Team ,
We are doing below code to access iceberg table from glue catalog and data
storage as S3
var spark = SparkSession.builder().master("local[*]")
.config("spark.sql.def
19 matches
Mail list logo
