Hi All,
We are trying to send the spark logs using fluent-bit. We validated that
fluent-bit is able to move logs of all other pods except the driver/executor
pods.
It would be great if someone can guide us where should I look for spark logs in
Spark on Kubernetes with client/cluster mode
Hello Everyone,
We are trying to stream the changes in our Iceberg tables stored in AWS S3. We
are achieving this through Spark-Iceberg Connector and using JAR files for
Spark-AWS. Suddenly we have started receiving error "Connection pool shut down".
Spark Version: 3.4.1
Iceberg: 1.3.1
Any
Hello Everyone,
We have setup spark and setup Iceberg-Glue connectors as mentioned at
https://iceberg.apache.org/docs/latest/aws/ to integrate Spark, Iceberg, and
AWS Glue Catalog. We are able to read tables through this but we are unable to
read data through views. PFB, the error:
only.
Thank,
Sanket
From: Nagatomi Yasukazu
Sent: Friday, September 8, 2023 9:35 AM
To: Agrawal, Sanket
Cc: Chao Sun ; Yeachan Park ;
user@spark.apache.org
Subject: [EXT] Re: Spark 3.4.1 and Hive 3.1.3
Hi Sanket,
While migrating to Hive 3.1.3 may resolve many issues, the link below suggests
Hi, I tried replacing just this JAR but getting errors.
From: Nagatomi Yasukazu
Sent: Friday, September 8, 2023 9:35 AM
To: Agrawal, Sanket
Cc: Chao Sun ; Yeachan Park ;
user@spark.apache.org
Subject: [EXT] Re: Spark 3.4.1 and Hive 3.1.3
Hi Sanket,
While migrating to Hive 3.1.3 may resolve
Hi Chao,
The reason to migrate to Hive 3.1.3 is to remove a vulnerability from
hive-exec-2.3.9.jar.
Thanks
Sanket
From: Chao Sun
Sent: Thursday, September 7, 2023 10:23 PM
To: Agrawal, Sanket
Cc: Yeachan Park ; user@spark.apache.org
Subject: [EXT] Re: Spark 3.4.1 and Hive 3.1.3
Hi Sanket
is
completed successfully.
Has anyone tried building Spark 3.4.1 with Hive 3.1.3 or higher?
Thanks,
Sanket A.
From: Yeachan Park
Sent: Tuesday, September 5, 2023 8:52 PM
To: Agrawal, Sanket
Cc: user@spark.apache.org
Subject: [EXT] Re: Spark 3.4.1 and Hive 3.1.3
What's the full traceback when
I don’t see an entry in pom.xml while building spark. I think it is being
downloaded as part of some other dependency.
From: Sean Owen
Sent: Thursday, August 31, 2023 5:10 PM
To: Agrawal, Sanket
Cc: user@spark.apache.org
Subject: [EXT] Re: Okio Vulnerability in Spark 3.4.1
Does
Hi All,
Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in Spark
3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we try this
version of jar then the spark application is failing with below error:
py4j.protocol.Py4JJavaError: An error occurred while calling
