Following is the related JIRA , Can someone pls check
https://issues.apache.org/jira/browse/SPARK-24266
I am using 3.0.1 , It says fixed in 3.0.0 and 3.1.0 . Could you please
suggest what can be done to avoid this?
Kind Regards,
Sachit Murarka
On Sun, Jan 31, 2021 at 6:38 PM Sachit Murarka
Hi Arpan,
I presume you are interested in what client was doing.
If you have access to the edge node (where spark code is submitted), look
for the following file
${HOME/.spark_history
example
-rw-r--r--. 1 hduser hadoop 111997 Jun 2 2018 .spark_history
just use shell tools (cat, grep etc)
Hi Users,
I am running Spark application on Kubernetes and getting the following
exception in the driver pod. Though it is not affecting the output.
This exception is coming every 5 minutes and this is a structured streaming
job.
Could anyone please advise ?
21/01/29 06:33:15 WARN
Hi Arpan,
Launch spark shell and in the shell type ":history" , you will see the
query executed.
In the Spark UI under SQL Tab you can see the query plan when you click on
the details button(Though it won't show you the complete query). But by
looking at the plan you can get your query.
Hope
