Hi Prashant,
Yes I understand that will be available from Spark 3.0.2 . But I am using
Spark3.0.1
Could you please suggest any way to mitigate this exception?
Will it cause any issues from this exception in the spark application?
Thanks
Sachit
On Mon, 1 Feb 2021, 13:57 Prashant Sharma,
Did you guys find a way to retrieve schema while saving into external
database? I'm also stuck at the same place without any clear path forward.
Thanks,
Rahul
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
Hi Arpan,
log in as any user that has execution right for spark. type spark-shell, do
some simple commands then exit. go to home directory of that user and look
for that hidden file
${HOME/.spark_history
it will be there.
HTH,
LinkedIn *
Application wise it wont show as such.
You can try to corelate it with explain plain output using some filters or
attribute.
Or else if you do not have too much queries in history. Just take queries
and find plan of those queries and match it with shown in UI.
I know thats the tedious task. But
Sachit,
That is showing all the queries that got executed, but how it would get
mapped to specific application Id it was associated with ?
Thanks,
Arpan Bhandari
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
Hi arpan,
In spark shell when you type
:history.
then also it is not showing?
Thanks
Sachit
On Mon, 1 Feb 2021, 21:13 Arpan Bhandari, wrote:
> Hey Sachit,
>
> It shows the query plan, which is difficult to diagnose out and depict the
> actual query.
>
>
> Thanks,
> Arpan Bhandari
>
>
>
> --
>
Sean,
Thank you for the quick response. So just to clarify, if there is a
vulnerability or when Oracle releases their quarterly updates, there should be
no issue with applying the patch/update for Java for Spark, correct?
Thank you,
Aaron Mathews
Application Products Center of Excellence
There's no Java for / in Spark, but, yes I can't see a scenario where a
minor JVM update would break Spark.
Not 100% impossible - I'm thinking of obscure timezone definition updates,
changes to supported crypto algorithms, etc.
But that should be extremely rare.
I think we'd always recommend the
Hey Mich,
Thanks for the suggestions, but i don't see any such folder created on the
edge node.
Thanks,
Arpan Bhandari
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail:
Hey Sachit,
It shows the query plan, which is difficult to diagnose out and depict the
actual query.
Thanks,
Arpan Bhandari
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
-
To unsubscribe e-mail:
The Spark distro does not include Java. That has to be present in the
environment where the Spark cluster is run.
It works with Java 8, and 11 in 3.x (Oracle and OpenJDK AFAIK). It seems to
99% work on 14+ even.
On Mon, Feb 1, 2021 at 9:11 AM
wrote:
> Hello,
>
>
>
> I am looking for information
Hello,
I am looking for information Oracle Java and Spark.
Is Java embedded with Spark? If it is, what steps need to be taken to update
Java to Oracle's latest release?
Thank you,
Aaron Mathews
Application Products Center of Excellence (APCoE) - Systems Operations Engineer
CTO Platform
Hi Sachit,
The fix verison on that JIRA says 3.0.2, so this fix is not yet released.
Soon, there will be a 3.1.1 release, in the meantime you can try out the
3.1.1-rc which also has the fix and let us know your findings.
Thanks,
On Mon, Feb 1, 2021 at 10:24 AM Sachit Murarka
wrote:
>
13 matches
Mail list logo
