Thanks Zahid, Yes I am using history server to see previous UIs. However, my question still remains on viewing old thread dumps, as I cannot see them on the old completed spark UIs, only when spark context is running.
On Wed, Apr 8, 2020 at 4:01 PM Zahid Rahman <zahidr1...@gmail.com> wrote: > Spark UI is only available while SparkContext is running. > > However You can get to the Spark UI after your application completes or > crashes. > > To do this Spark includes a tool called the Spark History Server that > allows you to reconstruct the Spark UI. > > You can find up to date information on how to use this tool in the spark > documentation https://spark.apache.org/docs/latest/monitoring.html > > > > > > On Wed, 8 Apr 2020, 23:47 Ruijing Li, <liruijin...@gmail.com> wrote: > >> Hi all, >> >> As stated in title, currently when I view the spark UI of a completed >> spark job, I see there are thread dump links in the executor tab, but >> clicking on them does nothing. Is it possible to see the thread dumps >> somehow even if the job finishes? On spark 2.4.5. >> >> Thanks. >> -- >> Cheers, >> Ruijing Li >> > -- Cheers, Ruijing Li
