Re: Spark master driver UI: How to keep it after process finished?
Hi Saif, You need to run your application with `spark.eventLog.enabled` set to true. Then if you are using standalone mode, you can view the Master UI at port 8080. Otherwise, you may start a history server through `sbin/start-history-server.sh`, which by default starts the history UI at port 18080. For more information on how to set this up, visit: http://spark.apache.org/docs/latest/monitoring.html -Andrew 2015-08-07 13:16 GMT-07:00 François Pelletier newslett...@francoispelletier.org: look at spark.history.ui.port, if you use standalone spark.yarn.historyServer.address, if you use YARN in your Spark config file Mine is located at /etc/spark/conf/spark-defaults.conf If you use Apache Ambari you can find this settings in the Spark / Configs / Advanced spark-defaults tab François Le 2015-08-07 15:58, saif.a.ell...@wellsfargo.com a écrit : Hello, thank you, but that port is unreachable for me. Can you please share where can I find that port equivalent in my environment? Thank you Saif *From:* François Pelletier [mailto:newslett...@francoispelletier.org newslett...@francoispelletier.org] *Sent:* Friday, August 07, 2015 4:38 PM *To:* user@spark.apache.org *Subject:* Re: Spark master driver UI: How to keep it after process finished? Hi, all spark processes are saved in the Spark History Server look at your host on port 18080 instead of 4040 François Le 2015-08-07 15:26, saif.a.ell...@wellsfargo.com a écrit : Hi, A silly question here. The Driver Web UI dies when the spark-submit program finish. I would like some time to analyze after the program ends, as the page does not refresh it self, when I hit F5 I lose all the info. Thanks, Saif
Spark master driver UI: How to keep it after process finished?
Hi, A silly question here. The Driver Web UI dies when the spark-submit program finish. I would like some time to analyze after the program ends, as the page does not refresh it self, when I hit F5 I lose all the info. Thanks, Saif
Re: Spark master driver UI: How to keep it after process finished?
Hi, all spark processes are saved in the Spark History Server look at your host on port 18080 instead of 4040 François Le 2015-08-07 15:26, saif.a.ell...@wellsfargo.com a écrit : Hi, A silly question here. The Driver Web UI dies when the spark-submit program finish. I would like some time to analyze after the program ends, as the page does not refresh it self, when I hit F5 I lose all the info. Thanks, Saif
RE: Spark master driver UI: How to keep it after process finished?
Hello, thank you, but that port is unreachable for me. Can you please share where can I find that port equivalent in my environment? Thank you Saif From: François Pelletier [mailto:newslett...@francoispelletier.org] Sent: Friday, August 07, 2015 4:38 PM To: user@spark.apache.org Subject: Re: Spark master driver UI: How to keep it after process finished? Hi, all spark processes are saved in the Spark History Server look at your host on port 18080 instead of 4040 François Le 2015-08-07 15:26, saif.a.ell...@wellsfargo.commailto:saif.a.ell...@wellsfargo.com a écrit : Hi, A silly question here. The Driver Web UI dies when the spark-submit program finish. I would like some time to analyze after the program ends, as the page does not refresh it self, when I hit F5 I lose all the info. Thanks, Saif
RE: Spark master driver UI: How to keep it after process finished?
Verzonden vanaf mijn Sony Xperia™-smartphone
saif.a.ell...@wellsfargo.com schreef
!-- /* Font Definitions */ @font-face {font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal,
li.MsoNormal, div.MsoNormal{margin:0in;margin-bottom:.0001pt;
font-size:12.0pt; font-family:Times New Roman,serif; color:black;}
a:link, span.MsoHyperlink {mso-style-priority:99; color:blue;
text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99; color:purple; text-decoration:underline;}
p.emailquote, li.emailquote, div.emailquote
{mso-style-name:emailquote; mso-margin-top-alt:auto;
margin-right:0in; mso-margin-bottom-alt:auto; margin-left:1.0pt;
border:none;padding:0in;font-size:12.0pt; font-family:Times
New Roman,serif; color:black;} span.EmailStyle18
{mso-style-type:personal-reply; font-family:Calibri,sans-serif;
color:#1F497D;} .MsoChpDefault {mso-style-type:export-only;
font-size:10.0pt;} @page WordSection1 {size:8.5in 11.0in; margin:1.0in
1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} --
Hello, thank you, but that port is unreachable for me. Can you please share
where can I find that port equivalent in my environment?
Thank you
Saif
From: François Pelletier [mailto:newslett...@francoispelletier.org]
Sent: Friday, August 07, 2015 4:38 PM
To: user@spark.apache.org
Subject: Re: Spark master driver UI: How to keep it after process finished?
Hi, all spark processes are saved in the Spark History Server
look at your host on port 18080 instead of 4040
François
Le 2015-08-07 15:26, saif.a.ell...@wellsfargo.com a écrit :
Hi,
A silly question here. The Driver Web UI dies when the spark-submit program
finish. I would like some time to analyze after the program ends, as the page
does not refresh it self, when I hit F5 I lose all the info.
Thanks,
Saif
Re: Spark master driver UI: How to keep it after process finished?
look at spark.history.ui.port, if you use standalone spark.yarn.historyServer.address, if you use YARN in your Spark config file Mine is located at /etc/spark/conf/spark-defaults.conf If you use Apache Ambari you can find this settings in the Spark / Configs / Advanced spark-defaults tab François Le 2015-08-07 15:58, saif.a.ell...@wellsfargo.com a écrit : Hello, thank you, but that port is unreachable for me. Can you please share where can I find that port equivalent in my environment? Thank you Saif *From:*François Pelletier [mailto:newslett...@francoispelletier.org] *Sent:* Friday, August 07, 2015 4:38 PM *To:* user@spark.apache.org *Subject:* Re: Spark master driver UI: How to keep it after process finished? Hi, all spark processes are saved in the Spark History Server look at your host on port 18080 instead of 4040 François Le 2015-08-07 15:26, saif.a.ell...@wellsfargo.com mailto:saif.a.ell...@wellsfargo.com a écrit : Hi, A silly question here. The Driver Web UI dies when the spark-submit program finish. I would like some time to analyze after the program ends, as the page does not refresh it self, when I hit F5 I lose all the info. Thanks, Saif
