Hi Mich, I don't think so. There is support for a UI page refresh but I haven't seen it in use.
See StreamingPage [1] where it schedules refresh every 5 secs, i.e. Some(5000). In SparkUIUtils.headerSparkPage [2] there is refreshInterval but it's not used in any place in Spark. Time to fill an JIRA issue? What about REST API and httpie updating regularly [3]? Perhaps Metrics with ConsoleSink [4]? [1] https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala#L158 [2] https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/ui/UIUtils.scala#L202 [3] http://spark.apache.org/docs/latest/monitoring.html#rest-api [4] http://spark.apache.org/docs/latest/monitoring.html#metrics Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Thu, Aug 25, 2016 at 11:55 AM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi, > > This may be already there. > > A spark job opens up a UI on port specified by --conf "spark.ui.port=${SP}" > that defaults to 4040. > > However, on UI one needs to refresh the page to see the progress. > > Can this be polled so it is refreshed automatically > > Thanks > > > Dr Mich Talebzadeh > > > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > http://talebzadehmich.wordpress.com > > > Disclaimer: Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
