Yes, I tried it, but it simply does not work. so, my concern is *to use "ssh tunnel" to forward a port of cluster to localhost port. *
But in Spark UI, there are two ports which I should forward using "*ssh tunnel*". Considering a default port, 8080 is web-ui port to come into web-ui, and 4040 is web-monitoring port to see time execution like DAG in application details UI, as you probably know. But after finishing a job, I can see the list of a job on the web-ui on 8080, but when I click "application details UI" on port 4040 to see time excution, it does not work. Any suggestion? I really need to see execution of DAG. Best, Phil On Tue, Jan 26, 2016 at 12:04 AM, Mohammed Guller <moham...@glassbeam.com> wrote: > I am not sure whether you can copy the log files from Spark workers to > your local machine and view it from the Web UI. In fact, if you are able to > copy the log files locally, you can just view them directly in any text > editor. > > > > I suspect what you really want to see is the application history. Here is > the relevant information from Spark’s monitoring page ( > http://spark.apache.org/docs/latest/monitoring.html) > > > > To view the web UI after the fact, set spark.eventLog.enabled to true > before starting the application. This configures Spark to log Spark events > that encode the information displayed in the UI to persisted storage. > > > > Mohammed > > Author: Big Data Analytics with Spark > <http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> > > > > *From:* Philip Lee [mailto:philjj...@gmail.com] > *Sent:* Monday, January 25, 2016 9:51 AM > *To:* user@spark.apache.org > *Subject:* Re: a question about web ui log > > > > As I mentioned before, I am tryint to see the spark log on a cluster via > ssh-tunnel > > > > 1) The error on application details UI is probably from monitoring porting > 4044. Web UI port is 8088, right? so how could I see job web ui view and > application details UI view in the web ui on my local machine? > > > > 2) still wondering how to see the log after copyting log file to my local. > > > > The error was metioned in previous mail. > > > > Thanks, > > Phil > > > > > > > > On Mon, Jan 25, 2016 at 5:36 PM, Philip Lee <philjj...@gmail.com> wrote: > > Hello, a questino about web UI log. > > > > I could see web interface log after forwarding the port on my cluster to > my local and click completed application, but when I clicked "application > detail UI" > > > > [image: Inline image 1] > > > > It happened to me. I do not know why. I also checked the specific log > folder. It has a log file in it. Actually, that's why I could click the > completed application link, right? > > > > So is it okay for me to copy the log file in my cluster to my local > machine. > > And after turning on spark Job Manger on my local by myself, I could see > application deatils UI in my local machine? > > > > Best, > > Phil > > >
