ah!! I think I know what you mean. My job was just in "accepted" stage for a long time as it was running a huge file. But now that it is in running stage , I can see it . I can see it at post 9046 though instead of 4040 . But I can see it. Thanks -roni
On Tue, Mar 3, 2015 at 1:19 PM, Zhan Zhang <zzh...@hortonworks.com> wrote: > In Yarn (Cluster or client), you can access the spark ui when the app is > running. After app is done, you can still access it, but need some extra > setup for history server. > > Thanks. > > Zhan Zhang > > On Mar 3, 2015, at 10:08 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > bq. changing the address with internal to the external one , but still > does not work. > Not sure what happened. > For the time being, you can use yarn command line to pull container log > (put in your appId and container Id): > yarn logs -applicationId application_1386639398517_0007 -containerId > container_1386639398517_0007_01_000019 > > Cheers > > On Tue, Mar 3, 2015 at 9:50 AM, roni <roni.epi...@gmail.com> wrote: > >> Hi Ted, >> I used s3://support.elasticmapreduce/spark/install-spark to install >> spark on my EMR cluster. It is 1.2.0. >> When I click on the link for history or logs it takes me to >> >> http://ip-172-31-43-116.us-west-2.compute.internal:9035/node/containerlogs/container_1424105590052_0070_01_000001/hadoop >> and I get - >> >> The server at *ip-172-31-43-116.us >> <http://ip-172-31-43-116.us>-west-2.compute.internal* can't be found, >> because the DNS lookup failed. DNS is the network service that translates a >> website's name to its Internet address. This error is most often caused by >> having no connection to the Internet or a misconfigured network. It can >> also be caused by an unresponsive DNS server or a firewall preventing Google >> Chrome from accessing the network. >> I tried changing the address with internal to the external one , but >> still does not work. >> Thanks >> _roni >> >> >> On Tue, Mar 3, 2015 at 9:05 AM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> bq. spark UI does not work for Yarn-cluster. >>> >>> Can you be a bit more specific on the error(s) you saw ? >>> >>> What Spark release are you using ? >>> >>> Cheers >>> >>> On Tue, Mar 3, 2015 at 8:53 AM, Rohini joshi <roni.epi...@gmail.com> >>> wrote: >>> >>>> Sorry , for half email - here it is again in full >>>> Hi , >>>> I have 2 questions - >>>> >>>> 1. I was trying to use Resource Manager UI for my SPARK application >>>> using yarn cluster mode as I observed that spark UI does not work for >>>> Yarn-cluster. >>>> IS that correct or am I missing some setup? >>>> >>>> 2. when I click on Application Monitoring or history , i get >>>> re-directed to some linked with internal Ip address. Even if I replace that >>>> address with the public IP , it still does not work. What kind of setup >>>> changes are needed for that? >>>> >>>> Thanks >>>> -roni >>>> >>>> On Tue, Mar 3, 2015 at 8:45 AM, Rohini joshi <roni.epi...@gmail.com> >>>> wrote: >>>> >>>>> Hi , >>>>> I have 2 questions - >>>>> >>>>> 1. I was trying to use Resource Manager UI for my SPARK application >>>>> using yarn cluster mode as I observed that spark UI does not work for >>>>> Yarn-cluster. >>>>> IS that correct or am I missing some setup? >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>> >> > >
