Hi ,
I have 2 questions -
1. I was trying to use Resource Manager UI for my SPARK application using
yarn cluster mode as I observed that spark UI does not work for
Yarn-cluster.
IS that correct or am I missing some setup?
Sorry , for half email - here it is again in full
Hi ,
I have 2 questions -
1. I was trying to use Resource Manager UI for my SPARK application using
yarn cluster mode as I observed that spark UI does not work for
Yarn-cluster.
IS that correct or am I missing some setup?
2. when I click on
bq. spark UI does not work for Yarn-cluster.
Can you be a bit more specific on the error(s) you saw ?
What Spark release are you using ?
Cheers
On Tue, Mar 3, 2015 at 8:53 AM, Rohini joshi roni.epi...@gmail.com wrote:
Sorry , for half email - here it is again in full
Hi ,
I have 2
bq. changing the address with internal to the external one , but still does
not work.
Not sure what happened.
For the time being, you can use yarn command line to pull container log
(put in your appId and container Id):
yarn logs -applicationId application_1386639398517_0007 -containerId
Hi Ted,
I used s3://support.elasticmapreduce/spark/install-spark to install spark
on my EMR cluster. It is 1.2.0.
When I click on the link for history or logs it takes me to
http://ip-172-31-43-116.us-west-2.compute.internal:9035/node/containerlogs/container_1424105590052_0070_01_01/hadoop
Ted,
If the application is running then the logs are not available. Plus what i
want to view is the details about the running app as in spark UI.
Do I have to open some ports or do some other setting changes?
On Tue, Mar 3, 2015 at 10:08 AM, Ted Yu yuzhih...@gmail.com wrote:
bq. changing the
In Yarn (Cluster or client), you can access the spark ui when the app is
running. After app is done, you can still access it, but need some extra setup
for history server.
Thanks.
Zhan Zhang
On Mar 3, 2015, at 10:08 AM, Ted Yu
yuzhih...@gmail.commailto:yuzhih...@gmail.com wrote:
bq.
ah!! I think I know what you mean. My job was just in accepted stage for
a long time as it was running a huge file.
But now that it is in running stage , I can see it . I can see it at post
9046 though instead of 4040 . But I can see it.
Thanks
-roni
On Tue, Mar 3, 2015 at 1:19 PM, Zhan Zhang
