[
https://issues.apache.org/jira/browse/SPARK-11147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961959#comment-14961959
]
Steve Loughran commented on SPARK-11147:
----------------------------------------
looks like the RM proxy is having the connection problems. What's happening is
that for security reasons, the spark UI 302's all HTTP requests from any host
other than the resource manager, to the RM proxy. It then proxies the HTTP GET
request
# have a look in the RM logs under {{org.apache.hadoop.yarn.server.webproxy}}
# spark app side, turn on debug for
{{org.apache.hadoop.yarn.server.webproxy.amfilter}} and below
# ssh in to the RM, and see if you can curl down the UI from there -as it
should bypass the redirection mechaniums
> HTTP 500 if try to access Spark UI in yarn-cluster
> --------------------------------------------------
>
> Key: SPARK-11147
> URL: https://issues.apache.org/jira/browse/SPARK-11147
> Project: Spark
> Issue Type: Bug
> Components: Web UI, YARN
> Affects Versions: 1.5.1
> Environment: HDP: 2.3.2.0-2950 (Hadoop 2.7.1.2.3.2.0-2950)
> Spark: 1.5.x (c27e1904)
> Reporter: Sebastian YEPES FERNANDEZ
>
> Hello,
> I am facing a similar issue as described in SPARK-5837, but is my case the
> SparkUI only work in "yarn-client" mode. If a run the same job using
> "yarn-cluster" I get the HTTP 500 error:
> {code}
> HTTP ERROR 500
> Problem accessing /proxy/application_1444297190346_0085/. Reason:
> Connection to http://XX.XX.XX.XX:55827 refused
> Caused by:
> org.apache.http.conn.HttpHostConnectException: Connection to
> http://XX.XX.XX.XX:55827 refused
> at
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
> at
> org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
> at
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
> at
> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> {code}
> I have verified that the UI port "55827" is actually Listening on the worker
> node, I can even run a "curl http://XX.XX.XX.XX:55827"; and it redirects me to
> another URL: http://YY.YY.YY.YY:8088/proxy/application_1444297190346_0082
> The strange thing is the its redirecting me to the app "_0082" and not the
> actually running job "_0085"
> Does anyone have any suggestions on what could be causing this issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
