HARIKRISHNAN CHENEPERTH KUNHUMVEETTIL created SPARK-21013:
-------------------------------------------------------------
Summary: Spark History Server does not show the logs of completed
Yarn Jobs
Key: SPARK-21013
URL: https://issues.apache.org/jira/browse/SPARK-21013
Project: Spark
Issue Type: Bug
Components: Web UI
Affects Versions: 2.1.0, 2.0.1, 1.6.1
Reporter: HARIKRISHNAN CHENEPERTH KUNHUMVEETTIL
I am facing issue when accessing the container logs of a completed Spark (Yarn)
application from the History Server.
Repro Steps:
1) Run the spark-shell in yarn client mode. Or run Pi job in Yarn mode.
2) Once the job is completed, (in the case of spark shell, exit after doing
some simple operations), try to access the STDOUT or STDERR logs of the
application from the Executors tab in the Spark History Server UI.
3) If yarn log aggregation is enabled, then logs won't be available in node
manager's log location. But history Server is trying to access the logs from
the nodemanager's log location giving below error in the UI:
Failed redirect for container_e31_1496881617682_0003_01_000002
ResourceManager
RM Home
NodeManager
Tools
Failed while trying to construct the redirect url to the log server. Log Server
url may not be configured
java.lang.Exception: Unknown container. Container either has not started or has
already completed or doesn't belong to this node at all.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
