[ https://issues.apache.org/jira/browse/SPARK-23121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332249#comment-16332249 ]
Sandor Murakozi commented on SPARK-23121: ----------------------------------------- [~guoxiaolongzte] found two separate problems, bot triggered by having a high number of jobs/stages. In such a situation the store of the history server drops various objects to save memory. It may happen that the job itself is in the store, but its stages or the RDDOperationGraph are not. In such cases rendering of the all jobs and the job pages fails. As a consequence, the jobs page may become inaccessible if the cluster processes many jobs, so I think the priority of this issue should be increased. What do you think [~srowen] ? > When the Spark Streaming app is running for a period of time, the page is > incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13' > and ui can not be accessed. > --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-23121 > URL: https://issues.apache.org/jira/browse/SPARK-23121 > Project: Spark > Issue Type: Bug > Components: Web UI > Affects Versions: 2.4.0 > Reporter: guoxiaolongzte > Priority: Major > Attachments: 1.png, 2.png > > > When the Spark Streaming app is running for a period of time, the page is > incorrectly reported when accessing '/ jobs /' or '/ jobs / job /? Id = 13' > and ui can not be accessed. > > Test command: > ./bin/spark-submit --class org.apache.spark.examples.streaming.HdfsWordCount > ./examples/jars/spark-examples_2.11-2.4.0-SNAPSHOT.jar /spark > > The app is running for a period of time, ui can not be accessed, please see > attachment. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org