[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334511#comment-17334511 ]
Kousuke Saruta commented on SPARK-35229: ---------------------------------------- [~tiehexue]Thanks for the report. I'll try to mitigate this issue by limiting the maximum number of the jobs/executors. The maximum number of the tasks are already limited by spark.ui.timeline.tasks.maximum so I'll try with the similar approach. > Spark Job web page is extremely slow while there are more than 1500 events in > timeline > -------------------------------------------------------------------------------------- > > Key: SPARK-35229 > URL: https://issues.apache.org/jira/browse/SPARK-35229 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 3.1.1 > Reporter: Wang Yuan > Priority: Blocker > > In a spark streaming application, there are 1000+ executors, more than 2000 > events (executors events, job events) generated, then the jobs/job web page > of spark is not able to show. The browser (chrome, firefox, safari) freezes. > I had to open another window to open stages, executors pages by their > addresses manually. > The jobs page is the home page, so this is rather annoyed. The problem is > that the vis-timeline rending is too slow. > Their are some suggestion: > 1) the page should not render timeline when the page is loading unless user > clicks the link. > 2) the executor group and job group should be separated, and user can choose > to show one, both or neither. > 3) the executor group should display executor event by time horizontally. > Currently it is displayed executors one line by one line. If more than 100 > executors, the page is not that good. > 4)the vis-timeline library is not maintained anymore since 2017. Should be > replaced a new one like [https://github.com/visjs/vis-timeline] > 5) it is also good to get recent events, e.g. 500, and load more if user want > see more, however the data could all be loaded once. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org