[ https://issues.apache.org/jira/browse/SPARK-25504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-25504: ------------------------------------ Assignee: (was: Apache Spark) > [Spark Job History]If the number of tasks exceeds spark.ui.retainedTask > value, then stage page of UI should display upto the value set in the > retainedTask. > ----------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-25504 > URL: https://issues.apache.org/jira/browse/SPARK-25504 > Project: Spark > Issue Type: Documentation > Components: Web UI > Affects Versions: 2.3.1 > Reporter: ABHISHEK KUMAR GUPTA > Priority: Minor > Attachments: AppStatusListener.png > > > *Steps* > 1. spark.ui.retainedTask =60000( spark-default.conf of Job History) > 2. Run a job for creating 100 task as below from Spark-shell > val rdd = sc.parallelize(1 to 49000, 49000) > rdd.count > 3. Launch Job History Page > 4. Click on corresponding spark-shell Application ID link in Job History and > Launch Job Page for the application > 5. Click on the Job ID description and Launch Stage Page > 6. Click on the Description of Stage ID to Launch Stage Page of specific > Stage ID > *Atcual Result* > Showing Total number of task above Pagination Panel as *Tasks(49000)* > It is correct as number of task is 49000 created. > Now Execute as below > 1. Run a job for creating 100010 task as below from Spark-shell > val rdd = sc.parallelize(1 to 100100, 100100)) > rdd.count > 2. Launch Job History Page > 3. Click on corresponding spark-shell Application ID link in Job History and > Launch Job Page for the application > 4. Click on the Job ID description and Launch Stage Page > 5. Click on the Description of Stage ID to Launch Stage Page of specific > Stage ID > *Atcual Result* > Showing Total number of task above Pagination Panel as *Tasks(58100, showing > 100100)* > *Expected:* > spark.ui.retainedTask =60000 > So it should show as *Tasks( 60000, Showing 100100)* as > spark.ui.retainedTask =60000 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org