[
https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Toth updated SPARK-50356:
-------------------------------
Priority: Minor (was: Major)
> Skipped stages are inconsitent on UI
> ------------------------------------
>
> Key: SPARK-50356
> URL: https://issues.apache.org/jira/browse/SPARK-50356
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, UI
> Affects Versions: 4.0.0
> Reporter: Peter Toth
> Priority: Minor
> Attachments: Screenshot 2024-11-19 at 16.06.48.png, Screenshot
> 2024-11-19 at 16.07.06.png, Screenshot 2024-11-19 at 16.07.20.png
>
>
> The following snipett produces inconsitent UI state:
> {noformat}
> import scala.collection.mutable
> import scala.concurrent.{ExecutionContext, Future, Await}
> import scala.concurrent.duration.Duration
> val rdd = sc.parallelize(1 to 1000, 100).repartition(10)
> rdd.count
> val futures = mutable.ListBuffer.empty[Future[Unit]]
> implicit val xc =
> ExecutionContext.fromExecutorService(java.util.concurrent.Executors.newFixedThreadPool(5))
> futures += Future { rdd.count }
> futures += Future { rdd.count }
> futures += Future { rdd.count }
> Await.result(Future.sequence(futures), Duration.Inf)
> {noformat}
> The jobs page doesn't show the skipped tasks oj job 2 and job 3 properly:
> !Screenshot 2024-11-19 at 16.06.48.png!
> The job 1 page looks good:
> !Screenshot 2024-11-19 at 16.07.06.png!
> But job 2 and 3 pages are inconsistent in DAG and skipped stages list:
> !Screenshot 2024-11-19 at 16.07.20.png!
> The issue affects only the UI, the result of the jobs are correct.
> Without parallel submission of jobs the UI is consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
