Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 @jiangxb1987 - yes that is the issue and you are right, we can avoid it by checking if the stageId is valid when we get a task end event. But I like this approach better because we can clean up the hack which sets the `numRunningTasks` to 0 when stage ends and also it is inline with the way we are doing bookkeeping in ExecutorAllocationListener i.e, keeping entry per stage. Let me know what you think.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org