Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16620
@markhamstra I prefer that approach to the approach in the existing PR, but
I still have some hesitation about that because of the inconsistencies between
the TaskSetManager (which still thinks tasks are running) and DAGScheduler
(which thinks the stage is done), as mentioned in my comment above. It sounds
like everyone else prefers that approach though -- perhaps we can at least add
some better commenting so future readers of the code know the DAGSched and TSM
will have different views of the world and that listeners may get duplicate
stage completed messages as a result?
Another argument for your approach is that it's *no worse* than the current
code, and is the smallest change (I think) that can fix the bug. We can fix
the larger issues in a separate PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
