Ngone51 commented on PR #43954: URL: https://github.com/apache/spark/pull/43954#issuecomment-1851530021
Damn! Barrier stage seems to be a special case. It called `killAllTaskAttempts()` to kill all the other tasks when there was a task failure but didn't abort the stage as it would have a retry later. In this PR, we replace `killAllTaskAttempts()` with `cancelTasks()` and enables stage abortion by default within `cancelTasks()`. This leads to the barrier stage failure instead of retry. It would only work for all the cases if we're confident enough to remove stage abortion from `cancelTasks()` thoroughly without any control flag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
