Hi community, this one should be an easy one: I have left spark.task.maxFailures to it's default (which should be 4). I see a job that shows the following statistics for Tasks: Succeeded/Total
7109/819 (1 failed) So there were 819 tasks to start with. I have 2 executors in that cluster. From Spark docs it says spark.task.maxFailures is the number of times to try a task before a job is given up. So I was imagining that 819*4 (i.e. 3276) would be the max number to ever see in the succeeded (accounting for retries on every possibly task). even that 3276*2 (6552, if it's per task per executor) does not account for 7109 successfull tasks. Could anyone help explain why I'm seeing such high number of succeeded tasks?
