Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21060 > The callback works for collect in R and Scala but Python doesn't. I think we should at least match the behaviour. I wonder why it's hard to say a bug when collect is detected in some APIs but not in some APIs. The behavior consistency among Python/Scala/R/JAVA does not mean a bug, right? > That's because the change was big and invasive. I wouldn't backport it too; however, this fix is relatively small. This is the reason why we did not backport that PR. We still can backport the minimal changes to the previous releases. > I think we usually use committer's judgement when we make an exception. I already have been seeing many backports that actually causes behaviour changes and I did this because it looks being backported in general. This is the reason why we should formally document it if this is actually the rule. I am not against this specific PR. All the committers need to be really careful when they make a decision to backport a behavior change. If any committer does it, we should jump in and stop the backport. This is what we should do.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org