Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21758 I mean rddC's partitions are derived from rddA and rddB, here assuming partitions in rddA requires barrier, but not required in rddB. So rddC's partitions are the half barrier and half not barrier. So how do you merge such conflict, did you mark rddC's all partitions as barrier, or only the ones coming from rddA? Obviously partitions from rddB doesn't require barrier (just normal tasks), forcing to barrier will require strict resource demand.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org