Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r213397293 --- Diff: core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala --- @@ -32,12 +32,16 @@ import org.apache.spark.{Partition, TaskContext} * doesn't modify the keys. * @param isFromBarrier Indicates whether this RDD is transformed from an RDDBarrier, a stage * containing at least one RDDBarrier shall be turned into a barrier stage. + * @param orderSensitiveFunc whether or not the function is order-sensitive. If it's order + * sensitive, it may return totally different result if the input order + * changed. Mostly stateful functions are order-sensitive. */ private[spark] class MapPartitionsRDD[U: ClassTag, T: ClassTag]( var prev: RDD[T], f: (TaskContext, Int, Iterator[T]) => Iterator[U], // (TaskContext, partition index, iterator) preservesPartitioning: Boolean = false, - isFromBarrier: Boolean = false) + isFromBarrier: Boolean = false, + orderSensitiveFunc: Boolean = false) --- End diff -- `orderSensitiveFunc` -> `isOrderSensitive`
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org