Repository: spark Updated Branches: refs/heads/master 77cc0d67d -> 4cc704b12
[CORE][MINOR] Improve the error message of checkpoint RDD verification ### What changes were proposed in this pull request? The original error message is pretty confusing. It is unable to tell which number is `number of partitions` and which one is the `RDD ID`. This PR is to improve the checkpoint checking. ### How was this patch tested? N/A Author: gatorsmile <gatorsm...@gmail.com> Closes #18796 from gatorsmile/improveErrMsgForCheckpoint. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4cc704b1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4cc704b1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/4cc704b1 Branch: refs/heads/master Commit: 4cc704b12194cfa70717fb35d90b8b2ccda29866 Parents: 77cc0d6 Author: gatorsmile <gatorsm...@gmail.com> Authored: Tue Aug 1 17:10:46 2017 -0700 Committer: Shixiong Zhu <shixi...@databricks.com> Committed: Tue Aug 1 17:10:46 2017 -0700 ---------------------------------------------------------------------- .../scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/4cc704b1/core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala b/core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala index 37c67ce..979152b 100644 --- a/core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala +++ b/core/src/main/scala/org/apache/spark/rdd/ReliableCheckpointRDD.scala @@ -152,8 +152,10 @@ private[spark] object ReliableCheckpointRDD extends Logging { sc, checkpointDirPath.toString, originalRDD.partitioner) if (newRDD.partitions.length != originalRDD.partitions.length) { throw new SparkException( - s"Checkpoint RDD $newRDD(${newRDD.partitions.length}) has different " + - s"number of partitions from original RDD $originalRDD(${originalRDD.partitions.length})") + "Checkpoint RDD has a different number of partitions from original RDD. Original " + + s"RDD [ID: ${originalRDD.id}, num of partitions: ${originalRDD.partitions.length}]; " + + s"Checkpoint RDD [ID: ${newRDD.id}, num of partitions: " + + s"${newRDD.partitions.length}].") } newRDD } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
