Repository: spark Updated Branches: refs/heads/branch-2.2 e35c53a97 -> 2695b9213
[SPARK-22315][SPARKR] Warn if SparkR package version doesn't match SparkContext ## What changes were proposed in this pull request? This PR adds a check between the R package version used and the version reported by SparkContext running in the JVM. The goal here is to warn users when they have a R package downloaded from CRAN and are using that to connect to an existing Spark cluster. This is raised as a warning rather than an error as users might want to use patch versions interchangeably (e.g., 2.1.3 with 2.1.2 etc.) ## How was this patch tested? Manually by changing the `DESCRIPTION` file Author: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Closes #19624 from shivaram/sparkr-version-check. (cherry picked from commit 65a8bf6036fe41a53b4b1e4298fa35d7fa4e9970) Signed-off-by: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2695b921 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2695b921 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2695b921 Branch: refs/heads/branch-2.2 Commit: 2695b9213d590201cb9937736134e94a11f48ba2 Parents: e35c53a Author: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Authored: Mon Nov 6 08:58:42 2017 -0800 Committer: Shivaram Venkataraman <shiva...@cs.berkeley.edu> Committed: Mon Nov 6 08:58:55 2017 -0800 ---------------------------------------------------------------------- R/pkg/R/sparkR.R | 12 ++++++++++++ 1 file changed, 12 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/2695b921/R/pkg/R/sparkR.R ---------------------------------------------------------------------- diff --git a/R/pkg/R/sparkR.R b/R/pkg/R/sparkR.R index d0a12b7..9ebd344 100644 --- a/R/pkg/R/sparkR.R +++ b/R/pkg/R/sparkR.R @@ -420,6 +420,18 @@ sparkR.session <- function( enableHiveSupport) assign(".sparkRsession", sparkSession, envir = .sparkREnv) } + + # Check if version number of SparkSession matches version number of SparkR package + jvmVersion <- callJMethod(sparkSession, "version") + # Remove -SNAPSHOT from jvm versions + jvmVersionStrip <- gsub("-SNAPSHOT", "", jvmVersion) + rPackageVersion <- paste0(packageVersion("SparkR")) + + if (jvmVersionStrip != rPackageVersion) { + warning(paste("Version mismatch between Spark JVM and SparkR package. JVM version was", + jvmVersion, ", while R package version was", rPackageVersion)) + } + sparkSession } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
