[GitHub] spark pull request #19624: [SPARKR][SPARK-22315] Warn if SparkR package vers...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/19624


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19624: [SPARKR][SPARK-22315] Warn if SparkR package vers...

[shivaram]

Github user shivaram commented on a diff in the pull request:

https://github.com/apache/spark/pull/19624#discussion_r148320092
  
--- Diff: R/pkg/R/sparkR.R ---
@@ -420,6 +420,18 @@ sparkR.session <- function(
 enableHiveSupport)
 assign(".sparkRsession", sparkSession, envir = .sparkREnv)
   }
+
+  # Check if version number of SparkSession matches version number of 
SparkR package
+  jvmVersion <- callJMethod(sparkSession, "version")
+  # Remove -SNAPSHOT from jvm versions
+  jvmVersion <- gsub("-SNAPSHOT", "", jvmVersion)
+  rPackageVersion <- paste0(packageVersion("SparkR"))
+
+  if (jvmVersion != rPackageVersion) {
+warning(paste("Version mismatch between Spark JVM and SparkR package. 
JVM version was",
+  jvmVersion, ", while R package version was", 
rPackageVersion))
--- End diff --

I wonder if that would be more confusing as somebody might think they just 
need to remove the `-snapshot` ?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19624: [SPARKR][SPARK-22315] Warn if SparkR package vers...

[felixcheung]

Github user felixcheung commented on a diff in the pull request:

https://github.com/apache/spark/pull/19624#discussion_r148192869
  
--- Diff: R/pkg/R/sparkR.R ---
@@ -420,6 +420,18 @@ sparkR.session <- function(
 enableHiveSupport)
 assign(".sparkRsession", sparkSession, envir = .sparkREnv)
   }
+
+  # Check if version number of SparkSession matches version number of 
SparkR package
+  jvmVersion <- callJMethod(sparkSession, "version")
+  # Remove -SNAPSHOT from jvm versions
+  jvmVersion <- gsub("-SNAPSHOT", "", jvmVersion)
+  rPackageVersion <- paste0(packageVersion("SparkR"))
+
+  if (jvmVersion != rPackageVersion) {
+warning(paste("Version mismatch between Spark JVM and SparkR package. 
JVM version was",
+  jvmVersion, ", while R package version was", 
rPackageVersion))
--- End diff --

not sure if it's important, perhaps print the original version (ie. with 
-SNAPSHOT) 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19624: [SPARKR][SPARK-22315] Warn is SparkR package vers...

[shivaram]

GitHub user shivaram opened a pull request:

https://github.com/apache/spark/pull/19624

[SPARKR][SPARK-22315] Warn is SparkR package version doesn't match 
SparkContext

## What changes were proposed in this pull request?

This PR adds a check between the R package version used and the version 
reported by SparkContext running in the JVM. The goal here is to warn users 
when they have a R package downloaded from CRAN and are using that to connect 
to an existing Spark cluster.

This is raised as a warning rather than an error as users might want to use 
patch versions interchangeably (e.g., 2.1.3 with 2.1.2 etc.)

## How was this patch tested?

Manually by changing the `DESCRIPTION` file

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shivaram/spark-1 sparkr-version-check

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19624.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19624


commit 6f6696762a8e779527339360c28f134da36940d6
Author: Shivaram Venkataraman 
Date:   2017-10-30T02:30:25Z

Warn if R pkg version doesn't match JVM version

commit 680a38ccd106145cd7dd72c10ed9f2b1b41b18f6
Author: Shivaram Venkataraman 
Date:   2017-10-31T20:11:46Z

Remove snapshot before comparing versions




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org