Repository: spark Updated Branches: refs/heads/master f5b7714e0 -> 7a702d8d5
[SPARK-21616][SPARKR][DOCS] update R migration guide and vignettes ## What changes were proposed in this pull request? update R migration guide and vignettes ## How was this patch tested? manually Author: Felix Cheung <felixcheun...@hotmail.com> Closes #20106 from felixcheung/rreleasenote23. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7a702d8d Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7a702d8d Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7a702d8d Branch: refs/heads/master Commit: 7a702d8d5ed830de5d2237f136b08bd18deae037 Parents: f5b7714 Author: Felix Cheung <felixcheun...@hotmail.com> Authored: Tue Jan 2 07:00:31 2018 +0900 Committer: hyukjinkwon <gurwls...@gmail.com> Committed: Tue Jan 2 07:00:31 2018 +0900 ---------------------------------------------------------------------- R/pkg/tests/fulltests/test_Windows.R | 1 + R/pkg/vignettes/sparkr-vignettes.Rmd | 3 +-- docs/sparkr.md | 6 ++++++ 3 files changed, 8 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/R/pkg/tests/fulltests/test_Windows.R ---------------------------------------------------------------------- diff --git a/R/pkg/tests/fulltests/test_Windows.R b/R/pkg/tests/fulltests/test_Windows.R index b2ec6c6..209827d 100644 --- a/R/pkg/tests/fulltests/test_Windows.R +++ b/R/pkg/tests/fulltests/test_Windows.R @@ -14,6 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. # + context("Windows-specific tests") test_that("sparkJars tag in SparkContext", { http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/R/pkg/vignettes/sparkr-vignettes.Rmd ---------------------------------------------------------------------- diff --git a/R/pkg/vignettes/sparkr-vignettes.Rmd b/R/pkg/vignettes/sparkr-vignettes.Rmd index 8c4ea2f..2e66242 100644 --- a/R/pkg/vignettes/sparkr-vignettes.Rmd +++ b/R/pkg/vignettes/sparkr-vignettes.Rmd @@ -391,8 +391,7 @@ We convert `mpg` to `kmpg` (kilometers per gallon). `carsSubDF` is a `SparkDataF ```{r} carsSubDF <- select(carsDF, "model", "mpg") -schema <- structType(structField("model", "string"), structField("mpg", "double"), - structField("kmpg", "double")) +schema <- "model STRING, mpg DOUBLE, kmpg DOUBLE" out <- dapply(carsSubDF, function(x) { x <- cbind(x, x$mpg * 1.61) }, schema) head(collect(out)) ``` http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/docs/sparkr.md ---------------------------------------------------------------------- diff --git a/docs/sparkr.md b/docs/sparkr.md index a3254e7..997ea60 100644 --- a/docs/sparkr.md +++ b/docs/sparkr.md @@ -657,3 +657,9 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - By default, derby.log is now saved to `tempdir()`. This will be created when instantiating the SparkSession with `enableHiveSupport` set to `TRUE`. - `spark.lda` was not setting the optimizer correctly. It has been corrected. - Several model summary outputs are updated to have `coefficients` as `matrix`. This includes `spark.logit`, `spark.kmeans`, `spark.glm`. Model summary outputs for `spark.gaussianMixture` have added log-likelihood as `loglik`. + +## Upgrading to SparkR 2.3.0 + + - The `stringsAsFactors` parameter was previously ignored with `collect`, for example, in `collect(createDataFrame(iris), stringsAsFactors = TRUE))`. It has been corrected. + - For `summary`, option for statistics to compute has been added. Its output is changed from that from `describe`. + - A warning can be raised if versions of SparkR package and the Spark JVM do not match. --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org