GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/19165
[SPARKR][BACKPORT-2.1] backporting package and test changes ## What changes were proposed in this pull request? cherrypick or manually porting changes to 2.1 ## How was this patch tested? Jenkins You can merge this pull request into a Git repository by running: $ git pull https://github.com/felixcheung/spark rbackportpkg21 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/19165.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #19165 ---- commit b3062ebcf09b37f826449a6f88f3162cb772eea2 Author: Wayne Zhang <actuaryzh...@uber.com> Date: 2017-05-08T06:16:30Z cherrypick 2fdaeb52bbe2ed1a9127ac72917286e505303c85 commit 5e9483871ad34fa8d653b69a77f066bd3183bd47 Author: Felix Cheung <felixcheun...@hotmail.com> Date: 2017-05-08T06:10:18Z cherrypick c24bdaab5a234d18b273544cefc44cc4005bf8fc commit 15c289477fe8313920f3656734fdd0eba1cfb3dc Author: Felix Cheung <felixcheun...@hotmail.com> Date: 2017-09-08T07:48:23Z manual port of 0b0be47e7b742d96810c60b19a9aa920242e5224 commit 79c82c773cbd8b1dbb59f8329c621bdbe9e2c287 Author: Felix Cheung <felixcheun...@hotmail.com> Date: 2017-09-08T07:50:16Z port subset of 9f4ff9552470fb97ca38bb56bbf43be49a9a316c commit a74ba0276705e9ac84f56b57a558b2211cdf947b Author: hyukjinkwon <gurwls...@gmail.com> Date: 2017-06-18T18:26:27Z [SPARK-21128][R] Remove both "spark-warehouse" and "metastore_db" before listing files in R tests ## What changes were proposed in this pull request? This PR proposes to list the files in test _after_ removing both "spark-warehouse" and "metastore_db" so that the next run of R tests pass fine. This is sometimes a bit annoying. ## How was this patch tested? Manually running multiple times R tests via `./R/run-tests.sh`. **Before** Second run: ``` SparkSQL functions: Spark package found in SPARK_HOME: .../spark ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ....................................................................................................1234....................... Failed ------------------------------------------------------------------------- 1. Failure: No extra files are created in SPARK_HOME by starting session and making calls (test_sparkSQL.R#3384) length(list1) not equal to length(list2). 1/1 mismatches [1] 25 - 23 == 2 2. Failure: No extra files are created in SPARK_HOME by starting session and making calls (test_sparkSQL.R#3384) sort(list1, na.last = TRUE) not equal to sort(list2, na.last = TRUE). 10/25 mismatches x[16]: "metastore_db" y[16]: "pkg" x[17]: "pkg" y[17]: "R" x[18]: "R" y[18]: "README.md" x[19]: "README.md" y[19]: "run-tests.sh" x[20]: "run-tests.sh" y[20]: "SparkR_2.2.0.tar.gz" x[21]: "metastore_db" y[21]: "pkg" x[22]: "pkg" y[22]: "R" x[23]: "R" y[23]: "README.md" x[24]: "README.md" y[24]: "run-tests.sh" x[25]: "run-tests.sh" y[25]: "SparkR_2.2.0.tar.gz" 3. Failure: No extra files are created in SPARK_HOME by starting session and making calls (test_sparkSQL.R#3388) length(list1) not equal to length(list2). 1/1 mismatches [1] 25 - 23 == 2 4. Failure: No extra files are created in SPARK_HOME by starting session and making calls (test_sparkSQL.R#3388) sort(list1, na.last = TRUE) not equal to sort(list2, na.last = TRUE). 10/25 mismatches x[16]: "metastore_db" y[16]: "pkg" x[17]: "pkg" y[17]: "R" x[18]: "R" y[18]: "README.md" x[19]: "README.md" y[19]: "run-tests.sh" x[20]: "run-tests.sh" y[20]: "SparkR_2.2.0.tar.gz" x[21]: "metastore_db" y[21]: "pkg" x[22]: "pkg" y[22]: "R" x[23]: "R" y[23]: "README.md" x[24]: "README.md" y[24]: "run-tests.sh" x[25]: "run-tests.sh" y[25]: "SparkR_2.2.0.tar.gz" DONE =========================================================================== ``` **After** Second run: ``` SparkSQL functions: Spark package found in SPARK_HOME: .../spark ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................................................... ............................................................................................................................... ``` Author: hyukjinkwon <gurwls...@gmail.com> Closes #18335 from HyukjinKwon/SPARK-21128. commit 129a9798a98ddd86601c876fb83eeb5f5995d814 Author: Felix Cheung <felixcheun...@hotmail.com> Date: 2017-08-24T04:35:17Z [SPARK-21805][SPARKR] Disable R vignettes code on Windows ## What changes were proposed in this pull request? Code in vignettes requires winutils on windows to run, when publishing to CRAN or building from source, winutils might not be available, so it's better to disable code run (so resulting vigenttes will not have output from code, but text is still there and code is still there) fix * checking re-building of vignette outputs ... WARNING and > %LOCALAPPDATA% not found. Please define the environment variable or restart and enter an installation path in localDir. ## How was this patch tested? jenkins, appveyor, r-hub before: https://artifacts.r-hub.io/SparkR_2.2.0.tar.gz-49cecef3bb09db1db130db31604e0293/SparkR.Rcheck/00check.log after: https://artifacts.r-hub.io/SparkR_2.2.0.tar.gz-86a066c7576f46794930ad114e5cff7c/SparkR.Rcheck/00check.log Author: Felix Cheung <felixcheun...@hotmail.com> Closes #19016 from felixcheung/rvigwind. commit b53f812d729bb56929876bde9872cf1f2451e63a Author: Felix Cheung <felixcheun...@hotmail.com> Date: 2017-09-08T08:01:02Z not cran ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org