spark git commit: Copy pyspark and SparkR packages to latest release dir too

Repository: spark Updated Branches: refs/heads/branch-2.1 e8f351f9a -> 2c88e1dc3 Copy pyspark and SparkR packages to latest release dir too ## What changes were proposed in this pull request? Copy pyspark and SparkR packages to latest release dir, as per comment

spark git commit: Copy pyspark and SparkR packages to latest release dir too

Repository: spark Updated Branches: refs/heads/master 934035ae7 -> c074c96dc Copy pyspark and SparkR packages to latest release dir too ## What changes were proposed in this pull request? Copy pyspark and SparkR packages to latest release dir, as per comment

spark git commit: Copy the SparkR source package with LFTP

Repository: spark Updated Branches: refs/heads/branch-2.1 4ceed95b4 -> e8f351f9a Copy the SparkR source package with LFTP This PR adds a line in release-build.sh to copy the SparkR source archive using LFTP Author: Shivaram Venkataraman Closes #16226 from

spark git commit: Copy the SparkR source package with LFTP

Repository: spark Updated Branches: refs/heads/master 9338aa4f8 -> 934035ae7 Copy the SparkR source package with LFTP This PR adds a line in release-build.sh to copy the SparkR source archive using LFTP Author: Shivaram Venkataraman Closes #16226 from

spark git commit: [SPARK-18697][BUILD] Upgrade sbt plugins

Repository: spark Updated Branches: refs/heads/master 86a96034c -> 9338aa4f8 [SPARK-18697][BUILD] Upgrade sbt plugins ## What changes were proposed in this pull request? This PR is to upgrade sbt plugins. The following sbt plugins will be upgraded: ``` sbteclipse-plugin: 4.0.0 -> 5.0.1

spark git commit: [SPARK-18349][SPARKR] Update R API documentation on ml model summary

Repository: spark Updated Branches: refs/heads/branch-2.1 ef5646b4c -> 4ceed95b4 [SPARK-18349][SPARKR] Update R API documentation on ml model summary ## What changes were proposed in this pull request? In this PR, the document of `summary` method is improved in the format: returns summary

spark git commit: [SPARK-18349][SPARKR] Update R API documentation on ml model summary

Repository: spark Updated Branches: refs/heads/master 4ac8b20bf -> 86a96034c [SPARK-18349][SPARKR] Update R API documentation on ml model summary ## What changes were proposed in this pull request? In this PR, the document of `summary` method is improved in the format: returns summary

spark git commit: [SPARKR][PYSPARK] Fix R source package name to match Spark version. Remove pip tar.gz from distribution

Repository: spark Updated Branches: refs/heads/branch-2.1 1cafc76ea -> ef5646b4c [SPARKR][PYSPARK] Fix R source package name to match Spark version. Remove pip tar.gz from distribution ## What changes were proposed in this pull request? Fixes name of R source package so that the `cp` in

spark git commit: [SPARKR][PYSPARK] Fix R source package name to match Spark version. Remove pip tar.gz from distribution

Repository: spark Updated Branches: refs/heads/master 458fa3325 -> 4ac8b20bf [SPARKR][PYSPARK] Fix R source package name to match Spark version. Remove pip tar.gz from distribution ## What changes were proposed in this pull request? Fixes name of R source package so that the `cp` in

spark git commit: [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled (branch 2.1)

Repository: spark Updated Branches: refs/heads/branch-2.1 fcd22e538 -> 1cafc76ea [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled (branch 2.1) ## What changes were proposed in this pull request? Backport #16203 to branch 2.1. ## How was this patch

spark git commit: [SPARK-18776][SS] Make Offset for FileStreamSource corrected formatted in json

Repository: spark Updated Branches: refs/heads/master 202fcd21c -> 458fa3325 [SPARK-18776][SS] Make Offset for FileStreamSource corrected formatted in json ## What changes were proposed in this pull request? - Changed FileStreamSource to use new FileStreamSourceOffset rather than

spark git commit: [SPARK-18776][SS] Make Offset for FileStreamSource corrected formatted in json

Repository: spark Updated Branches: refs/heads/branch-2.1 e43209fe2 -> fcd22e538 [SPARK-18776][SS] Make Offset for FileStreamSource corrected formatted in json ## What changes were proposed in this pull request? - Changed FileStreamSource to use new FileStreamSourceOffset rather than

spark git commit: [SPARK-18590][SPARKR] Change the R source build to Hadoop 2.6

Repository: spark Updated Branches: refs/heads/master 3261e25da -> 202fcd21c [SPARK-18590][SPARKR] Change the R source build to Hadoop 2.6 This PR changes the SparkR source release tarball to be built using the Hadoop 2.6 profile. Previously it was using the without hadoop profile which

spark git commit: [SPARK-18590][SPARKR] Change the R source build to Hadoop 2.6

Repository: spark Updated Branches: refs/heads/branch-2.1 9483242f4 -> e43209fe2 [SPARK-18590][SPARKR] Change the R source build to Hadoop 2.6 This PR changes the SparkR source release tarball to be built using the Hadoop 2.6 profile. Previously it was using the without hadoop profile which

spark git commit: [SPARK-18760][SQL] Consistent format specification for FileFormats

Repository: spark Updated Branches: refs/heads/branch-2.1 a03564418 -> 9483242f4 [SPARK-18760][SQL] Consistent format specification for FileFormats ## What changes were proposed in this pull request? This patch fixes the format specification in explain for file sources (Parquet and Text

spark git commit: [SPARK-18751][CORE] Fix deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

Repository: spark Updated Branches: refs/heads/master c3d3a9d0e -> 26432df9c [SPARK-18751][CORE] Fix deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext ## What changes were proposed in this pull request? When `SparkContext.stop` is called in

spark git commit: [SPARK-18751][CORE] Fix deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext

Repository: spark Updated Branches: refs/heads/branch-2.1 d69df9073 -> a03564418 [SPARK-18751][CORE] Fix deadlock when SparkContext.stop is called in Utils.tryOrStopSparkContext ## What changes were proposed in this pull request? When `SparkContext.stop` is called in

spark git commit: [SPARK-18590][SPARKR] build R source package when making distribution

Repository: spark Updated Branches: refs/heads/branch-2.1 e0173f14e -> d69df9073 [SPARK-18590][SPARKR] build R source package when making distribution This PR has 2 key changes. One, we are building source package (aka bundle package) for SparkR which could be released on CRAN. Two, we

spark git commit: [SPARK-18590][SPARKR] build R source package when making distribution

Repository: spark Updated Branches: refs/heads/master 3c68944b2 -> c3d3a9d0e [SPARK-18590][SPARKR] build R source package when making distribution ## What changes were proposed in this pull request? This PR has 2 key changes. One, we are building source package (aka bundle package) for

spark git commit: [SPARK-16589] [PYTHON] Chained cartesian produces incorrect number of records

Repository: spark Updated Branches: refs/heads/branch-2.1 726217eb7 -> e0173f14e [SPARK-16589] [PYTHON] Chained cartesian produces incorrect number of records ## What changes were proposed in this pull request? Fixes a bug in the python implementation of rdd cartesian product related to

spark git commit: [SPARK-16589] [PYTHON] Chained cartesian produces incorrect number of records

Repository: spark Updated Branches: refs/heads/master ed8869ebb -> 3c68944b2 [SPARK-16589] [PYTHON] Chained cartesian produces incorrect number of records ## What changes were proposed in this pull request? Fixes a bug in the python implementation of rdd cartesian product related to

spark git commit: [SPARK-8617][WEBUI] HistoryServer: Include in-progress files during cleanup

Repository: spark Updated Branches: refs/heads/master b44d1b8fc -> ed8869ebb [SPARK-8617][WEBUI] HistoryServer: Include in-progress files during cleanup ## What changes were proposed in this pull request? - Removed the`attempt.completed ` filter so cleaner would include the orphan inprogress

spark git commit: [SPARK-18662][HOTFIX] Add new resource-managers directories to SparkLauncher.

Repository: spark Updated Branches: refs/heads/master 6a5a7254d -> b44d1b8fc [SPARK-18662][HOTFIX] Add new resource-managers directories to SparkLauncher. These directories are added to the classpath of applications when testing or using SPARK_PREPEND_CLASSES, otherwise updated classes are

spark git commit: [SPARK-18667][PYSPARK][SQL] Change the way to group row in BatchEvalPythonExec so input_file_name function can work with UDF in pyspark

Repository: spark Updated Branches: refs/heads/branch-2.1 9095c152e -> 726217eb7 [SPARK-18667][PYSPARK][SQL] Change the way to group row in BatchEvalPythonExec so input_file_name function can work with UDF in pyspark ## What changes were proposed in this pull request? `input_file_name`

spark git commit: [SPARK-18667][PYSPARK][SQL] Change the way to group row in BatchEvalPythonExec so input_file_name function can work with UDF in pyspark

Repository: spark Updated Branches: refs/heads/master 7f3c778fd -> 6a5a7254d [SPARK-18667][PYSPARK][SQL] Change the way to group row in BatchEvalPythonExec so input_file_name function can work with UDF in pyspark ## What changes were proposed in this pull request? `input_file_name` doesn't

spark git commit: [SPARK-18718][TESTS] Skip some test failures due to path length limitation and fix tests to pass on Windows

Repository: spark Updated Branches: refs/heads/master 9bf8f3cd4 -> 7f3c778fd [SPARK-18718][TESTS] Skip some test failures due to path length limitation and fix tests to pass on Windows ## What changes were proposed in this pull request? There are some tests failed on Windows due to the

spark git commit: [SPARK-18325][SPARKR][ML] SparkR ML wrappers example code and user guide

Repository: spark Updated Branches: refs/heads/branch-2.1 48aa6775d -> 9095c152e [SPARK-18325][SPARKR][ML] SparkR ML wrappers example code and user guide ## What changes were proposed in this pull request? * Add all R examples for ML wrappers which were added during 2.1 release cycle. * Split

spark git commit: [SPARK-18325][SPARKR][ML] SparkR ML wrappers example code and user guide

Repository: spark Updated Branches: refs/heads/master b47b892e4 -> 9bf8f3cd4 [SPARK-18325][SPARKR][ML] SparkR ML wrappers example code and user guide ## What changes were proposed in this pull request? * Add all R examples for ML wrappers which were added during 2.1 release cycle. * Split the