spark git commit: [SPARK-20544][SPARKR] skip tests when running on CRAN

2017-05-03 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 d8bd213f1 -> 5fe9313d7 [SPARK-20544][SPARKR] skip tests when running on CRAN General rule on skip or not: skip if - RDD tests - tests could run long or complicated (streaming, hivecontext) - tests on error conditions - tests won't

spark git commit: [SPARK-20543][SPARKR] skip tests when running on CRAN

2017-05-03 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 02bbe7311 -> fc472bddd [SPARK-20543][SPARKR] skip tests when running on CRAN ## What changes were proposed in this pull request? General rule on skip or not: skip if - RDD tests - tests could run long or complicated (streaming,

spark git commit: [SPARK-20584][PYSPARK][SQL] Python generic hint support

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master 13eb37c86 -> 02bbe7311 [SPARK-20584][PYSPARK][SQL] Python generic hint support ## What changes were proposed in this pull request? Adds `hint` method to PySpark `DataFrame`. ## How was this patch tested? Unit tests, doctests. Author:

spark git commit: [SPARK-20584][PYSPARK][SQL] Python generic hint support

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.2 a3a5fcfef -> d8bd213f1 [SPARK-20584][PYSPARK][SQL] Python generic hint support ## What changes were proposed in this pull request? Adds `hint` method to PySpark `DataFrame`. ## How was this patch tested? Unit tests, doctests.

[2/2] spark git commit: Preparing development version 2.2.1-SNAPSHOT

2017-05-03 Thread pwendell
Preparing development version 2.2.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a3a5fcfe Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a3a5fcfe Diff:

[1/2] spark git commit: Preparing Spark release v2.2.0-rc2

2017-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-2.2 2629e7c7a -> a3a5fcfef Preparing Spark release v2.2.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1d4017b4 Tree:

spark git commit: [MINOR][SQL] Fix the test title from =!= to <=>, remove a duplicated test and add a test for =!=

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6b9e49d12 -> 13eb37c86 [MINOR][SQL] Fix the test title from =!= to <=>, remove a duplicated test and add a test for =!= ## What changes were proposed in this pull request? This PR proposes three things as below: - This test looks not

spark git commit: [MINOR][SQL] Fix the test title from =!= to <=>, remove a duplicated test and add a test for =!=

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.2 36d807906 -> 2629e7c7a [MINOR][SQL] Fix the test title from =!= to <=>, remove a duplicated test and add a test for =!= ## What changes were proposed in this pull request? This PR proposes three things as below: - This test looks

spark git commit: [SPARK-19965][SS] DataFrame batch reader may fail to infer partitions when reading FileStreamSink's output

2017-05-03 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 527fc5d0c -> 6b9e49d12 [SPARK-19965][SS] DataFrame batch reader may fail to infer partitions when reading FileStreamSink's output ## The Problem Right now DataFrame batch reader may fail to infer partitions when reading FileStreamSink's

spark git commit: [SPARK-19965][SS] DataFrame batch reader may fail to infer partitions when reading FileStreamSink's output

2017-05-03 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.2 f0e80aa2d -> 36d807906 [SPARK-19965][SS] DataFrame batch reader may fail to infer partitions when reading FileStreamSink's output ## The Problem Right now DataFrame batch reader may fail to infer partitions when reading

spark-website git commit: trigger resync

2017-05-03 Thread marmbrus
Repository: spark-website Updated Branches: refs/heads/asf-site d4f0c34ac -> 7b32b181f trigger resync Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/7b32b181 Tree:

[spark-website] Git Push Summary

2017-05-03 Thread marmbrus
Repository: spark-website Updated Branches: refs/heads/spark-2.1.1 [created] d4f0c34ac - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-20576][SQL] Support generic hint function in Dataset/DataFrame

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.2 b1a732fea -> f0e80aa2d [SPARK-20576][SQL] Support generic hint function in Dataset/DataFrame ## What changes were proposed in this pull request? We allow users to specify hints (currently only "broadcast" is supported) in SQL and

spark git commit: [SPARK-20576][SQL] Support generic hint function in Dataset/DataFrame

2017-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master 27f543b15 -> 527fc5d0c [SPARK-20576][SQL] Support generic hint function in Dataset/DataFrame ## What changes were proposed in this pull request? We allow users to specify hints (currently only "broadcast" is supported) in SQL and

spark git commit: [SPARK-20441][SPARK-20432][SS] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation

2017-05-03 Thread brkyvz
Repository: spark Updated Branches: refs/heads/branch-2.2 b5947f5c3 -> b1a732fea [SPARK-20441][SPARK-20432][SS] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation ## What changes were proposed in this pull request? Within the

spark git commit: [SPARK-20441][SPARK-20432][SS] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation

2017-05-03 Thread brkyvz
Repository: spark Updated Branches: refs/heads/master 7f96f2d7f -> 27f543b15 [SPARK-20441][SPARK-20432][SS] Within the same streaming query, one StreamingRelation should only be transformed to one StreamingExecutionRelation ## What changes were proposed in this pull request? Within the same

spark git commit: [SPARK-16957][MLLIB] Use midpoints for split values.

2017-05-03 Thread srowen
Repository: spark Updated Branches: refs/heads/master 16fab6b0e -> 7f96f2d7f [SPARK-16957][MLLIB] Use midpoints for split values. ## What changes were proposed in this pull request? Use midpoints for split values now, and maybe later to make it weighted. ## How was this patch tested? + [x]

spark git commit: [SPARK-20523][BUILD] Clean up build warnings for 2.2.0 release

2017-05-03 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.2 4f647ab66 -> b5947f5c3 [SPARK-20523][BUILD] Clean up build warnings for 2.2.0 release ## What changes were proposed in this pull request? Fix build warnings primarily related to Breeze 0.13 operator changes, Java style problems ##

spark git commit: [SPARK-20523][BUILD] Clean up build warnings for 2.2.0 release

2017-05-03 Thread srowen
Repository: spark Updated Branches: refs/heads/master db2fb84b4 -> 16fab6b0e [SPARK-20523][BUILD] Clean up build warnings for 2.2.0 release ## What changes were proposed in this pull request? Fix build warnings primarily related to Breeze 0.13 operator changes, Java style problems ## How

spark git commit: [SPARK-6227][MLLIB][PYSPARK] Implement PySpark wrappers for SVD and PCA (v2)

2017-05-03 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 6235132a8 -> db2fb84b4 [SPARK-6227][MLLIB][PYSPARK] Implement PySpark wrappers for SVD and PCA (v2) Add PCA and SVD to PySpark's wrappers for `RowMatrix` and `IndexedRowMatrix` (SVD only). Based on #7963, updated. ## How was this patch

spark git commit: [SPARK-6227][MLLIB][PYSPARK] Implement PySpark wrappers for SVD and PCA (v2)

2017-05-03 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.2 c80242ab9 -> 4f647ab66 [SPARK-6227][MLLIB][PYSPARK] Implement PySpark wrappers for SVD and PCA (v2) Add PCA and SVD to PySpark's wrappers for `RowMatrix` and `IndexedRowMatrix` (SVD only). Based on #7963, updated. ## How was this