spark git commit: [SPARK-11958][SPARK-11957][ML][DOC] SQLTransformer user guide and example code

Repository: spark Updated Branches: refs/heads/branch-1.6 8652fc03c -> 5c8216920 [SPARK-11958][SPARK-11957][ML][DOC] SQLTransformer user guide and example code Add ```SQLTransformer``` user guide, example code and make Scala API doc more clear. Author: Yanbo Liang

spark git commit: [SPARK-11958][SPARK-11957][ML][DOC] SQLTransformer user guide and example code

Repository: spark Updated Branches: refs/heads/master 7d05a6245 -> 4a39b5a1b [SPARK-11958][SPARK-11957][ML][DOC] SQLTransformer user guide and example code Add ```SQLTransformer``` user guide, example code and make Scala API doc more clear. Author: Yanbo Liang Closes

spark git commit: [SPARK-10259][ML] Add @since annotation to ml.classification

Repository: spark Updated Branches: refs/heads/master 73896588d -> 7d05a6245 [SPARK-10259][ML] Add @since annotation to ml.classification Add since annotation to ml.classification Author: Takahashi Hiroshi Closes #8534 from taishi-oss/issue10259. Project:

spark git commit: [SPARK-10259][ML] Add @since annotation to ml.classification

Repository: spark Updated Branches: refs/heads/branch-1.6 3c683ed5f -> 8652fc03c [SPARK-10259][ML] Add @since annotation to ml.classification Add since annotation to ml.classification Author: Takahashi Hiroshi Closes #8534 from taishi-oss/issue10259.

[3/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Replace example code in ml-features.md using include_example

[SPARK-11551][DOC][EXAMPLE] Replace example code in ml-features.md using include_example Made new patch contaning only markdown examples moved to exmaple/folder. Ony three java code were not shfted since they were contaning compliation error ,these classes are 1)StandardScale

[2/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Replace example code in ml-features.md using include_example

http://git-wip-us.apache.org/repos/asf/spark/blob/3c683ed5/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java -- diff --git

[1/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Replace example code in ml-features.md using include_example

Repository: spark Updated Branches: refs/heads/master 3e7e05f5e -> 78209b0cc http://git-wip-us.apache.org/repos/asf/spark/blob/78209b0c/examples/src/main/scala/org/apache/spark/examples/ml/StringIndexerExample.scala -- diff

spark git commit: [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib - 1.5 backport

Repository: spark Updated Branches: refs/heads/branch-1.5 3868ab644 -> 2f30927a5 [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib - 1.5 backport This backports [https://github.com/apache/spark/pull/10161] to Spark 1.5, with the difference that ChiSqSelector does not require

spark git commit: Closes #10098

Repository: spark Updated Branches: refs/heads/master 78209b0cc -> 73896588d Closes #10098 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/73896588 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/73896588 Diff:

spark git commit: [SPARK-12060][CORE] Avoid memory copy in JavaSerializerInstance.serialize

Repository: spark Updated Branches: refs/heads/master 5d80d8c6a -> 3f4efb5c2 [SPARK-12060][CORE] Avoid memory copy in JavaSerializerInstance.serialize Merged #10051 again since #10083 is resolved. This reverts commit 328b757d5d4486ea3c2e246780792d7a57ee85e5. Author: Shixiong Zhu

spark git commit: [SPARK-12101][CORE] Fix thread pools that cannot cache tasks in Worker and AppClient (backport 1.5)

Repository: spark Updated Branches: refs/heads/branch-1.5 93a0510a5 -> 3868ab644 [SPARK-12101][CORE] Fix thread pools that cannot cache tasks in Worker and AppClient (backport 1.5) backport #10108 to branch 1.5 Author: Shixiong Zhu Closes #10135 from

spark git commit: [SPARK-11963][DOC] Add docs for QuantileDiscretizer

Repository: spark Updated Branches: refs/heads/branch-1.6 539914f1a -> c8aa5f201 [SPARK-11963][DOC] Add docs for QuantileDiscretizer https://issues.apache.org/jira/browse/SPARK-11963 Author: Xusen Yin Closes #9962 from yinxusen/SPARK-11963. (cherry picked from commit

spark git commit: [SPARK-11884] Drop multiple columns in the DataFrame API

Repository: spark Updated Branches: refs/heads/master 871e85d9c -> 84b809445 [SPARK-11884] Drop multiple columns in the DataFrame API See the thread Ben started: http://search-hadoop.com/m/q3RTtveEuhjsr7g/ This PR adds drop() method to DataFrame which accepts multiple column names Author:

spark git commit: [SPARK-12184][PYTHON] Make python api doc for pivot consistant with scala doc

Repository: spark Updated Branches: refs/heads/master 84b809445 -> 36282f78b [SPARK-12184][PYTHON] Make python api doc for pivot consistant with scala doc In SPARK-11946 the API for pivot was changed a bit and got updated doc, the doc changes were not made for the python api though. This PR

spark git commit: [SPARK-12184][PYTHON] Make python api doc for pivot consistant with scala doc

Repository: spark Updated Branches: refs/heads/branch-1.6 c8aa5f201 -> cdeb89b34 [SPARK-12184][PYTHON] Make python api doc for pivot consistant with scala doc In SPARK-11946 the API for pivot was changed a bit and got updated doc, the doc changes were not made for the python api though. This

[2/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

http://git-wip-us.apache.org/repos/asf/spark/blob/39d677c8/R/pkg/inst/tests/testthat/test_binaryFile.R -- diff --git a/R/pkg/inst/tests/testthat/test_binaryFile.R b/R/pkg/inst/tests/testthat/test_binaryFile.R new file mode 100644

[4/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

[SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases. This PR: 1. Suppress all known warnings. 2. Cleanup test cases and fix some errors in test cases. 3. Fix errors in HiveContext related test cases. These test cases are actually not run previously due to a bug of creating

[3/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

http://git-wip-us.apache.org/repos/asf/spark/blob/39d677c8/R/pkg/inst/tests/test_sparkSQL.R -- diff --git a/R/pkg/inst/tests/test_sparkSQL.R b/R/pkg/inst/tests/test_sparkSQL.R deleted file mode 100644 index 6ef03ae..000 ---

spark git commit: [SPARK-11932][STREAMING] Partition previous TrackStateRDD if partitioner not present

Repository: spark Updated Branches: refs/heads/master ef3f047c0 -> 5d80d8c6a [SPARK-11932][STREAMING] Partition previous TrackStateRDD if partitioner not present The reason is that TrackStateRDDs generated by trackStateByKey expect the previous batch's TrackStateRDDs to have a partitioner.

[1/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

Repository: spark Updated Branches: refs/heads/master 9cde7d5fa -> 39d677c8f http://git-wip-us.apache.org/repos/asf/spark/blob/39d677c8/R/pkg/inst/tests/testthat/test_sparkSQL.R -- diff --git

[2/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

http://git-wip-us.apache.org/repos/asf/spark/blob/3f230f7b/R/pkg/inst/tests/testthat/test_binaryFile.R -- diff --git a/R/pkg/inst/tests/testthat/test_binaryFile.R b/R/pkg/inst/tests/testthat/test_binaryFile.R new file mode 100644

[1/4] spark git commit: [SPARK-12034][SPARKR] Eliminate warnings in SparkR test cases.

Repository: spark Updated Branches: refs/heads/branch-1.6 c54b698ec -> 3f230f7b3 http://git-wip-us.apache.org/repos/asf/spark/blob/3f230f7b/R/pkg/inst/tests/testthat/test_sparkSQL.R -- diff --git

spark git commit: [SPARK-12132] [PYSPARK] raise KeyboardInterrupt inside SIGINT handler

Repository: spark Updated Branches: refs/heads/branch-1.6 3f230f7b3 -> fed453821 [SPARK-12132] [PYSPARK] raise KeyboardInterrupt inside SIGINT handler Currently, the current line is not cleared by Cltr-C After this patch ``` >>> asdfasdf^C Traceback (most recent call last): File

spark git commit: [SPARK-11932][STREAMING] Partition previous TrackStateRDD if partitioner not present

Repository: spark Updated Branches: refs/heads/branch-1.6 fed453821 -> 539914f1a [SPARK-11932][STREAMING] Partition previous TrackStateRDD if partitioner not present The reason is that TrackStateRDDs generated by trackStateByKey expect the previous batch's TrackStateRDDs to have a

spark git commit: [SPARK-12132] [PYSPARK] raise KeyboardInterrupt inside SIGINT handler

Repository: spark Updated Branches: refs/heads/master 39d677c8f -> ef3f047c0 [SPARK-12132] [PYSPARK] raise KeyboardInterrupt inside SIGINT handler Currently, the current line is not cleared by Cltr-C After this patch ``` >>> asdfasdf^C Traceback (most recent call last): File

spark git commit: [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib

Repository: spark Updated Branches: refs/heads/branch-1.6 cdeb89b34 -> 115bfbdae [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib Switched from using SQLContext constructor to using getOrCreate, mainly in model save/load methods. This covers all instances in spark.mllib. There were

spark git commit: [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib

Repository: spark Updated Branches: refs/heads/master 36282f78b -> 3e7e05f5e [SPARK-12160][MLLIB] Use SQLContext.getOrCreate in MLlib Switched from using SQLContext constructor to using getOrCreate, mainly in model save/load methods. This covers all instances in spark.mllib. There were no