spark git commit: [SPARK-14607] [SPARK-14484] [SQL] fix case-insensitive predicates in FileSourceStrategy

2016-04-13 Thread davies
Repository: spark Updated Branches: refs/heads/master fc3cd2f50 -> 62b7f306f [SPARK-14607] [SPARK-14484] [SQL] fix case-insensitive predicates in FileSourceStrategy ## What changes were proposed in this pull request? When prune the partitions or push down predicates, case-sensitivity is not

spark git commit: [SPARK-14472][PYSPARK][ML] Cleanup ML JavaWrapper and related class hierarchy

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 781df4998 -> fc3cd2f50 [SPARK-14472][PYSPARK][ML] Cleanup ML JavaWrapper and related class hierarchy Currently, JavaWrapper is only a wrapper class for pipeline classes that have Params and JavaCallable is a separate mixin that provides me

spark git commit: [SPARK-13089][ML] [Doc] spark.ml Naive Bayes user guide and examples

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master fcdd69260 -> 781df4998 [SPARK-13089][ML] [Doc] spark.ml Naive Bayes user guide and examples jira: https://issues.apache.org/jira/browse/SPARK-13089 Add section in ml-classification.md for NaiveBayes DataFrame-based API, plus example code

spark git commit: [SPARK-14509][DOC] Add python CountVectorizerExample

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master a91aaf5a8 -> fcdd69260 [SPARK-14509][DOC] Add python CountVectorizerExample ## What changes were proposed in this pull request? Add python CountVectorizerExample ## How was this patch tested? manual tests Author: Zheng RuiFeng Closes #1

spark git commit: [SPARK-14375][ML] Unit test for spark.ml KMeansSummary

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 0d17593b3 -> a91aaf5a8 [SPARK-14375][ML] Unit test for spark.ml KMeansSummary ## What changes were proposed in this pull request? * Modify ```KMeansSummary.clusterSizes``` method to make it robust to empty clusters. * Add unit test for spa

spark git commit: [SPARK-14461][ML] GLM training summaries should provide solver

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master b0adb9f54 -> 0d17593b3 [SPARK-14461][ML] GLM training summaries should provide solver ## What changes were proposed in this pull request? GLM training summaries should provide solver. ## How was this patch tested? Unit tests. cc jkbradley

spark git commit: [SPARK-10386][MLLIB] PrefixSpanModel supports save/load

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master dbbe14907 -> b0adb9f54 [SPARK-10386][MLLIB] PrefixSpanModel supports save/load ```PrefixSpanModel``` supports ```save/load```. It's similar with #9267. cc jkbradley Author: Yanbo Liang Closes #10664 from yanboliang/spark-10386. Projec

spark git commit: [SPARK-14581] [SQL] push predicatese through more logical plans

2016-04-13 Thread davies
Repository: spark Updated Branches: refs/heads/master f9d578eaa -> dbbe14907 [SPARK-14581] [SQL] push predicatese through more logical plans ## What changes were proposed in this pull request? Right now, filter push down only works with Project, Aggregate, Generate and Join, they can't be pu

spark git commit: [SPARK-13783][ML] Model export/import for spark.ml: GBTs

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 7d2ed8cc0 -> f9d578eaa [SPARK-13783][ML] Model export/import for spark.ml: GBTs ## What changes were proposed in this pull request? * Added save/load for ```GBTClassifier/GBTClassificationModel/GBTRegressor/GBTRegressionModel```. * Meanwhi

spark git commit: [SPARK-14388][SQL] Implement CREATE TABLE

2016-04-13 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 1018a1c1e -> 7d2ed8cc0 [SPARK-14388][SQL] Implement CREATE TABLE ## What changes were proposed in this pull request? This patch implements the `CREATE TABLE` command using the `SessionCatalog`. Previously we handled only `CTAS` and `CREAT

spark git commit: [SPARK-14568][ML] Instrumentation framework for logistic regression

2016-04-13 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 323e7390a -> 1018a1c1e [SPARK-14568][ML] Instrumentation framework for logistic regression ## What changes were proposed in this pull request? This adds extra logging information about a `LogisticRegression` estimator when being fit on a

spark git commit: Revert "[SPARK-14154][MLLIB] Simplify the implementation for Kolmogorov–Smirnov test"

2016-04-13 Thread meng
Repository: spark Updated Branches: refs/heads/master dd11e401e -> 323e7390a Revert "[SPARK-14154][MLLIB] Simplify the implementation for Kolmogorov–Smirnov test" This reverts commit d2a819a6363190b946986ebf6f8001d520098c3b. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commi

spark git commit: [SPARK-14537][CORE] Make TaskSchedulerImpl waiting fail if context is shut down

2016-04-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 23f93f559 -> dd11e401e [SPARK-14537][CORE] Make TaskSchedulerImpl waiting fail if context is shut down This patch makes the postStartHook throw an IllegalStateException if the SparkContext is shutdown while it is waiting for the backend to