spark git commit: [SPARK-17310][SQL] Add an option to disable record-level filter in Parquet-side

2017-11-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d8741b2b0 -> 673c67046 [SPARK-17310][SQL] Add an option to disable record-level filter in Parquet-side ## What changes were proposed in this pull request? There is a concern that Spark-side codegen row-by-row filtering might be faster tha

spark git commit: [SPARK-17074][SQL] Generate equi-height histogram in column statistics

2017-11-14 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 673c67046 -> 11b60af73 [SPARK-17074][SQL] Generate equi-height histogram in column statistics ## What changes were proposed in this pull request? Equi-height histogram is effective in cardinality estimation, and more accurate than basic c

[1/5] spark git commit: [SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend.

2017-11-14 Thread irashid
Repository: spark Updated Branches: refs/heads/master 11b60af73 -> 4741c0780 http://git-wip-us.apache.org/repos/asf/spark/blob/4741c078/core/src/test/resources/HistoryServerExpectations/stage_task_list_w__sortBy_expectation.json --

[2/5] spark git commit: [SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend.

2017-11-14 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/4741c078/core/src/test/resources/HistoryServerExpectations/job_list_from_multi_attempt_app_json_2__expectation.json -- diff --git a/core/src/test/resources/HistoryServerExpectation

[3/5] spark git commit: [SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend.

2017-11-14 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/4741c078/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala -- diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/

[4/5] spark git commit: [SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend.

2017-11-14 Thread irashid
http://git-wip-us.apache.org/repos/asf/spark/blob/4741c078/core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala -- diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala b/core/src/main/scala/org

[5/5] spark git commit: [SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend.

2017-11-14 Thread irashid
[SPARK-20648][CORE] Port JobsTab and StageTab to the new UI backend. This change is a little larger because there's a whole lot of logic behind these pages, all really tied to internal types and listeners, and some of that logic had to be implemented in the new listener and the needed data exposed

[1/2] spark git commit: [SPARK-20652][SQL] Store SQL UI data in the new app status store.

2017-11-14 Thread irashid
Repository: spark Updated Branches: refs/heads/master 4741c0780 -> 0ffa7c488 http://git-wip-us.apache.org/repos/asf/spark/blob/0ffa7c48/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala -- diff --

[2/2] spark git commit: [SPARK-20652][SQL] Store SQL UI data in the new app status store.

2017-11-14 Thread irashid
[SPARK-20652][SQL] Store SQL UI data in the new app status store. This change replaces the SQLListener with a new implementation that saves the data to the same store used by the SparkContext's status store. For that, the types used by the old SQLListener had to be updated a bit so that they're mo

spark git commit: [SPARK-22519][YARN] Remove unnecessary stagingDirPath null check in ApplicationMaster.cleanupStagingDir()

2017-11-14 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 0ffa7c488 -> eaff295a2 [SPARK-22519][YARN] Remove unnecessary stagingDirPath null check in ApplicationMaster.cleanupStagingDir() ## What changes were proposed in this pull request? Removed the unnecessary stagingDirPath null check in Appl

spark git commit: [SPARK-22511][BUILD] Update maven central repo address

2017-11-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master eaff295a2 -> b00972259 [SPARK-22511][BUILD] Update maven central repo address ## What changes were proposed in this pull request? Use repo.maven.apache.org repo address; use latest ASF parent POM version 18 ## How was this patch tested?

spark git commit: [SPARK-22511][BUILD] Update maven central repo address

2017-11-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.2 3ea6fd0c4 -> 210f2922b [SPARK-22511][BUILD] Update maven central repo address Use repo.maven.apache.org repo address; use latest ASF parent POM version 18 Existing tests; no functional change Author: Sean Owen Closes #19742 from sro

spark git commit: [SPARK-21087][ML] CrossValidator, TrainValidationSplit expose sub models after fitting: Scala

2017-11-14 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master b00972259 -> 774398045 [SPARK-21087][ML] CrossValidator, TrainValidationSplit expose sub models after fitting: Scala ## What changes were proposed in this pull request? We add a parameter whether to collect the full model list when Cross

spark git commit: [SPARK-12375][ML] VectorIndexerModel support handle unseen categories via handleInvalid

2017-11-14 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 774398045 -> 1e6f76059 [SPARK-12375][ML] VectorIndexerModel support handle unseen categories via handleInvalid ## What changes were proposed in this pull request? Support skip/error/keep strategy, similar to `StringIndexer`. Implemented v