spark git commit: [SPARK-23091][ML] Incorrect unit test for approxQuantile

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 793841c6b -> 396cdfbea [SPARK-23091][ML] Incorrect unit test for approxQuantile ## What changes were proposed in this pull request? Narrow bound on approx quantile test to epsilon from 2*epsilon to match paper ## How was this patch

spark git commit: [SPARK-23091][ML] Incorrect unit test for approxQuantile

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 c647f918b -> 0cde5212a [SPARK-23091][ML] Incorrect unit test for approxQuantile ## What changes were proposed in this pull request? Narrow bound on approx quantile test to epsilon from 2*epsilon to match paper ## How was this patch

svn commit: r24332 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_19_18_01-c647f91-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Sat Jan 20 02:15:17 2018 New Revision: 24332 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_19_18_01-c647f91 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24331 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_19_16_01-793841c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Sat Jan 20 00:14:43 2018 New Revision: 24331 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_19_16_01-793841c docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-21771][SQL] remove useless hive client in SparkSQLEnv

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 f9ad00a5a -> c647f918b [SPARK-21771][SQL] remove useless hive client in SparkSQLEnv ## What changes were proposed in this pull request? Once a meta hive client is created, it generates its SessionState which creates a lot of session

spark git commit: [SPARK-21771][SQL] remove useless hive client in SparkSQLEnv

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/master f6da41b01 -> 793841c6b [SPARK-21771][SQL] remove useless hive client in SparkSQLEnv ## What changes were proposed in this pull request? Once a meta hive client is created, it generates its SessionState which creates a lot of session

svn commit: r24330 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_19_14_01-f9ad00a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 22:15:08 2018 New Revision: 24330 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_19_14_01-f9ad00a docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23135][UI] Fix rendering of accumulators in the stage page.

2018-01-19 Thread sameerag
Repository: spark Updated Branches: refs/heads/master aa3a1276f -> f6da41b01 [SPARK-23135][UI] Fix rendering of accumulators in the stage page. This follows the behavior of 2.2: only named accumulators with a value are rendered. Screenshot:

spark git commit: [SPARK-23135][UI] Fix rendering of accumulators in the stage page.

2018-01-19 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 d0cb19873 -> f9ad00a5a [SPARK-23135][UI] Fix rendering of accumulators in the stage page. This follows the behavior of 2.2: only named accumulators with a value are rendered. Screenshot:

svn commit: r24328 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_19_12_01-aa3a127-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 20:15:04 2018 New Revision: 24328 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_19_12_01-aa3a127 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23103][CORE] Ensure correct sort order for negative values in LevelDB.

2018-01-19 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-2.3 4b79514c9 -> d0cb19873 [SPARK-23103][CORE] Ensure correct sort order for negative values in LevelDB. The code was sorting "0" as "less than" negative values, which is a little wrong. Fix is simple, most of the changes are the added

spark git commit: [SPARK-23103][CORE] Ensure correct sort order for negative values in LevelDB.

2018-01-19 Thread irashid
Repository: spark Updated Branches: refs/heads/master fed2139f0 -> aa3a1276f [SPARK-23103][CORE] Ensure correct sort order for negative values in LevelDB. The code was sorting "0" as "less than" negative values, which is a little wrong. Fix is simple, most of the changes are the added test

spark git commit: [SPARK-20664][CORE] Delete stale application data from SHS.

2018-01-19 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-2.3 ffe45913d -> 4b79514c9 [SPARK-20664][CORE] Delete stale application data from SHS. Detect the deletion of event log files from storage, and remove data about the related application attempt in the SHS. Also contains code to fix

spark git commit: [SPARK-20664][CORE] Delete stale application data from SHS.

2018-01-19 Thread irashid
Repository: spark Updated Branches: refs/heads/master 07296a61c -> fed2139f0 [SPARK-20664][CORE] Delete stale application data from SHS. Detect the deletion of event log files from storage, and remove data about the related application attempt in the SHS. Also contains code to fix

spark git commit: [INFRA] Close stale PR.

2018-01-19 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 73d3b230f -> 07296a61c [INFRA] Close stale PR. Closes #20185. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/07296a61 Tree:

spark git commit: [SPARK-23104][K8S][DOCS] Changes to Kubernetes scheduler documentation

2018-01-19 Thread vanzin
Repository: spark Updated Branches: refs/heads/master d8aaa771e -> 73d3b230f [SPARK-23104][K8S][DOCS] Changes to Kubernetes scheduler documentation ## What changes were proposed in this pull request? Docs changes: - Adding a warning that the backend is experimental. - Removing a defunct

spark git commit: [SPARK-23104][K8S][DOCS] Changes to Kubernetes scheduler documentation

2018-01-19 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.3 55efeffd7 -> ffe45913d [SPARK-23104][K8S][DOCS] Changes to Kubernetes scheduler documentation ## What changes were proposed in this pull request? Docs changes: - Adding a warning that the backend is experimental. - Removing a defunct

svn commit: r24327 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_19_10_01-55efeff-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 18:15:05 2018 New Revision: 24327 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_19_10_01-55efeff docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23149][SQL] polish ColumnarBatch

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 606a7485f -> d8aaa771e [SPARK-23149][SQL] polish ColumnarBatch ## What changes were proposed in this pull request? Several cleanups in `ColumnarBatch` * remove `schema`. The `ColumnVector`s inside `ColumnarBatch` already have the data

spark git commit: [SPARK-23149][SQL] polish ColumnarBatch

2018-01-19 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 8d6845cf9 -> 55efeffd7 [SPARK-23149][SQL] polish ColumnarBatch ## What changes were proposed in this pull request? Several cleanups in `ColumnarBatch` * remove `schema`. The `ColumnVector`s inside `ColumnarBatch` already have the

svn commit: r24325 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_19_08_01-606a748-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 16:15:01 2018 New Revision: 24325 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_19_08_01-606a748 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sparse

2018-01-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6c39654ef -> 606a7485f [SPARK-23085][ML] API parity for mllib.linalg.Vectors.sparse ## What changes were proposed in this pull request? `ML.Vectors#sparse(size: Int, elements: Seq[(Int, Double)])` support zero-length ## How was this patch

spark git commit: [SPARK-23000][TEST] Keep Derby DB Location Unchanged After Session Cloning

2018-01-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 b7a81999d -> 8d6845cf9 [SPARK-23000][TEST] Keep Derby DB Location Unchanged After Session Cloning ## What changes were proposed in this pull request? After session cloning in `TestHive`, the conf of the singleton SparkContext for

spark git commit: [SPARK-23000][TEST] Keep Derby DB Location Unchanged After Session Cloning

2018-01-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/master e1c33b6cd -> 6c39654ef [SPARK-23000][TEST] Keep Derby DB Location Unchanged After Session Cloning ## What changes were proposed in this pull request? After session cloning in `TestHive`, the conf of the singleton SparkContext for derby DB

[2/2] spark-website git commit: Add notice that site is licensed as AL2

2018-01-19 Thread srowen
Add notice that site is licensed as AL2 Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/464ddc70 Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/464ddc70 Diff:

[1/2] spark-website git commit: Add notice that site is licensed as AL2

2018-01-19 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site b3ecc858f -> 464ddc703 http://git-wip-us.apache.org/repos/asf/spark-website/blob/464ddc70/site/news/submit-talks-to-spark-summit-2014.html -- diff --git

spark git commit: [SPARK-23024][WEB-UI] Spark ui about the contents of the form need to have hidden and show features, when the table records very much.

2018-01-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master e41400c3c -> e1c33b6cd [SPARK-23024][WEB-UI] Spark ui about the contents of the form need to have hidden and show features, when the table records very much. ## What changes were proposed in this pull request? Spark ui about the contents

svn commit: r24322 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_19_06_01-b7a8199-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 14:14:36 2018 New Revision: 24322 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_19_06_01-b7a8199 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24315 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_19_04_01-e41400c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 12:19:02 2018 New Revision: 24315 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_19_04_01-e41400c docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23089][STS] Recreate session log directory if it doesn't exist

2018-01-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/master b74366481 -> e41400c3c [SPARK-23089][STS] Recreate session log directory if it doesn't exist ## What changes were proposed in this pull request? When creating a session directory, Thrift should create the parent directory (i.e.

spark git commit: [SPARK-23048][ML] Add OneHotEncoderEstimator document and examples

2018-01-19 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.3 e58223171 -> ef7989d55 [SPARK-23048][ML] Add OneHotEncoderEstimator document and examples ## What changes were proposed in this pull request? We have `OneHotEncoderEstimator` now and `OneHotEncoder` will be deprecated since 2.3.0. We

spark git commit: [SPARK-23048][ML] Add OneHotEncoderEstimator document and examples

2018-01-19 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 60203fca6 -> b74366481 [SPARK-23048][ML] Add OneHotEncoderEstimator document and examples ## What changes were proposed in this pull request? We have `OneHotEncoderEstimator` now and `OneHotEncoder` will be deprecated since 2.3.0. We

spark git commit: [SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter

2018-01-19 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 9c4b99861 -> 60203fca6 [SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter Update user guide entry for `FeatureHasher` to match the Scala / Python doc, to describe the `categoricalCols` parameter. ## How was this

spark git commit: [SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter

2018-01-19 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.3 54c1fae12 -> e58223171 [SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter Update user guide entry for `FeatureHasher` to match the Scala / Python doc, to describe the `categoricalCols` parameter. ## How was

svn commit: r24312 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_19_02_01-54c1fae-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-19 Thread pwendell
Author: pwendell Date: Fri Jan 19 10:14:54 2018 New Revision: 24312 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_19_02_01-54c1fae docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [BUILD][MINOR] Fix java style check issues

2018-01-19 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 541dbc00b -> 54c1fae12 [BUILD][MINOR] Fix java style check issues ## What changes were proposed in this pull request? This patch fixes a few recently introduced java style check errors in master and release branch. As an aside,

spark git commit: [BUILD][MINOR] Fix java style check issues

2018-01-19 Thread sameerag
Repository: spark Updated Branches: refs/heads/master 568055da9 -> 9c4b99861 [BUILD][MINOR] Fix java style check issues ## What changes were proposed in this pull request? This patch fixes a few recently introduced java style check errors in master and release branch. As an aside, given