spark git commit: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by default

2018-05-09 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 62d01391f -> e3d434994 [SPARK-22279][SQL] Enable `convertMetastoreOrc` by default ## What changes were proposed in this pull request? We reverted `spark.sql.hive.convertMetastoreOrc` at https://github.com/apache/spark/pull/20536 because

[2/2] spark git commit: [SPARK-24073][SQL] Rename DataReaderFactory to InputPartition.

2018-05-09 Thread lixiao
[SPARK-24073][SQL] Rename DataReaderFactory to InputPartition. ## What changes were proposed in this pull request? Renames: * `DataReaderFactory` to `InputPartition` * `DataReader` to `InputPartitionReader` * `createDataReaderFactories` to `planInputPartitions` *

[1/2] spark git commit: [SPARK-24073][SQL] Rename DataReaderFactory to InputPartition.

2018-05-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 9341c951e -> 62d01391f http://git-wip-us.apache.org/repos/asf/spark/blob/62d01391/sql/core/src/test/java/test/org/apache/spark/sql/sources/v2/JavaSchemaRequiredDataSource.java

spark git commit: [SPARK-23852][SQL] Add test that fails if PARQUET-1217 is not fixed

2018-05-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 9e3bb3136 -> 9341c951e [SPARK-23852][SQL] Add test that fails if PARQUET-1217 is not fixed ## What changes were proposed in this pull request? Add a new test that triggers if PARQUET-1217 - a predicate pushdown bug - is not fixed in

svn commit: r26820 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_09_16_01-9e3bb31-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-09 Thread pwendell
Author: pwendell Date: Wed May 9 23:15:57 2018 New Revision: 26820 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_09_16_01-9e3bb31 docs [This commit notification would consist of 1461 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBackend.killExecutors

2018-05-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/master fd1179c17 -> 9e3bb3136 [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBackend.killExecutors ## What changes were proposed in this pull request? In method *CoarseGrainedSchedulerBackend.killExecutors()*, `numPendingExecutors` should

svn commit: r26819 - in /dev/spark/2.3.1-SNAPSHOT-2018_05_09_14_01-8889d78-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-09 Thread pwendell
Author: pwendell Date: Wed May 9 21:15:26 2018 New Revision: 26819 Log: Apache Spark 2.3.1-SNAPSHOT-2018_05_09_14_01-8889d78 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r26817 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_09_12_01-fd1179c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-09 Thread pwendell
Author: pwendell Date: Wed May 9 19:16:51 2018 New Revision: 26817 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_09_12_01-fd1179c docs [This commit notification would consist of 1461 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24214][SS] Fix toJSON for StreamingRelationV2/StreamingExecutionRelation/ContinuousExecutionRelation

2018-05-09 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.3 aba52f449 -> 8889d7864 [SPARK-24214][SS] Fix toJSON for StreamingRelationV2/StreamingExecutionRelation/ContinuousExecutionRelation ## What changes were proposed in this pull request? We should overwrite "otherCopyArgs" to provide the

spark git commit: [SPARK-24214][SS] Fix toJSON for StreamingRelationV2/StreamingExecutionRelation/ContinuousExecutionRelation

2018-05-09 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 7aaa148f5 -> fd1179c17 [SPARK-24214][SS] Fix toJSON for StreamingRelationV2/StreamingExecutionRelation/ContinuousExecutionRelation ## What changes were proposed in this pull request? We should overwrite "otherCopyArgs" to provide the

spark git commit: [SPARK-14682][ML] Provide evaluateEachIteration method or equivalent for spark.ml GBTs

2018-05-09 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 628c7b517 -> 7aaa148f5 [SPARK-14682][ML] Provide evaluateEachIteration method or equivalent for spark.ml GBTs ## What changes were proposed in this pull request? Provide evaluateEachIteration method or equivalent for spark.ml GBTs. ##

spark git commit: [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6

2018-05-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 866270ea5 -> f9d6a16ce [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6 This PR aims to bump Py4J in order to fix the following float/double bug. Py4J 0.10.5 fixes this (https://github.com/bartdag/py4j/issues/272) and the latest Py4J is

spark git commit: [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6

2018-05-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.1 c4ecc04c6 -> 8177b2148 [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6 This PR aims to bump Py4J in order to fix the following float/double bug. Py4J 0.10.5 fixes this (https://github.com/bartdag/py4j/issues/272) and the latest Py4J is

[1/2] spark git commit: [SPARKR] Match pyspark features in SparkR communication protocol.

2018-05-09 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 94155d039 -> 628c7b517 [SPARKR] Match pyspark features in SparkR communication protocol. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/628c7b51 Tree:

[2/2] spark git commit: [PYSPARK] Update py4j to version 0.10.7.

2018-05-09 Thread vanzin
[PYSPARK] Update py4j to version 0.10.7. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cc613b55 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/cc613b55 Diff:

spark git commit: [MINOR][ML][DOC] Improved Naive Bayes user guide explanation

2018-05-09 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 6ea582e36 -> 94155d039 [MINOR][ML][DOC] Improved Naive Bayes user guide explanation ## What changes were proposed in this pull request? This copies the material from the spark.mllib user guide page for Naive Bayes to the spark.ml user

spark git commit: [SPARK-24181][SQL] Better error message for writing sorted data

2018-05-09 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master cac9b1dea -> 6ea582e36 [SPARK-24181][SQL] Better error message for writing sorted data ## What changes were proposed in this pull request? The exception message should clearly distinguish sorting and bucketing in `save` and `jdbc` write.

svn commit: r26781 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_09_00_01-cac9b1d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-09 Thread pwendell
Author: pwendell Date: Wed May 9 07:17:02 2018 New Revision: 26781 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_09_00_01-cac9b1d docs [This commit notification would consist of 1461 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]