spark git commit: [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveDirCommand output schema in Parquet issue

2018-09-09 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 0782dfa14 -> c9ca35943 [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveDirCommand output schema in Parquet issue ## What changes were proposed in this pull request? How to reproduce: ```scala spark.sql("CREATE TABLE tbl(id long)")

spark git commit: [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveDirCommand output schema in Parquet issue

2018-09-09 Thread wenchen
Repository: spark Updated Branches: refs/heads/master a0aed475c -> f8b4d5aaf [SPARK-25313][SQL][FOLLOW-UP] Fix InsertIntoHiveDirCommand output schema in Parquet issue ## What changes were proposed in this pull request? How to reproduce: ```scala spark.sql("CREATE TABLE tbl(id long)")

svn commit: r29236 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_22_02-0782dfa-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Mon Sep 10 05:16:41 2018 New Revision: 29236 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_22_02-0782dfa docs [This commit notification would consist of 1477 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29235 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_20_02-a0aed47-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Mon Sep 10 03:16:13 2018 New Revision: 29235 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_20_02-a0aed47 docs [This commit notification would consist of 1484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore

2018-09-09 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 c1c1bda3c -> 0782dfa14 [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore ## What changes were proposed in this pull request? Apache Spark doesn't create Hive

spark git commit: [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore

2018-09-09 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 77c996403 -> a0aed475c [SPARK-25175][SQL] Field resolution should fail if there is ambiguity for ORC native data source table persisted in metastore ## What changes were proposed in this pull request? Apache Spark doesn't create Hive

svn commit: r29234 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_12_02-77c9964-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Sun Sep 9 19:16:22 2018 New Revision: 29234 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_12_02-77c9964 docs [This commit notification would consist of 1484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29233 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_10_02-c1c1bda-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Sun Sep 9 17:19:13 2018 New Revision: 29233 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_10_02-c1c1bda docs [This commit notification would consist of 1477 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29232 - in /dev/spark/2.3.3-SNAPSHOT-2018_09_09_10_02-5ad644a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Sun Sep 9 17:17:21 2018 New Revision: 29232 Log: Apache Spark 2.3.3-SNAPSHOT-2018_09_09_10_02-5ad644a docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result

2018-09-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 5b8b6b4e9 -> 5ad644a4c [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result How to reproduce: ```scala val df1 = spark.createDataFrame(Seq( (1, 1) )).toDF("a", "b").withColumn("c", lit(null).cast("int")) val df2 =

spark git commit: [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result

2018-09-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 6b7ea78ae -> c1c1bda3c [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result ## What changes were proposed in this pull request? How to reproduce: ```scala val df1 = spark.createDataFrame(Seq( (1, 1) )).toDF("a",

spark git commit: [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result

2018-09-09 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 88a930dfa -> 77c996403 [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result ## What changes were proposed in this pull request? How to reproduce: ```scala val df1 = spark.createDataFrame(Seq( (1, 1) )).toDF("a",

svn commit: r29230 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_08_02-88a930d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Sun Sep 9 15:16:46 2018 New Revision: 29230 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_08_02-88a930d docs [This commit notification would consist of 1484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark-website git commit: Add a note on PMC criteria

2018-09-09 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 0bfb69b9b -> 9d5aa3ea4 Add a note on PMC criteria Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/9d5aa3ea Tree:

spark git commit: [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeasure` method

2018-09-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0b9ccd55c -> 88a930dfa [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeasure` method ## What changes were proposed in this pull request? Remove `BisectingKMeansModel.setDistanceMeasure` method. In `BisectingKMeansModel` set this

spark git commit: [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeasure` method

2018-09-09 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.4 a00a160e1 -> 6b7ea78ae [MINOR][ML] Remove `BisectingKMeansModel.setDistanceMeasure` method ## What changes were proposed in this pull request? Remove `BisectingKMeansModel.setDistanceMeasure` method. In `BisectingKMeansModel` set this

[2/2] spark git commit: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317]

2018-09-09 Thread wenchen
Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317] ## What changes were proposed in this pull request? When running TPC-DS benchmarks on 2.4 release, npoggi and winglungngai saw more than 10% performance regression on the following queries: q67, q24a and q24b. After we applying

[1/2] spark git commit: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317]

2018-09-09 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 8f7d8a097 -> a00a160e1 http://git-wip-us.apache.org/repos/asf/spark/blob/a00a160e/core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java

[1/2] spark git commit: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317]

2018-09-09 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 1cfda4482 -> 0b9ccd55c http://git-wip-us.apache.org/repos/asf/spark/blob/0b9ccd55/core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeInMemorySorter.java

[2/2] spark git commit: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317]

2018-09-09 Thread wenchen
Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-25317] ## What changes were proposed in this pull request? When running TPC-DS benchmarks on 2.4 release, npoggi and winglungngai saw more than 10% performance regression on the following queries: q67, q24a and q24b. After we applying

svn commit: r29220 - in /dev/spark/2.4.0-SNAPSHOT-2018_09_09_00_02-1cfda44-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-09 Thread pwendell
Author: pwendell Date: Sun Sep 9 07:16:56 2018 New Revision: 29220 Log: Apache Spark 2.4.0-SNAPSHOT-2018_09_09_00_02-1cfda44 docs [This commit notification would consist of 1484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]