spark git commit: [SPARK-23072][SQL][TEST] Add a Unicode schema test for file-based data sources

2018-01-16 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 00c744e40 -> 8ef323c57 [SPARK-23072][SQL][TEST] Add a Unicode schema test for file-based data sources ## What changes were proposed in this pull request? After [SPARK-20682](https://github.com/apache/spark/pull/19651), Apache Spark

spark git commit: [SPARK-23072][SQL][TEST] Add a Unicode schema test for file-based data sources

2018-01-16 Thread wenchen
Repository: spark Updated Branches: refs/heads/master a963980a6 -> a0aedb0de [SPARK-23072][SQL][TEST] Add a Unicode schema test for file-based data sources ## What changes were proposed in this pull request? After [SPARK-20682](https://github.com/apache/spark/pull/19651), Apache Spark 2.3

spark git commit: Fix merge between 07ae39d0ec and 1667057851

2018-01-16 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 50345a2aa -> a963980a6 Fix merge between 07ae39d0ec and 1667057851 ## What changes were proposed in this pull request? The first commit added a new test, and the second refactored the class the test was in. The automatic merge put the

spark git commit: Revert "[SPARK-23020][CORE] Fix races in launcher code, test."

2018-01-16 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 0a441d2ed -> b9339eee1 Revert "[SPARK-23020][CORE] Fix races in launcher code, test." This reverts commit 20c69816a63071b82b1035d4b48798c358206421. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

svn commit: r24252 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_16_22_01-0a441d2e-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _

2018-01-16 Thread pwendell
Author: pwendell Date: Wed Jan 17 06:15:25 2018 New Revision: 24252 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_16_22_01-0a441d2e docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: Revert "[SPARK-23020][CORE] Fix races in launcher code, test."

2018-01-16 Thread sameerag
Repository: spark Updated Branches: refs/heads/master 166705785 -> 50345a2aa Revert "[SPARK-23020][CORE] Fix races in launcher code, test." This reverts commit 66217dac4f8952a9923625908ad3dcb030763c81. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

svn commit: r24251 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_20_01-1667057-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Wed Jan 17 04:14:54 2018 New Revision: 24251 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_20_01-1667057 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[2/2] spark git commit: [SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader

2018-01-16 Thread tdas
[SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader ## What changes were proposed in this pull request? The Kafka reader is now interruptible and can close itself. ## How was this patch tested? I locally ran one of the

[1/2] spark git commit: [SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader

2018-01-16 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.3 08252bb38 -> 0a441d2ed http://git-wip-us.apache.org/repos/asf/spark/blob/0a441d2e/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala

[2/2] spark git commit: [SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader

2018-01-16 Thread tdas
[SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader ## What changes were proposed in this pull request? The Kafka reader is now interruptible and can close itself. ## How was this patch tested? I locally ran one of the

[1/2] spark git commit: [SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader

2018-01-16 Thread tdas
Repository: spark Updated Branches: refs/heads/master a9b845ebb -> 166705785 http://git-wip-us.apache.org/repos/asf/spark/blob/16670578/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala

spark git commit: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 41d1a323c -> 08252bb38 [SPARK-22361][SQL][TEST] Add unit test for Window Frames ## What changes were proposed in this pull request? There are already quite a few integration tests using window frames, but the unit tests coverage is

spark git commit: [SPARK-22361][SQL][TEST] Add unit test for Window Frames

2018-01-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0c2ba427b -> a9b845ebb [SPARK-22361][SQL][TEST] Add unit test for Window Frames ## What changes were proposed in this pull request? There are already quite a few integration tests using window frames, but the unit tests coverage is not

spark git commit: [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException

2018-01-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 7022ef800 -> d09eecccf [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException ## What changes were proposed in this pull request? The following SQL involving scalar correlated query returns a

spark git commit: [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException

2018-01-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 5ae91 -> 0c2ba427b [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException ## What changes were proposed in this pull request? The following SQL involving scalar correlated query returns a map

spark git commit: [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException

2018-01-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 833a584bb -> 41d1a323c [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException ## What changes were proposed in this pull request? The following SQL involving scalar correlated query returns a

spark git commit: [SPARK-23044] Error handling for jira assignment

2018-01-16 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 4371466b3 -> 5ae91 [SPARK-23044] Error handling for jira assignment ## What changes were proposed in this pull request? * If there is any error while trying to assign the jira, prompt again * Filter out the "Apache Spark" choice *

svn commit: r24250 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_16_01-4371466-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Wed Jan 17 00:14:40 2018 New Revision: 24250 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_16_01-4371466 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24246 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_16_14_01-833a584-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 22:14:49 2018 New Revision: 24246 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_16_14_01-833a584 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23045][ML][SPARKR] Update RFormula to use OneHotEncoderEstimator.

2018-01-16 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.3 863ffdc8a -> 833a584bb [SPARK-23045][ML][SPARKR] Update RFormula to use OneHotEncoderEstimator. ## What changes were proposed in this pull request? RFormula should use VectorSizeHint & OneHotEncoderEstimator in its pipeline to avoid

spark git commit: [SPARK-23045][ML][SPARKR] Update RFormula to use OneHotEncoderEstimator.

2018-01-16 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 12db365b4 -> 4371466b3 [SPARK-23045][ML][SPARKR] Update RFormula to use OneHotEncoderEstimator. ## What changes were proposed in this pull request? RFormula should use VectorSizeHint & OneHotEncoderEstimator in its pipeline to avoid

spark-website git commit: Note update of master branch SNAPSHOT versions as part of cutting new release branch

2018-01-16 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site ab2bcbf28 -> b3ecc858f Note update of master branch SNAPSHOT versions as part of cutting new release branch Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit:

svn commit: r24245 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_12_01-12db365-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 20:15:16 2018 New Revision: 24245 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_12_01-12db365 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-16139][TEST] Add logging functionality for leaked threads in tests

2018-01-16 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 75db14864 -> 12db365b4 [SPARK-16139][TEST] Add logging functionality for leaked threads in tests ## What changes were proposed in this pull request? Lots of our tests don't properly shutdown everything they create, and end up leaking

svn commit: r24240 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_08_01-75db148-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 16:14:49 2018 New Revision: 24240 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_08_01-75db148 docs [This commit notification would consist of 1441 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-16 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 5c06ee2d4 -> 863ffdc8a [SPARK-22392][SQL] data source v2 columnar batch reader ## What changes were proposed in this pull request? a new Data Source V2 interface to allow the data source to return `ColumnarBatch` during the scan. ##

spark git commit: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-16 Thread wenchen
Repository: spark Updated Branches: refs/heads/master b85eb946a -> 75db14864 [SPARK-22392][SQL] data source v2 columnar batch reader ## What changes were proposed in this pull request? a new Data Source V2 interface to allow the data source to return `ColumnarBatch` during the scan. ## How

svn commit: r24238 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_16_06_01-5c06ee2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 14:15:07 2018 New Revision: 24238 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_16_06_01-5c06ee2 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24237 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_04_01-b85eb94-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 12:20:21 2018 New Revision: 24237 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_04_01-b85eb94 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22978][PYSPARK] Register Vectorized UDFs for SQL Statement

2018-01-16 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 20c69816a -> 5c06ee2d4 [SPARK-22978][PYSPARK] Register Vectorized UDFs for SQL Statement ## What changes were proposed in this pull request? Register Vectorized UDFs for SQL Statement. For example, ```Python >>> from

spark git commit: [SPARK-22978][PYSPARK] Register Vectorized UDFs for SQL Statement

2018-01-16 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 66217dac4 -> b85eb946a [SPARK-22978][PYSPARK] Register Vectorized UDFs for SQL Statement ## What changes were proposed in this pull request? Register Vectorized UDFs for SQL Statement. For example, ```Python >>> from pyspark.sql.functions

svn commit: r24235 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_16_02_01-20c6981-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 10:15:28 2018 New Revision: 24235 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_16_02_01-20c6981 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24233 - in /dev/spark/2.4.0-SNAPSHOT-2018_01_16_00_01-66217da-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-16 Thread pwendell
Author: pwendell Date: Tue Jan 16 08:16:30 2018 New Revision: 24233 Log: Apache Spark 2.4.0-SNAPSHOT-2018_01_16_00_01-66217da docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]