spark git commit: [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9.

2018-03-25 Thread jshao
Repository: spark Updated Branches: refs/heads/master 087fb3142 -> eb48edf9c [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9. This particular test assumed that Hadoop libraries did not support http as a file system. Hadoop 2.9 does, so the test failed. The test n

svn commit: r25961 - in /dev/spark/2.3.1-SNAPSHOT-2018_03_25_22_02-328dea6-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-03-25 Thread pwendell
Author: pwendell Date: Mon Mar 26 05:16:36 2018 New Revision: 25961 Log: Apache Spark 2.3.1-SNAPSHOT-2018_03_25_22_02-328dea6 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_udf` with keyword args

2018-03-25 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 2fd7acabf -> 328dea6f8 [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_udf` with keyword args ## What changes were proposed in this pull request? Add documentation about the limitations of `pandas_udf` with keyword arguments an

spark git commit: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_udf` with keyword args

2018-03-25 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master a9350d709 -> 087fb3142 [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_udf` with keyword args ## What changes were proposed in this pull request? Add documentation about the limitations of `pandas_udf` with keyword arguments and re

spark git commit: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-25 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master e4bec7cb8 -> a9350d709 [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql ## What changes were proposed in this pull request? This cleans up unused imports, mainly from pyspark.sql module. Added a note in function.py that imports `User

svn commit: r25960 - in /dev/spark/2.4.0-SNAPSHOT-2018_03_25_20_01-e4bec7c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-03-25 Thread pwendell
Author: pwendell Date: Mon Mar 26 03:15:33 2018 New Revision: 25960 Log: Apache Spark 2.4.0-SNAPSHOT-2018_03_25_20_01-e4bec7c docs [This commit notification would consist of 1451 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r25959 - in /dev/spark/2.3.1-SNAPSHOT-2018_03_25_18_01-2fd7aca-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-03-25 Thread pwendell
Author: pwendell Date: Mon Mar 26 01:15:28 2018 New Revision: 25959 Log: Apache Spark 2.3.1-SNAPSHOT-2018_03_25_18_01-2fd7aca docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-23549][SQL] Cast to timestamp when comparing timestamp with date

2018-03-25 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 5f653d4f7 -> e4bec7cb8 [SPARK-23549][SQL] Cast to timestamp when comparing timestamp with date ## What changes were proposed in this pull request? This PR fixes an incorrect comparison in SQL between timestamp and date. This is because bo

spark git commit: [HOT-FIX] Fix SparkOutOfMemoryError: Unable to acquire 262144 bytes of memory, got 224631

2018-03-25 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.3 57026a185 -> 2fd7acabf [HOT-FIX] Fix SparkOutOfMemoryError: Unable to acquire 262144 bytes of memory, got 224631 ## What changes were proposed in this pull request? https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88

spark git commit: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Random Numbers

2018-03-25 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.3 523fcafc5 -> 57026a185 [SPARK-23599][SQL] Add a UUID generator from Pseudo-Random Numbers ## What changes were proposed in this pull request? This patch adds a UUID generator from Pseudo-Random Numbers. We can use it later to have det

svn commit: r25958 - in /dev/spark/2.4.0-SNAPSHOT-2018_03_25_12_01-5f653d4-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-03-25 Thread pwendell
Author: pwendell Date: Sun Mar 25 19:15:22 2018 New Revision: 25958 Log: Apache Spark 2.4.0-SNAPSHOT-2018_03_25_12_01-5f653d4 docs [This commit notification would consist of 1451 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

[1/2] spark git commit: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQuerySuite

2018-03-25 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 816a5496b -> 5f653d4f7 http://git-wip-us.apache.org/repos/asf/spark/blob/5f653d4f/sql/core/src/test/resources/tpcds-v2.7.0/q98.sql -- diff --git a/sql/core/src/test/resource

[2/2] spark git commit: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQuerySuite

2018-03-25 Thread lixiao
[SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQuerySuite ## What changes were proposed in this pull request? This pr added TPCDS v2.7 (latest) queries in `TPCDSQuerySuite` because the current `TPCDSQuerySuite` tests older one (v1.4) and some queries are different from v1.4 and v2.7. Since th