svn commit: r29758 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_20_02-3b7395f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Fri Sep 28 03:16:57 2018 New Revision: 29758 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_20_02-3b7395f docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25459][SQL] Add viewOriginalText back to CatalogTable

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 5fd22d053 -> 3b7395fe0 [SPARK-25459][SQL] Add viewOriginalText back to CatalogTable ## What changes were proposed in this pull request? The `show create table` will show a lot of generated attributes for views that created by older Spark

svn commit: r29751 - in /dev/spark/2.3.3-SNAPSHOT-2018_09_27_14_03-f13565b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 21:19:30 2018 New Revision: 29751 Log: Apache Spark 2.3.3-SNAPSHOT-2018_09_27_14_03-f13565b docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29750 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_14_03-a43a082-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 21:18:34 2018 New Revision: 29750 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_14_03-a43a082 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29749 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_12_02-5fd22d0-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 19:17:11 2018 New Revision: 29749 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_12_02-5fd22d0 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only

2018-09-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.3 f40e4c71c -> f13565b6e [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only Currently, In the spark UI, when there are failed jobs or failed stages, display message for the

spark git commit: [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only

2018-09-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.4 0256f8a09 -> a43a082e0 [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only Currently, In the spark UI, when there are failed jobs or failed stages, display message for the

svn commit: r29747 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_10_03-0256f8a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 17:17:35 2018 New Revision: 29747 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_10_03-0256f8a docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM.

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 659ecb54a -> 0256f8a09 [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM. Caching the value of that config means different instances of SparkEnv will always use whatever was the first value to be read. It also

svn commit: r29741 - in /dev/spark/v2.4.0-rc2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-09-27 Thread wenchen
Author: wenchen Date: Thu Sep 27 15:45:37 2018 New Revision: 29741 Log: Apache Spark v2.4.0-rc2 docs [This commit notification would consist of 1474 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29740 - /dev/spark/v2.4.0-rc2-bin/

2018-09-27 Thread wenchen
Author: wenchen Date: Thu Sep 27 15:24:41 2018 New Revision: 29740 Log: Apache Spark v2.4.0-rc2 Added: dev/spark/v2.4.0-rc2-bin/ dev/spark/v2.4.0-rc2-bin/SparkR_2.4.0.tar.gz (with props) dev/spark/v2.4.0-rc2-bin/SparkR_2.4.0.tar.gz.asc

svn commit: r29739 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_08_02-a1adde5-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 15:17:14 2018 New Revision: 29739 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_08_02-a1adde5 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[2/2] spark git commit: Preparing development version 2.4.1-SNAPSHOT

2018-09-27 Thread wenchen
Preparing development version 2.4.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/659ecb54 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/659ecb54 Diff:

[1/2] spark git commit: Preparing Spark release v2.4.0-rc2

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 3c78ea258 -> 659ecb54a Preparing Spark release v2.4.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/42f25f30 Tree:

[spark] Git Push Summary

2018-09-27 Thread wenchen
Repository: spark Updated Tags: refs/tags/v2.4.0-rc2 [created] 42f25f309 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-24341][SQL][FOLLOWUP] remove duplicated error checking

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f856fe483 -> a1adde540 [SPARK-24341][SQL][FOLLOWUP] remove duplicated error checking ## What changes were proposed in this pull request? There are 2 places we check for problematic `InSubquery`: the rule `ResolveSubquery` and

svn commit: r29737 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_06_02-3c78ea2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 13:17:19 2018 New Revision: 29737 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_06_02-3c78ea2 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-21436][CORE] Take advantage of known partitioner for distinct on RDDs to avoid a shuffle

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master dd8f6b1ce -> f856fe483 [SPARK-21436][CORE] Take advantage of known partitioner for distinct on RDDs to avoid a shuffle ## What changes were proposed in this pull request? Special case the situation where we know the partioner and the

spark git commit: [SPARK-25541][SQL][FOLLOWUP] Remove overriding filterKeys in CaseInsensitiveMap

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 86a2450e0 -> dd8f6b1ce [SPARK-25541][SQL][FOLLOWUP] Remove overriding filterKeys in CaseInsensitiveMap ## What changes were proposed in this pull request? As per the discussion in

spark git commit: [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 53eb85854 -> 3c78ea258 [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function ## What changes were proposed in this pull request? In ElementAt, when first argument is MapType, we should coerce the key type

spark-website git commit: Update my affiliation

2018-09-27 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 74d902cdc -> 8b7444182 Update my affiliation Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/8b744418 Tree:

spark git commit: [SPARK-25551][SQL] Remove unused InSubquery expression

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 2a8cbfddb -> 86a2450e0 [SPARK-25551][SQL] Remove unused InSubquery expression ## What changes were proposed in this pull request? The PR removes the `InSubquery` expression which was introduced a long time ago and its only usage was

svn commit: r29736 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_04_02-2a8cbfd-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 11:17:10 2018 New Revision: 29736 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_04_02-2a8cbfd docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29734 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_02_02-53eb858-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 09:16:59 2018 New Revision: 29734 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_02_02-53eb858 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29733 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_00_02-f309b28-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 07:17:48 2018 New Revision: 29733 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_00_02-f309b28 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 0b4e58187 -> 53eb85854 [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions ## What changes were proposed in this pull request? Thanks for bahchis reporting this. It is more like a follow

spark git commit: [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d03e0af80 -> 2a8cbfddb [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions ## What changes were proposed in this pull request? Thanks for bahchis reporting this. It is more like a follow up

spark git commit: [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master ff876137f -> d03e0af80 [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function ## What changes were proposed in this pull request? In ElementAt, when first argument is MapType, we should coerce the key type and

spark git commit: [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 0cf4c5bbe -> 0b4e58187 [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp ## What changes were proposed in this pull request? We have an agreement that the behavior of `from/to_utc_timestamp` is corrected, although

spark git commit: [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f309b28bd -> ff876137f [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp ## What changes were proposed in this pull request? We have an agreement that the behavior of `from/to_utc_timestamp` is corrected, although the

spark git commit: [SPARK-25485][SQL][TEST] Refactor UnsafeProjectionBenchmark to use main method

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 8b727994e -> f309b28bd [SPARK-25485][SQL][TEST] Refactor UnsafeProjectionBenchmark to use main method ## What changes were proposed in this pull request? Refactor `UnsafeProjectionBenchmark` to use main method. Generate benchmark result: