spark git commit: [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f309b28bd -> ff876137f [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp ## What changes were proposed in this pull request? We have an agreement that the behavior of `from/to_utc_timestamp` is corrected, although the fun

spark git commit: [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 0cf4c5bbe -> 0b4e58187 [SPARK-23715][SQL][DOC] improve document for from/to_utc_timestamp ## What changes were proposed in this pull request? We have an agreement that the behavior of `from/to_utc_timestamp` is corrected, although the

spark git commit: [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master ff876137f -> d03e0af80 [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function ## What changes were proposed in this pull request? In ElementAt, when first argument is MapType, we should coerce the key type and

spark git commit: [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d03e0af80 -> 2a8cbfddb [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions ## What changes were proposed in this pull request? Thanks for bahchis reporting this. It is more like a follow up wor

spark git commit: [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 0b4e58187 -> 53eb85854 [SPARK-25314][SQL] Fix Python UDF accessing attributes from both side of join in join conditions ## What changes were proposed in this pull request? Thanks for bahchis reporting this. It is more like a follow up

svn commit: r29733 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_00_02-f309b28-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 07:17:48 2018 New Revision: 29733 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_00_02-f309b28 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r29734 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_02_02-53eb858-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 09:16:59 2018 New Revision: 29734 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_02_02-53eb858 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r29736 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_04_02-2a8cbfd-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 11:17:10 2018 New Revision: 29736 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_04_02-2a8cbfd docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-25551][SQL] Remove unused InSubquery expression

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 2a8cbfddb -> 86a2450e0 [SPARK-25551][SQL] Remove unused InSubquery expression ## What changes were proposed in this pull request? The PR removes the `InSubquery` expression which was introduced a long time ago and its only usage was remov

spark-website git commit: Update my affiliation

2018-09-27 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 74d902cdc -> 8b7444182 Update my affiliation Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/8b744418 Tree: http://git-wip-us.apache.org/repos

spark git commit: [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 53eb85854 -> 3c78ea258 [SPARK-25522][SQL] Improve type promotion for input arguments of elementAt function ## What changes were proposed in this pull request? In ElementAt, when first argument is MapType, we should coerce the key type

spark git commit: [SPARK-25541][SQL][FOLLOWUP] Remove overriding filterKeys in CaseInsensitiveMap

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 86a2450e0 -> dd8f6b1ce [SPARK-25541][SQL][FOLLOWUP] Remove overriding filterKeys in CaseInsensitiveMap ## What changes were proposed in this pull request? As per the discussion in https://github.com/apache/spark/pull/22553#pullrequestrevi

spark git commit: [SPARK-21436][CORE] Take advantage of known partitioner for distinct on RDDs to avoid a shuffle

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master dd8f6b1ce -> f856fe483 [SPARK-21436][CORE] Take advantage of known partitioner for distinct on RDDs to avoid a shuffle ## What changes were proposed in this pull request? Special case the situation where we know the partioner and the numb

svn commit: r29737 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_06_02-3c78ea2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 13:17:19 2018 New Revision: 29737 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_06_02-3c78ea2 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24341][SQL][FOLLOWUP] remove duplicated error checking

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f856fe483 -> a1adde540 [SPARK-24341][SQL][FOLLOWUP] remove duplicated error checking ## What changes were proposed in this pull request? There are 2 places we check for problematic `InSubquery`: the rule `ResolveSubquery` and `InSubquery.

[spark] Git Push Summary

2018-09-27 Thread wenchen
Repository: spark Updated Tags: refs/tags/v2.4.0-rc2 [created] 42f25f309 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v2.4.0-rc2

2018-09-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 3c78ea258 -> 659ecb54a Preparing Spark release v2.4.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/42f25f30 Tree: http://git-wip-us.apache.org/repos/asf/spar

[2/2] spark git commit: Preparing development version 2.4.1-SNAPSHOT

2018-09-27 Thread wenchen
Preparing development version 2.4.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/659ecb54 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/659ecb54 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/659

svn commit: r29739 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_08_02-a1adde5-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 15:17:14 2018 New Revision: 29739 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_08_02-a1adde5 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r29740 - /dev/spark/v2.4.0-rc2-bin/

2018-09-27 Thread wenchen
Author: wenchen Date: Thu Sep 27 15:24:41 2018 New Revision: 29740 Log: Apache Spark v2.4.0-rc2 Added: dev/spark/v2.4.0-rc2-bin/ dev/spark/v2.4.0-rc2-bin/SparkR_2.4.0.tar.gz (with props) dev/spark/v2.4.0-rc2-bin/SparkR_2.4.0.tar.gz.asc dev/spark/v2.4.0-rc2-bin/SparkR_2.4.0.tar.g

svn commit: r29741 - in /dev/spark/v2.4.0-rc2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-09-27 Thread wenchen
Author: wenchen Date: Thu Sep 27 15:45:37 2018 New Revision: 29741 Log: Apache Spark v2.4.0-rc2 docs [This commit notification would consist of 1474 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] -

spark git commit: [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM.

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master a1adde540 -> 5fd22d053 [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM. Caching the value of that config means different instances of SparkEnv will always use whatever was the first value to be read. It also breaks tes

spark git commit: [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM.

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 659ecb54a -> 0256f8a09 [SPARK-25546][CORE] Don't cache value of EVENT_LOG_CALLSITE_LONG_FORM. Caching the value of that config means different instances of SparkEnv will always use whatever was the first value to be read. It also breaks

svn commit: r29747 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_10_03-0256f8a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 17:17:35 2018 New Revision: 29747 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_10_03-0256f8a docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only

2018-09-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.4 0256f8a09 -> a43a082e0 [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only Currently, In the spark UI, when there are failed jobs or failed stages, display message for the c

spark git commit: [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only

2018-09-27 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.3 f40e4c71c -> f13565b6e [SPARK-25533][CORE][WEBUI] AppSummary should hold the information about succeeded Jobs and completed stages only Currently, In the spark UI, when there are failed jobs or failed stages, display message for the c

svn commit: r29749 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_12_02-5fd22d0-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 19:17:11 2018 New Revision: 29749 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_12_02-5fd22d0 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r29750 - in /dev/spark/2.4.1-SNAPSHOT-2018_09_27_14_03-a43a082-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 21:18:34 2018 New Revision: 29750 Log: Apache Spark 2.4.1-SNAPSHOT-2018_09_27_14_03-a43a082 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r29751 - in /dev/spark/2.3.3-SNAPSHOT-2018_09_27_14_03-f13565b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Thu Sep 27 21:19:30 2018 New Revision: 29751 Log: Apache Spark 2.3.3-SNAPSHOT-2018_09_27_14_03-f13565b docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-25459][SQL] Add viewOriginalText back to CatalogTable

2018-09-27 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 5fd22d053 -> 3b7395fe0 [SPARK-25459][SQL] Add viewOriginalText back to CatalogTable ## What changes were proposed in this pull request? The `show create table` will show a lot of generated attributes for views that created by older Spark

svn commit: r29758 - in /dev/spark/2.5.0-SNAPSHOT-2018_09_27_20_02-3b7395f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-09-27 Thread pwendell
Author: pwendell Date: Fri Sep 28 03:16:57 2018 New Revision: 29758 Log: Apache Spark 2.5.0-SNAPSHOT-2018_09_27_20_02-3b7395f docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---