spark git commit: [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of value arguments should be true.

2018-08-16 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 8af61fba0 -> c1ffb3c10 [SPARK-23938][SQL][FOLLOW-UP][TEST] Nullabilities of value arguments should be true. ## What changes were proposed in this pull request? This is a follow-up pr of #22017 which added `map_zip_with` function. In the

spark git commit: [SPARK-25122][SQL] Deduplication of supports equals code

2018-08-16 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f16140975 -> 8af61fba0 [SPARK-25122][SQL] Deduplication of supports equals code ## What changes were proposed in this pull request? The method ```*supportEquals``` determining whether elements of a data type could be used as items in a

svn commit: r28789 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_16_20_02-f161409-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-08-16 Thread pwendell
Author: pwendell Date: Fri Aug 17 03:15:57 2018 New Revision: 28789 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_16_20_02-f161409 docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23940][SQL] Add transform_values SQL function

2018-08-16 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 9251c61bd -> f16140975 [SPARK-23940][SQL] Add transform_values SQL function ## What changes were proposed in this pull request? This pr adds `transform_values` function which applies the function to each entry of the map and transforms

spark git commit: [SPARK-24665][PYSPARK][FOLLOWUP] Use SQLConf in PySpark to manage all sql configs

2018-08-16 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 30be71e91 -> 9251c61bd [SPARK-24665][PYSPARK][FOLLOWUP] Use SQLConf in PySpark to manage all sql configs ## What changes were proposed in this pull request? Follow up for SPARK-24665, find some others hard code during code review. ##

spark git commit: [DOCS] Fix cloud-integration.md Typo

2018-08-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master 709f541dd -> 30be71e91 [DOCS] Fix cloud-integration.md Typo Corrected typo; changed spark-default.conf to spark-defaults.conf Closes #22125 from KraFusion/patch-2. Authored-by: Joey Krabacher Signed-off-by: Sean Owen Project:

spark git commit: [DOCS] Update configuration.md

2018-08-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master e59dd8fa0 -> 709f541dd [DOCS] Update configuration.md changed $SPARK_HOME/conf/spark-default.conf to $SPARK_HOME/conf/spark-defaults.conf no testing necessary as this was a change to documentation. Closes #22116 from KraFusion/patch-1.

svn commit: r28788 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_16_16_02-e59dd8f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-08-16 Thread pwendell
Author: pwendell Date: Thu Aug 16 23:15:58 2018 New Revision: 28788 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_16_16_02-e59dd8f docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark-website git commit: Mention correctness issues as blockers in developer docs; mention 18 month release branch maintenance guideline

2018-08-16 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site fff22b750 -> 0bef76657 Mention correctness issues as blockers in developer docs; mention 18 month release branch maintenance guideline Author: Sean Owen Closes #136 from srowen/Correctness. Project:

spark git commit: [SPARK-25092][SQL][FOLLOWUP] Add RewriteCorrelatedScalarSubquery in list of nonExcludableRules

2018-08-16 Thread lixiao
Repository: spark Updated Branches: refs/heads/master e50192494 -> e59dd8fa0 [SPARK-25092][SQL][FOLLOWUP] Add RewriteCorrelatedScalarSubquery in list of nonExcludableRules ## What changes were proposed in this pull request? Add RewriteCorrelatedScalarSubquery in the list of

spark-website git commit: For projects using names that are likely proscribed, either update to use current conforming name or remove the link, for now

2018-08-16 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 8eb764260 -> fff22b750 For projects using names that are likely proscribed, either update to use current conforming name or remove the link, for now Author: Sean Owen Closes #137 from srowen/SparkNames. Project:

spark git commit: [SPARK-24555][ML] logNumExamples in KMeans/BiKM/GMM/AFT/NB

2018-08-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master b3e6fe7c4 -> e50192494 [SPARK-24555][ML] logNumExamples in KMeans/BiKM/GMM/AFT/NB ## What changes were proposed in this pull request? logNumExamples in KMeans/BiKM/GMM/AFT/NB ## How was this patch tested? existing tests Closes #21561

spark git commit: [SPARK-23654][BUILD] remove jets3t as a dependency of spark

2018-08-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master ea63a7a16 -> b3e6fe7c4 [SPARK-23654][BUILD] remove jets3t as a dependency of spark ## What changes were proposed in this pull request? Remove jets3t dependency, and bouncy castle which it brings in; update licenses and deps Note this

svn commit: r28782 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_16_08_02-ea63a7a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-08-16 Thread pwendell
Author: pwendell Date: Thu Aug 16 15:16:14 2018 New Revision: 28782 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_16_08_02-ea63a7a docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23932][SQL] Higher order function zip_with

2018-08-16 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 5b4a38d82 -> ea63a7a16 [SPARK-23932][SQL] Higher order function zip_with ## What changes were proposed in this pull request? Merges the two given arrays, element-wise, into a single array using function. If one array is shorter, nulls are

svn commit: r28776 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_16_04_02-5b4a38d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-08-16 Thread pwendell
Author: pwendell Date: Thu Aug 16 11:21:07 2018 New Revision: 28776 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_16_04_02-5b4a38d docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23939][SQL] Add transform_keys function

2018-08-16 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 7822c3f8d -> 5b4a38d82 [SPARK-23939][SQL] Add transform_keys function ## What changes were proposed in this pull request? This pr adds transform_keys function which applies the function to each entry of the map and transforms the keys.

svn commit: r28766 - in /dev/spark/2.4.0-SNAPSHOT-2018_08_16_00_02-7822c3f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-08-16 Thread pwendell
Author: pwendell Date: Thu Aug 16 07:16:37 2018 New Revision: 28766 Log: Apache Spark 2.4.0-SNAPSHOT-2018_08_16_00_02-7822c3f docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25082][SQL] improve the javadoc for expm1()

2018-08-16 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master a791c29bd -> 7822c3f8d [SPARK-25082][SQL] improve the javadoc for expm1() ## What changes were proposed in this pull request? Correct the javadoc for expm1() function. ## How was this patch tested? None. It is a minor issue. Closes