spark git commit: [SPARK-22892][SQL] Simplify some estimation logic by using double instead of decimal

2017-12-28 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d4f0b1d2c -> 224375c55 [SPARK-22892][SQL] Simplify some estimation logic by using double instead of decimal ## What changes were proposed in this pull request? Simplify some estimation logic by using double instead of decimal. ## How

spark git commit: [SPARK-22834][SQL] Make insertion commands have real children to fix UI issues

2017-12-28 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 67ea11ea0 -> d4f0b1d2c [SPARK-22834][SQL] Make insertion commands have real children to fix UI issues ## What changes were proposed in this pull request? With #19474, children of insertion commands are missing in UI. To fix it: 1. Create

spark git commit: [SPARK-22891][SQL] Make hive client creation thread safe

2017-12-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 796e48c60 -> 67ea11ea0 [SPARK-22891][SQL] Make hive client creation thread safe ## What changes were proposed in this pull request? This is to walk around the hive issue: https://issues.apache.org/jira/browse/HIVE-11935 ## How was this

spark git commit: [SPARK-22313][PYTHON][FOLLOWUP] Explicitly import warnings namespace in flume.py

2017-12-28 Thread ueshin
Repository: spark Updated Branches: refs/heads/master c74573084 -> 796e48c60 [SPARK-22313][PYTHON][FOLLOWUP] Explicitly import warnings namespace in flume.py ## What changes were proposed in this pull request? This PR explicitly imports the missing `warnings` in `flume.py`. ## How was this

svn commit: r23942 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_20_01-c745730-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Fri Dec 29 04:14:53 2017 New Revision: 23942 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_20_01-c745730 docs [This commit notification would consist of 1425 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22905][MLLIB] Fix ChiSqSelectorModel save implementation

2017-12-28 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master ffe6fd77a -> c74573084 [SPARK-22905][MLLIB] Fix ChiSqSelectorModel save implementation ## What changes were proposed in this pull request? Currently, in `ChiSqSelectorModel`, save: ```

svn commit: r23941 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_16_01-ffe6fd7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Fri Dec 29 00:14:40 2017 New Revision: 23941 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_16_01-ffe6fd7 docs [This commit notification would consist of 1425 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22818][SQL] csv escape of quote escape

2017-12-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master cfcd74668 -> ffe6fd77a [SPARK-22818][SQL] csv escape of quote escape ## What changes were proposed in this pull request? Escape of escape should be considered when using the UniVocity csv encoding/decoding library. Ref:

spark git commit: [SPARK-11035][CORE] Add in-process Spark app launcher.

2017-12-28 Thread irashid
Repository: spark Updated Branches: refs/heads/master 613b71a12 -> cfcd74668 [SPARK-11035][CORE] Add in-process Spark app launcher. This change adds a new launcher that allows applications to be run in a separate thread in the same process as the calling code. To achieve that, some code from

spark git commit: [SPARK-22890][TEST] Basic tests for DateTimeOperations

2017-12-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 9c21ece35 -> 613b71a12 [SPARK-22890][TEST] Basic tests for DateTimeOperations ## What changes were proposed in this pull request? Test Coverage for `DateTimeOperations`, this is a Sub-tasks for

spark git commit: [SPARK-22836][UI] Show driver logs in UI when available.

2017-12-28 Thread irashid
Repository: spark Updated Branches: refs/heads/master 8f6d5734d -> 9c21ece35 [SPARK-22836][UI] Show driver logs in UI when available. Port code from the old executors listener to the new one, so that the driver logs present in the application start event are kept. Author: Marcelo Vanzin

svn commit: r23939 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_12_01-8f6d573-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Thu Dec 28 20:14:52 2017 New Revision: 23939 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_12_01-8f6d573 docs [This commit notification would consist of 1422 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22875][BUILD] Assembly build fails for a high user id

2017-12-28 Thread srowen
Repository: spark Updated Branches: refs/heads/master 5536f3181 -> 8f6d5734d [SPARK-22875][BUILD] Assembly build fails for a high user id ## What changes were proposed in this pull request? Add tarLongFileMode=posix configuration for the assembly plugin ## How was this patch tested? Reran

svn commit: r23934 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_08_01-5536f31-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Thu Dec 28 16:17:34 2017 New Revision: 23934 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_08_01-5536f31 docs [This commit notification would consist of 1422 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [MINOR][BUILD] Fix Java linter errors

2017-12-28 Thread srowen
Repository: spark Updated Branches: refs/heads/master 287781742 -> 5536f3181 [MINOR][BUILD] Fix Java linter errors ## What changes were proposed in this pull request? This PR cleans up a few Java linter errors for Apache Spark 2.3 release. ## How was this patch tested? ```bash $

spark git commit: [SPARK-22917][SQL] Should not try to generate histogram for empty/null columns

2017-12-28 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 755f2f518 -> 287781742 [SPARK-22917][SQL] Should not try to generate histogram for empty/null columns ## What changes were proposed in this pull request? For empty/null column, the result of `ApproximatePercentile` is null. Then in

spark git commit: [SPARK-20392][SQL][FOLLOWUP] should not add extra AnalysisBarrier

2017-12-28 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 1eebfbe19 -> 755f2f518 [SPARK-20392][SQL][FOLLOWUP] should not add extra AnalysisBarrier ## What changes were proposed in this pull request? I found this problem while auditing the analyzer code. It's dangerous to introduce extra

svn commit: r23932 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_04_01-1eebfbe-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Thu Dec 28 12:16:27 2017 New Revision: 23932 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_04_01-1eebfbe docs [This commit notification would consist of 1422 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-21208][R] Adds setLocalProperty and getLocalProperty in R

2017-12-28 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 76e8a1d7e -> 1eebfbe19 [SPARK-21208][R] Adds setLocalProperty and getLocalProperty in R ## What changes were proposed in this pull request? This PR adds `setLocalProperty` and `getLocalProperty`in R. ```R > df <- createDataFrame(iris) >

spark git commit: [SPARK-22843][R] Adds localCheckpoint in R

2017-12-28 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master ded6d27e4 -> 76e8a1d7e [SPARK-22843][R] Adds localCheckpoint in R ## What changes were proposed in this pull request? This PR proposes to add `localCheckpoint(..)` in R API. ```r df <- localCheckpoint(createDataFrame(iris)) ``` ## How

svn commit: r23929 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_28_00_01-ded6d27-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-28 Thread pwendell
Author: pwendell Date: Thu Dec 28 08:15:36 2017 New Revision: 23929 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_28_00_01-ded6d27 docs [This commit notification would consist of 1422 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]