spark git commit: [SPARK-21345][SQL][TEST][TEST-MAVEN][BRANCH-2.1] SparkSessionBuilderSuite should clean up stopped sessions.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 7f7b63bb6 -> 5e2bfd5bc [SPARK-21345][SQL][TEST][TEST-MAVEN][BRANCH-2.1] SparkSessionBuilderSuite should clean up stopped sessions. ## What changes were proposed in this pull request? `SparkSessionBuilderSuite` should clean up stopped

spark git commit: [SPARK-21093][R] Terminate R's worker processes in the parent of R's daemon to prevent a leak

2017-07-08 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master c3712b77a -> 08e0d033b [SPARK-21093][R] Terminate R's worker processes in the parent of R's daemon to prevent a leak ## What changes were proposed in this pull request? This is a retry for #18320. This PR was reverted due to unexpected te

spark git commit: [SPARK-21307][REVERT][SQL] Remove SQLConf parameters from the parser-related classes

2017-07-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 062c336d0 -> c3712b77a [SPARK-21307][REVERT][SQL] Remove SQLConf parameters from the parser-related classes ## What changes were proposed in this pull request? Since we do not set active sessions when parsing the plan, we are unable to co

spark git commit: [SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 c8d7855b9 -> 964332b28 [SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem. ## What changes were proposed in this pull request? In current code, reducer can break the old shuffle service when `spark.reducer.maxR

spark git commit: [SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 9131bdb7e -> 062c336d0 [SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem. ## What changes were proposed in this pull request? In current code, reducer can break the old shuffle service when `spark.reducer.maxReqSi

spark git commit: [SPARK-20342][CORE] Update task accumulators before sending task end event.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 a64f10800 -> c8d7855b9 [SPARK-20342][CORE] Update task accumulators before sending task end event. This makes sures that listeners get updated task information; otherwise it's possible to write incomplete task information into event log

spark git commit: [SPARK-20342][CORE] Update task accumulators before sending task end event.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 9fccc3627 -> 9131bdb7e [SPARK-20342][CORE] Update task accumulators before sending task end event. This makes sures that listeners get updated task information; otherwise it's possible to write incomplete task information into event logs, f

spark git commit: [SPARK-21083][SQL] Store zero size and row count when analyzing empty table

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 0b8dd2d08 -> 9fccc3627 [SPARK-21083][SQL] Store zero size and row count when analyzing empty table ## What changes were proposed in this pull request? We should be able to store zero size and row count after analyzing empty table. This pr

spark git commit: [SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 7d0b1c927 -> a64f10800 [SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions. `SparkSessionBuilderSuite` should clean up stopped sessions. Otherwise, it leaves behind some stopped `SparkContext

spark git commit: [SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions.

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 330bf5c99 -> 0b8dd2d08 [SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions. ## What changes were proposed in this pull request? `SparkSessionBuilderSuite` should clean up stopped sessions. Otherw

spark git commit: [SPARK-21228][SQL][BRANCH-2.2] InSet incorrect handling of structs

2017-07-08 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 ab12848d6 -> 7d0b1c927 [SPARK-21228][SQL][BRANCH-2.2] InSet incorrect handling of structs ## What changes were proposed in this pull request? This is backport of https://github.com/apache/spark/pull/18455 When data type is struct, InSe

spark git commit: [SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a test in ALSCleanerSuite

2017-07-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 01f183e84 -> 330bf5c99 [SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a test in ALSCleanerSuite ## What changes were proposed in this pull request? This PR is similar to #17869. Once` 'spark.local.dir'` is set.

spark git commit: Mesos doc fixes

2017-07-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master f5f02d213 -> 01f183e84 Mesos doc fixes ## What changes were proposed in this pull request? Some link fixes for the documentation [Running Spark on Mesos](https://spark.apache.org/docs/latest/running-on-mesos.html): * Updated Link to Meso