spark git commit: [SPARK-18538][SQL] Fix Concurrent Table Fetching Using DataFrameReader JDBC APIs

2016-11-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 2eb6764fb -> b28fe4a4a [SPARK-18538][SQL] Fix Concurrent Table Fetching Using DataFrameReader JDBC APIs ### What changes were proposed in this pull request? The following two `DataFrameReader` JDBC APIs ignore the user-specified

spark git commit: [SPARK-18476][SPARKR][ML] SparkR Logistic Regression should should support output original label.

2016-11-30 Thread yliang
Repository: spark Updated Branches: refs/heads/branch-2.1 7d4596734 -> e8d8e3509 [SPARK-18476][SPARKR][ML] SparkR Logistic Regression should should support output original label. ## What changes were proposed in this pull request? Similar to SPARK-18401, as a classification algorithm,

spark git commit: [SPARK-18476][SPARKR][ML] SparkR Logistic Regression should should support output original label.

2016-11-30 Thread yliang
Repository: spark Updated Branches: refs/heads/master 0a811210f -> 2eb6764fb [SPARK-18476][SPARKR][ML] SparkR Logistic Regression should should support output original label. ## What changes were proposed in this pull request? Similar to SPARK-18401, as a classification algorithm, logistic

spark git commit: [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly

2016-11-30 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 6e2e987bd -> 7d4596734 [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly ## What changes were proposed in this pull request? Fixed the potential SparkContext leak in

spark git commit: [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly

2016-11-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master c4979f6ea -> 0a811210f [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly ## What changes were proposed in this pull request? Fixed the potential SparkContext leak in

spark git commit: [SPARK-18655][SS] Ignore Structured Streaming 2.0.2 logs in history server

2016-11-30 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 c2c2fdcb7 -> 6e2e987bd [SPARK-18655][SS] Ignore Structured Streaming 2.0.2 logs in history server ## What changes were proposed in this pull request? As `queryStatus` in StreamingQueryListener events was removed in #15954, parsing

spark git commit: [SPARK-18655][SS] Ignore Structured Streaming 2.0.2 logs in history server

2016-11-30 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 93e9d880b -> c4979f6ea [SPARK-18655][SS] Ignore Structured Streaming 2.0.2 logs in history server ## What changes were proposed in this pull request? As `queryStatus` in StreamingQueryListener events was removed in #15954, parsing 2.0.2

spark git commit: [SPARK-18546][CORE] Fix merging shuffle spills when using encryption.

2016-11-30 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.1 9e96ac5a9 -> c2c2fdcb7 [SPARK-18546][CORE] Fix merging shuffle spills when using encryption. The problem exists because it's not possible to just concatenate encrypted partition data from different spill files; currently each partition

spark git commit: [SPARK-18546][CORE] Fix merging shuffle spills when using encryption.

2016-11-30 Thread vanzin
Repository: spark Updated Branches: refs/heads/master f135b70fd -> 93e9d880b [SPARK-18546][CORE] Fix merging shuffle spills when using encryption. The problem exists because it's not possible to just concatenate encrypted partition data from different spill files; currently each partition

spark git commit: [SPARK-18251][SQL] the type of Dataset can't be Option of non-flat type

2016-11-30 Thread lian
Repository: spark Updated Branches: refs/heads/branch-2.1 f542df310 -> 9e96ac5a9 [SPARK-18251][SQL] the type of Dataset can't be Option of non-flat type ## What changes were proposed in this pull request? For input object of non-flat type, we can't encode it to row if it's null, as Spark

spark git commit: [SPARK-18251][SQL] the type of Dataset can't be Option of non-flat type

2016-11-30 Thread lian
Repository: spark Updated Branches: refs/heads/master 60022bfd6 -> f135b70fd [SPARK-18251][SQL] the type of Dataset can't be Option of non-flat type ## What changes were proposed in this pull request? For input object of non-flat type, we can't encode it to row if it's null, as Spark SQL

spark git commit: [SPARK][EXAMPLE] Added missing semicolon in quick-start-guide example

2016-11-30 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 1b1c849bf -> 5ecd3c23a [SPARK][EXAMPLE] Added missing semicolon in quick-start-guide example ## What changes were proposed in this pull request? Added missing semicolon in quick-start-guide java example code which wasn't compiling

spark git commit: [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors

2016-11-30 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 8b33aa089 -> 1b1c849bf [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors ## What changes were proposed in this pull request? The method `TaskSchedulerImpl.runningTasksByExecutors()` accesses the mutable

spark git commit: [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors

2016-11-30 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.1 eae85da38 -> 7c0e2962d [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors ## What changes were proposed in this pull request? The method `TaskSchedulerImpl.runningTasksByExecutors()` accesses the mutable

spark git commit: [SPARK-18220][SQL] read Hive orc table with varchar column should not fail

2016-11-30 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 6e044ab9a -> 3de93fb48 [SPARK-18220][SQL] read Hive orc table with varchar column should not fail ## What changes were proposed in this pull request? Spark SQL only has `StringType`, when reading hive table with varchar column, we

spark git commit: [SPARK-18220][SQL] read Hive orc table with varchar column should not fail

2016-11-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master c24076dcf -> 3f03c90a8 [SPARK-18220][SQL] read Hive orc table with varchar column should not fail ## What changes were proposed in this pull request? Spark SQL only has `StringType`, when reading hive table with varchar column, we will

spark git commit: [SPARK-18612][MLLIB] Delete broadcasted variable in LBFGS CostFun

2016-11-30 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.1 7043c6b69 -> 05ba5eed7 [SPARK-18612][MLLIB] Delete broadcasted variable in LBFGS CostFun ## What changes were proposed in this pull request? Fix a broadcasted variable leak occurring at each invocation of CostFun in L-BFGS. ## How

spark git commit: [SPARK-18612][MLLIB] Delete broadcasted variable in LBFGS CostFun

2016-11-30 Thread srowen
Repository: spark Updated Branches: refs/heads/master fe854f2e4 -> c5a64d760 [SPARK-18612][MLLIB] Delete broadcasted variable in LBFGS CostFun ## What changes were proposed in this pull request? Fix a broadcasted variable leak occurring at each invocation of CostFun in L-BFGS. ## How was

spark git commit: [SPARK-18366][PYSPARK][ML] Add handleInvalid to Pyspark for QuantileDiscretizer and Bucketizer

2016-11-30 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 56c82edab -> fe854f2e4 [SPARK-18366][PYSPARK][ML] Add handleInvalid to Pyspark for QuantileDiscretizer and Bucketizer ## What changes were proposed in this pull request? added the new handleInvalid param for these transformers to Python

spark git commit: [SPARK-18366][PYSPARK][ML] Add handleInvalid to Pyspark for QuantileDiscretizer and Bucketizer

2016-11-30 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.1 5e4afbfb6 -> 7043c6b69 [SPARK-18366][PYSPARK][ML] Add handleInvalid to Pyspark for QuantileDiscretizer and Bucketizer ## What changes were proposed in this pull request? added the new handleInvalid param for these transformers to