spark git commit: [SPARK-18949][SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread rxin
Repository: spark Updated Branches: refs/heads/master b2dd8ec6b -> 24c0c9412 [SPARK-18949][SQL] Add recoverPartitions API to Catalog ### What changes were proposed in this pull request? Currently, we only have a SQL interface for recovering all the partitions in the directory of a table and

spark git commit: [SPARK-18900][FLAKY-TEST] StateStoreSuite.maintenance

2016-12-20 Thread tdas
Repository: spark Updated Branches: refs/heads/master 047a9d92c -> b2dd8ec6b [SPARK-18900][FLAKY-TEST] StateStoreSuite.maintenance ## What changes were proposed in this pull request? It was pretty flaky before 10 days ago.

spark git commit: [SPARK-18900][FLAKY-TEST] StateStoreSuite.maintenance

2016-12-20 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.1 3857d5ba8 -> 063a98e52 [SPARK-18900][FLAKY-TEST] StateStoreSuite.maintenance ## What changes were proposed in this pull request? It was pretty flaky before 10 days ago.

spark git commit: [SPARK-18928][BRANCH-2.0] Check TaskContext.isInterrupted() in FileScanRDD, JDBCRDD & UnsafeSorter

2016-12-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 678d91c1d -> 2aae220b5 [SPARK-18928][BRANCH-2.0] Check TaskContext.isInterrupted() in FileScanRDD, JDBCRDD & UnsafeSorter This is a branch-2.0 backport of #16340; the original description follows: ## What changes were proposed in

spark git commit: [SPARK-18761][BRANCH-2.0] Introduce "task reaper" to oversee task killing in executors

2016-12-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 1f0c5fa75 -> 678d91c1d [SPARK-18761][BRANCH-2.0] Introduce "task reaper" to oversee task killing in executors Branch-2.0 backport of #16189; original description follows: ## What changes were proposed in this pull request? Spark's

spark git commit: [SPARK-18576][PYTHON] Add basic TaskContext information to PySpark

2016-12-20 Thread rxin
Repository: spark Updated Branches: refs/heads/master caed89321 -> 047a9d92c [SPARK-18576][PYTHON] Add basic TaskContext information to PySpark ## What changes were proposed in this pull request? Adds basic TaskContext information to PySpark. ## How was this patch tested? New unit tests to

spark git commit: [SPARK-18927][SS] MemorySink for StructuredStreaming can't recover from checkpoint if location is provided in SessionConf

2016-12-20 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 cd297c390 -> 3857d5ba8 [SPARK-18927][SS] MemorySink for StructuredStreaming can't recover from checkpoint if location is provided in SessionConf ## What changes were proposed in this pull request? Checkpoint Location can be defined

spark git commit: [SPARK-18927][SS] MemorySink for StructuredStreaming can't recover from checkpoint if location is provided in SessionConf

2016-12-20 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 95c95b71e -> caed89321 [SPARK-18927][SS] MemorySink for StructuredStreaming can't recover from checkpoint if location is provided in SessionConf ## What changes were proposed in this pull request? Checkpoint Location can be defined for a

spark git commit: [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator

2016-12-20 Thread davies
Repository: spark Updated Branches: refs/heads/master 150d26cad -> 95c95b71e [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator ## What changes were proposed in this pull request? There is a timeout failure when using `rdd.toLocalIterator()` or

spark git commit: [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator

2016-12-20 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.1 2971ae564 -> cd297c390 [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator ## What changes were proposed in this pull request? There is a timeout failure when using `rdd.toLocalIterator()`

spark git commit: [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator

2016-12-20 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 2a5ab1490 -> 1f0c5fa75 [SPARK-18281] [SQL] [PYSPARK] Remove timeout for reading data through socket for local iterator ## What changes were proposed in this pull request? There is a timeout failure when using `rdd.toLocalIterator()`

spark git commit: [SPARK-18761][CORE] Introduce "task reaper" to oversee task killing in executors

2016-12-20 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-2.1 f07e989c0 -> 2971ae564 [SPARK-18761][CORE] Introduce "task reaper" to oversee task killing in executors ## What changes were proposed in this pull request? Spark's current task cancellation / task killing mechanism is "best effort"