spark git commit: [SPARK-18024][SQL] Introduce an internal commit protocol API

2016-10-31 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7d6c87155 -> d9d146500 [SPARK-18024][SQL] Introduce an internal commit protocol API ## What changes were proposed in this pull request? This patch introduces an internal commit protocol API that is used by the batch data source to do

spark git commit: [SPARK-18167][SQL] Retry when the SQLQuerySuite test flakes

2016-10-31 Thread rxin
Repository: spark Updated Branches: refs/heads/master efc254a82 -> 7d6c87155 [SPARK-18167][SQL] Retry when the SQLQuerySuite test flakes ## What changes were proposed in this pull request? This will re-run the flaky test a few times after it fails. This will help determine if it's due to

spark git commit: [SPARK-18087][SQL] Optimize insert to not require REPAIR TABLE

2016-10-31 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6633b97b5 -> efc254a82 [SPARK-18087][SQL] Optimize insert to not require REPAIR TABLE ## What changes were proposed in this pull request? When inserting into datasource tables with partitions managed by the hive metastore, we need to

spark git commit: [SPARK-18167][SQL] Also log all partitions when the SQLQuerySuite test flakes

2016-10-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/master de3f87fa7 -> 6633b97b5 [SPARK-18167][SQL] Also log all partitions when the SQLQuerySuite test flakes ## What changes were proposed in this pull request? One possibility for this test flaking is that we have corrupted the partition schema

spark git commit: [SPARK-18030][TESTS] Fix flaky FileStreamSourceSuite by not deleting the files

2016-10-31 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 8bfc3b7aa -> de3f87fa7 [SPARK-18030][TESTS] Fix flaky FileStreamSourceSuite by not deleting the files ## What changes were proposed in this pull request? The test `when schema inference is turned on, should read partition data` should

spark git commit: [SPARK-18030][TESTS] Fix flaky FileStreamSourceSuite by not deleting the files

2016-10-31 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 300d596a5 -> e06f43e33 [SPARK-18030][TESTS] Fix flaky FileStreamSourceSuite by not deleting the files ## What changes were proposed in this pull request? The test `when schema inference is turned on, should read partition data`

spark git commit: [SPARK-18143][SQL] Ignore Structured Streaming event logs to avoid breaking history server (branch 2.0)

2016-10-31 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 9f924747d -> 300d596a5 [SPARK-18143][SQL] Ignore Structured Streaming event logs to avoid breaking history server (branch 2.0) ## What changes were proposed in this pull request? Backport #15663 to branch-2.0 and fixed conflicts in

spark git commit: [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate large query plans

2016-10-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 26b07f190 -> 8bfc3b7aa [SPARK-17972][SQL] Add Dataset.checkpoint() to truncate large query plans ## What changes were proposed in this pull request? ### Problem Iterative ML code may easily create query plans that grow exponentially. We

spark git commit: [BUILD] Close stale Pull Requests.

2016-10-31 Thread srowen
Repository: spark Updated Branches: refs/heads/master d2923f173 -> 26b07f190 [BUILD] Close stale Pull Requests. Closes #11610 Closes #15411 Closes #15501 Closes #12613 Closes #12518 Closes #12026 Closes #15524 Closes #12693 Closes #12358 Closes #15588 Closes #15635 Closes #15678 Closes #14699

spark git commit: [SPARK-18143][SQL] Ignore Structured Streaming event logs to avoid breaking history server

2016-10-31 Thread tdas
Repository: spark Updated Branches: refs/heads/master 7c3786929 -> d2923f173 [SPARK-18143][SQL] Ignore Structured Streaming event logs to avoid breaking history server ## What changes were proposed in this pull request? Because of the refactoring work in Structured Streaming, the event logs