spark git commit: [SPARK-16968][SQL][BACKPORT-2.0] Add additional options in jdbc when creating a new table

2017-01-18 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.0 ee4e8faff -> 9fc053c30 [SPARK-16968][SQL][BACKPORT-2.0] Add additional options in jdbc when creating a new table ### What changes were proposed in this pull request? This PR is to backport the PRs

spark git commit: Update known_translations for contributor names

2017-01-18 Thread yhuai
Repository: spark Updated Branches: refs/heads/master fe409f31d -> 0c9231858 Update known_translations for contributor names ## What changes were proposed in this pull request? Update known_translations per https://github.com/apache/spark/pull/16423#issuecomment-269739634 Author: Yin Huai

spark git commit: [SPARK-14975][ML] Fixed GBTClassifier to predict probability per training instance and fixed interfaces

2017-01-18 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master a81e336f1 -> fe409f31d [SPARK-14975][ML] Fixed GBTClassifier to predict probability per training instance and fixed interfaces ## What changes were proposed in this pull request? For all of the classifiers in MLLib we can predict

spark git commit: [SPARK-19182][DSTREAM] Optimize the lock in StreamingJobProgressListener to not block UI when generating Streaming jobs

2017-01-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 569e50680 -> a81e336f1 [SPARK-19182][DSTREAM] Optimize the lock in StreamingJobProgressListener to not block UI when generating Streaming jobs ## What changes were proposed in this pull request? When DStreamGraph is generating a job, it

spark git commit: [SPARK-19168][STRUCTURED STREAMING] StateStore should be aborted upon error

2017-01-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master c050c1227 -> 569e50680 [SPARK-19168][STRUCTURED STREAMING] StateStore should be aborted upon error ## What changes were proposed in this pull request? We should call `StateStore.abort()` when there should be any error before the store is

spark git commit: [SPARK-19168][STRUCTURED STREAMING] StateStore should be aborted upon error

2017-01-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 047506bae -> 4cff0b504 [SPARK-19168][STRUCTURED STREAMING] StateStore should be aborted upon error ## What changes were proposed in this pull request? We should call `StateStore.abort()` when there should be any error before the

spark git commit: [SPARK-19113][SS][TESTS] Ignore StreamingQueryException thrown from awaitInitialization to avoid breaking tests

2017-01-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 77202a6c5 -> 047506bae [SPARK-19113][SS][TESTS] Ignore StreamingQueryException thrown from awaitInitialization to avoid breaking tests ## What changes were proposed in this pull request? #16492 missed one race condition:

spark git commit: [SPARK-19113][SS][TESTS] Ignore StreamingQueryException thrown from awaitInitialization to avoid breaking tests

2017-01-18 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 33791a8ce -> c050c1227 [SPARK-19113][SS][TESTS] Ignore StreamingQueryException thrown from awaitInitialization to avoid breaking tests ## What changes were proposed in this pull request? #16492 missed one race condition:

spark git commit: [SPARK-18113] Use ask to replace askWithRetry in canCommit and make receiver idempotent.

2017-01-18 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 278fa1eb3 -> 33791a8ce [SPARK-18113] Use ask to replace askWithRetry in canCommit and make receiver idempotent. ## What changes were proposed in this pull request? Method canCommit sends AskPermissionToCommitOutput using askWithRetry. If

spark git commit: [SPARK-19231][SPARKR] add error handling for download and untar for Spark release

2017-01-18 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master d06172b88 -> 278fa1eb3 [SPARK-19231][SPARKR] add error handling for download and untar for Spark release ## What changes were proposed in this pull request? When R is starting as a package and it needs to download the Spark release

spark git commit: [SPARK-19231][SPARKR] add error handling for download and untar for Spark release

2017-01-18 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.1 29b954bba -> 77202a6c5 [SPARK-19231][SPARKR] add error handling for download and untar for Spark release ## What changes were proposed in this pull request? When R is starting as a package and it needs to download the Spark release

spark git commit: [SPARK-19223][SQL][PYSPARK] Fix InputFileBlockHolder for datasources which are based on HadoopRDD or NewHadoopRDD

2017-01-18 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f85f29608 -> d06172b88 [SPARK-19223][SQL][PYSPARK] Fix InputFileBlockHolder for datasources which are based on HadoopRDD or NewHadoopRDD ## What changes were proposed in this pull request? For some datasources which are based on

spark git commit: [SPARK-19024][SQL] Implement new approach to write a permanent view

2017-01-18 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 17ce0b5b3 -> f85f29608 [SPARK-19024][SQL] Implement new approach to write a permanent view ## What changes were proposed in this pull request? On CREATE/ALTER a view, it's no longer needed to generate a SQL text string from the

spark git commit: [SPARK-18782][BUILD] Bump Hadoop 2.6 version to use Hadoop 2.6.5

2017-01-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master eefdf9f9d -> 17ce0b5b3 [SPARK-18782][BUILD] Bump Hadoop 2.6 version to use Hadoop 2.6.5 **What changes were proposed in this pull request?** Use Hadoop 2.6.5 for the Hadoop 2.6 profile, I see a bunch of fixes including security ones in

spark git commit: [SPARK-19227][SPARK-19251] remove unused imports and outdated comments

2017-01-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4494cd971 -> eefdf9f9d [SPARK-19227][SPARK-19251] remove unused imports and outdated comments ## What changes were proposed in this pull request? remove ununsed imports and outdated comments, and fix some minor code style issue. ## How