spark git commit: [SPARK-20855][Docs][DStream] Update the Spark kinesis docs to use the KinesisInputDStream builder instead of deprecated KinesisUtils

2017-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7f295059c -> 4f77c0623 [SPARK-20855][Docs][DStream] Update the Spark kinesis docs to use the KinesisInputDStream builder instead of deprecated KinesisUtils ## What changes were proposed in this pull request? The examples and docs for Spar

spark-website git commit: Fixes git command and description for resolving conflicts

2017-07-25 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site f57f0702b -> efcfd7fc8 Fixes git command and description for resolving conflicts Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/efcfd7fc Tree

spark-website git commit: Describe AppVeyor CI build in "Pull Request" of controbuting guide

2017-07-25 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site efcfd7fc8 -> 6ff5039f3 Describe AppVeyor CI build in "Pull Request" of controbuting guide Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/6ff5

spark git commit: [SPARK-21498][EXAMPLES] quick start -> one py demo have some bug in code

2017-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4f77c0623 -> 996a809c5 [SPARK-21498][EXAMPLES] quick start -> one py demo have some bug in code I find a bug about 'quick start',and created a new issues,Sean Owen let me to make a pull request, and I do ## What changes were proposed in t

spark git commit: [SPARK-21175] Reject OpenBlocks when memory shortage on shuffle service.

2017-07-25 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 996a809c5 -> 799e13161 [SPARK-21175] Reject OpenBlocks when memory shortage on shuffle service. ## What changes were proposed in this pull request? A shuffle service can serves blocks from multiple apps/tasks. Thus the shuffle service can

spark git commit: [SPARK-21383][YARN] Fix the YarnAllocator allocates more Resource

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 799e13161 -> 8de080d9f [SPARK-21383][YARN] Fix the YarnAllocator allocates more Resource When NodeManagers launching Executors, the `missing` value will exceed the real value when the launch is slow, this can lead to YARN allocates more re

spark git commit: [SPARK-21383][YARN] Fix the YarnAllocator allocates more Resource

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 62ca13dca -> e5ec3390c [SPARK-21383][YARN] Fix the YarnAllocator allocates more Resource When NodeManagers launching Executors, the `missing` value will exceed the real value when the launch is slow, this can lead to YARN allocates more

spark git commit: [SPARK-21447][WEB UI] Spark history server fails to render compressed

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 8de080d9f -> 06a979379 [SPARK-21447][WEB UI] Spark history server fails to render compressed inprogress history file in some cases. Add failure handling for EOFException that can be thrown during decompression of an inprogress spark histor

spark git commit: [SPARK-21447][WEB UI] Spark history server fails to render compressed

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 e5ec3390c -> c91191bed [SPARK-21447][WEB UI] Spark history server fails to render compressed inprogress history file in some cases. Add failure handling for EOFException that can be thrown during decompression of an inprogress spark hi

spark git commit: [SPARK-21491][GRAPHX] Enhance GraphX performance: breakOut instead of .toMap

2017-07-25 Thread srowen
Repository: spark Updated Branches: refs/heads/master 06a979379 -> 9b4da7b79 [SPARK-21491][GRAPHX] Enhance GraphX performance: breakOut instead of .toMap ## What changes were proposed in this pull request? `Traversable.toMap` changed to 'collections.breakOut', that eliminates intermediate tu

spark git commit: [SPARK-20586][SQL] Add deterministic to ScalaUDF

2017-07-25 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 9b4da7b79 -> ebc24a9b7 [SPARK-20586][SQL] Add deterministic to ScalaUDF ### What changes were proposed in this pull request? Like [Hive UDFType](https://hive.apache.org/javadocs/r2.0.1/api/org/apache/hadoop/hive/ql/udf/UDFType.html), we s

spark git commit: [SPARK-21494][NETWORK] Use correct app id when authenticating to external service.

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.2 c91191bed -> 1bfd1a83b [SPARK-21494][NETWORK] Use correct app id when authenticating to external service. There was some code based on the old SASL handler in the new auth client that was incorrectly using the SASL user as the user to

spark git commit: [SPARK-21494][NETWORK] Use correct app id when authenticating to external service.

2017-07-25 Thread vanzin
Repository: spark Updated Branches: refs/heads/master ebc24a9b7 -> 300807c6e [SPARK-21494][NETWORK] Use correct app id when authenticating to external service. There was some code based on the old SASL handler in the new auth client that was incorrectly using the SASL user as the user to auth

spark git commit: [SPARK-21517][CORE] Avoid copying memory when transfer chunks remotely

2017-07-25 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 300807c6e -> 16612638f [SPARK-21517][CORE] Avoid copying memory when transfer chunks remotely ## What changes were proposed in this pull request? In our production cluster,oom happens when NettyBlockRpcServer receive OpenBlocks message.Th