spark git commit: [SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation.

2017-07-21 Thread holden
Repository: spark Updated Branches: refs/heads/branch-2.2 88dccda39 -> da403b953 [SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation. Update the Quickstart and RDD programming guides to mention pip. Built docs locally. Author: Holden Karau Closes #18698 from

spark git commit: [SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation.

2017-07-21 Thread holden
Repository: spark Updated Branches: refs/heads/master 113399b8b -> cc00e99d5 [SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation. ## What changes were proposed in this pull request? Update the Quickstart and RDD programming guides to mention pip. ## How was this patch tested? Built

spark git commit: [SPARK-21243][CORE] Limit no. of map outputs in a shuffle fetch

2017-07-21 Thread tgraves
Repository: spark Updated Branches: refs/heads/branch-2.2 9949fed1c -> 88dccda39 [SPARK-21243][CORE] Limit no. of map outputs in a shuffle fetch For configurations with external shuffle enabled, we have observed that if a very large no. of blocks are being fetched from a remote host, it puts

spark git commit: [SPARK-19810][BUILD][FOLLOW-UP] jcl-over-slf4j dependency needs to be compile scope for SBT build

2017-07-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 2f1468429 -> 113399b8b [SPARK-19810][BUILD][FOLLOW-UP] jcl-over-slf4j dependency needs to be compile scope for SBT build ## What changes were proposed in this pull request? jcl-over-slf4j dependency needs to be compile scope for SBT

spark git commit: [SPARK-21472][SQL][FOLLOW-UP] Introduce ArrowColumnVector as a reader for Arrow vectors.

2017-07-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master c57dfaef3 -> 2f1468429 [SPARK-21472][SQL][FOLLOW-UP] Introduce ArrowColumnVector as a reader for Arrow vectors. ## What changes were proposed in this pull request? This is a follow-up of #18680. In some environment, a compile error

spark git commit: [MINOR][SS][DOCS] Minor doc change for kafka integration

2017-07-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master 3ac609308 -> c57dfaef3 [MINOR][SS][DOCS] Minor doc change for kafka integration ## What changes were proposed in this pull request? Minor change to kafka integration document for structured streaming. ## How was this patch tested? N/A,