spark git commit: Revert "[SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling"

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 d995dac1c -> 3ecef2491 Revert "[SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling" This reverts commit d995dac1cdeec940364453675f59ce5cf2b53684. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commi

spark git commit: [SPARK-20889][SPARKR] Grouped documentation for COLLECTION column methods

2017-06-29 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master fddb63f46 -> 52981715b [SPARK-20889][SPARKR] Grouped documentation for COLLECTION column methods ## What changes were proposed in this pull request? Grouped documentation for column collection methods. Author: actuaryzhang Author: Wayne

spark git commit: [SPARK-20889][SPARKR] Grouped documentation for MISC column methods

2017-06-29 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master e2f32ee45 -> fddb63f46 [SPARK-20889][SPARKR] Grouped documentation for MISC column methods ## What changes were proposed in this pull request? Grouped documentation for column misc methods. Author: actuaryzhang Author: Wayne Zhang Close

spark git commit: [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 083adb07f -> d995dac1c [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling ## What changes were proposed in this pull request? `WindowExec` currently improperly stores complex objects (UnsafeRow, UnsafeArrayData,

spark git commit: [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 d16e2620d -> 8b08fd06c [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling ## What changes were proposed in this pull request? `WindowExec` currently improperly stores complex objects (UnsafeRow, UnsafeArrayData,

spark git commit: [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master cfc696f4a -> e2f32ee45 [SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling ## What changes were proposed in this pull request? `WindowExec` currently improperly stores complex objects (UnsafeRow, UnsafeArrayData, Uns

spark git commit: [SPARK-21253][CORE][HOTFIX] Fix Scala 2.10 build

2017-06-29 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.2 c6ba64793 -> d16e2620d [SPARK-21253][CORE][HOTFIX] Fix Scala 2.10 build ## What changes were proposed in this pull request? A follow up PR to fix Scala 2.10 build for #18472 ## How was this patch tested? Jenkins Author: Shixiong Zhu

spark git commit: [SPARK-21253][CORE][HOTFIX] Fix Scala 2.10 build

2017-06-29 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 88a536bab -> cfc696f4a [SPARK-21253][CORE][HOTFIX] Fix Scala 2.10 build ## What changes were proposed in this pull request? A follow up PR to fix Scala 2.10 build for #18472 ## How was this patch tested? Jenkins Author: Shixiong Zhu C

spark git commit: [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 80f7ac3a6 -> 88a536bab [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8 ## What changes were proposed in this pull request? Please see also https://issues.apache.org/jira/browse/SPARK-21176 This chan

spark git commit: [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 26f4f340c -> 083adb07f [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8 ## What changes were proposed in this pull request? Please see also https://issues.apache.org/jira/browse/SPARK-21176 This

spark git commit: [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 8de67e369 -> c6ba64793 [SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8 ## What changes were proposed in this pull request? Please see also https://issues.apache.org/jira/browse/SPARK-21176 This

spark git commit: [SPARK-21253][CORE] Disable spark.reducer.maxReqSizeShuffleToMem

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 20cf51194 -> 8de67e369 [SPARK-21253][CORE] Disable spark.reducer.maxReqSizeShuffleToMem Disable spark.reducer.maxReqSizeShuffleToMem because it breaks the old shuffle service. Credits to wangyum Closes #18466 Jenkins Author: Shixio

spark git commit: [SPARK-21253][CORE] Disable spark.reducer.maxReqSizeShuffleToMem

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4996c5394 -> 80f7ac3a6 [SPARK-21253][CORE] Disable spark.reducer.maxReqSizeShuffleToMem ## What changes were proposed in this pull request? Disable spark.reducer.maxReqSizeShuffleToMem because it breaks the old shuffle service. Credits t

spark git commit: [SPARK-21253][CORE] Fix a bug that StreamCallback may not be notified if network errors happen

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 17a04b900 -> 20cf51194 [SPARK-21253][CORE] Fix a bug that StreamCallback may not be notified if network errors happen ## What changes were proposed in this pull request? If a network error happens before processing StreamResponse/Stre

spark git commit: [SPARK-21253][CORE] Fix a bug that StreamCallback may not be notified if network errors happen

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f9151bebc -> 4996c5394 [SPARK-21253][CORE] Fix a bug that StreamCallback may not be notified if network errors happen ## What changes were proposed in this pull request? If a network error happens before processing StreamResponse/StreamFa

spark git commit: [SPARK-21188][CORE] releaseAllLocksForTask should synchronize the whole method

2017-06-29 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 18066f2e6 -> f9151bebc [SPARK-21188][CORE] releaseAllLocksForTask should synchronize the whole method ## What changes were proposed in this pull request? Since the objects `readLocksByTask`, `writeLocksByTask` and `info`s are coupled and

spark git commit: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 29bd251dd -> 18066f2e6 [SPARK-21052][SQL] Add hash map metrics to join ## What changes were proposed in this pull request? This adds the average hash map probe metrics to join operator such as `BroadcastHashJoin` and `ShuffledHashJoin`.

spark git commit: [SPARK-21225][CORE] Considering CPUS_PER_TASK when allocating task slots for each WorkerOffer

2017-06-29 Thread wenchen
Repository: spark Updated Branches: refs/heads/master d7da2b94d -> 29bd251dd [SPARK-21225][CORE] Considering CPUS_PER_TASK when allocating task slots for each WorkerOffer JIRA Issue:https://issues.apache.org/jira/browse/SPARK-21225 In the function "resourceOffers", It declare a variable "

spark git commit: [SPARK-21135][WEB UI] On history server page，duration of incompleted applications should be hidden instead of showing up as 0

2017-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master d106a74c5 -> d7da2b94d [SPARK-21135][WEB UI] On history server page，duration of incompleted applications should be hidden instead of showing up as 0 ## What changes were proposed in this pull request? Hide duration of incompleted applic

spark git commit: [SPARK-21240] Fix code style for constructing and stopping a SparkContext in UT.

2017-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master 70085e83d -> d106a74c5 [SPARK-21240] Fix code style for constructing and stopping a SparkContext in UT. ## What changes were proposed in this pull request? Same with SPARK-20985. Fix code style for constructing and stopping a `SparkContext

spark git commit: [SPARK-21210][DOC][ML] Javadoc 8 fixes for ML shared param traits

2017-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master a2d562354 -> 70085e83d [SPARK-21210][DOC][ML] Javadoc 8 fixes for ML shared param traits PR #15999 included fixes for doc strings in the ML shared param traits (occurrences of `>` and `>=`). This PR simply uses the HTML-escaped version of

spark git commit: [SPARK-21210][DOC][ML] Javadoc 8 fixes for ML shared param traits

2017-06-29 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.2 970f68c05 -> 17a04b900 [SPARK-21210][DOC][ML] Javadoc 8 fixes for ML shared param traits PR #15999 included fixes for doc strings in the ML shared param traits (occurrences of `>` and `>=`). This PR simply uses the HTML-escaped versio

spark git commit: [SPARK-20889][SPARKR] Grouped documentation for NONAGGREGATE column methods

2017-06-29 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 9f6b3e65c -> a2d562354 [SPARK-20889][SPARKR] Grouped documentation for NONAGGREGATE column methods ## What changes were proposed in this pull request? Grouped documentation for nonaggregate column methods. Author: actuaryzhang Author: Wa