[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16760248 --- Diff: core/src/main/scala/org/apache/spark/shuffle/hash/HashShuffleManager.scala --- @@ -25,6 +25,9 @@ import org.apache.spark.shuffle._ * mapper (poss

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16760213 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16760122 --- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala --- @@ -87,4 +99,12 @@ class CacheManagerSuite extends FunSuite with BeforeAndAfter

[GitHub] spark pull request: Added support for :cp that was broken in...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1929#issuecomment-53533093 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19293/consoleFull) for PR 1929 at commit [`f420cbf`](https://github.com/a

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16760052 --- Diff: core/src/main/scala/org/apache/spark/ui/storage/StorageTab.scala --- @@ -70,8 +70,10 @@ class StorageListener(storageStatusListener: StorageStat

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16760036 --- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala --- @@ -68,7 +68,9 @@ private[spark] class CacheManager(blockManager: BlockManager) exten

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16760032 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16760004 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-3241][SQL] create NumberFormat instance...

2014-08-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2157#issuecomment-53532690 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759963 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread andrewor14
Github user andrewor14 closed the pull request at: https://github.com/apache/spark/pull/2156 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759951 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-3241][SQL] create NumberFormat instance...

2014-08-26 Thread baishuo
GitHub user baishuo opened a pull request: https://github.com/apache/spark/pull/2157 [SPARK-3241][SQL] create NumberFormat instance by threadsafe way You can merge this pull request into a Git repository by running: $ git pull https://github.com/baishuo/spark patch-threadlocal

[GitHub] spark pull request: [SPARK-3243] Don't use stale spark-driver.* sy...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2154#issuecomment-53532443 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19290/consoleFull) for PR 2154 at commit [`17ec6fc`](https://github.com/a

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53532419 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19291/consoleFull) for PR 2129 at commit [`881a8f0`](https://github.com/a

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759871 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759807 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759788 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759777 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockManager.scala --- @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Found

[GitHub] spark pull request: [SPARK-3239] [PySpark] randomize the dirs for ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2152#issuecomment-53532166 QA tests have started for PR 2152. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19297/consoleFull --- If

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759699 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleBlockManager.scala --- @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759644 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -121,20 +129,16 @@ private[spark] class DiskStore(blockManager: BlockManager, diskM

[GitHub] spark pull request: [SPARK-3239] [PySpark] randomize the dirs for ...

2014-08-26 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/2152#issuecomment-53531754 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759577 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -89,25 +89,33 @@ private[spark] class DiskStore(blockManager: BlockManager, diskMan

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1241#issuecomment-53531388 Thanks for doing this. To help with the review, can you write a short design doc discussing the interfaces between different components, similar to the one attached here htt

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759331 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -89,25 +89,33 @@ private[spark] class DiskStore(blockManager: BlockManager, diskMan

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759291 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -573,7 +634,7 @@ private[spark] class BlockManager( serializer: Seria

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759276 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -573,7 +634,7 @@ private[spark] class BlockManager( serializer: Seria

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759266 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -107,6 +153,11 @@ private[spark] class BlockManager( } }

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759256 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -85,18 +87,62 @@ private[spark] class BlockManager( val tachyonStorePat

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759235 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -34,8 +34,11 @@ import org.apache.spark.io.CompressionCodec import org.apac

[GitHub] spark pull request: [SPARK-2805] akka 2.3.4

2014-08-26 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1685#issuecomment-53530947 This looks good to me, given automated tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-1812] Enable cross build for scala 2.11...

2014-08-26 Thread ScrapCodes
Github user ScrapCodes closed the pull request at: https://github.com/apache/spark/pull/996 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759141 --- Diff: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala --- @@ -17,14 +17,18 @@ package org.apache.spark.shuffle.sort

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53530699 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19296/consoleFull) for PR 2143 at commit [`9c84202`](https://github.com/ap

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759034 --- Diff: core/src/main/scala/org/apache/spark/shuffle/FileShuffleBlockManager.scala --- @@ -15,21 +15,24 @@ * limitations under the License. */

[GitHub] spark pull request: [SPARK-2288] Hide ShuffleBlockManager behind S...

2014-08-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1241#discussion_r16759047 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleBlockManager.scala --- @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53530257 The failed unit test is unrelated to this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53530260 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-53530198 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19287/consoleFull) for PR 1846 at commit [`ae1e072`](https://github.com/a

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53530146 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19289/consoleFull) for PR 2143 at commit [`9c84202`](https://github.com/a

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/2156 [SPARK-3167] Handle special driver configs in Windows (Branch 1.1) This is an effort to bring the Windows scripts up to speed after recent splashing changes in #1845. You can merge this pull req

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2065#issuecomment-53530019 This now looks quite good to me. There were a few more formatting issues, should take 5 minutes to solve :) --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758820 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeBatchFetcher.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2155#issuecomment-53529888 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19292/consoleFull) for PR 2155 at commit [`bec42d2`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-26 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2155#issuecomment-53529874 This is a follow up of #2114 after discussing with @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Added support for :cp that was broken in...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1929#issuecomment-53529843 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19293/consoleFull) for PR 1929 at commit [`f420cbf`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758798 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeBatchFetcher.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758766 --- Diff: external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeBatchFetcher.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to the Apache So

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758756 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/TransactionProcessor.scala --- @@ -225,4 +237,16 @@ private class Transaction

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758729 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -63,18 +68,34 @@ private[flume] class S

[GitHub] spark pull request: [SPARK-3198] [SQL] Remove the TreeNode.id

2014-08-26 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/2155 [SPARK-3198] [SQL] Remove the TreeNode.id Thus id property of the TreeNode API does save time in a faster way to compare 2 TreeNodes, it is kind of performance bottleneck during the expressio

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758724 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -45,7 +47,8 @@ private[flume] class Spa

[GitHub] spark pull request: [SPARK-3224] FetchFailed reduce stages should ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2127#issuecomment-53529562 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19288/consoleFull) for PR 2127 at commit [`effb1ce`](https://github.com/a

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/2065#discussion_r16758698 --- Diff: external/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala --- @@ -63,18 +68,34 @@ private[flume] class S

[GitHub] spark pull request: Added support for :cp that was broken in...

2014-08-26 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1929#issuecomment-53529460 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2129 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2871] [PySpark] add RDD.lookup(key)

2014-08-26 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/2093#issuecomment-53529444 @davies Thanks, just saw your patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-2845] Add timestamps to block manager e...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/654#issuecomment-53529354 @vanzin Hey I've been busy with the 1.1 blockers but after that I'll take another look at this one --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53529332 Okay I'm gonna merge this - thanks Andrew. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53529009 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19291/consoleFull) for PR 2129 at commit [`881a8f0`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3243] Don't use stale spark-driver.* sy...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2154#issuecomment-53528999 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19290/consoleFull) for PR 2154 at commit [`17ec6fc`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16758471 --- Diff: core/src/main/scala/org/apache/spark/ui/storage/StorageTab.scala --- @@ -70,8 +70,10 @@ class StorageListener(storageStatusListener: StorageStat

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16758470 --- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala --- @@ -87,4 +99,12 @@ class CacheManagerSuite extends FunSuite with BeforeAndAfter

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16758476 --- Diff: core/src/main/scala/org/apache/spark/ui/storage/StorageTab.scala --- @@ -70,8 +70,10 @@ class StorageListener(storageStatusListener: StorageStat

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2131#discussion_r16758468 --- Diff: core/src/test/scala/org/apache/spark/ui/storage/StorageTabSuite.scala --- @@ -162,4 +163,29 @@ class StorageTabSuite extends FunSuite with Befor

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53528924 Looks like this failed one of your own tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-3243] Don't use stale spark-driver.* sy...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2154#issuecomment-53528855 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53528831 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3197] [SQL] Reduce the Expression tree ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2113#issuecomment-53528690 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19285/consoleFull) for PR 2113 at commit [`db40395`](https://github.com/a

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2129#discussion_r16758263 --- Diff: python/pyspark/java_gateway.py --- @@ -69,6 +70,22 @@ def preexec_func(): error_msg += "

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2129#discussion_r16758239 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitDriverBootstrapper.scala --- @@ -133,17 +133,24 @@ private[spark] object SparkSubmitDrive

[GitHub] spark pull request: Dimension Independent Matrix Square Using MapR...

2014-08-26 Thread rezazadeh
Github user rezazadeh closed the pull request at: https://github.com/apache/spark/pull/336 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request: Dimension Independent Matrix Square Using MapR...

2014-08-26 Thread rezazadeh
Github user rezazadeh commented on the pull request: https://github.com/apache/spark/pull/336#issuecomment-53528432 Moved to https://github.com/apache/spark/pull/1778 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53527919 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19289/consoleFull) for PR 2143 at commit [`9c84202`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2143#discussion_r16757841 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -76,6 +76,20 @@ private[spark] class ContextCleaner(sc: SparkContext) extends Logg

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53527699 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-3224] FetchFailed reduce stages should ...

2014-08-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2127 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-3243] Don't use stale spark-driver.* sy...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2154#issuecomment-53527608 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19286/consoleFull) for PR 2154 at commit [`0140836`](https://github.com/a

[GitHub] spark pull request: [SPARK-3138][SQL] sqlContext.parquetFile shoul...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2044#issuecomment-53527605 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19281/consoleFull) for PR 2044 at commit [`4ae477f`](https://github.com/a

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2065#issuecomment-53527293 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19283/consoleFull) for PR 2065 at commit [`d7427cc`](https://github.com/a

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53527156 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19284/consoleFull) for PR 2131 at commit [`56ea488`](https://github.com/a

[GitHub] spark pull request: [SPARK-3224] FetchFailed reduce stages should ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2127#issuecomment-53527001 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19288/consoleFull) for PR 2127 at commit [`effb1ce`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3154][STREAMING] Make FlumePollingInput...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2065#issuecomment-53526713 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19280/consoleFull) for PR 2065 at commit [`a0a8852`](https://github.com/a

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-53526598 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19287/consoleFull) for PR 1846 at commit [`ae1e072`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3224] FetchFailed reduce stages should ...

2014-08-26 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2127#discussion_r16757427 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1045,21 +1045,19 @@ class DAGScheduler( stage.pendingTasks +

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53526552 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19279/consoleFull) for PR 2129 at commit [`22b1acd`](https://github.com/a

[GitHub] spark pull request: [SPARK-3224] FetchFailed reduce stages should ...

2014-08-26 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/2127#discussion_r16757359 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1045,21 +1045,19 @@ class DAGScheduler( stage.pendingTa

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2129#issuecomment-53526021 Hey @andrewor14 - a few minor comments, but otherwise LGTM. At this point you are the foremost export on windows scripting, so can't add much value reviewing that. ---

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2129#discussion_r16757138 --- Diff: python/pyspark/java_gateway.py --- @@ -69,6 +70,22 @@ def preexec_func(): error_msg += "--

[GitHub] spark pull request: [SPARK-3167] Handle special driver configs in ...

2014-08-26 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2129#discussion_r16757019 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitDriverBootstrapper.scala --- @@ -133,17 +133,24 @@ private[spark] object SparkSubmitDriverB

[GitHub] spark pull request: [SPARK-3139] Made ContextCleaner to not block ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2143#issuecomment-53525603 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19276/consoleFull) for PR 2143 at commit [`9c84202`](https://github.com/a

[GitHub] spark pull request: [SPARK-3243] Don't use stale spark-driver.* sy...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2154#issuecomment-53525556 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19286/consoleFull) for PR 2154 at commit [`0140836`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3012] Standardized Distance Functions b...

2014-08-26 Thread yu-iskw
Github user yu-iskw commented on the pull request: https://github.com/apache/spark/pull/1964#issuecomment-53525367 @mengxr, Will you please review the detail on this pull request. Thank you for your trouble in advance. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-3197] [SQL] Reduce the Expression tree ...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2113#issuecomment-53525340 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19285/consoleFull) for PR 2113 at commit [`db40395`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3243] Don't use stale `spark-driver.*` ...

2014-08-26 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/2154 [SPARK-3243] Don't use stale `spark-driver.*` system properties If we set both `spark.driver.extraClassPath` and `--driver-class-path`, then the latter correctly overrides the former. However, th

[GitHub] spark pull request: [SPARK-3197] [SQL] Reduce the Expression tree ...

2014-08-26 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2113#issuecomment-53525154 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53525126 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19284/consoleFull) for PR 2131 at commit [`56ea488`](https://github.com/ap

[GitHub] spark pull request: [SPARK-3170][CORE][BUG]:RDD info loss in "Stor...

2014-08-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2131#issuecomment-53525012 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-2608][Core] Fixed command line option p...

2014-08-26 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/2145#discussion_r16756703 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala --- @@ -18,19 +18,21 @@ package org.apache.spa

  1   2   3   4   5   6   >