[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/11659 [SPARK-13830] prefer block manager than direct result for large result ## What changes were proposed in this pull request? The current RPC can't handle large blocks very well, it's very slow

[GitHub] spark pull request: [SPARK-13207] [SQL] Make partitioning discover...

2016-03-11 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11088#issuecomment-195522100 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11646#issuecomment-195522580 **[Test build #52939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52939/consoleFull)** for PR 11646 at commit [`65596df`](https://gi

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11659#issuecomment-195522563 **[Test build #52938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52938/consoleFull)** for PR 11659 at commit [`e982656`](https://gi

[GitHub] spark pull request: [SPARK-13207] [SQL] Make partitioning discover...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11088#issuecomment-195522607 **[Test build #52940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52940/consoleFull)** for PR 11088 at commit [`2460716`](https://gi

[GitHub] spark pull request: [SPARK-10907][SPARK-6157][WIP] Remove pendingU...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195524921 **[Test build #52941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52941/consoleFull)** for PR 11613 at commit [`ebb25f9`](https://gi

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/11646#discussion_r55881083 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -454,19 +454,60 @@ trait FileFormat { requiredColumns: Arr

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11646#issuecomment-195530198 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11646#issuecomment-195530204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11646#issuecomment-195530072 **[Test build #52939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52939/consoleFull)** for PR 11646 at commit [`65596df`](https://g

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55883199 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195534728 @andrewor14 @nongli, this should now be ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195536492 **[Test build #52942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52942/consoleFull)** for PR 11613 at commit [`dbca8cf`](https://gi

[GitHub] spark pull request: [SPARK-13578] [core] Modify launch scripts to ...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11591#issuecomment-195539011 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13578] [core] Modify launch scripts to ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11591#issuecomment-195541177 **[Test build #52943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52943/consoleFull)** for PR 11591 at commit [`79e1bf5`](https://gi

[GitHub] spark pull request: [SPARK-13576] Don't create assembly for exampl...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11452#issuecomment-195542456 **[Test build #52933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52933/consoleFull)** for PR 11452 at commit [`fda639b`](https://g

[GitHub] spark pull request: [SPARK-13576] Don't create assembly for exampl...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11452#issuecomment-195543244 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13576] Don't create assembly for exampl...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11452#issuecomment-195543238 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13653] Split disk writer into separate ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11498#issuecomment-195543538 **[Test build #52944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52944/consoleFull)** for PR 11498 at commit [`8b8f4cb`](https://gi

[GitHub] spark pull request: [SPARK-13811] [SQL] No Push-Down of Constraint...

2016-03-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195544498 I guess my question is why we are doing this for filters at all. It makes sense for joins, but perhaps we should just eliminate that part of the rule. --- If your pr

[GitHub] spark pull request: [SPARK-13653] Split disk writer into separate ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11498#issuecomment-195544738 **[Test build #52945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52945/consoleFull)** for PR 11498 at commit [`c04d3d7`](https://gi

[GitHub] spark pull request: [SPARK-XXXX][SQL] Add a strategy for planning ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11646#issuecomment-195547936 **[Test build #52946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52946/consoleFull)** for PR 11646 at commit [`35be8d5`](https://gi

[GitHub] spark pull request: [SPARK-13653] Split disk writer into separate ...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11498#issuecomment-195549353 **[Test build #52947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52947/consoleFull)** for PR 11498 at commit [`9fdde9f`](https://gi

[GitHub] spark pull request: [SPARK-13810] [CORE] Add Port Configuration Su...

2016-03-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11644#issuecomment-195549744 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55888190 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/11615#discussion_r55888314 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -270,9 +270,25 @@ private[spark] object JettyUtils extends Logging {

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11573#discussion_r55888485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala --- @@ -64,10 +83,86 @@ private[sql] class SparkQl(conf: ParserConf = SimplePar

[GitHub] spark pull request: [SPARK-13810] [CORE] Add Port Configuration Su...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11644#issuecomment-195551694 **[Test build #52948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52948/consoleFull)** for PR 11644 at commit [`8177e69`](https://gi

[GitHub] spark pull request: [SPARK-7425] [ML] spark.ml Predictor should su...

2016-03-11 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/10355#issuecomment-195557577 @MLnick you were right, thanks for spotting that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11573#discussion_r55889875 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala --- @@ -64,10 +83,86 @@ private[sql] class SparkQl(conf: ParserConf = SimplePar

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11659#discussion_r55889878 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -97,9 +97,9 @@ private[spark] class Executor( // Set the classloader f

[GitHub] spark pull request: [SPARK-13776][WebUI]Limit the max number of ac...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11615#issuecomment-195560080 **[Test build #52949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52949/consoleFull)** for PR 11615 at commit [`e772aed`](https://gi

[GitHub] spark pull request: SPARK-13779: Avoid cancelling non-local contai...

2016-03-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11612#discussion_r55891228 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -265,25 +265,52 @@ private[yarn] class YarnAllocator( // For

[GitHub] spark pull request: [SPARK-13396] Stop using our internal deprecat...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11544#issuecomment-195563762 **[Test build #52934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52934/consoleFull)** for PR 11544 at commit [`228e078`](https://g

[GitHub] spark pull request: [SPARK-13396] Stop using our internal deprecat...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11544#issuecomment-195564066 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13626] [core] Avoid duplicate config de...

2016-03-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11510#issuecomment-195564195 I assume no one has any problems with this, so I'll merge when tests pass again. retest this please --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-13396] Stop using our internal deprecat...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11544#issuecomment-195564067 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13811] [SQL] No Push-Down of Constraint...

2016-03-11 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195564292 @marmbrus Are you saying we should not add null filters for `Filter`? https://github.com/apache/spark/pull/11372 The original PR shows it can benefit some

[GitHub] spark pull request: [SPARK-529] [sql] Modify SQLConf to use new co...

2016-03-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11570#issuecomment-195564408 Ping anyone? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-13207] [SQL] Make partitioning discover...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11088#issuecomment-195564366 **[Test build #52940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52940/consoleFull)** for PR 11088 at commit [`2460716`](https://g

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11659#issuecomment-195564685 **[Test build #52950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52950/consoleFull)** for PR 11659 at commit [`142b43e`](https://gi

[GitHub] spark pull request: [SPARK-13207] [SQL] Make partitioning discover...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11088#issuecomment-195564715 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13626] [core] Avoid duplicate config de...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11510#issuecomment-195564730 **[Test build #52951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52951/consoleFull)** for PR 11510 at commit [`d7f1ffa`](https://gi

[GitHub] spark pull request: [SPARK-13207] [SQL] Make partitioning discover...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11088#issuecomment-195564713 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11573#discussion_r55892525 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala --- @@ -29,7 +29,26 @@ private[sql] class SparkQl(conf: ParserConf = SimplePars

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195565806 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195565781 cc @yhuai @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11573#discussion_r55892958 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AlterTableCommandParser.scala --- @@ -0,0 +1,428 @@ +/* + * Licensed to the A

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/11573#discussion_r55893285 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala --- @@ -29,7 +29,26 @@ private[sql] class SparkQl(conf: ParserConf = SimplePars

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195567000 **[Test build #52952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52952/consoleFull)** for PR 11208 at commit [`e060dea`](https://gi

[GitHub] spark pull request: [Spark-13814][PySpark] Delete unnecessary impo...

2016-03-11 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11651#issuecomment-195567314 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [Spark-13814][PySpark] Delete unnecessary impo...

2016-03-11 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11651#issuecomment-195567401 Merging this into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195569065 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195569069 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-195569043 **[Test build #52952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52952/consoleFull)** for PR 11208 at commit [`e060dea`](https://g

[GitHub] spark pull request: [SPARK-13139][SQL] Parse Hive DDL commands our...

2016-03-11 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11573#issuecomment-195570543 I have left a few comments. It is a good starting point. Thank you for working on this! --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195570778 **[Test build #52935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52935/consoleFull)** for PR 11613 at commit [`858aea2`](https://g

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195571056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195571052 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55895725 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -520,3 +447,42 @@ private[spark] class MemoryStore( logMem

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55896680 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -793,7 +798,12 @@ private[spark] class BlockManager( // We w

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55896316 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -520,3 +447,42 @@ private[spark] class MemoryStore( logMem

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55897125 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11659#issuecomment-195574451 **[Test build #52938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52938/consoleFull)** for PR 11659 at commit [`e982656`](https://g

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55896526 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -717,8 +717,13 @@ private[spark] class BlockManager( level: St

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11659#issuecomment-195575304 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55897629 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-XXXXX] Guard against race condition whe...

2016-03-11 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/11660 [SPARK-X] Guard against race condition when re-caching disk blocks in memory When reading data from the DiskStore and attempting to cache it back into the memory store, we should guard again

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55897743 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -520,3 +447,42 @@ private[spark] class MemoryStore( logMemo

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55897722 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -520,3 +447,42 @@ private[spark] class MemoryStore( logMemo

[GitHub] spark pull request: [SPARK-13830] prefer block manager than direct...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11659#issuecomment-195575309 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13833] Guard against race condition whe...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11660#issuecomment-195576832 **[Test build #52953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52953/consoleFull)** for PR 11660 at commit [`5342712`](https://gi

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55897952 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-12583][Mesos] Mesos shuffle service: Do...

2016-03-11 Thread tnachen
Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/11272#issuecomment-195576917 @andrewor14 I'll try to create a test to verify this, when is the 2.0 closing date? --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898084 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898188 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898378 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13811] [SQL] No Push-Down of Constraint...

2016-03-11 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195579496 In this commit, three parts are done: - removed the `IsNotNull` constraints for compound expressions - added `IsNotNull` constraints support for all the `Binar

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195581011 **[Test build #52941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52941/consoleFull)** for PR 11613 at commit [`ebb25f9`](https://g

[GitHub] spark pull request: [SPARK-13811] [SQL] No Push-Down of Constraint...

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195580608 **[Test build #52954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52954/consoleFull)** for PR 11649 at commit [`baa2cda`](https://gi

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195582626 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898638 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11613#issuecomment-195582619 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898608 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898706 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13833] Guard against race condition whe...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11660#issuecomment-195583417 /cc @andrewor14. In addition to fixing that race-condition, this patch also addresses that duplicated code that you pointed out in a review of my previous PR. --- I

[GitHub] spark pull request: [SPARK-12313] [SQL] [RFC] use alt metastore pa...

2016-03-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/11607#discussion_r55898782 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -346,14 +376,18 @@ private[client] class Shim_v0_13 extends Shim_v0_1

[GitHub] spark pull request: [SPARK-12313] [SQL] [RFC] use alt metastore pa...

2016-03-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/11607#issuecomment-195583935 This makes sense to me if it helps with the original issue; did you get a chance to try that out? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898874 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898848 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55898978 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13834] Update sbt for 2.x.

2016-03-11 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/11661 [SPARK-13834] Update sbt for 2.x. ## What changes were proposed in this pull request? For 2.0.0, we had better bump `sbt`, too. SBT 0.13.11 fixes wrong warnings and improve increment

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55899254 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13811] [SQL] Removed IsNotNull Constrai...

2016-03-11 Thread sameeragarwal
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195585134 @marmbrus by adding additional `isNotNull` checks, we are hoping to generate code that can quickly short circuit the filter condition checks in the generated code

[GitHub] spark pull request: [SPARK-13834] Update sbt for 2.x.

2016-03-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11661#issuecomment-195586138 **[Test build #52955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52955/consoleFull)** for PR 11661 at commit [`e3e62ba`](https://gi

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55899657 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13811][SPARK-13836] [SQL] Removed IsNot...

2016-03-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195588783 I see. We should probably comment on that in `NullFiltering` if that is the intention. Anytime there are implicit contracts between components like this it makes thi

[GitHub] spark pull request: [SPARK-10907][SPARK-6157] Remove pendingUnroll...

2016-03-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11613#discussion_r55900156 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -100,48 +92,136 @@ private[spark] class MemoryStore( */

[GitHub] spark pull request: [SPARK-13811][SPARK-13836] [SQL] Removed IsNot...

2016-03-11 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/11649#issuecomment-195589358 Also, does codegen really need us to add redundant expressions into the query plan? Can't it just look at the constraints and pick those that are useful for it? ---

[GitHub] spark pull request: [Spark-13814][PySpark] Delete unnecessary impo...

2016-03-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11651 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

<    1   2   3   4   5   6   7   >