[GitHub] spark pull request: [SPARK-9104][SPARK-9105][SPARK-9106][SPARK-910...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7753#issuecomment-153218542 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153217910 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153217626 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153217627 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11424] Guard against double-close() of ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9423#issuecomment-153217594 [Test build #44852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44852/consoleFull) for PR 9423 at commit [`cf008ca`](https://git

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153216201 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153216204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153216140 **[Test build #44849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44849/consoleFull)** for PR 9393 at commit [`d2ee5ee`](https://git

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-02 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9206#discussion_r43709190 --- Diff: network/common/src/main/java/org/apache/spark/network/client/StreamInterceptor.java --- @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9404#discussion_r43709146 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Exchange.scala --- @@ -194,12 +194,13 @@ case class Exchange(newPartitioning: Partitioning, ch

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9404#discussion_r43709117 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -353,4 +354,62 @@ class CachedTableSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9393#issuecomment-153215454 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43709078 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala --- @@ -381,7 +381,7 @@ class ParquetIOSuite exte

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153215419 **[Test build #44864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44864/consoleFull)** for PR 9422 at commit [`8a0ae14`](https://gith

[GitHub] spark pull request: [SPARK-11235] [network] Add ability to stream ...

2015-11-02 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9206#discussion_r43709051 --- Diff: network/common/src/main/java/org/apache/spark/network/client/StreamInterceptor.java --- @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Sof

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153214490 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43708876 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala --- @@ -381,7 +381,7 @@ class ParquetIOSuite extends

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153214517 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9428#issuecomment-153214364 **[Test build #44863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44863/consoleFull)** for PR 9428 at commit [`647162f`](https://gith

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9404#discussion_r43708751 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Exchange.scala --- @@ -229,12 +230,33 @@ private[sql] case class EnsureRequirements(sqlContext

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43708749 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetIOSuite.scala --- @@ -381,7 +381,7 @@ class ParquetIOSuite exte

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9428#issuecomment-153213809 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153213799 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153213800 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9428#issuecomment-153213792 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153213662 **[Test build #44846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44846/consoleFull)** for PR 9344 at commit [`f976faa`](https://git

[GitHub] spark pull request: [SPARK-9516][UI] Improvement of Thread Dump Pa...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7910#issuecomment-153213648 **[Test build #44861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44861/consoleFull)** for PR 7910 at commit [`6106c7f`](https://gith

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread zsxwing
Github user zsxwing closed the pull request at: https://github.com/apache/spark/pull/9258 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9428#issuecomment-153213596 /cc @andrewor14 @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-8582][Core]Optimize checkpointing to av...

2015-11-02 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9258#issuecomment-153213531 @andrewor14 I just opened #9428 to take over #7021 and fix potential issues in the previous PR. --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: Pr7021

2015-11-02 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/9428 Pr7021 Took over #7021 and fixed the following potential issues in the previous PR: 1. Make sure checkpointing all data even if the Iterator is not drained. 2. Make sure checkpointing al

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153213191 **[Test build #44862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44862/consoleFull)** for PR 9343 at commit [`886acc7`](https://gith

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153212674 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9343#issuecomment-153212605 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9104][SPARK-9105][SPARK-9106][SPARK-910...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7753#issuecomment-153212301 **[Test build #44860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44860/consoleFull)** for PR 7753 at commit [`f2f0e64`](https://gith

[GitHub] spark pull request: [SPARK-9516][UI] Improvement of Thread Dump Pa...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7910#issuecomment-153211864 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9516][UI] Improvement of Thread Dump Pa...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7910#issuecomment-153211850 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9516][UI] Improvement of Thread Dump Pa...

2015-11-02 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/7910#discussion_r43707691 --- Diff: core/src/main/resources/org/apache/spark/ui/static/table.js --- @@ -30,3 +30,75 @@ function stripeSummaryTable() { } });

[GitHub] spark pull request: [SPARK-9104][SPARK-9105][SPARK-9106][SPARK-910...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7753#issuecomment-153211298 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9104][SPARK-9105][SPARK-9106][SPARK-910...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7753#issuecomment-153211274 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153211178 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153211177 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-15322 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153211087 **[Test build #44857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44857/consoleFull)** for PR 9426 at commit [`aa60ae4`](https://git

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-15326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9344#issuecomment-153211013 **[Test build #44844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44844/consoleFull)** for PR 9344 at commit [`b0d569d`](https://git

[GitHub] spark pull request: [SPARK-11389][CORE] Add support for off-heap m...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9344#discussion_r43707433 --- Diff: core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java --- @@ -342,18 +361,25 @@ public long cleanUpAllAllocatedMemory() { synchr

[GitHub] spark pull request: [SPARK-10181][SQL] Do kerberos login for crede...

2015-11-02 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/9272#discussion_r43707087 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -520,6 +520,8 @@ object SparkSubmit { } if (args.principal

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153210300 [Test build #44859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44859/consoleFull) for PR 9427 at commit [`c897b75`](https://gith

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153210174 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153210196 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153208863 [Test build #44858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44858/consoleFull) for PR 9427 at commit [`46ef394`](https://gith

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153208742 @pwendell @yhuai Thanks for the pointers and reviewing this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9395 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [DOC] Missing link to R DataFrame API doc

2015-11-02 Thread Lewuathe
Github user Lewuathe commented on the pull request: https://github.com/apache/spark/pull/9394#issuecomment-153208639 @shivaram Sure. I'll submit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153208299 Got it- thanks I can merge it then. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Update Netty's call to PooledByteBufAllocator

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9389#issuecomment-153208265 Ping @slachiewicz, do you mind either addressing Sean's comments or closing this PR? --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43705834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpres

[GitHub] spark pull request: [SPARK-10181][SQL] Do kerberos login for crede...

2015-11-02 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/9272#discussion_r43705844 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -520,6 +520,8 @@ object SparkSubmit { } if (args.principal

[GitHub] spark pull request: [SPARK-11329] [SQL] Support star expansion for...

2015-11-02 Thread nongli
Github user nongli commented on a diff in the pull request: https://github.com/apache/spark/pull/9343#discussion_r43705776 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -166,26 +166,63 @@ abstract class Star extends LeafExpres

[GitHub] spark pull request: [SPARK-11432][GraphX] Personalized PageRank sh...

2015-11-02 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/9386#issuecomment-153207973 This is actually a pretty serious error since it could lead to mass being accumulated on unreachable sub-graphs. The performance implications of the above branch shoul

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/9260 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9260#issuecomment-153207836 @yhuai I think it's pretty low-risk. I'm going to close this PR and will continue work over at #9427, which is opened against branch-1.5. --- If your project is set u

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153207783 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9427#issuecomment-153207799 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9427 [SPARK-11293] Fix shuffle memory leaks in Spillable collections and UnsafeShuffleWriter (branch-1.5) This patch fixes multiple memory leaks in `Spillable` collections, as well as a leak in `Unsaf

[GitHub] spark pull request: [SPARK-11432][GraphX] Personalized PageRank sh...

2015-11-02 Thread dwmclary
Github user dwmclary commented on the pull request: https://github.com/apache/spark/pull/9386#issuecomment-153207454 Yes, I agree, it shouldn't add overhead. Sent from my iPhone > On Nov 2, 2015, at 4:35 PM, DB Tsai wrote: > > I think the branching should be

[GitHub] spark pull request: [SPARK-11275][SQL][WIP] Rollup and Cube Genera...

2015-11-02 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9419#issuecomment-153207195 @gatorsmile thanks for working on this fixing, can you also add the unit test? --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153207007 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153207003 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153206930 **[Test build #44854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44854/consoleFull)** for PR 9422 at commit [`5c0da8e`](https://git

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153206695 @pwendell We changed the default hadoop version from 2.2. MIMA was pulling in `hadoop2.2` jars into the `lib_managed` directory when it built `oldDeps`. In `spark

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread vectorijk
Github user vectorijk commented on the pull request: https://github.com/apache/spark/pull/9311#issuecomment-153205774 Thanks @mengxr @dbtsai ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10592] [ML] [PySpark] Deprecate weights...

2015-11-02 Thread vectorijk
Github user vectorijk commented on a diff in the pull request: https://github.com/apache/spark/pull/9311#discussion_r43704773 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/SparkRWrappers.scala --- @@ -51,13 +51,22 @@ private[r] object SparkRWrappers { pipeline.fit(d

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205634 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205632 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153205520 **[Test build #44834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44834/consoleFull)** for PR 9383 at commit [`df44fc6`](https://git

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153205125 **[Test build #44857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44857/consoleFull)** for PR 9426 at commit [`aa60ae4`](https://gith

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153204595 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153204617 Ok, thanks for explanation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153204553 **[Test build #44853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44853/consoleFull)** for PR 9424 at commit [`9526e7c`](https://git

[GitHub] spark pull request: [SPARK-11467] [SQL] add Python API for stddev/...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9424#issuecomment-153204597 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9425#issuecomment-153204264 **[Test build #44856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44856/consoleFull)** for PR 9425 at commit [`b17debf`](https://gith

[GitHub] spark pull request: [SPARK-11432][GraphX] Personalized PageRank sh...

2015-11-02 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/9386#issuecomment-153204018 I think the branching should be fine since it's just initialization. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [test-maven][test-hadoop1.0][SPARK-11236][CORE...

2015-11-02 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/9395#issuecomment-153203911 Hey @calvinjia seems okay to merge since this is just triggering other failures. Can you explain more though how this works around the mima issue - the patch seems the

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153203729 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153203737 `unhandledFilter` will not see filters using partitioning columns. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153203733 **[Test build #44855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44855/consoleFull)** for PR 9404 at commit [`8428541`](https://gith

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9426#issuecomment-153203706 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11349] [ML] Support transform string la...

2015-11-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/9302#discussion_r43703873 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala --- @@ -200,6 +200,13 @@ class RFormulaModel private[feature]( dataset.sch

[GitHub] spark pull request: [MINOR][ML] removed the old `getModelWeights` ...

2015-11-02 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/9426 [MINOR][ML] removed the old `getModelWeights` function Removed the old `getModelWeights` function which was private and renamed into `getModelCoefficients` You can merge this pull request into a Git

[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9425#issuecomment-153203185 The failure should not be related to the version of hadoop, so testing with 2.x should be fine. --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9425#issuecomment-153202972 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9425#issuecomment-153202952 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153202825 Ok, actually I was planning to optimize the expression with partition key, which will introduce the `ConstantFolding`, as the partition key will be a constant val

[GitHub] spark pull request: [SPARK-11466] [core] Avoid mockito in multi-th...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9425#issuecomment-153202719 Thanks for the PR! btw, if you want jenkins to test hadoop1, you can add `[test-hadoop1.0]` to the title. --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: [SPARK-10429] [SQL] make mutableProjection ato...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9422#issuecomment-153202611 **[Test build #44854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44854/consoleFull)** for PR 9422 at commit [`5c0da8e`](https://gith

[GitHub] spark pull request: [SPARK-11293] Fix shuffle memory leaks in Spil...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9260#issuecomment-153202532 Is it risky? Looks not? If it is safe patch, wow about we add the check in `Task.scala` first to make sure that 1.5 does throw the exception? Then, we add the fix

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153202279 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153202255 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

<    1   2   3   4   5   6   7   8   9   10   >