[GitHub] spark pull request: [SPARK-6918][YARN] Secure HBase support.

2015-04-19 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/5586#issuecomment-94367970 Yeah, LGTM, I need this function. can we put hbase's config into hbase-site.xml, right? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: Common interfaces between RDD, DStream, and Da...

2015-04-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5565#issuecomment-94366735 Hey @nkronenfeld - simply put, we cannot make binary-incompatible changes to API's in Spark due to our API guarantees, so this rules out many of your proposed changes. T

[GitHub] spark pull request: [PYSPARK] Fix a typo in "fold" function in rdd...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5587#issuecomment-94366385 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [PYSPARK] Fix a typo in "fold" function in rdd...

2015-04-19 Thread AiHe
GitHub user AiHe opened a pull request: https://github.com/apache/spark/pull/5587 [PYSPARK] Fix a typo in "fold" function in rdd.py This will make the “fold” function consistent with the "fold" in rdd.scala and other "aggregate" functions where “acc” goes first. Otherwise,

[GitHub] spark pull request: [SPARK-6973]modify total stages/tasks on the a...

2015-04-19 Thread XuTingjun
GitHub user XuTingjun reopened a pull request: https://github.com/apache/spark/pull/5550 [SPARK-6973]modify total stages/tasks on the allJobsPage Though totalStages = allStages - skippedStages is understandable. But consider the problem [SPARK-6973], I think totalStages = allStages

[GitHub] spark pull request: [SPARK-6973]modify total stages/tasks on the a...

2015-04-19 Thread XuTingjun
Github user XuTingjun closed the pull request at: https://github.com/apache/spark/pull/5550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-6973]modify total stages/tasks on the a...

2015-04-19 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/5550#issuecomment-94364746 @srowen , do you mean we should count the first time of stages into total/completed stages? @JoshRosen , Do you have any suggestions? --- If your project is set up

[GitHub] spark pull request: [SPARK-6661] Python type errors should print t...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5361#issuecomment-94364266 [Test build #30576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30576/consoleFull) for PR 5361 at commit [`73c5d79`](https://githu

[GitHub] spark pull request: [SPARK-6661] Python type errors should print t...

2015-04-19 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/5361#issuecomment-94363938 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-19 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-94362346 @srowen I tried setting httpclient in spark-parent's dependencyManagement and removing my direct dependency, but I encounter the same problem as before (httpclient

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5149#issuecomment-94361972 This is looking good in terms of functionality. I made some comments to help simplify the code a bit. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/5149#discussion_r28664977 --- Diff: dev/merge_spark_pr.py --- @@ -286,68 +281,145 @@ def resolve_jira_issues(title, merge_branches, comment): resolve_jira_issue(merge_bra

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/5149#discussion_r28664956 --- Diff: dev/merge_spark_pr.py --- @@ -286,68 +281,145 @@ def resolve_jira_issues(title, merge_branches, comment): resolve_jira_issue(merge_bra

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/5149#discussion_r28664855 --- Diff: dev/merge_spark_pr.py --- @@ -286,68 +281,145 @@ def resolve_jira_issues(title, merge_branches, comment): resolve_jira_issue(merge_bra

[GitHub] spark pull request: [SPARK-6113] [ml] Small cleanups after origina...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5567#issuecomment-94360227 [Test build #690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/690/consoleFull) for PR 5567 at commit [`bb9f610`](https://githu

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/5149#discussion_r28664751 --- Diff: dev/merge_spark_pr.py --- @@ -286,68 +281,145 @@ def resolve_jira_issues(title, merge_branches, comment): resolve_jira_issue(merge_bra

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94358790 [Test build #30575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30575/consoleFull) for PR 5584 at commit [`8699680`](https://gith

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94358797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-7000] [ml] Refactor prediction and tree...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5585#issuecomment-94358565 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-7000] [ml] Refactor prediction and tree...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5585#issuecomment-94358561 [Test build #30574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30574/consoleFull) for PR 5585 at commit [`b2cb6ad`](https://gith

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94357202 [Test build #30573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30573/consoleFull) for PR 5584 at commit [`37ce656`](https://gith

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94357210 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [Project Infra] SPARK-1684: Merge script shoul...

2015-04-19 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/5149#discussion_r28663536 --- Diff: dev/merge_spark_pr.py --- @@ -286,68 +281,145 @@ def resolve_jira_issues(title, merge_branches, comment): resolve_jira_issue(merge_bra

[GitHub] spark pull request: [SPARK-6918][YARN] Secure HBase support.

2015-04-19 Thread deanchen
Github user deanchen commented on the pull request: https://github.com/apache/spark/pull/5586#issuecomment-94352574 @XuTingjun noticed you were also interested in this feature on https://github.com/apache/spark/pull/5031 --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: [SPARK-6918][YARN] Secure HBase support.

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5586#issuecomment-94352454 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-6918][YARN] Secure HBase support.

2015-04-19 Thread deanchen
GitHub user deanchen opened a pull request: https://github.com/apache/spark/pull/5586 [SPARK-6918][YARN] Secure HBase support. Obtain HBase security token with Kerberos credentials locally to be sent to executors. Similar to obtainTokenForNamenodes. Fails gracefully if HBa

[GitHub] spark pull request: Common interfaces between RDD, DStream, and Da...

2015-04-19 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/5565#issuecomment-94351472 No, we do that at the moment. But doing it that way results in a few rather ugly constructs in the application code that can be rather painful, as soon as one

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-94351145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-94351140 [Test build #30571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30571/consoleFull) for PR 5491 at commit [`cb45105`](https://gith

[GitHub] spark pull request: [doc][mllib] Fix typo of the page title in Iso...

2015-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5581 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [doc][mllib] Fix typo of the page title in Iso...

2015-04-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5581#issuecomment-94350914 Good catch. I'll hand-test this since Jenkins wouldn't test it anyway, but looks fine. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-6985][streaming] Receiver maxRate over ...

2015-04-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/5559#issuecomment-94350661 Jenkins, this is okay to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Common interfaces between RDD, DStream, and Da...

2015-04-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5565#issuecomment-94350194 @nkronenfeld the thrust of my comment comes from the fact that `DStream`s give you an `RDD`, which you can do whatever you want with, including make a `DStream` again with

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-94350049 Should do; give it a try. This should be how you control the version of an artifact that is used without creating an actual dependency on it. --- If your project is set u

[GitHub] spark pull request: [SPARK-6979][Streaming] Replace JobScheduler.e...

2015-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5554 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6979][Streaming] Replace JobScheduler.e...

2015-04-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5554#issuecomment-94349784 LGTM. I'm going to merge it in master since it doesn't conflict. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-6113] [ml] Small cleanups after origina...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5567#issuecomment-94349642 @mengxr Let's get [https://github.com/apache/spark/pull/5585] merged before this since it's blocking my ensemble stuff more. Thanks! --- If your project is set up fo

[GitHub] spark pull request: [SPARK-6065] [MLlib] Optimize word2vec.findSyn...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5467#issuecomment-94348677 @MechCoder There are 2 pieces of code which we're talking about: (1) converting the Map to a Matrix and (2) converting the Matrix to a Map. I suppose that either appr

[GitHub] spark pull request: [SPARK-6983][Streaming] Update ReceiverTracker...

2015-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5557 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6983][Streaming] Update ReceiverTracker...

2015-04-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5557#issuecomment-94347997 I'm going to merge this in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-5924] Add the ability to specify withMe...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4704#issuecomment-94347932 @jrabary Can you please run dev/scalastyle to see where the style issue is? Thanks! --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/5582 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94347892 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94347885 [Test build #30575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30575/consoleFull) for PR 5584 at commit [`8699680`](https://githu

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94347795 yup, seems fine to me too, this was probably an unintentional omission, as both `StreamingLinearRegressionWithSGD` and `StreamingLogisticRegressionWithSGD` do extend

[GitHub] spark pull request: [SPARK-6113] [ml] Small cleanups after origina...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5567#issuecomment-94347713 [Test build #690 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/690/consoleFull) for PR 5567 at commit [`bb9f610`](https://github

[GitHub] spark pull request: [SPARK-6113] [ml] Small cleanups after origina...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5567#issuecomment-94347671 Jenkins didn't post it here, but it looks like the tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94347681 alright I'm going to merge this. if there is a problem we can always revert it. Thanks for taking a look. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94347562 I don't see a reason not to make it Serializable, but I haven't done much with streaming yet... Seems Ok to me --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-7000] [ml] Refactor prediction and tree...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5585#issuecomment-94347422 [Test build #30574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30574/consoleFull) for PR 5585 at commit [`b2cb6ad`](https://githu

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94347409 All comments addressed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94347386 I'll take a look, but also: CC @freeman-lab --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-7000] [ml] Refactor prediction and tree...

2015-04-19 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/5585 [SPARK-7000] [ml] Refactor prediction and tree abstractions to be under ml.prediction subpackage From JIRA: > spark.ml prediction abstractions are currently not gathered; they are in both ml.

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662270 --- Diff: network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java --- @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662265 --- Diff: network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java --- @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662180 --- Diff: network/common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662171 --- Diff: network/common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662130 --- Diff: network/common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662125 --- Diff: network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java --- @@ -0,0 +1,277 @@ +/* + * Licensed to the Ap

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662096 --- Diff: network/common/src/test/java/org/apache/spark/network/RequestTimeoutIntegrationSuite.java --- @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94345661 [Test build #30573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30573/consoleFull) for PR 5584 at commit [`37ce656`](https://githu

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662058 --- Diff: network/common/src/main/java/org/apache/spark/network/util/MapConfigProvider.java --- @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28662027 --- Diff: network/common/src/main/java/org/apache/spark/network/server/TransportChannelHandler.java --- @@ -93,4 +103,25 @@ public void channelRead0(ChannelHand

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28661995 --- Diff: network/common/src/main/java/org/apache/spark/network/client/TransportResponseHandler.java --- @@ -50,13 +50,17 @@ private final Map outs

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/5584#discussion_r28661988 --- Diff: network/common/src/main/java/org/apache/spark/network/client/TransportResponseHandler.java --- @@ -50,13 +50,17 @@ private final Map outs

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94344935 [Test build #30572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30572/consoleFull) for PR 5584 at commit [`aa5278b`](https://githu

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94344944 [Test build #30572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30572/consoleFull) for PR 5584 at commit [`aa5278b`](https://gith

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94344947 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/5584#issuecomment-94344712 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: [SPARK-7003] Improve reliability of connection...

2015-04-19 Thread aarondav
GitHub user aarondav opened a pull request: https://github.com/apache/spark/pull/5584 [SPARK-7003] Improve reliability of connection failure detection between Netty block transfer service endpoints Currently we rely on the assumption that an exception will be raised and the channel

[GitHub] spark pull request: [SPARK-6969][SQL] Refresh the cached table whe...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5583#issuecomment-94343647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6969][SQL] Refresh the cached table whe...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5583#issuecomment-94343624 [Test build #30570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30570/consoleFull) for PR 5583 at commit [`92b2498`](https://gith

[GitHub] spark pull request: [SPARK-6156][CORE]Not cache in memory again wh...

2015-04-19 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4886#issuecomment-94342126 @srowen I fine to close this patch. eh...I not sure I fully got your point. I agree this means "a level which does not use memory" and ` This branch is for the case

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-19 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-94341589 @vanzin Added another temporary ListBuffer `leftToClean` to store the apps that wasn't deleted succesfully and avoid editing `appsToClean` in its iterator. -

[GitHub] spark pull request: [SPARK-6879][HistoryServer]check if app is com...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5491#issuecomment-94341483 [Test build #30571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30571/consoleFull) for PR 5491 at commit [`cb45105`](https://githu

[GitHub] spark pull request: [SPARK-6955][NETWORK]Do not let Yarn Shuffle S...

2015-04-19 Thread SaintBacchus
Github user SaintBacchus closed the pull request at: https://github.com/apache/spark/pull/5537 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature i

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread SaintBacchus
Github user SaintBacchus commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94338939 I seems to be a better way. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-19 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-94333039 Does dependency management work for transitive dependencies? I'm trying to ensure the higher version wins out. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: Common interfaces between RDD, DStream, and Da...

2015-04-19 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/5565#issuecomment-94332668 Ok, I think there are only 3 real API changes, though one of them appears in multiple places. * `RDD` * `flatMap[U](f: T => TraversableOnce[U])` changed

[GitHub] spark pull request: [SPARK-6869][PySpark] Add pyspark archives pat...

2015-04-19 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on the pull request: https://github.com/apache/spark/pull/5478#issuecomment-94331295 @lianhuiwang OK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Common interfaces between RDD, DStream, and Da...

2015-04-19 Thread nkronenfeld
Github user nkronenfeld commented on the pull request: https://github.com/apache/spark/pull/5565#issuecomment-94330557 Thanks for taking a look at it. I'll take a look and write up here exactly what API changes this makes - while I don't think there are many, there aren't none

[GitHub] spark pull request: [SPARK-6969][SQL] Refresh the cached table whe...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5583#issuecomment-94330093 [Test build #30570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30570/consoleFull) for PR 5583 at commit [`92b2498`](https://githu

[GitHub] spark pull request: [SPARK-6969][SQL] Refresh the cached table whe...

2015-04-19 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/5583 [SPARK-6969][SQL] Refresh the cached table when REFRESH TABLE is used https://issues.apache.org/jira/browse/SPARK-6969 You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94329386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94329380 [Test build #30569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30569/consoleFull) for PR 5575 at commit [`df60b80`](https://gith

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-19 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-94329327 I dont think it should be a direct dependency just for this purpose, no. This is what the dependencyManagement section is for. What are you trying to achieve with this? Ju

[GitHub] spark pull request: [SPARK-6122][Core] Upgrade tachyon-client vers...

2015-04-19 Thread calvinjia
Github user calvinjia commented on the pull request: https://github.com/apache/spark/pull/5354#issuecomment-94327839 @srowen Thanks for taking another look! For the Jackson library, spark already has a dependency managed version, but for transitive dependencies I need to exclud

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94321419 [Test build #30569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30569/consoleFull) for PR 5575 at commit [`df60b80`](https://githu

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94318343 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94318339 **[Test build #30568 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30568/consoleFull)** for PR 5575 at commit [`a3dc42d`](https://git

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94317730 cc @mengxr and @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94308606 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6998][MLlib] Make StreamingKMeans 'Seri...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5582#issuecomment-94308583 [Test build #30566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30566/consoleFull) for PR 5582 at commit [`67c2a14`](https://gith

[GitHub] spark pull request: [SPARK-6979][Streaming] Replace JobScheduler.e...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5554#issuecomment-94307254 [Test build #30565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30565/consoleFull) for PR 5554 at commit [`5304350`](https://gith

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94307271 [Test build #30568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30568/consoleFull) for PR 5575 at commit [`a3dc42d`](https://githu

[GitHub] spark pull request: [SPARK-6979][Streaming] Replace JobScheduler.e...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5554#issuecomment-94307259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-6955] Perform port retries at NettyBloc...

2015-04-19 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/5575#issuecomment-94307038 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-5924] Add the ability to specify withMe...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4704#issuecomment-94305535 [Test build #30567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30567/consoleFull) for PR 4704 at commit [`8896b0e`](https://gith

[GitHub] spark pull request: [SPARK-5924] Add the ability to specify withMe...

2015-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4704#issuecomment-94305536 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30

[GitHub] spark pull request: [SPARK-5924] Add the ability to specify withMe...

2015-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4704#issuecomment-94305471 [Test build #30567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30567/consoleFull) for PR 4704 at commit [`8896b0e`](https://githu

  1   2   >