[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator-framewor...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137205192 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator-framewor...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137205291 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator-framewor...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137207813 [Test build #41938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41938/consoleFull) for PR 8577 at commit

[GitHub] spark pull request: [SPARK-9516][UI] Improvement of Thread Dump Pa...

2015-09-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/7910#issuecomment-137207794 I would strongly prefer to do the grep in the browser; all of the data that we're filtering is present there anyways and it will lead to better UX because it can

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread nssalian
Github user nssalian closed the pull request at: https://github.com/apache/spark/pull/8054 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137212445 [Test build #41940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41940/consoleFull) for PR 8577 at commit

[GitHub] spark pull request: [SPARK-9821][PYSPARK][WIP] pyspark-reduceByKey...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8569#issuecomment-137218184 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9821][PYSPARK][WIP] pyspark-reduceByKey...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8569#issuecomment-137218151 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38576199 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38576175 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38576831 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38581664 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38581621 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38581610 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38583232 --- Diff: mllib/src/test/java/org/apache/spark/ml/source/JavaLibSVMRelationSuite.java --- @@ -0,0 +1,59 @@ +package org.apache.spark.ml.source; +

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38583127 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-6157][CORE]Unroll unsuccessful memory_a...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4887#issuecomment-137243514 @suyanNone once you address the comments I'll merge this. Thanks for catching this optimization. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-10034][SQL] add regression test for Sor...

2015-09-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8231 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137211736 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137211793 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10247] [core] improve readability of a ...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8434#issuecomment-137213971 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8512#issuecomment-137213741 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10247] [core] improve readability of a ...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8434#issuecomment-137215674 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10411][SQL]Move visualization above exp...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8570#issuecomment-137215884 @pwendell do you mean the triangle, e.g. `> Event timeline` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8512#issuecomment-137216010 [Test build #41941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41941/consoleFull) for PR 8512 at commit

[GitHub] spark pull request: [SPARK-9821][PYSPARK][WIP] pyspark-reduceByKey...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8569#issuecomment-137225444 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-137225646 These pyspark tests fail on every other PR. I'll merge this since I haven't seen any more feedback. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38577664 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38577741 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137231756 [Test build #41945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41945/consoleFull) for PR 8567 at commit

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38581619 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-137238198 Hi @andrewor14, This patch is not very useful outside YARN. On Standalone, all apps run as the same user, and authenticate using the same user and secret. So

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38582857 --- Diff: mllib/src/test/java/org/apache/spark/ml/source/JavaLibSVMRelationSuite.java --- @@ -0,0 +1,59 @@ +package org.apache.spark.ml.source; +

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8537#discussion_r38582919 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38566341 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,280 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-10381] Fix mixup of taskAttemptNumber &...

2015-09-02 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/8544#discussion_r38569507 --- Diff: core/src/test/scala/org/apache/spark/scheduler/OutputCommitCoordinatorIntegrationSuite.scala --- @@ -0,0 +1,68 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-9730][SQL] Add Full Outer Join support ...

2015-09-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/8383#issuecomment-137207810 @viirya Thanks for working on this, this patch is still over complicated, do you mind me to take over it? --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-9926] [SPARK-10340] [SQL] Use S3 bulk l...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8512#issuecomment-137213702 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10247] [core] improve readability of a ...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8434#issuecomment-137215638 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-10074 Include Float in @specialized anno...

2015-09-02 Thread tedyu
Github user tedyu closed the pull request at: https://github.com/apache/spark/pull/8259 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-10074 Include Float in @specialized anno...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8259#issuecomment-137215267 Ok, @tedyu would you mind closing this then? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10422] [SQL] String column in InMemoryC...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8578#issuecomment-137224617 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38575521 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1240,10 +1243,17 @@ class DAGScheduler( if

[GitHub] spark pull request: [SPARK-10422] [SQL] String column in InMemoryC...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8578#issuecomment-137225914 [Test build #41944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41944/consoleFull) for PR 8578 at commit

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38576963 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38577784 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-10353] [MLLIB] (1.3 backport) BLAS gemm...

2015-09-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8572#issuecomment-137236670 LGTM and confirmed that Jenkins build passed. Merged into branch-1.3. Thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-10117][MLLIB] Implement SQL data source...

2015-09-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/8537#issuecomment-137242200 @Lewuathe I made another pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38586126 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1240,10 +1243,17 @@ class DAGScheduler( if

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137250921 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/8218#discussion_r38587368 --- Diff: network/common/src/main/java/org/apache/spark/network/server/OneForOneStreamManager.java --- @@ -109,15 +111,34 @@ public void

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137250800 [Test build #41940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41940/console) for PR 8577 at commit

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/7884#issuecomment-137251765 @dbtsai I made one pass:) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/7884#issuecomment-137251694 1. `weight` vs. `sampleWeight`. I don't think `sampleWeight` solves the problem. I would rather change `weights` in linear models to `coefficients` instead. 2. Using

[GitHub] spark pull request: [SPARK-9730][SQL] Add Full Outer Join support ...

2015-09-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8383#discussion_r38589087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeOuterJoin.scala --- @@ -271,3 +298,224 @@ private class RightOuterIterator(

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-137257022 > it seems then that there are two kinds of authentication No, there's one kind of authentication. I don't know what's this "handshake" you talk about. Whenever

[GitHub] spark pull request: [SPARK-10261][Documentation, ML] Fixed @Since ...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8554#discussion_r38589981 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/BinaryClassificationEvaluator.scala --- @@ -30,16 +30,19 @@ import

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137258901 What if you want to run your driver UI on exactly one port, but need the executors to try many ports because they may collide with other executors on the cluster from

[GitHub] spark pull request: [SPARK-9730] [SQL] Add Full Outer Join support...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8579#issuecomment-137259513 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9730] [SQL] Add Full Outer Join support...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8579#issuecomment-137259497 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8389#discussion_r38591851 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala --- @@ -273,6 +273,10 @@ object SparkSQLConfTest extends Logging {

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8389#discussion_r38591695 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSuite.scala --- @@ -17,15 +17,15 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request: [SPARK-9730] [SQL] Add Full Outer Join support...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8579#issuecomment-137260752 [Test build #41948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41948/consoleFull) for PR 8579 at commit

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137265373 [Test build #41950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41950/consoleFull) for PR 8567 at commit

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/8218#discussion_r38595554 --- Diff: network/common/src/main/java/org/apache/spark/network/server/OneForOneStreamManager.java --- @@ -109,15 +111,34 @@ public void

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137268180 (EDIT: I posted a suggestion that I didn't like in retrospect, so I deleted it. Please disregard.) --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-10422] [SQL] String column in InMemoryC...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8578#issuecomment-137268254 [Test build #41944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41944/console) for PR 8578 at commit

[GitHub] spark pull request: [SPARK-10422] [SQL] String column in InMemoryC...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8578#issuecomment-137268317 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-10422] [SQL] String column in InMemoryC...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8578#issuecomment-137268318 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38596112 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala --- @@ -781,4 +808,117 @@ class LogisticRegressionSuite

[GitHub] spark pull request: [SPARK-5945] Spark should not retry a stage in...

2015-09-02 Thread ilganeli
Github user ilganeli commented on a diff in the pull request: https://github.com/apache/spark/pull/5636#discussion_r38597257 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -473,6 +473,283 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-9730] [SQL] Add Full Outer Join support...

2015-09-02 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/8579 [SPARK-9730] [SQL] Add Full Outer Join support for SortMergeJoin This PR is based on #8383 , thanks to @viirya JIRA: https://issues.apache.org/jira/browse/SPARK-9730 This patch

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137254438 Either way I agree with @vanzin that this is not the right fix. I've run Spark in a fire-walled environment where I had to assign a special port to each component

[GitHub] spark pull request: [SPARK-10261][Documentation, ML] Fixed @Since ...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8554#discussion_r38589979 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/BinaryClassificationEvaluator.scala --- @@ -30,16 +30,19 @@ import

[GitHub] spark pull request: [SPARK-10261][Documentation, ML] Fixed @Since ...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8554#discussion_r38590071 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/MulticlassClassificationEvaluator.scala --- @@ -29,10 +29,12 @@ import

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38590409 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -218,31 +217,51 @@ class LogisticRegression(override val

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137260281 > What if you want to run your driver UI on exactly one port Ok, I can somewhat see that argument, although I don't see why someone would require that Spark's UI

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8389#discussion_r38592158 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala --- @@ -51,6 +51,11 @@ object TestHive // SPARK-8910

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8389#discussion_r38592181 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/ListTablesSuite.scala --- @@ -19,12 +19,12 @@ package org.apache.spark.sql.hive

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8389#discussion_r38592397 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -25,16 +27,8 @@ import org.apache.spark.sql.{Row, SQLConf,

[GitHub] spark pull request: [SPARK-10176][SQL] Show partially analyzed pla...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8389#issuecomment-137262771 Hi @marmbrus @cloud-fan all my comments are minor and this is mergeable as is, but I have two high level comments: (1) I think we should always put `with

[GitHub] spark pull request: [SPARK-10311][Streaming]Reload appId and attem...

2015-09-02 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/8477#issuecomment-137263619 +1. This looks good. @vanzin - I assume your concern is with the reloading. Reloading here is to ignore the old values from the serlialized spark conf

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137264525 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137266772 > Ok, I can somewhat see that argument, although I don't see why someone would require that Spark's UI bind to a specific port. If they're building some application

[GitHub] spark pull request: [SPARK-10004] [shuffle] Perform auth checks wh...

2015-09-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8218#issuecomment-137267685 Yes, and if you look at that code, that secret is the return value of `SecurityManager.getSecretKey()`, which on YARN is stored in the `UserGroupInformation` object.

[GitHub] spark pull request: [SPARK-4449][Core]Specify port range in spark

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/5722#issuecomment-137267811 OK I think one thing we all agree is that SPARK-4449 means have different port retries across different ports. One way to do it is to have the special syntax

[GitHub] spark pull request: [SPARK-8137][core] Improve treeAggregate to co...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/7461#discussion_r3859 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1080,6 +1080,21 @@ abstract class RDD[T: ClassTag]( var partiallyAggregated

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137244071 [Test build #41945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41945/console) for PR 8567 at commit

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137244218 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8567#issuecomment-137244216 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38586801 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -803,13 +829,17 @@ private class LogisticAggregator(

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38586731 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -218,31 +217,51 @@ class LogisticRegression(override val

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38586752 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -499,22 +518,26 @@ class LogisticRegressionModel

[GitHub] spark pull request: [SPARK-7685][ML] Apply weights to different sa...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/7884#discussion_r38586798 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -803,13 +829,17 @@ private class LogisticAggregator(

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137251232 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137251230 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-9851] Support submitting map stages ind...

2015-09-02 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/8180#discussion_r38587515 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -746,6 +848,63 @@ class DAGScheduler( submitWaitingStages()

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137251173 [Test build #41938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41938/console) for PR 8577 at commit

[GitHub] spark pull request: [SPARK-10421] [build] Exclude curator artifact...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8577#issuecomment-137250923 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-9723][ML] params getordefault should th...

2015-09-02 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/8567#discussion_r38589547 --- Diff: mllib/src/main/scala/org/apache/spark/ml/param/params.scala --- @@ -501,7 +501,13 @@ trait Params extends Identifiable with Serializable { */

<    1   2   3   4   5   6   7   >