[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15684457 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15684474 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50897333 QA tests have started for PR 1361. This patch DID NOT merge cleanly! brView progress:

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50898186 QA results for PR 1361:br- This patch FAILED unit tests.brbrFor more information see test

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50898744 @freeman-lab Could you try to merge the latest master and resolve conflicts? It may be because of the change to constructors. --- If your project is set up for it, you

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50908639 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17681/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50908914 @mengxr think I fixed it, issue was in ``GradientDescent`` (where I added the check for an empty RDD) --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15707657 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15707941 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15708929 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala --- @@ -193,6 +195,17 @@ object MLUtils { loadLabeledPoints(sc, dir,

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50914178 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50922127 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17690/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50928131 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15718348 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15718670 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50934313 LGTM!!! My only concern in the documentation, but that we can try fixing independently (but we have to). And the unit test uses files, which is not the best way

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50934956 @freeman-lab I think the static methods `StreamingLinearRegressionWithSGD.start` are not necessary, and these methods actually do not start anything. Do you mind removing

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15719184 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50943077 @mengxr done! removed the static methods (and made the class public), and added those usage notes to ``StreamingLinearAlgorithm`` --- If your project is set up for

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15723222 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -162,45 +162,55 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15723261 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -162,45 +162,55 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15723317 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724226 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -162,45 +162,55 @@ object GradientDescent extends

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724230 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -162,45 +162,55 @@ object GradientDescent extends

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50946141 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17716/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724418 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -162,45 +162,55 @@ object GradientDescent extends

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50946889 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17718/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724782 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -174,17 +182,18 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724803 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724885 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15724913 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50948711 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15725513 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15725538 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15725602 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -174,17 +182,18 @@ object GradientDescent extends

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50949034 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15725743 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearAlgorithm.scala --- @@ -0,0 +1,94 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50950926 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17732/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15726365 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionWithSGD.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-5095 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds the following public classes (experimental):brclass

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50952454 LGTM. Merged into master. Thanks a lot for putting Streaming and MLlib together! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50952474 Yay!!! On Fri, Aug 1, 2014 at 8:12 PM, Xiangrui Meng notificati...@github.com wrote: LGTM. Merged into master. Thanks a lot for putting Streaming

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-08-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1361 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628281 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628342 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628354 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628373 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628387 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628405 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628426 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15628448 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15630544 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15630602 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15630645 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15630714 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15630905 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15671040 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15671309 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-31 Thread freeman-lab
Github user freeman-lab commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15682567 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-30 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-50647207 @mengxr and @mateiz thanks for the feedback! I've started working on fixing all this. I'll leave a couple notes / questions above, otherwise changes are great and

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15561486 --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15561528 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565489 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565590 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565612 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565687 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565686 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565706 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565728 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565789 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/util/MLStreamingUtils.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565824 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565834 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565864 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565941 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565956 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15565983 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/StreamingLinearRegressionSuite.scala --- @@ -0,0 +1,127 @@ +/* + * Licensed to the

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15566035 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15566050 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingLinearRegression.scala --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15566064 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15566077 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-26 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/1361#discussion_r15436152 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/StreamingRegression.scala --- @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49272156 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49272169 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49272423 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16774/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49287245 QA results for PR 1361:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49344174 Looks like the basic test for correct final params passes, but not the stricter test for improvement on every update. Both pass locally. My guess is that it's

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49384594 @mengxr mind retesting? I tried to make the convergence test more robust in a couple ways. If we still have issues we might need to rethink that test further.

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49384924 QA tests have started for PR 1361. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16799/consoleFull ---

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-49389728 QA results for PR 1361:br- This patch PASSES unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-14 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-48935929 @mengxr I added two tests, they check that parameter estimates are accurate, and improve over time. The tests use temporary file writing / file streams, which is

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-12 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-48803931 @freeman-lab Could you add some unit tests? There should be some examples under streaming and mllib. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: Streaming mllib [SPARK-2438][MLLIB]

2014-07-10 Thread freeman-lab
Github user freeman-lab commented on the pull request: https://github.com/apache/spark/pull/1361#issuecomment-48664477 @mengxr great! Just created a JIRA (https://issues.apache.org/jira/browse/SPARK-2438) and added to the title. --- If your project is set up for it, you can reply to