[GitHub] spark pull request: SPARK-1496: Have jarOfClass return Option[Stri...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/438#issuecomment-41009317 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Honor default fs name when initializing event ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/450#issuecomment-41009229 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Honor default fs name when initializing event ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/450#issuecomment-41009230 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14319/ --- If your project is set up for it, you can r

[GitHub] spark pull request: version number fix

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/467#issuecomment-41008424 I merged this. @andrewor14 this PR is actually into branch 0.9 so it's not possible to remove the commnet. --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41008412 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41008416 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41008347 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41008236 `(math.random * 1000).toInt` can only produce 1000 values, which is very few. You should use a much bigger number than 1000, e.g. `1e12`, and then do `toLong`. Or you can c

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11840079 --- Diff: python/pyspark/rddsampler.py --- @@ -19,7 +19,7 @@ import random class RDDSampler(object): -def __init__(self, withReplacement,

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11840059 --- Diff: python/pyspark/rddsampler.py --- @@ -19,7 +19,7 @@ import random class RDDSampler(object): -def __init__(self, withReplacement,

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11840021 --- Diff: python/pyspark/rddsampler.py --- @@ -19,7 +19,7 @@ import random class RDDSampler(object): -def __init__(self, withReplacem

[GitHub] spark pull request: Minor optimizations. Use safer take, tail meth...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/473#issuecomment-41007764 Jenkins, test this please. Thanks for the clean-up! Looks good to me, pending tests... --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread smartnut007
Github user smartnut007 commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11839813 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -465,7 +465,13 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread smartnut007
Github user smartnut007 commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11839800 --- Diff: python/pyspark/rddsampler.py --- @@ -19,7 +19,7 @@ import random class RDDSampler(object): -def __init__(self, withReplace

[GitHub] spark pull request: [SPARK-1550] Fixed - Successive creation of sp...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/478#issuecomment-41007441 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-1550] Fixed - Successive creation of sp...

2014-04-21 Thread prabinb
GitHub user prabinb opened a pull request: https://github.com/apache/spark/pull/478 [SPARK-1550] Fixed - Successive creation of spark context fails in pyspark, if the previous initialization of spark context had failed. You can merge this pull request into a Git repository by runn

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread advancedxy
Github user advancedxy commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41007103 ```seed: Int = (math.random * 1000).toInt)``` hi, @mateiz should we use Long instead of Int to avoid collision. --- If your project is set up for it, you can reply

[GitHub] spark pull request: Honor default fs name when initializing event ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/450#issuecomment-41007019 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11839631 --- Diff: python/pyspark/rddsampler.py --- @@ -19,7 +19,7 @@ import random class RDDSampler(object): -def __init__(self, withReplacement,

[GitHub] spark pull request: Honor default fs name when initializing event ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/450#issuecomment-41007015 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/477#discussion_r11839619 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -465,7 +465,13 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41006940 Hey, FYI, it's not a good idea to use System.nanoTime as the seed because multiple RDDs created at the same time (which can easily happen due to lazy evaluation) would have

[GitHub] spark pull request: [SPARK-1459] Use local path (and not complete ...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/375#issuecomment-41006943 Thanks, I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41006955 Same thing applies in Python, don't use the current time, call their random function. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: Fix org.scala-lang: * inconsistent versions de...

2014-04-21 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/465#issuecomment-41006929 @srowen I'm sorry, I submitted a modified on the sql/catalyst/pom.xml,sql/hive/pom.xml,sql/core/pom.xml (Four spaces formatted into two spaces). --- If your project

[GitHub] spark pull request: Honor default fs name when initializing event ...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/450#issuecomment-41006843 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: JIRA issue: [SPARK-1405] Gibbs sampling based ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/476#issuecomment-41006728 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14318/ --- If your project is set up for it, you can r

[GitHub] spark pull request: JIRA issue: [SPARK-1405] Gibbs sampling based ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/476#issuecomment-41006725 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41006583 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14317/ --- If your project is set up for it, you can r

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41006582 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: version number fix

2014-04-21 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/467#issuecomment-41005988 (you could also remove it in this PR) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread smartnut007
Github user smartnut007 commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41005731 @advancedxy If consistency is important then I could set default of long(time.time() * 1e9) in RDDSampler (python api) constructor like you suggested. --- If your p

[GitHub] spark pull request: [SPARK-1439, SPARK-1440] Generate unified Scal...

2014-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/457 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: SPARK-1438 RDD make seed optional in RDD metho...

2014-04-21 Thread smartnut007
Github user smartnut007 closed the pull request at: https://github.com/apache/spark/pull/462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1438 RDD make seed optional in RDD metho...

2014-04-21 Thread smartnut007
Github user smartnut007 commented on the pull request: https://github.com/apache/spark/pull/462#issuecomment-41005398 new PR against master instead of 1.0 https://github.com/apache/spark/pull/477 --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [Fix #274] Document + fix annotation usages

2014-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/477#issuecomment-41005327 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: SPARK-1438 RDD.sample() make seed param option...

2014-04-21 Thread smartnut007
GitHub user smartnut007 opened a pull request: https://github.com/apache/spark/pull/477 SPARK-1438 RDD.sample() make seed param optional copying form previous pull request https://github.com/apache/spark/pull/462 Its probably better to let the underlying language implementat

[GitHub] spark pull request: JIRA issue: [SPARK-1405] Gibbs sampling based ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/476#issuecomment-41005124 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: JIRA issue: [SPARK-1405] Gibbs sampling based ...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/476#issuecomment-41005119 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: JIRA issue: [SPARK-1405](https://issues.apache...

2014-04-21 Thread yinxusen
GitHub user yinxusen opened a pull request: https://github.com/apache/spark/pull/476 JIRA issue: [SPARK-1405](https://issues.apache.org/jira/browse/SPARK-1405) Gibbs sampling based Latent Dirichlet Allocation (LDA) for MLlib (This PR is based on a joint work done with @liancheng fou

[GitHub] spark pull request: [Fix #274] Document + fix annotation usages

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/470#issuecomment-41004962 Thanks I'm merging this - there is an issue with the binary checker for sql... we'll need to fix it. --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/469#discussion_r11838597 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -236,6 +236,21 @@ abstract class RDD[T: ClassTag]( } /** + * Retu

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/469#discussion_r11838553 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -236,6 +236,21 @@ abstract class RDD[T: ClassTag]( } /** + * Retu

[GitHub] spark pull request: [SPARK-1439, SPARK-1440] Generate unified Scal...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/457#issuecomment-41003865 Okay I tested this locally and I'm going to merge it. One thing I noticed is that in catalyst/hive a ton of things are exposed that we might want to either make package-p

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41003059 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41003054 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41003027 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-41002706 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-41002708 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14314/ --- If your project is set up for it, you can r

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41002705 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41002707 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14315/ --- If your project is set up for it, you can r

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/475#issuecomment-41002344 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14316/ --- If your project is set up for it, you can r

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/475#issuecomment-41002343 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/475#issuecomment-41002278 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/475#issuecomment-41002274 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread coderxiang
Github user coderxiang commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-41002133 @yinxusen Just sent the data/code to you. For the running time, from the web UI, it appears to be just (nIter * average reduce/aggregate), . The aggregate time for SGD

[GitHub] spark pull request: SPARK-1544 Add support for deep decision trees...

2014-04-21 Thread manishamde
GitHub user manishamde opened a pull request: https://github.com/apache/spark/pull/475 SPARK-1544 Add support for deep decision trees. etrain and I came with a PR for arbitrarily deep decision trees at the cost of multiple passes over the data at deep tree levels. To summa

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41001038 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14313/ --- If your project is set up for it, you can r

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-41001037 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41000213 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/474#issuecomment-41000208 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-41000185 Cool, could you share your data-generator code to me, and let me take care of the `Nan` problem? Besides, could you provide the total running time of SGD and ADMM when th

[GitHub] spark pull request: [HOTFIX] SPARK-1399: move outdated comments

2014-04-21 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/474 [HOTFIX] SPARK-1399: move outdated comments as the original PR was merged before this mistake is foundfix here, Sorry about that @pwendell, @andrewor14, I will be more careful next tim

[GitHub] spark pull request: version number fix

2014-04-21 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/467#issuecomment-40999877 @andrewor14 , ah, sorry about that, I will submit a "hotfix", --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40999182 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40999175 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1332] Improve Spark Streaming's Network...

2014-04-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/300 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread coderxiang
Github user coderxiang commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-40997859 @etrain that's a good point if ADMM implementations of other algorithms are going to be added in MLlib. Fortunately, for lasso, ridge regression and sparse logistic reg

[GitHub] spark pull request: [SQL]Append some missing types for HiveUDF

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/459#issuecomment-40997571 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL]Append some missing types for HiveUDF

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/459#issuecomment-40997572 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14312/ --- If your project is set up for it, you can r

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread coderxiang
Github user coderxiang commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-40997301 @mengxr @yinxusen Here are some comparison results between ADMM and SGD. These results are just for these particular parameter settings and data sets and no over-gene

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-40996991 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-40996982 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-40996994 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1332] Improve Spark Streaming's Network...

2014-04-21 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/300#issuecomment-40996870 Thanks - I've merged this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1438 RDD make seed optional in RDD metho...

2014-04-21 Thread advancedxy
Github user advancedxy commented on a diff in the pull request: https://github.com/apache/spark/pull/462#discussion_r11835682 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -466,6 +466,12 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40996418 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40996419 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14311/ --- If your project is set up for it, you can r

[GitHub] spark pull request: Minor optimizations. Use safer take, tail meth...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/473#issuecomment-40994587 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-40994380 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1538] Fix SparkUI incorrectly hiding pe...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/469#issuecomment-40994383 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14310/ --- If your project is set up for it, you can r

[GitHub] spark pull request: Minor optimizations. Use safer take, tail meth...

2014-04-21 Thread izendejas
GitHub user izendejas opened a pull request: https://github.com/apache/spark/pull/473 Minor optimizations. Use safer take, tail methods. Per an email thread I initiated and after feedback and clearing my ICLA, I'm requesting some minor changes be pulled to use the more readable and

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread etrain
Github user etrain commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-40994263 Hey, this looks awesome! One high-level issue I see is that the ADMM optimizer has embedded in it knowledge of the loss function it's trying to minimize. ADMM is much more

[GitHub] spark pull request: [Fix #274] Document + fix annotation usages

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/470#issuecomment-40993649 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Fix #274] Document + fix annotation usages

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/470#issuecomment-40993650 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14309/ --- If your project is set up for it, you can r

[GitHub] spark pull request: [SQL]Append some missing types for HiveUDF

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/459#issuecomment-40993061 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL]Append some missing types for HiveUDF

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/459#issuecomment-40993057 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SQL]Append some missing types for HiveUDF

2014-04-21 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/459#issuecomment-40992985 sorry, my bad. Updated "java.lang.BigDecimal" => "java.math.BigDecimal" --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40992799 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-40992792 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread coderxiang
Github user coderxiang commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-40992367 @yinxusen Thanks for the comments. I'm running some comparison between SGD and ADMM right now and will try to post them later. It would also be great if you could prov

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/458#issuecomment-40991905 @coderxiang It is better to have a detailed test, just like what @mengxr said. I can do a favor on testing if you need. --- If your project is set up for it, you can rep

[GitHub] spark pull request: SPARK-1235: manage the DAGScheduler EventProce...

2014-04-21 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/186#discussion_r11833395 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -678,7 +530,7 @@ class DAGScheduler( * We run the operation in a s

[GitHub] spark pull request: SPARK-693

2014-04-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/472#issuecomment-40991777 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/458#discussion_r11833298 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/LassoSuite.scala --- @@ -44,8 +44,11 @@ class LassoSuite extends FunSuite with LocalSparkCont

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/458#discussion_r11833315 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/regression/LassoSuite.scala --- @@ -113,9 +116,100 @@ class LassoSuite extends FunSuite with LocalSpark

[GitHub] spark pull request: SPARK-693

2014-04-21 Thread chu11
GitHub user chu11 opened a pull request: https://github.com/apache/spark/pull/472 SPARK-693 If SPARK_CONF_DIR environment variable is specified, use spark-env.sh specified in $SPARK_CONF_DIR when starting daemons. You can merge this pull request into a Git repository by running:

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/458#discussion_r11833279 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/Lasso.scala --- @@ -189,3 +230,70 @@ object LassoWithSGD { sc.stop() } }

[GitHub] spark pull request: [SPARK-1543][MLlib] Add ADMM for solving Lasso...

2014-04-21 Thread yinxusen
Github user yinxusen commented on a diff in the pull request: https://github.com/apache/spark/pull/458#discussion_r11833249 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/regression/Lasso.scala --- @@ -189,3 +230,70 @@ object LassoWithSGD { sc.stop() } }

  1   2   3   4   >