[GitHub] spark pull request: [SPARK-2059][SQL] Don't throw TreeNodeExceptio...

2014-07-03 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1294#issuecomment-47989149 Can you paste the error messages thrown by this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47987610 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16316/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47987609 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47987148 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14538915 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47987156 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47986687 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread concretevitamin
Github user concretevitamin commented on a diff in the pull request: https://github.com/apache/spark/pull/1238#discussion_r14538726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala --- @@ -26,6 +26,18 @@ import org.apache.spark.sq

[GitHub] spark pull request: [SQL] [WIP] Prototype implementation of size e...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1238#issuecomment-47986699 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2059][SQL] Don't throw TreeNodeExceptio...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1294#issuecomment-47984822 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16314/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2059][SQL] Don't throw TreeNodeExceptio...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1294#issuecomment-47984820 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-2082] stratified sampling in PairRDDFun...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1025#issuecomment-47984396 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2082] stratified sampling in PairRDDFun...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1025#issuecomment-47984385 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14537682 --- Diff: bin/run-example --- @@ -34,9 +34,9 @@ else fi if [ -f "$FWDIR/RELEASE" ]; then - export SPARK_EXAMPLES_JAR=`ls "$FWDIR"/lib/spar

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14537586 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SPARK-2059][SQL] Don't throw TreeNodeExceptio...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1294#issuecomment-47983823 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2109] Setting SPARK_MEM for bin/pyspark...

2014-07-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1050#issuecomment-47983864 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-2059][SQL] Don't throw TreeNodeExceptio...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1294#issuecomment-47983810 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14537335 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SPARK-2059][SQL] Add analysis checks

2014-07-03 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1265#issuecomment-47983619 PR #1294 should fix this problem, after merger #1294, this one can also be merged safely. --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: Don't throw TreeNodeException in `execution.Ex...

2014-07-03 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/1294 Don't throw TreeNodeException in `execution.ExplainCommand` This is a fix for the problem revealed by PR #1265. Currently `HiveComparisonSuite` ignores output of `ExplainCommand` since Ca

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14537279 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SPARK-2082] stratified sampling in PairRDDFun...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1025#issuecomment-47983319 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2082] stratified sampling in PairRDDFun...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1025#issuecomment-47983304 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14537201 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982767 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982776 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14536966 --- Diff: project/SparkBuild.scala --- @@ -15,393 +15,194 @@ * limitations under the License. */ +import scala.util.Properties +impor

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/772#discussion_r14536931 --- Diff: project/SparkBuild.scala --- @@ -434,250 +235,41 @@ object SparkBuild extends Build { "-noqualifier", "java.lang" ) )

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982406 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-47981340 @ScrapCodes thanks for updating this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-47981128 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1776] Have Spark's SBT build read depen...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/772#issuecomment-47981107 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-2059][SQL] Add analysis checks

2014-07-03 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1265#issuecomment-47981028 `HiveComparisonTest` simply ignores all `ExplainCommand` and don't check their output. We should catch the exception in `execution.ExplainCommand` and report the except

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47980908 Thanks, I merged this into master and 1.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1283 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47978602 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16310/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47978600 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16308/ --- If your project

[GitHub] spark pull request: [SPARK-1199][REPL] Remove VALId and use the or...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1179#issuecomment-47978596 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-1199][REPL] Remove VALId and use the or...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1179#issuecomment-47978599 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16309/ --- If your project

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47978597 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47978598 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-1097] Workaround Hadoop conf Concurrent...

2014-07-03 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1273#discussion_r14533232 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -141,7 +141,7 @@ class HadoopRDD[K, V]( // local process. The local cache

[GitHub] spark pull request: Update SQLConf.scala

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1272#issuecomment-47970749 @rxin wins because he says that SQLConf will become a Thread-local variable. This looks good, the only thing to change for thread-safety is to add a synchronized for get

[GitHub] spark pull request: SPARK-1675. Make clear whether computePrincipa...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1171#issuecomment-47970667 Merged. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1675. Make clear whether computePrincipa...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1171#issuecomment-47970639 Merged. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1675. Make clear whether computePrincipa...

2014-07-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1171 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [MLLIB] SPARK-1682: Add gradient descent w/o s...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/643#issuecomment-47969925 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16307/ --- If your project

[GitHub] spark pull request: [MLLIB] SPARK-1682: Add gradient descent w/o s...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/643#issuecomment-47969923 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-1199][REPL] Remove VALId and use the or...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1179#issuecomment-47968906 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1199][REPL] Remove VALId and use the or...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1179#issuecomment-47968891 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968895 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968908 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968583 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47967682 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47967655 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-2342] Evaluation helper's output type d...

2014-07-03 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/1283#issuecomment-47966853 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-07-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14529947 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -371,6 +371,19 @@ class SQLQuerySuite extends QueryTest { (3,

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-07-03 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14529948 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -296,3 +298,5 @@ private[sql] abstract class SparkStrategies ex

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-07-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14529904 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -135,13 +137,12 @@ class SqlParser extends StandardTokenParsers w

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47965718 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47965725 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16305/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1782: svd for sparse matrix using ARPACK

2014-07-03 Thread vrilleup
Github user vrilleup commented on the pull request: https://github.com/apache/spark/pull/964#issuecomment-47964502 @yangliuyu did you persist the underlying RDD[Vector] behind RowMatrix? This RDD is used in every iteration, caching will improve run time a lot. From my tests, if the R

[GitHub] spark pull request: [MLLIB] SPARK-1682: Add gradient descent w/o s...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/643#issuecomment-47963771 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [MLLIB] SPARK-1682: Add gradient descent w/o s...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/643#issuecomment-47963757 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1782: svd for sparse matrix using ARPACK

2014-07-03 Thread vrilleup
Github user vrilleup commented on the pull request: https://github.com/apache/spark/pull/964#issuecomment-47963684 @mengxr the plan sounds good, I will make the changes. I had some experiences on designing a generic ParameterSet, including constraints on individual parameter, constrai

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962624 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962626 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16306/ --- If your project is set up for it, you can

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962409 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962391 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962057 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [spark-2352] Implementation of an 1-hidden lay...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-47962042 @bgreeven Please add `[MLLIB]` to your PR, following https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark . It makes easier for people who want to searc

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47961794 @kellrott Please create a JIRA and then add both the JIRA number and `[MLLIB]` to the title of this PR, following https://cwiki.apache.org/confluence/display/SPARK/Contrib

[GitHub] spark pull request: [SPARK-2355] Add checker for the number of clu...

2014-07-03 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/1293#issuecomment-47961640 The problem lies in `initKMeansParallel`, the implementation of k-means|| algorithm. Since it selects at most the centers as many as the data number, when calling `LocalKM

[GitHub] spark pull request: [SPARK-2350] Don't NPE while launching drivers

2014-07-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1289#issuecomment-47959923 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK] Fix NPE for ExternalAppendOnlyMap

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1288#issuecomment-47959809 This fixes [SPARK-2349](https://issues.apache.org/jira/browse/SPARK-2349). --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] spark pull request: [SPARK] Fix NPE for ExternalAppendOnlyMap

2014-07-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1288 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK] Fix NPE for ExternalAppendOnlyMap

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1288#issuecomment-47959481 LGTM, merging into master and branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-2355] Add checker for the number of clu...

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1293#issuecomment-47959115 data.count() is actually a very expensive operation, as it has to scan all the data. If cached, it may not be as much a problem, but it is still probably not worth it fo

[GitHub] spark pull request: [SPARK-2355] Add checker for the number of clu...

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1293#issuecomment-47958358 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-2324] SparkContext should not exit dire...

2014-07-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1274 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2324] SparkContext should not exit dire...

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1274#issuecomment-47958114 LGTM. Merging into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2355] Add checker for the number of clu...

2014-07-03 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/1293 [SPARK-2355] Add checker for the number of clusters When the number of clusters given to perform with org.apache.spark.mllib.clustering.KMeans under parallel initial mode is greater than data number

[GitHub] spark pull request: [SPARK-2350] Don't NPE while launching drivers

2014-07-03 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1289#issuecomment-47957745 This should be merged into branch-1.0 and branch-0.9 as well, as this bug disallows queuing up standalone drivers. --- If your project is set up for it, you can reply t

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47951045 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47951037 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47948720 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47948723 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16304/ --- If your project is set up for it, you can

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47948510 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47948524 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47947236 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16303/ --- If your project is set up for it, you can

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47947235 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47947062 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1292#issuecomment-47947046 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Mllib grouped optimization

2014-07-03 Thread kellrott
GitHub user kellrott opened a pull request: https://github.com/apache/spark/pull/1292 Mllib grouped optimization The purpose of this patch is the enable MLLib to better handle scenarios where the user would want to do learning on multiple feature/label sets at the same time. Rather

[GitHub] spark pull request: rename the TestRelation to LocalRelation

2014-07-03 Thread scwf
Github user scwf closed the pull request at: https://github.com/apache/spark/pull/645 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled

[GitHub] spark pull request: SPARK-1782: svd for sparse matrix using ARPACK

2014-07-03 Thread yangliuyu
Github user yangliuyu commented on the pull request: https://github.com/apache/spark/pull/964#issuecomment-47886598 @mengxr k is 100, rCond, tol and maxIterations are all defaults, i.e. 1e-9, 1e-10, 300. Change iteration number from 300 to 200 will not reduce too much time cost <10% (

[GitHub] spark pull request: [MLLIB] [SPARK-2222] Add multiclass evaluation...

2014-07-03 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1155#issuecomment-47886626 @avulanov Thanks for adding metrics for multiclass classification! I made one pass. Besides the inline comments, how can we handle models that output raw scores for each c

[GitHub] spark pull request: [MLLIB] [SPARK-2222] Add multiclass evaluation...

2014-07-03 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1155#discussion_r14505016 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark pull request: [MLLIB] [SPARK-2222] Add multiclass evaluation...

2014-07-03 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/1155#discussion_r14505019 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/evaluation/MulticlassMetricsSuite.scala --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Soft

<    1   2   3   4   >