Re: [VOTE] Release Apache Spark 0.9.2 (RC1)

2014-07-18 Thread Patrick Wendell
+1 - Looked through the release commits - Looked through JIRA issues - Ran the audit tests (one issue with the maven app test, but was also an issue with 0.9.1 so I think it's my environment) - Checked sigs/sums On Thu, Jul 17, 2014 at 11:13 PM, Xiangrui Meng wrote: > UPDATE: > > The staging rep

Re: [VOTE] Release Apache Spark 0.9.2 (RC1)

2014-07-18 Thread Andrew Or
+1, tested on standalone cluster and ran spark shell, pyspark and SparkPi 2014-07-18 0:03 GMT-07:00 Patrick Wendell : > +1 > > - Looked through the release commits > - Looked through JIRA issues > - Ran the audit tests (one issue with the maven app test, but was also > an issue with 0.9.1 so I t

Re: Current way to include hive in a build

2014-07-18 Thread Sean Owen
This build invocation works just as you have it, for me. (At least, it gets through Hive; Examples fails for a different unrelated reason.) commons-logging 1.0.4 exists in Maven for sure. Maybe there is some temporary problem accessing Maven's repo? On Fri, Jul 18, 2014 at 12:00 AM, Stephen Boesc

RE: Feature selection interface

2014-07-18 Thread Ulanov, Alexander
FYI This is my first take on feature selection, filtering and chi-squared: https://github.com/apache/spark/pull/1484 -Original Message- From: Ulanov, Alexander Sent: Thursday, July 10, 2014 9:39 PM To: dev@spark.apache.org Subject: Feature selection interface Hi, I've implemented a cla

Re: Contributing to MLlib: Proposal for Clustering Algorithms

2014-07-18 Thread RJ Nowling
Nice to meet you, Jeremy! This is great! Hierarchical clustering was next on my list -- currently trying to get my PR for MiniBatch KMeans accepted. If it's cool with you, I'll try converting your code to fit in with the existing MLLib code as you suggest. I also need to review the Decision Tree

Re: [VOTE] Release Apache Spark 0.9.2 (RC1)

2014-07-18 Thread Sean McNamara
+1 On Jul 18, 2014, at 2:08 AM, Andrew Or wrote: > +1, tested on standalone cluster and ran spark shell, pyspark and SparkPi > > > 2014-07-18 0:03 GMT-07:00 Patrick Wendell : > >> +1 >> >> - Looked through the release commits >> - Looked through JIRA issues >> - Ran the audit tests (one iss

Re: Building Spark against Scala 2.10.1 virtualized

2014-07-18 Thread Meisam Fathi
Sorry for resurrecting this thread but project/SparkBuild.scala is completely rewritten recently (after this commit https://github.com/apache/spark/tree/628932b). Should library dependencies be defined in pox.xml files after this commit? Thanks Meisam On Thu, Jun 5, 2014 at 4:51 PM, Matei Zaharia

OWLQN

2014-07-18 Thread Debasish Das
Hi, I thought OWLQN is already merged to mllib optimization but I don't see it in the master yet... Are there any issues in merging it in ? I see there are some merge conflicts right now... https://github.com/apache/spark/pull/840/ Thanks. Deb

Re: Current way to include hive in a build

2014-07-18 Thread Stephen Boesch
Thanks v much Patrick and Sean. I have the build working now as follows: mvn -Pyarn -Pcdh5 -Phive -DskipTests clean package in Addition, I am in the midst of running some tests and so far so good. The pom.xml changes: Added to main/parent directory pom.xml: cdh5 2.

Re: Building Spark against Scala 2.10.1 virtualized

2014-07-18 Thread Reynold Xin
Yes. On Fri, Jul 18, 2014 at 12:50 PM, Meisam Fathi wrote: > Sorry for resurrecting this thread but project/SparkBuild.scala is > completely rewritten recently (after this commit > https://github.com/apache/spark/tree/628932b). Should library > dependencies be defined in pox.xml files after thi

Re: OWLQN

2014-07-18 Thread DB Tsai
I'm working on it with weighted regularization. The problem is that OWLQN doesn't work nicely with Updater now since all the L1 logic should be in OWLQN instead of L1Updater. Sincerely, DB Tsai --- My Blog: https://www.dbtsai.com LinkedIn: https

How to set Java options -Xmn

2014-07-18 Thread npanj
Hi, I am trying to set -Xmn to control GC in spark.executor.extraJavaOptions (as recommended by tuning guide), but I am getting error that "spark.executor.extraJavaOptions is not allowed to alter memory settings". It seems that extraJavaOptions takes just one number, not list of java options. Ho

Re: Contributing to MLlib: Proposal for Clustering Algorithms

2014-07-18 Thread Jeremy Freeman
Hi RJ, that sounds like a great idea. I'd be happy to look over what you put together. -- Jeremy -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Contributing-to-MLlib-Proposal-for-Clustering-Algorithms-tp7212p7418.html Sent from the Apache Spark Devel