Re: feature selection and sparse vector support

2014-04-11 Thread Héctor Mouriño-Talín
Hi, Regarding the implementation of feature selection techniques, I'm implementing some iterative algorithms based on a paper by Gavin Brown et al. [1]. In this paper, he proposes a common framework for many Information Theory-based criteria, namely those that use relevancy (mutual information

Re: RFC: varargs in Logging.scala?

2014-04-11 Thread Marcelo Vanzin
On Thu, Apr 10, 2014 at 5:46 PM, Michael Armbrust mich...@databricks.com wrote: ... all of the suffer from the fact that the log message needs to be built even though it might not be used. This is not true of the current implementation (and this is actually why Spark has a logging trait

Re: RFC: varargs in Logging.scala?

2014-04-11 Thread David Hall
Another usage that's nice is: logDebug { val timeS = timeMillis/1000.0 sTime: $timeS } which can be useful for more complicated expressions. On Thu, Apr 10, 2014 at 5:55 PM, Michael Armbrust mich...@databricks.comwrote: BTW... You can do calculations in string interpolation: sTime:

Re: feature selection and sparse vector support

2014-04-11 Thread Ignacio Zendejas
Thanks for the response, Xiangrui. And sounds good, Héctor. Look forward to working on this together. A common interface is definitely required. I'll create a JIRA shortly and will explore design options myself to bring ideas to the table. cheers. On Fri, Apr 11, 2014 at 5:44 AM, Héctor

Re: Suggestion

2014-04-11 Thread Sandy Ryza
Hi Priya, Here's a good place to start: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark -Sandy On Fri, Apr 11, 2014 at 12:05 PM, priya arora arora.priya4...@gmail.comwrote: Hi, May I know how one can contribute in this project http://spark.apache.org/mllib/ or in

Re: feature selection and sparse vector support

2014-04-11 Thread Ignacio Zendejas
Here's the JIRA: https://issues.apache.org/jira/browse/SPARK-1473 Future discussions should take place in its comments section. Thanks. On Fri, Apr 11, 2014 at 11:26 AM, Ignacio Zendejas ignacio.zendejas...@gmail.com wrote: Thanks for the response, Xiangrui. And sounds good, Héctor.

Re: Building Spark AMI

2014-04-11 Thread Mayur Rustagi
I am creating one fully configured synced one. But you still need to send over configuration. Do you plan to use chef for that ? On Apr 10, 2014 6:58 PM, Jim Ancona j...@anconafamily.com wrote: Are there scripts to build the AMI used by the spark-ec2 script? Alternatively, is there a place