Re: Scala style questions

2014-06-19 Thread Nick Pentreath
Using infix notation is exactly that - notation. There is no effect on code etc. I also personally tend to favour the more verbose dot notation (perhaps due to java background ). More readable IMO with the exception of DSLs where it can be quite nice. FWIW Spark prefers dot notation for rea

Re: Mahout on Spark?

2014-02-19 Thread Nick Pentreath
>>>> >>>>>> I'm also convinced that Spark is a superior platform for executing >>>>>>> distributed ML algorithms. We've had a discussion about a change from >>>>>>> Hadoop to another platform some time ago, but at that

Re: interesting matrix decomposition idea

2013-09-08 Thread Nick Pentreath
I suspect this will be coming to Spark Mllib soon :) — Sent from Mailbox for iPhone On Mon, Sep 9, 2013 at 1:40 AM, Ted Dunning wrote: > http://arxiv.org/pdf/1107.0789v6.pdf > The basic idea is to use randomized column sampling to divide the matrix > into parts which are then decomposed usi

Re: Proposal: scala DSL module for Mahout linear algebra.

2013-07-26 Thread Nick Pentreath
Thanks for the update on that PR I will definitely take a look. I wonder if they will run into the exact same Colt issues as mahout did?! This DSL looks great, I'm gonna play around with it as soon as I get a chance. One question - breeze has quite a similar syntax that is a bit simpler in s

Re: Mahout vectors/matrices/solvers on spark

2013-07-05 Thread Nick Pentreath
Hi Dmitry ​You can take a look at using the update "magic" method which is similar to apply but handles assignment.  ​If you want to keep the := as assignment I think you could do  def :=(value: Double) = update ... (I don't have my laptop around at the moment so can't check this works).

Re: Mahout vectors/matrices/solvers on spark

2013-06-24 Thread Nick Pentreath
te: > On Mon, Jun 24, 2013 at 1:46 PM, Nick Pentreath > wrote: >> That looks great Dmitry! >> >> >> The thing about Breeze that drives the complexity in it is partly >> specialization for Float, Double and Int matrices, and partly getting the >> syntax to

Re: Mahout vectors/matrices/solvers on spark

2013-06-24 Thread Nick Pentreath
That looks great Dmitry!  ​The thing about Breeze that drives the complexity in it is partly specialization for Float, Double and Int matrices, and partly getting the syntax to "just work" for all combinations of matrix types and operands etc. mostly it does "just work" but occasionally not.

Re: Mahout vectors/matrices/solvers on spark

2013-06-19 Thread Nick Pentreath
Hi Dmitriy I'd be interested to look at helping with this potentially (time permitting). I've recently been working on a port of Mahout's ALS implementation to Spark. I spent a bit of time thinking about how much of mahout-math to use. For now I found that using the Breeze linear algebra library

Re: Discussion Of ML environment/MR, Mahout

2013-03-13 Thread Nick Pentreath
t; On Wed, Mar 13, 2013 at 1:01 AM, Nick Pentreath > wrote: > >> The main point of interest in this context is that I intend to build a >> minimal first-cut machine learning library for Spark. This is likely to >> involve porting / using parts of Mahout where it makes sense

Re: Discussion Of ML environment/MR, Mahout

2013-03-13 Thread Nick Pentreath
I for one have been using Spark extensively for the past few months - admittedly not in full production, mostly for testing, prototyping - and love it. Over the past two releases they have come a huge way, adding a near-complete Python API and Spark Streaming (comparable to Storm - well more like t