Re: Scala style questions

2014-06-19 Thread Nick Pentreath
Using infix notation is exactly that - notation. There is no effect on code etc. I also personally tend to favour the more verbose dot notation (perhaps due to java background ). More readable IMO with the exception of DSLs where it can be quite nice. FWIW Spark prefers dot notation for

Re: Mahout on Spark?

2014-02-19 Thread Nick Pentreath
with that. On 02/19/2014 08:04 AM, Nick Pentreath wrote: I know the Spark/Mllib devs can occasionally be quite set in ways of doing certain things, but we'd welcome as many Mahout devs as possible to work together. It may be too late, but perhaps a GSoC project to look at a port

Re: interesting matrix decomposition idea

2013-09-08 Thread Nick Pentreath
I suspect this will be coming to Spark Mllib soon :) — Sent from Mailbox for iPhone On Mon, Sep 9, 2013 at 1:40 AM, Ted Dunning ted.dunn...@gmail.com wrote: http://arxiv.org/pdf/1107.0789v6.pdf The basic idea is to use randomized column sampling to divide the matrix into parts which are

Re: Proposal: scala DSL module for Mahout linear algebra.

2013-07-27 Thread Nick Pentreath
Thanks for the update on that PR I will definitely take a look. I wonder if they will run into the exact same Colt issues as mahout did?! This DSL looks great, I'm gonna play around with it as soon as I get a chance. One question - breeze has quite a similar syntax that is a bit simpler in

Re: Mahout vectors/matrices/solvers on spark

2013-07-05 Thread Nick Pentreath
Hi Dmitry ​You can take a look at using the update magic method which is similar to apply but handles assignment.  ​If you want to keep the := as assignment I think you could do  def :=(value: Double) = update ... (I don't have my laptop around at the moment so can't check this works).

Re: Mahout vectors/matrices/solvers on spark

2013-06-24 Thread Nick Pentreath
That looks great Dmitry!  ​The thing about Breeze that drives the complexity in it is partly specialization for Float, Double and Int matrices, and partly getting the syntax to just work for all combinations of matrix types and operands etc. mostly it does just work but occasionally not.

Re: Mahout vectors/matrices/solvers on spark

2013-06-24 Thread Nick Pentreath
24, 2013 at 1:46 PM, Nick Pentreath nick.pentre...@gmail.comwrote: That looks great Dmitry! The thing about Breeze that drives the complexity in it is partly specialization for Float, Double and Int matrices, and partly getting the syntax to just work for all combinations of matrix types

Re: Mahout vectors/matrices/solvers on spark

2013-06-19 Thread Nick Pentreath
Hi Dmitriy I'd be interested to look at helping with this potentially (time permitting). I've recently been working on a port of Mahout's ALS implementation to Spark. I spent a bit of time thinking about how much of mahout-math to use. For now I found that using the Breeze linear algebra

Re: Discussion Of ML environment/MR, Mahout

2013-03-13 Thread Nick Pentreath
I for one have been using Spark extensively for the past few months - admittedly not in full production, mostly for testing, prototyping - and love it. Over the past two releases they have come a huge way, adding a near-complete Python API and Spark Streaming (comparable to Storm - well more like

Re: Discussion Of ML environment/MR, Mahout

2013-03-13 Thread Nick Pentreath
, Mar 13, 2013 at 1:01 AM, Nick Pentreath nick.pentre...@gmail.comwrote: The main point of interest in this context is that I intend to build a minimal first-cut machine learning library for Spark. This is likely to involve porting / using parts of Mahout where it makes sense (or at the very