Re: sparse matrix library

2008-02-02 Thread gopi
checkout the source code from: svn checkout http://svn.apache.org/repos/asf/lucene/mahout/trunk Lucene/mahout/trunk Chaitanya On Feb 2, 2008 10:01 PM, Jason Rennie <[EMAIL PROTECTED]> wrote: > How do you get the code? SVN is asking for username/password... > > What is the constant hit you take

Re: sparse matrix library

2008-02-02 Thread Jason Rennie
How do you get the code? SVN is asking for username/password... What is the constant hit you take for using HBase vs. an in-memory sparse matrix library? Jason On Thu, Jan 31, 2008 at 2:39 AM, Lukas Vlcek <[EMAIL PROTECTED]> wrote: > BTW: Have you seen http://wiki.apache.org/hadoop/Matrix yet?

Re: sparse matrix library

2008-01-31 Thread Grant Ingersoll
This is a good idea. Many of these things will be needed in Hadoop for others, so ideally, we can try to help along some of these things in Hadoop by testing them out, patching, etc. This will help keep our deps. down and make Hadoop better, as well. -Grant On Jan 31, 2008, at 2:39 AM, L

Re: sparse matrix library

2008-01-30 Thread Lukas Vlcek
BTW: Have you seen http://wiki.apache.org/hadoop/Matrix yet? On Jan 30, 2008 11:23 PM, Jason Rennie <[EMAIL PROTECTED]> wrote: > On Jan 30, 2008 2:49 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > > > For efficient row and column access, I built a hybrid that has both CSR > > and > > CSC represenat

Re: sparse matrix library

2008-01-30 Thread Jason Rennie
On Jan 30, 2008 2:49 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > For efficient row and column access, I built a hybrid that has both CSR > and > CSC represenations under the hood. I also don't care much about mutation. Cool :) OS? Or, something you built for VEOH? Probably best to move off-

Re: sparse matrix library

2008-01-30 Thread Ted Dunning
For efficient row and column access, I built a hybrid that has both CSR and CSC represenations under the hood. I also don't care much about mutation. Relative to API, I think we need to support (at least): A * v v * A A * A A' * A rowSum(A) columnSum(A) sum(A) forEachNo

Re: sparse matrix library

2008-01-30 Thread Jason Rennie
On Jan 30, 2008 1:58 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > Many of my matrices need both fast column AND fast row access. They are > also binary. Yet another implementation. > Do you know a good storage format where both column and row access is efficient? Hadn't found a good one yet, b

Re: sparse matrix library

2008-01-30 Thread Ted Dunning
On 1/30/08 10:37 AM, "Jason Rennie" <[EMAIL PROTECTED]> wrote: > Btw, not familiar w/ RSR. Didn't mean CSC, did you? I did mean CSC. Forgot that the last letter was row/column choice. Many of my matrices need both fast column AND fast row access. They are also binary. Yet another impleme

Re: sparse matrix library

2008-01-30 Thread Jason Rennie
On Jan 30, 2008 12:40 PM, Ted Dunning <[EMAIL PROTECTED]> wrote: > I wrote my own package as well based roughly on Colt's high level > structure. Has a sparse matrix code-based been chosen for Mahout? If not, we might as well start the discussion :) It turns out that for many data-mining proje

Re: sparse matrix library

2008-01-30 Thread Ted Dunning
I wrote my own package as well based roughly on Colt's high level structure. It turns out that for many data-mining projects you need several flavors of sparse matrices ([CSR, RSR] x [binary values, real values, complex values] x ...). It also helps a lot of have row and column labels a la R.