Yes, the goal is to handle really huge matrices, for example, matrix operations for large-scale statistical processing, matrix decomposition of huge web link graph/social graph.
It's the tests on 5 nodes and 10 nodes. In the future, I'll try them on a thousand nodes. On Thu, Jun 4, 2009 at 1:19 AM, tog <[email protected]> wrote: > Hi Edward, > > I had a look to the benchmarks ... > Well a 5000 dense matrix multiply is roughly 30 seconds on my laptop. I have > been doing out-of-core parallel matrix factor on solve with dense systems up > to 350000 > so I guess this is at least probably for larger matrix that Hama could be > interesting > Do you plan to do such tests with really huge matrices ? > Otherwise what is your business case ? > > Cheers > Guillaume > > On Wed, Jun 3, 2009 at 6:14 PM, Edward J. Yoon <[email protected]>wrote: > >> FYI, I ran some benchmarks - >> http://wiki.apache.org/hama/PerformanceEvaluation >> >> If you need any help, Pls let us know. >> >> Thanks. >> >> On Wed, Jun 3, 2009 at 6:55 PM, tog <[email protected]> wrote: >> > Yes I understand the difference between MPI and Hadoop - I have been >> using >> > MPI before it actually exists :) >> > But as you phrased it, I had the impression that Hama was working on a 1 >> > node/core cluster !! >> > >> > Regards >> > Guillaume >> > >> > On Wed, Jun 3, 2009 at 5:44 PM, Edward J. Yoon <[email protected] >> >wrote: >> > >> >> Hi, >> >> >> >> There is some difference between Map/Reduce and MPI programming. MPI >> >> is based on and designed for fast parallel computing using network >> >> communication on small cluster. Since MPI requires network >> >> communication, Increased node numbers, there is a linear increase of >> >> network cost at same time. On the contrary, Map/Reduce is designed to >> >> distributed processing by connecting many commodity computers >> >> together. Therefore, The algorithms should avoid large amounts of >> >> communication for best performance and that key is the 'sequential >> >> process'. >> >> >> >> Thanks. >> >> >> >> On Wed, Jun 3, 2009 at 6:07 PM, tog <[email protected]> wrote: >> >> > Hi Edward >> >> > >> >> > I have a test to do which is basically Sparce Mat Vec multiplication >> and >> >> Mat >> >> > norm computation. So that should be possible with Hama in its current >> >> state >> >> > I guess. >> >> > What do you mean by "sequentially executed" >> >> > >> >> > Cheers >> >> > Guillaume >> >> > >> >> > On Wed, Jun 3, 2009 at 5:00 PM, Edward J. Yoon <[email protected] >> >> >wrote: >> >> > >> >> >> Hi, >> >> >> >> >> >> Currently, the basic matrix operations are implemented based on the >> >> >> map/reduce programming model. For example, the matrix get/set >> methods, >> >> >> the matrix norms, matrix-matrix multiplication/addition, matrix >> >> >> transpose. In near future, SVD, Eigenvalue decomposition and some >> >> >> graph algorithms will be implemented. All the operations are >> >> >> sequentially executed. >> >> >> >> >> >> Thanks. >> >> >> >> >> >> On Wed, Jun 3, 2009 at 5:45 PM, tog <[email protected]> >> wrote: >> >> >> > >> >> >> > Hi, >> >> >> > >> >> >> > I would like to know what is the status of Hama ? >> >> >> > What am I able to do with it ? >> >> >> > What are the future directions ? >> >> >> > >> >> >> > Cheers >> >> >> > Guillaume >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Best Regards, Edward J. Yoon @ NHN, corp. >> >> >> [email protected] >> >> >> http://blog.udanax.org >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > >> >> > PGP KeyID: 1024D/47172155 >> >> > FingerPrint: C739 8B3C 5ABF 127F CCFA 5835 F673 370B 4717 2155 >> >> > >> >> > http://cheztog.blogspot.com >> >> > >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon @ NHN, corp. >> >> [email protected] >> >> http://blog.udanax.org >> >> >> > >> > >> > >> > -- >> > >> > PGP KeyID: 1024D/47172155 >> > FingerPrint: C739 8B3C 5ABF 127F CCFA 5835 F673 370B 4717 2155 >> > >> > http://cheztog.blogspot.com >> > >> >> >> >> -- >> Best Regards, Edward J. Yoon @ NHN, corp. >> [email protected] >> http://blog.udanax.org >> > > > > -- > > PGP KeyID: 1024D/47172155 > FingerPrint: C739 8B3C 5ABF 127F CCFA 5835 F673 370B 4717 2155 > > http://cheztog.blogspot.com > -- Best Regards, Edward J. Yoon @ NHN, corp. [email protected] http://blog.udanax.org
