Re: Schema to store graph

2009-04-01 Thread Dave Latham
infully slow, then you should be good > moving > forward with 0.19 and then watch everything get 10+X faster when you > upgrade > to 0.20 :) > > Hope that helps. > > JG > > > -----Original Message----- > > From: Dave Latham [mailto:lat...@davelink.net] > > Sent

RE: Schema to store graph

2009-04-01 Thread Jonathan Gray
t: Wednesday, April 01, 2009 11:58 AM > To: hbase-user@hadoop.apache.org > Subject: Re: Schema to store graph > > Can someone clarify the issues with the number of columns per column > family > that HBase 0.19 can handle? I'm a bit confused, because I feel like > ther

Re: Schema to store graph

2009-04-01 Thread Amandeep Khurana
Alright. Got it. Thanks. Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Wed, Apr 1, 2009 at 1:33 AM, Edward J. Yoon wrote: > Let's assume the graph looks like presented below: > > 1 - 2 - 3 > | / > 4 > > We can now represent as: > > | 1 2 3 4 >

Re: Schema to store graph

2009-04-01 Thread Dave Latham
Can someone clarify the issues with the number of columns per column family that HBase 0.19 can handle? I'm a bit confused, because I feel like there's some conflicting information. In this post (Dec. 20), St.Ack says low hundreds of columns per family are recommended, and refers to a bug (I'm gu

Re: Schema to store graph

2009-04-01 Thread Edward J. Yoon
Plus, The row URLs and anchor family of webTable that mentioned in BigTable paper is same with above structure. It's the web-link graph which is represented as an adjacency matrix. On Wed, Apr 1, 2009 at 5:33 PM, Edward J. Yoon wrote: > Let's assume the graph looks like presented below: > > 1 - 2

Re: Schema to store graph

2009-04-01 Thread Edward J. Yoon
Let's assume the graph looks like presented below: 1 - 2 - 3 | / 4 We can now represent as: | 1 2 3 4 --+-- 1 | 0 1 0 1 2 | 1 0 1 1 3 | 0 1 0 0 4 | 1 0 1 0 We don't need to store the zeros, Hbase is ideal in storing sparse matrices. So, It can be simply implemented usi

Re: Schema to store graph

2009-04-01 Thread Amandeep Khurana
Right. Edward, I didnt understand what you were trying to say with this: Anyway, I guess If you store the graph like that, you'll only need update the row 'v/w' to add v to w's/w to v's list of neighbors. Can you explain it please? Thanks Amandeep Amandeep Khurana Computer Science Graduate Stu

Re: Schema to store graph

2009-04-01 Thread stack
Edward is referring to https://issues.apache.org/jira/browse/HBASE-867. We need to fix it for 0.20.0 hbase release. St.Ack On Wed, Apr 1, 2009 at 5:02 AM, Edward J. Yoon wrote: > One thing is Hbase 0.19 doesn't work with over 5,000 qualifier of one > column so I couldn't test/benchmark for larg

Re: Schema to store graph

2009-03-31 Thread Edward J. Yoon
One thing is Hbase 0.19 doesn't work with over 5,000 qualifier of one column so I couldn't test/benchmark for large scale. On Tue, Mar 31, 2009 at 6:04 PM, Amandeep Khurana wrote: > Response below > > > Amandeep Khurana > Computer Science Graduate Student > University of California, Santa Cruz >

Re: Schema to store graph

2009-03-31 Thread Amandeep Khurana
Response below Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Tue, Mar 31, 2009 at 1:58 AM, Edward J. Yoon wrote: > Hama store the sparse graph using Hbase as an sparse adjacency matrix. > One of reason is to perform matrix decomposition for large sp

Re: Schema to store graph

2009-03-31 Thread Edward J. Yoon
Hama store the sparse graph using Hbase as an sparse adjacency matrix. One of reason is to perform matrix decomposition for large sparse graphs. Anyway, I guess If you store the graph like that, you'll only need update the row 'v/w' to add v to w's/w to v's list of neighbors. Just FYI, You also ma