Re: coprocessors WAS -> Re: Parallel computing on HBase

2010-10-10 Thread William Kang
to do it? And would you please share >>>> more >>>> information about the development process and how may I contribute to >>>> it? >>>> Many thanks. >>>> >>>> >>>> William >>>> >>>> On Wed, Oct 6,

Re: coprocessors WAS -> Re: Parallel computing on HBase

2010-10-10 Thread Mingjie Lai
Base is in progress, yes. We have some initial patches up for review which are the start of this. Best regards, - Andy --- On Tue, 10/5/10, Ryan Rawson wrote: From: Ryan Rawson Subject: Re: Parallel computing on HBase To: user@hbase.apache.org Date: Tuesday, October 5, 2010, 11:10 PM You u

Re: coprocessors WAS -> Re: Parallel computing on HBase

2010-10-07 Thread Stack
;> >> >> I think you are asking about HBASE-2000: >> >> https://issues.apache.org/jira/browse/HBASE-2000 >> >> >> >> Work on an in-process parallel execution framework for HBase is in >> >> progress, yes. We have some initial patches up for revi

Re: coprocessors WAS -> Re: Parallel computing on HBase

2010-10-07 Thread William Kang
re asking about HBASE-2000: > >> https://issues.apache.org/jira/browse/HBASE-2000 > >> > >> Work on an in-process parallel execution framework for HBase is in > >> progress, yes. We have some initial patches up for review which are the > >> start o

coprocessors WAS -> Re: Parallel computing on HBase

2010-10-07 Thread Stack
on an in-process parallel execution framework for HBase is in >> progress, yes. We have some initial patches up for review which are the >> start of this. >> >> Best regards, >> >>    - Andy >> >> >> --- On Tue, 10/5/10, Ryan Rawson wrote: >&

Re: Parallel computing on HBase

2010-10-06 Thread William Kang
parallel execution framework for HBase is in > progress, yes. We have some initial patches up for review which are the > start of this. > > Best regards, > >- Andy > > > --- On Tue, 10/5/10, Ryan Rawson wrote: > > > From: Ryan Rawson > > Subject: Re: Parall

Re: Parallel computing on HBase

2010-10-06 Thread Andrew Purtell
Tue, 10/5/10, Ryan Rawson wrote: > From: Ryan Rawson > Subject: Re: Parallel computing on HBase > To: user@hbase.apache.org > Date: Tuesday, October 5, 2010, 11:10 PM > You understand the hbase data model > yes?  Each region gets a mapper > and each mapper reads the rows for t

Re: Parallel computing on HBase

2010-10-05 Thread Ryan Rawson
You understand the hbase data model yes? Each region gets a mapper and each mapper reads the rows for that region feeding it into the map functions. On the output side, each reducer just writes to hbase. The parallelism can support millions of row reads/second. I don't understand the rest of you

Re: Parallel computing on HBase

2010-10-05 Thread William Kang
Can you tell me a little about how HBase works with MR? If the MR source/sink has to go through just ONE region client, then it is not I am looking for. But if MR can plug directly with the region server containing specific rows, then it might work. Furthermore, MR is a heavy weight process with lo

Re: Parallel computing on HBase

2010-10-05 Thread Jeff Zhang
You can incorporate map reduce with hbase for parallel computing. On Wed, Oct 6, 2010 at 11:24 AM, William Kang wrote: > Hi guys, > Is there any project going on co-processing on region servers? Right now, we > have to transfer all data from region servers to region client after query, > is tha

Parallel computing on HBase

2010-10-05 Thread William Kang
Hi guys, Is there any project going on co-processing on region servers? Right now, we have to transfer all data from region servers to region client after query, is that right? This can be slow. Furthermore, the cpus on the region servers are not fully used. If we could distribute the computation a