Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-29 Thread Andrew Purtell
>> From: Andrew Purtell >> To: dev@hbase.apache.org >> Cc: >> Sent: Tuesday, August 28, 2012 3:05 AM >> Subject: Re: Improving Coprocessor postSplit/postOpen synchronization >> >> Never mind, I went to look at the code. Should have done that first. &

RE: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Ramkrishna.S.Vasudevan
urt...@apache.org > Subject: Re: Improving Coprocessor postSplit/postOpen synchronization > > Hello again everyone, > > Thanks for responding! I really appreciate all of the advice that's > been > given so far. :) > > Just to clarify Andrew do you have a prototype

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Kevin Shin
> > - Original Message - > From: Andrew Purtell > To: dev@hbase.apache.org > Cc: > Sent: Tuesday, August 28, 2012 3:05 AM > Subject: Re: Improving Coprocessor postSplit/postOpen synchronization > > Never mind, I went to look at the code. Should have do

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread lars hofhansl
That approach sounds good to me. - Original Message - From: Andrew Purtell To: dev@hbase.apache.org Cc: Sent: Tuesday, August 28, 2012 3:05 AM Subject: Re: Improving Coprocessor postSplit/postOpen synchronization Never mind, I went to look at the code. Should have done that first

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Andrew Purtell
Never mind, I went to look at the code. Should have done that first. Looking at 0.94 sources, in SplitTransaction, first we notify the master that the split has happened, and wait for the master to process it (which opens daughters), and then call up to the CP with the daughter regions as argument

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Andrew Purtell
(from postSplit) On Tue, Aug 28, 2012 at 12:53 PM, Andrew Purtell wrote: > What about writing a marker (a file) into the region at split (from > preSplit) which is then existence checked and read at open (postOpen)? This > file would contain whatever indexing metadata is required. > > Also, split

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-28 Thread Andrew Purtell
What about writing a marker (a file) into the region at split (from preSplit) which is then existence checked and read at open (postOpen)? This file would contain whatever indexing metadata is required. Also, splits are nearly instant because the daughters are created with reference files to the p

RE: Improving Coprocessor postSplit/postOpen synchronization

2012-08-27 Thread Ramkrishna.S.Vasudevan
cs.com] > Sent: Tuesday, August 28, 2012 3:40 AM > To: dev@hbase.apache.org > Subject: Re: Improving Coprocessor postSplit/postOpen synchronization > > Thanks Ted, > > As a better approach instead of adding code to pre/postOpen, we're > going to > see if we can add one mo

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-27 Thread Kevin Shin
Thanks Ted, As a better approach instead of adding code to pre/postOpen, we're going to see if we can add one more coprocessor call instead to enforce modularity between splits and opens. Will submit patch soon. Best, Kevin On Mon, Aug 27, 2012 at 1:49 PM, Ted Yu wrote: > Ramkrishna recently c

Re: Improving Coprocessor postSplit/postOpen synchronization

2012-08-27 Thread Ted Yu
Ramkrishna recently checked in HBASE-6633 But that doesn't seem to address your use case. Go ahead and file a JIRA. On Mon, Aug 27, 2012 at 1:29 PM, Kevin Shin < kevin.s...@thinkbiganalytics.com> wrote: > Hi everyone, > > A colleague and I were working with HBase coprocessors for secondary > in

Improving Coprocessor postSplit/postOpen synchronization

2012-08-27 Thread Kevin Shin
Hi everyone, A colleague and I were working with HBase coprocessors for secondary indexes and ran into an interesting problem regarding splits and synchronizing the corresponding parent/daughter regions. The goal with splits is to create two new daughter regions with the corresponding splits of t