Re: HBase Developer's Pow-wow.

Thanks for being accommodating, guys. See you on the 11th. I'll have to bring some elephant swag for the HW office. - Andy On Thu, Aug 30, 2012 at 9:12 AM, Devaraj Das wrote: > Sure, Jon (assuming it works for everyone else). I will start the ball > rolling on the logistics. > > Sent from m

Re: Improving Coprocessor postSplit/postOpen synchronization

Hi Kevin, > Just to clarify Andrew do you have a prototype patch up that could potentially be worked on to either move postSplit() or add new hooks into the framework/are planning on submitting it sometime in the near future? No, I meant one of the patches I put up on HBASE-2000. Basically CP des

Build failed in Jenkins: HBase-0.92-security #124

See Changes: [jmhsieh] HBASE-6686 HFile Quarantine fails with missing dirs in hadoop 2.0 -- [...truncated 7211 lines...] Forking command line: /bin/sh -c cd /x1/jenkins/jenkins-slave/workspac

Re: [Discuss] Release 0.92.2?

On Wed, Aug 29, 2012 at 11:03 AM, Ted Yu wrote: > According to: > https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/519/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSplitBeforeSettingSplittingInZK/ > > The test failure was due to: > > java.lang

Re: HBase Developer's Pow-wow.

Sure, Jon (assuming it works for everyone else). I will start the ball rolling on the logistics. Sent from my iPhone On Aug 29, 2012, at 4:13 PM, Jonathan Hsieh wrote: > Great! > > If we are going down to Sunnyvale, I'd prefer a Tuesday. DD, could we > schedule for a Tuesday 9/11 afternoon? > >

Re: HBase Developer's Pow-wow.

On Wed, Aug 29, 2012 at 9:35 PM, Ramkrishna.S.Vasudevan wrote: > It should be ok, atleast for one day :). > Yeah, for the next one, we promise that we'll all get up at 4AM so you can dial in at a reasonable 5PM (smile). St.Ack

RE: HBase Developer's Pow-wow.

It should be ok, atleast for one day :). Regards Ram > -Original Message- > From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of > Stack > Sent: Thursday, August 30, 2012 9:56 AM > To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow-wow. > > On Wed, Aug 29, 2012

RE: A general question on maxVersion handling when we have Secondary index tables

Reg , the collocation part of the main table regions and index table regions, that is pretty much necessary. Reg, how secondary index feature can be supported either as external or core-> I would say that seeing the current things that we have done it can be like security means secondary index ca

Re: HBase Developer's Pow-wow.

On Wed, Aug 29, 2012 at 9:21 PM, Ramkrishna.S.Vasudevan wrote: > I would be interested on this may be for the Secondary index related > discussion. > > Can I attend it over phone? Or someother way ? > We would love to have you Ram. 5PM our time is 5AM your time? Thats kinda early Ram. St.Ack

RE: HBase Developer's Pow-wow.

I would be interested on this may be for the Secondary index related discussion. Can I attend it over phone? Or someother way ? Regards Ram > -Original Message- > From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of > Stack > Sent: Thursday, August 30, 2012 2:03 AM > To: d

RE: A general question on maxVersion handling when we have Secondary index tables

Yes Jon. You got it right. This is the problem. But all the implementation we need to have some type of mechanism where we go thro all the rows in the sec index table. Suppose in the below example if I say in my main table maxVersions is 5. So I will scan the top 5 values from the sec index tabl

Re: HBase Developer's Pow-wow.

Great! If we are going down to Sunnyvale, I'd prefer a Tuesday. DD, could we schedule for a Tuesday 9/11 afternoon? Jon. On Wed, Aug 29, 2012 at 2:43 PM, Devaraj Das wrote: > Cool! I'll make sure the beer is good :-) > > On Aug 29, 2012, at 1:32 PM, Stack wrote: > > > On Wed, Aug 29, 2012 at 1

Re: A general question on maxVersion handling when we have Secondary index tables

On Tue, Aug 28, 2012 at 9:03 AM, Ted Yu wrote: > I think this discussion should be on HBASE JIRA. I disagree (I've already quoted you the 'discussion on mailing list, not in JIRA' section from the Karl Fogel book). St.Ack

Build failed in Jenkins: HBase-TRUNK #3293

See Changes: [stack] HBASE-5965 Move replication znodes to pb -- [...truncated 2615 lines...] Running org.apache.hadoop.hbase.master.TestRestartCluster Tests run: 2, Failures: 0, Errors: 0, Skipped:

Re: HBase Developer's Pow-wow.

Cool! I'll make sure the beer is good :-) On Aug 29, 2012, at 1:32 PM, Stack wrote: > On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: >> We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? >> >> On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: >> >>> There are a couple d

Re: HBase Developer's Pow-wow.

On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > >> There are a couple discussions brewing and major changes being it would be >> good to have a face-to-face pow-wow t

Re: HBase Developer's Pow-wow.

+1 I can be up week of the 10th if that's convenient. - Andy On Wednesday, August 29, 2012, Jimmy Xiang wrote: > +1 > > On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das > > > wrote: > > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > > > On Aug 29, 2012, at 11:31 AM, Jon

Re: HBase Developer's Pow-wow.

+1 On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > >> There are a couple discussions brewing and major changes being it would be >> good to have a face-to-face pow-w

Re: HBase Developer's Pow-wow.

We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > There are a couple discussions brewing and major changes being it would be > good to have a face-to-face pow-wow to demo, to discuss designs, and to > talk about project goals

Re: HBase Developer's Pow-wow.

I would vote for Cloudera PA office. Thanks Jon for this initiative. On Wed, Aug 29, 2012 at 11:30 AM, Jonathan Hsieh wrote: > There are a couple discussions brewing and major changes being it would be > good to have a face-to-face pow-wow to demo, to discuss designs, and to > talk about projec

HBase Developer's Pow-wow.

There are a couple discussions brewing and major changes being it would be good to have a face-to-face pow-wow to demo, to discuss designs, and to talk about project goals and policies. This would be mostly focused on project internals and maybe last half a day. Here are some suggestions for agen

Re: A general question on maxVersion handling when we have Secondary index tables

We should have a hbave dev meeting and bring this as one of the topics of discussion to bring up. I'll start another thread on that. On Wed, Aug 29, 2012 at 10:03 AM, Jesse Yates wrote: > Client library style stuff is _nice_ but one of the things everyone asks of > database is that we provide an

Re: [Discuss] Release 0.92.2?

According to: https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92/519/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransactionOnCluster/testSplitBeforeSettingSplittingInZK/ The test failure was due to: java.lang.RuntimeException: java.io.FileNotFoundException: /home/hud

Build failed in Jenkins: HBase-TRUNK #3292

See Changes: [Tedyu] HBASE-6564 HDFS space is not reclaimed when a column family is deleted (J Mohamed) -- [...truncated 2616 lines...] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed

Jenkins build is back to normal : HBase-0.92-security #123

See

Re: A general question on maxVersion handling when we have Secondary index tables

Let me rephrase to make sure I'm on the same page for the ram's question: We do three inserts on row 1 at different times to the same column (which is being indexed in a secondary table) (Are we assuming only a 1-to-1 secondary->primary mapping?) t1< t2 wrote: > When we have many to one mappin

Re: [0.92.2] Consistent error : Is a split parent in META, in HDFS, and not deployed

Maybe you can dig up master / region server logs to see if there was any of the following happening at time of merge ? region server failover load balancer run zookeeper session expiration. I feel we should make region merging more robust. Cheers On Wed, Aug 29, 2012 at 9:44 AM, Shrijeet Paliwal

Build failed in Jenkins: HBase-TRUNK #3291

See Changes: [Tedyu] HBASE-6659 Port HBASE-6508 Filter out edits at log split time [nkeywal] HBASE-6655 thrift.TestThriftServerCmdLine takes too much time: 733.972 sec -- [...truncated 2650 lines..

Re: A general question on maxVersion handling when we have Secondary index tables

I agree with Jesse. For the initial implementation, we can pick a common use case. When users present more use cases, we add more support in HBase core. On Wed, Aug 29, 2012 at 10:03 AM, Jesse Yates wrote: > Client library style stuff is _nice_ but one of the things everyone asks of > database is

Re: A general question on maxVersion handling when we have Secondary index tables

Client library style stuff is _nice_ but one of the things everyone asks of database is that we provide an index (cassandra has it, riak has it, mysql has it...hbase doesn't? Yes, different systems,etc.,etc., but the point is we could do it). Further, if we build it as a part of hbase, we can make

Re: [0.92.2] Consistent error : Is a split parent in META, in HDFS, and not deployed

Ted, Most likely Yes. On Wed, Aug 29, 2012 at 9:41 AM, Ted Yu wrote: > Thanks for sharing your experience, Shrijeet. > > Was the region cited below involved in your previous merging activity ? > > Cheers > > On Wed, Aug 29, 2012 at 9:36 AM, Shrijeet Paliwal > wrote: > > > Hi Devs, > > > > I did

Re: [0.92.2] Consistent error : Is a split parent in META, in HDFS, and not deployed

Thanks for sharing your experience, Shrijeet. Was the region cited below involved in your previous merging activity ? Cheers On Wed, Aug 29, 2012 at 9:36 AM, Shrijeet Paliwal wrote: > Hi Devs, > > I did an upgrade to HBase 0.92.2 recently. It ran fine for couple of days, > then I did online mer

[0.92.2] Consistent error : Is a split parent in META, in HDFS, and not deployed

Hi Devs, I did an upgrade to HBase 0.92.2 recently. It ran fine for couple of days, then I did online merge of regions. This got into a bit of inconsistent state. To fix it the corruption I ran hbck and managed to get into a situation described in http://search-hadoop.com/m/sm2APK8FMo1&subj=Re+Won

Re: A general question on maxVersion handling when we have Secondary index tables

For the secondary index based on state portion of address example, I wonder if we can achieve comparable performance using scan with proper filter. Cheers On Wed, Aug 29, 2012 at 9:11 AM, Jonathan Hsieh wrote: > Ted, > > Ram's summarizes the concern succinctly -- to answer the specific question

Re: A general question on maxVersion handling when we have Secondary index tables

Ted, Ram's summarizes the concern succinctly -- to answer the specific question it isn't for versions -- it is for the case where a secondary index can point to many many primary rows. (let's say we have a rowkey userid and we want to have a 2ndary index based on the state portion of there addres

Build failed in Jenkins: HBase-0.92-security #122

See -- [...truncated 7213 lines...] Running org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.934 sec Forking command line:

RE: A general question on maxVersion handling when we have Secondary index tables

When we have many to one mapping between main and secondary index table may be we will end up in hitting many RS. If there is one to one mapping may be that is not a problem. Basically my intention of this discussion was mainly to discuss on the version maintenance on any type of secondary index p

Re: A general question on maxVersion handling when we have Secondary index tables

Thanks for the detailed response, Jon. bq. it would mean that a query based on secondary index would potentially have to hit every region server that has a region in the primary table. Can you elaborate on the above a little bit ? Is this because secondary index would point us to more than one re

Re: A general question on maxVersion handling when we have Secondary index tables

I'm more of a fan of having secondary indexes added as an external feature (coproc or new client library on top of our current client library) and focusing on only adding apis necessary to make 2ndary indexes possible and correct on/in HBase. There are many different use patterns and requirements