Re: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-14 Thread Stack
On Fri, Sep 14, 2012 at 7:26 AM, Jonathan Hsieh wrote: > On Wed, Sep 12, 2012 at 11:21 AM, Stack wrote: > >> >> + Should there be more friction around committing? >> >> Suggestion that we try making lieutenants responsible for particular >> areas in the code base. These folks would have to +1 an

Re: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-14 Thread Jonathan Hsieh
On Wed, Sep 12, 2012 at 11:21 AM, Stack wrote: > > + Should there be more friction around committing? > > Suggestion that we try making lieutenants responsible for particular > areas in the code base. These folks would have to +1 anything that > touches their area. Formalize it in JIRA so these

Re: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-12 Thread Stack
On Wed, Sep 12, 2012 at 9:41 PM, Ramkrishna.S.Vasudevan wrote: > Thanks for the write up Stack. Some of my comments I can add up in a new > thread? > Up to you mighty Ram. St.Ack

RE: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-12 Thread Ramkrishna.S.Vasudevan
ubject: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase > Developer's Pow-wow] > > Here are notes from yesterday's contributor's meetup: > http://www.meetup.com/hbaseusergroup/events/80621872/ > > The notes below are spotty. I kept forgettin

Re: Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-12 Thread Enis Söztutar
Thanks Stack for the notes, and thanks everyone for coming. Cheers, Enis On Wed, Sep 12, 2012 at 11:21 AM, Stack wrote: > Here are notes from yesterday's contributor's meetup: > http://www.meetup.com/hbaseusergroup/events/80621872/ > > The notes below are spotty. I kept forgetting to take them

Notes from yesterday's Contributors Pow Wow [WAS --> Re: HBase Developer's Pow-wow]

2012-09-12 Thread Stack
Here are notes from yesterday's contributor's meetup: http://www.meetup.com/hbaseusergroup/events/80621872/ The notes below are spotty. I kept forgetting to take them. I also was unable to keep up w/ the rate of exchange so in many parts the reporting drops or a speaker's nuanced argument is cra

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Matt Corgan
t; Regards > Ram > > > > -Original Message----- > > From: Andrew Purtell [mailto:apurt...@apache.org] > > Sent: Tuesday, September 11, 2012 9:52 AM > > To: dev@hbase.apache.org > > Subject: Re: HBase Developer's Pow-wow. > > > > Regarding th

RE: HBase Developer's Pow-wow.

2012-09-10 Thread Ramkrishna.S.Vasudevan
> To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow-wow. > > Regarding this: > > On Mon, Sep 10, 2012 at 12:13 PM, Matt Corgan > wrote: > > 1) Per-region or Per-table > [...] > > 1) > > - Per-region: the index entries are stored on the same mac

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Andrew Purtell
Regarding this: On Mon, Sep 10, 2012 at 12:13 PM, Matt Corgan wrote: > 1) Per-region or Per-table [...] > 1) > - Per-region: the index entries are stored on the same machine as the > primary rows > - Per-table: each index is stored in a separate table, requiring > cross-server consistency LarsH

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Jacques
On Mon, Sep 10, 2012 at 6:20 PM, Matt Corgan wrote: > ... snipping lots of helpful use cases... It seems like portions of what you discussed would probably be nominally impacted by indexes while other would be very impacted. Also seems like compound-qualifier indexing would potentially be of i

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Matt Corgan
One sparse use case for us is rate limit detection. We store user events in an Event table whose primary key is a unique timestamp (sharded to avoid hotspotting) and which has eventType and ipAddress columns. We manually keep a separate table (the index, also sharded) called EventByDateIpType wit

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Devaraj Das
iscussion. > > Can I attend it over phone? Or someother way ? > > Regards > Ram > > > -Original Message- > > From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of > > Stack > > Sent: Thursday, August 30, 2012 2:03 AM > > To: de

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Jacques
> > All of my use-cases would require Per-table indexes. Per-region is easier > to keep consistent at write-time, but is seems useless to me for the large > tables that hbase is designed for (because you have to hit every region for > each read). > Can you expound on use cases? The pros and cons

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Matt Corgan
Can indexing be boiled down to these questions to start? 1) Per-region or Per-table 2) Sync or Async 3) Client-managed or Server-managed 4) Schema or Schema-less Definitions: 1) - Per-region: the index entries are stored on the same machine as the primary rows - Per-table: each index is stored i

Re: HBase Developer's Pow-wow.

2012-09-10 Thread lars hofhansl
) Let me know. I won't have time to prepare much for this, though. So it would be an ad hoc discussion, maybe with some white boarding. -- Lars - Original Message - From: Jesse Yates To: dev@hbase.apache.org Cc: Sent: Sunday, September 9, 2012 3:11 PM Subject: Re: HBase Developer&

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Jacques
> > > The use cases considered, at least over here at TM, all come down to > range scanning over values (e.g. WHERE INTEGER($value) < 50). So we > need a mapping such that a scan over the index returns either lists of > pointers to row:family:qualifier, or the value itself embedded in the > index,

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Jacques
See below On Mon, Sep 10, 2012 at 10:51 AM, Ted Yu wrote: > Jacques: > Thanks for your sharing. > > bq. row-level sharding as opposed to term > > Please elaborate on the above a little more: what is term sharding ? > If an index is basically a value (or term) pointing back to a row, there are t

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Andrew Purtell
On Mon, Sep 10, 2012 at 12:03 AM, Jacques wrote: >- How important is indexing column qualifiers themselves (similar to >Cassandra where people frequently utilize column qualifiers as "values" >with no actual values stored)? It would be good to have a secondary indexing option that can

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Andrew Purtell
Hi Jaques, > Does family level indexing make sense or is the real need for qualifier > level indexing? The use cases considered, at least over here at TM, all come down to range scanning over values (e.g. WHERE INTEGER($value) < 50). So we need a mapping such that a scan over the index returns ei

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Ted Yu
Jacques: Thanks for your sharing. bq. row-level sharding as opposed to term Please elaborate on the above a little more: what is term sharding ? bq. for what I will call a "local shadow family" I like this idea. User may request more than one index. Currently HBase is not so good at serving hig

Re: HBase Developer's Pow-wow.

2012-09-10 Thread Jacques
more food for thought on secondary indexing... *Additional questions*: - How important is indexing column qualifiers themselves (similar to Cassandra where people frequently utilize column qualifiers as "values" with no actual values stored)? - How important is indexing cell timestamp

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Stack
On Sun, Sep 9, 2012 at 8:03 PM, Jacques wrote: > Some random thoughts/questions bubbling around in my mind regarding > secondary indexes/indices. > Nice list Jacques. (Jesse, here is your chance to look real good. You are getting the questions in advance! When Jacques stands up to start asking

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Jacques
Some random thoughts/questions bubbling around in my mind regarding secondary indexes/indices. What are the top 5 use cases people are trying to solve? What solves more of these needs: synchronous 'transactional' or asynchronous best-effort (or delayed durable) index commit? Does family level inde

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Stack
On Sun, Sep 9, 2012 at 3:25 PM, Jesse Yates wrote: > On Sun, Sep 9, 2012 at 3:21 PM, Stack wrote: > >> On Sun, Sep 9, 2012 at 3:11 PM, Jesse Yates >> wrote: >> > I think we talked about wanting to do secondary indexing as well, as >> least >> > what that means for HBase (and maybe some of the _h

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Jesse Yates
On Sun, Sep 9, 2012 at 3:21 PM, Stack wrote: > On Sun, Sep 9, 2012 at 3:11 PM, Jesse Yates > wrote: > > I think we talked about wanting to do secondary indexing as well, as > least > > what that means for HBase (and maybe some of the _how_ it would work > too). > > > > Mind leading it Jesse? Yo

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Stack
On Sun, Sep 9, 2012 at 3:11 PM, Jesse Yates wrote: > I think we talked about wanting to do secondary indexing as well, as least > what that means for HBase (and maybe some of the _how_ it would work too). > Mind leading it Jesse? You have the necessary qualifications (smile). Would suggest you

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Jesse Yates
> > We are missing fellas to lead a chat on process change ideas (How to > have it so Jenkins is more blue than red; How do we enforce more rigor > around what gets committer, etc.). Anyone want to volunteer? I'd > volunteer LarsH since he was last to float these eternally recurring > notions but

Re: HBase Developer's Pow-wow.

2012-09-09 Thread Stack
On Mon, Sep 3, 2012 at 8:40 AM, Stack wrote: > On Fri, Aug 31, 2012 at 3:59 PM, Stack wrote: >> I'll put a post up on meetup.com for bay area hbase. > > I put the meetup up here: > http://www.meetup.com/hbaseusergroup/events/80621872/ (2pm at HWX). > Let me know if any of the details are off (Tha

Re: HBase Developer's Pow-wow.

2012-09-04 Thread Stack
On Tue, Sep 4, 2012 at 9:18 PM, Ramkrishna.S.Vasudevan wrote: > Stack, I may not be able to join seeing the time 2pm which is 2AM over here. > Anyway I can share my thoughts after the discussions are drafted in a > writeup. > Understood (Pardon our insensitivity arriving at a 2AM, for you, start

RE: HBase Developer's Pow-wow.

2012-09-04 Thread Ramkrishna.S.Vasudevan
Sent: Monday, September 03, 2012 9:11 PM > To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow-wow. > > On Fri, Aug 31, 2012 at 3:59 PM, Stack wrote: > > I'll put a post up on meetup.com for bay area hbase. > > I put the meetup up here: > http://www.meetu

Re: HBase Developer's Pow-wow.

2012-09-03 Thread Stack
On Fri, Aug 31, 2012 at 3:59 PM, Stack wrote: > I'll put a post up on meetup.com for bay area hbase. I put the meetup up here: http://www.meetup.com/hbaseusergroup/events/80621872/ (2pm at HWX). Let me know if any of the details are off (Thanks to Jon for the bulk of the text). St.Ack

Re: HBase Developer's Pow-wow.

2012-08-31 Thread Stack
On Thu, Aug 30, 2012 at 3:42 PM, Stack wrote: > On Thu, Aug 30, 2012 at 3:36 PM, Devaraj Das wrote: >> Should we move it to that week to accommodate Lars? >> > > We could. We had it set for the week of 10th so Andrew could come. > Andrew could you come the following week? An off-list exchange h

Re: HBase Developer's Pow-wow.

2012-08-30 Thread Stack
On Thu, Aug 30, 2012 at 3:36 PM, Devaraj Das wrote: > Should we move it to that week to accommodate Lars? > We could. We had it set for the week of 10th so Andrew could come. Andrew could you come the following week? St.Ack

Re: HBase Developer's Pow-wow.

2012-08-30 Thread Devaraj Das
Should we move it to that week to accommodate Lars? On Aug 30, 2012, at 3:04 PM, Lars George wrote: > Bummer, I will be in PA the week after, i.e. 9/17. It would have been great > to see you all again. > > Lars > > On Aug 29, 2012, at 10:15 PM, Andrew Purtell wrote: > >> +1 >> >> I can be u

Re: HBase Developer's Pow-wow.

2012-08-30 Thread Lars George
Bummer, I will be in PA the week after, i.e. 9/17. It would have been great to see you all again. Lars On Aug 29, 2012, at 10:15 PM, Andrew Purtell wrote: > +1 > > I can be up week of the 10th if that's convenient. > >- Andy > > On Wednesday, August 29, 2012, Jimmy Xiang wrote: > >> +1

Re: HBase Developer's Pow-wow.

2012-08-30 Thread Ted Yu
For 'Strategies for keeping Jenkins blue', we can take this opportunity to fix a few known flaky tests. Cheers On Wed, Aug 29, 2012 at 11:30 AM, Jonathan Hsieh wrote: > There are a couple discussions brewing and major changes being it would be > good to have a face-to-face pow-wow to demo, to d

Re: HBase Developer's Pow-wow.

2012-08-30 Thread N Keywal
gt; > To: dev@hbase.apache.org > > Subject: Re: HBase Developer's Pow-wow. > > > > Sure, Jon (assuming it works for everyone else). I will start the ball > > rolling on the logistics. > > > > Sent from my iPhone > > > > On Aug 29, 2012, at 4:13 PM, Jo

RE: HBase Developer's Pow-wow.

2012-08-30 Thread Ramkrishna.S.Vasudevan
Hi Latest improvement, Tuesday should be best for me. +1 for Tuesday. Regards Ram > -Original Message- > From: Devaraj Das [mailto:d...@hortonworks.com] > Sent: Thursday, August 30, 2012 11:42 AM > To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Andrew Purtell
Thanks for being accommodating, guys. See you on the 11th. I'll have to bring some elephant swag for the HW office. - Andy On Thu, Aug 30, 2012 at 9:12 AM, Devaraj Das wrote: > Sure, Jon (assuming it works for everyone else). I will start the ball > rolling on the logistics. > > Sent from m

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Devaraj Das
Sure, Jon (assuming it works for everyone else). I will start the ball rolling on the logistics. Sent from my iPhone On Aug 29, 2012, at 4:13 PM, Jonathan Hsieh wrote: > Great! > > If we are going down to Sunnyvale, I'd prefer a Tuesday. DD, could we > schedule for a Tuesday 9/11 afternoon? > >

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Stack
On Wed, Aug 29, 2012 at 9:35 PM, Ramkrishna.S.Vasudevan wrote: > It should be ok, atleast for one day :). > Yeah, for the next one, we promise that we'll all get up at 4AM so you can dial in at a reasonable 5PM (smile). St.Ack

RE: HBase Developer's Pow-wow.

2012-08-29 Thread Ramkrishna.S.Vasudevan
It should be ok, atleast for one day :). Regards Ram > -Original Message- > From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of > Stack > Sent: Thursday, August 30, 2012 9:56 AM > To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow-wow. &g

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Stack
On Wed, Aug 29, 2012 at 9:21 PM, Ramkrishna.S.Vasudevan wrote: > I would be interested on this may be for the Secondary index related > discussion. > > Can I attend it over phone? Or someother way ? > We would love to have you Ram. 5PM our time is 5AM your time? Thats kinda early Ram. St.Ack

RE: HBase Developer's Pow-wow.

2012-08-29 Thread Ramkrishna.S.Vasudevan
2:03 AM > To: dev@hbase.apache.org > Subject: Re: HBase Developer's Pow-wow. > > On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das > wrote: > > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > > > On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Jonathan Hsieh
Great! If we are going down to Sunnyvale, I'd prefer a Tuesday. DD, could we schedule for a Tuesday 9/11 afternoon? Jon. On Wed, Aug 29, 2012 at 2:43 PM, Devaraj Das wrote: > Cool! I'll make sure the beer is good :-) > > On Aug 29, 2012, at 1:32 PM, Stack wrote: > > > On Wed, Aug 29, 2012 at 1

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Devaraj Das
Cool! I'll make sure the beer is good :-) On Aug 29, 2012, at 1:32 PM, Stack wrote: > On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: >> We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? >> >> On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: >> >>> There are a couple d

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Stack
On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > >> There are a couple discussions brewing and major changes being it would be >> good to have a face-to-face pow-wow t

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Andrew Purtell
+1 I can be up week of the 10th if that's convenient. - Andy On Wednesday, August 29, 2012, Jimmy Xiang wrote: > +1 > > On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das > > > wrote: > > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > > > On Aug 29, 2012, at 11:31 AM, Jon

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Jimmy Xiang
+1 On Wed, Aug 29, 2012 at 1:06 PM, Devaraj Das wrote: > We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? > > On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > >> There are a couple discussions brewing and major changes being it would be >> good to have a face-to-face pow-w

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Devaraj Das
We could look at hosting here at Hortonworks, Sunnyvale. Thoughts? On Aug 29, 2012, at 11:31 AM, Jonathan Hsieh wrote: > There are a couple discussions brewing and major changes being it would be > good to have a face-to-face pow-wow to demo, to discuss designs, and to > talk about project goals

Re: HBase Developer's Pow-wow.

2012-08-29 Thread Ted Yu
I would vote for Cloudera PA office. Thanks Jon for this initiative. On Wed, Aug 29, 2012 at 11:30 AM, Jonathan Hsieh wrote: > There are a couple discussions brewing and major changes being it would be > good to have a face-to-face pow-wow to demo, to discuss designs, and to > talk about projec