Re: Moving 2.0 forward

2018-03-01 Thread Stack
In another thread in this list I flag the coming 2.0.0-beta-2. Look for an RC in the next day or so. After 2.0.0-beta-2 makes it out, I'll cut branch-2.0. Soon after will come our first hbase-2.0.0 release candidate. Expect this within a week or so after beta-2 after remaining blockers are address

Re: Moving 2.0 forward

2017-12-27 Thread Stack
Heads-up, I'm working on cutting a beta-1. Shout if there is something you think needs to make it in (The Chia-Ping Tsai cleanups look good). I'm mainly waiting on the hbase-thirdparty changes to go in. The rest can wait it. Thanks, S On Wed, Dec 13, 2017 at 11:28 PM, Stack wrote: > Update. Se

Re: Moving 2.0 forward

2017-12-13 Thread Stack
Update. See below. On Tue, Dec 5, 2017 at 10:43 AM, Stack wrote: > Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before > Christmas day. > > Still aiming for an hbase2 beta-1 showing up in your christmas stocking. > Here is the list [1]. We're doing pretty good. We've fixed a l

Re: Moving 2.0 forward

2017-12-05 Thread Stack
Update on hbase-2.0.0-beta-1. I'm going to put up a beta-1 RC before Christmas day. Here is the list [1]. We're doing pretty good. We've fixed a load since the last update including some nice cleanups two weeks ago (e.g. undoing client dependency on curator/zk watching making the dependence read-

Re: Moving 2.0 forward

2017-11-16 Thread Stack
On Thu, Nov 16, 2017 at 8:08 PM, ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > HBASE-18946 Stochastic load balancer assigns replica regions to the same RS > > Am on this today and probably will put up another patch. > > Regards > Ram > > Thanks Ram, St.Ack > On Fri, Nov 17,

Re: Moving 2.0 forward

2017-11-16 Thread ramkrishna vasudevan
HBASE-18946 Stochastic load balancer assigns replica regions to the same RS Am on this today and probably will put up another patch. Regards Ram On Fri, Nov 17, 2017 at 5:18 AM, Stack wrote: > hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last > remaining features and apply fi

Re: Moving 2.0 forward

2017-11-16 Thread Stack
hbase-2.0.0-beta-1 update (Reminder, beta-1 is where we finish last remaining features and apply final polish to API. There will be a beta-2 but it is about upgrade/rolling-upgrade and bug fixes ONLY). Myself and Mr. Drob did a pass over the outstanding hbase-2.0.0-beta-1 list this morning. See he

Re: Moving 2.0 forward

2017-10-31 Thread Stack
On Tue, Oct 31, 2017 at 2:31 PM, Mike Drob wrote: > Hoping to keep momentum going from our Stack working on alpha4, I tried to > take a stab at triaging some of the open beta-1 issues. > > I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner > then I'm happy to see it pulled

Re: Moving 2.0 forward

2017-10-31 Thread Mike Drob
Hoping to keep momentum going from our Stack working on alpha4, I tried to take a stab at triaging some of the open beta-1 issues. I moved some docs stuff out form beta-1 to 2.0-GA, if it gets done sooner then I'm happy to see it pulled back in. Trying to balance optimism with realism here, and kn

Re: Moving 2.0 forward

2017-10-31 Thread Josh Elser
+1 go from my POV. On 10/31/17 10:07 AM, Stack wrote: I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). They need more time. We'll pull them in for beta-1. CP API is basically done. There may be some changes for beta-1 but hopefully only changes informed by experience tr

Re: Moving 2.0 forward

2017-10-31 Thread Stack
I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). They need more time. We'll pull them in for beta-1. CP API is basically done. There may be some changes for beta-1 but hopefully only changes informed by experience trying to port an existing Coprocessor to hbase2. Shout if

Re: Moving 2.0 forward

2017-10-28 Thread Josh Elser
Yup, that was going to be my plan, Mike! Making a pass now, and will check back later tonight again. I see others have already done some work today on this front. On 10/27/17 11:38 PM, Mike Drob wrote: Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do it directly on th

Re: Moving 2.0 forward

2017-10-27 Thread Mike Drob
Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do it directly on the jenkins job, so you don't have to bother with JIRA uploads) If you're busy, then I can make time tomorrow or Sunday to kick off jobs, but I want to make sure we're not duplicating effort and jenkins cycles

Re: Moving 2.0 forward

2017-10-27 Thread Josh Elser
My turn to bump ;) By my take: HBASE-18770 and HBASE-19092 are the only issues that remain needing some more work. The rest are just awaiting a good QA run. Unless I hear otherwise, I'll try to keep an eye on things over the weekend, bump them along as necessary, and get them committed. Would

Re: Moving 2.0 forward

2017-10-26 Thread Guanghao Zhang
> > For public API we have rules on how to keep compatible so I think it is > less hurt for users, beta1 is fine. > Got it. Thanks for you explanation. When you think you'd be done w/ the sync of Admin and AsyncAdmin > Interfaces? Are the changes to Admin or to AsyncAdmin? There are two issues (H

Re: Moving 2.0 forward

2017-10-26 Thread Stack
On Thu, Oct 26, 2017 at 2:27 AM, Guanghao Zhang wrote: > I saw beta == no new features, no API changes, just fixes. And I am working > on HBASE-18805 to unify Admin and AsyncAdmin methods. The fix version was > 2.0-beta-1. But I thought this will introduce API change(deprecate some API > and intr

Re: Moving 2.0 forward

2017-10-26 Thread Duo Zhang
Alpha4 aims to freeze the CP related API, not the public API. We break everything for CP so we need to push out a stable(I hope) version to let CP users learn how to implement their project on top of the new APIs. For public API we have rules on how to keep compatible so I think it is less hurt fo

Re: Moving 2.0 forward

2017-10-26 Thread Guanghao Zhang
I saw beta == no new features, no API changes, just fixes. And I am working on HBASE-18805 to unify Admin and AsyncAdmin methods. The fix version was 2.0-beta-1. But I thought this will introduce API change(deprecate some API and introduce new one). So should I change the fix versions to 2.0-alpha-

Re: Moving 2.0 forward

2017-10-25 Thread Duo Zhang
OK, skimmed, we are in trouble! The in memory compaction just use the same constructor with normal compaction to construct a StoreScanner, and use it to do compaction... We have to provide several preXXX and postXXX for it, at least we should allow user reset TTL and max versions, and also do fil

Re: Moving 2.0 forward

2017-10-25 Thread Duo Zhang
When adding back the pre(Flush/Compact/Store)ScannerOpen methods in HBASE-19033, I found that there maybe a hole in our CP hools. It is the in memory compaction. As Anoop suggested in HBASE-19001, we still need to give user the ability to extend the max versions and TTL config, so I plan to add ba

Re: Moving 2.0 forward

2017-10-24 Thread Stack
Chatting with my coworker Mr. Mike Drob, we were batting back and forth what remains to be done. Surfacing our thoughts here so you all clued inOr if you think otherwise, please speak up. We have ~13 issues to land: https://issues.apache.org/jira/projects/HBASE/versions/12341594 About two are

Re: Moving 2.0 forward

2017-10-24 Thread ramkrishna vasudevan
Thanks Stack. Started working on it. Will post a patch ASAP. On Tue, Oct 24, 2017 at 11:38 AM, Stack wrote: > That'd be a good one to get in Ram. > S > > On Mon, Oct 23, 2017 at 9:42 PM, ramkrishna vasudevan < > ramkrishna.s.vasude...@gmail.com> wrote: > > > Hi Stack > > > > Do you want HBASE-18

Re: Moving 2.0 forward

2017-10-23 Thread Stack
That'd be a good one to get in Ram. S On Mon, Oct 23, 2017 at 9:42 PM, ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > Hi Stack > > Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal APIs > from CellUtil)? Because you had mentioned no more API changes. If so I

Re: Moving 2.0 forward

2017-10-23 Thread ramkrishna vasudevan
Hi Stack Do you want HBASE-18995 before the alpha-4 (REmoving exposed internal APIs from CellUtil)? Because you had mentioned no more API changes. If so I will start making changes and put up a patch ASAP. Regards Ram On Tue, Oct 24, 2017 at 3:22 AM, Stack wrote: > On Mon, Oct 23, 2017 at 11:5

Re: Moving 2.0 forward

2017-10-23 Thread Stack
On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser wrote: > +1 > > I was trying to work on helping out on the outstanding alpha-4 stuff last > week -- will be continuing to try to do the same this week. > > If you need any help, Stack, or if others need reviews where I haven't > noticed on my own: feel

Re: Moving 2.0 forward

2017-10-23 Thread Josh Elser
+1 I was trying to work on helping out on the outstanding alpha-4 stuff last week -- will be continuing to try to do the same this week. If you need any help, Stack, or if others need reviews where I haven't noticed on my own: feel free to @mention me. On 10/23/17 12:53 PM, Stack wrote: (R

Re: Moving 2.0 forward

2017-10-23 Thread Stack
(Reviving this thread) Lets push out alpha-4 this week. Alpha-4 is the release that has the refactor of the Coprocessor API shutting down access to internals marked InterfaceAudience.Private. The outstanding list is here: https://issues.apache.org/jira/projects/HBASE/versions/12341594 Please pus

Re: Moving 2.0 forward

2017-09-08 Thread Stack
I'll put up an alpha3 RC Monday, probably Monday night. That should be time, if we all sprint, for the public-facing API fixes to be done. I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 but it is plain that more time is needed (in spite of valiant effort so far by Anoop, Duo,

Re: Moving 2.0 forward

2017-08-17 Thread Stack
On Thu, Aug 17, 2017 at 12:45 PM, Mike Drob wrote: > I went ahead and tagged those as tentatively 2.0.0-alpha3 in jira so that > we can have them all in one place later. Folks can move additional issues > in and out as they see appropriate. > > Thanks M, S > On Thu, Aug 17, 2017 at 2:35 PM, St

Re: Moving 2.0 forward

2017-08-17 Thread Mike Drob
I went ahead and tagged those as tentatively 2.0.0-alpha3 in jira so that we can have them all in one place later. Folks can move additional issues in and out as they see appropriate. On Thu, Aug 17, 2017 at 2:35 PM, Stack wrote: > I put up the hbase-2.0.0-alpha2 release candidate. Please vote o

Re: Moving 2.0 forward

2017-08-17 Thread Stack
I put up the hbase-2.0.0-alpha2 release candidate. Please vote on it. For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to get a release out in the next week or so. I did a weeding of 2.0.0 issues over the last day. If folks are interested in helping out, below are the items I think we

Re: Moving 2.0 forward

2017-08-17 Thread Stack
On Wed, Aug 16, 2017 at 10:18 PM, Guanghao Zhang wrote: > About compatibility, there is one incompatible change about the replication > TableCFs' config. The old config is a string and it concatenate the list of > tables and column families in format "table1:cf1,cf2;table2:cfA,cfB" in > zookeeper

Re: Moving 2.0 forward

2017-08-16 Thread Guanghao Zhang
About compatibility, there is one incompatible change about the replication TableCFs' config. The old config is a string and it concatenate the list of tables and column families in format "table1:cf1,cf2;table2:cfA,cfB" in zookeeper for table-cf to replication peer mapping. When parse the config,

Re: Moving 2.0 forward

2017-08-14 Thread Stack
Heads-up: I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme is updated dependencies, reliance on relocated popular libs (guava, netty, protobuf), purge of checked-in generated src, and master-carries-no-regions by default. alpha3 I hope will follow soon after (end-of-August?). It

Re: Moving 2.0 forward

2017-08-02 Thread Stack
On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser wrote: > > > On 7/31/17 9:00 AM, Stack wrote: > >> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser wrote: >> >> ... >>> >>> I like the idea of this also hitting 2.0 as it would make the feature a >>> bit more "real", but am obviously a little nervous (I ha

Re: Moving 2.0 forward

2017-08-01 Thread Josh Elser
On 7/31/17 9:00 AM, Stack wrote: On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser wrote: ... I like the idea of this also hitting 2.0 as it would make the feature a bit more "real", but am obviously a little nervous (I have no reason to be nervous though). I am pretty happy with the feature in

Re: Moving 2.0 forward

2017-07-31 Thread Sean Busbey
What do folks think of this? I should start a separate thread on upgrade > expectations? > > St.Ack > > > >> I think that should not be imposed on the user. >> >> Regards, >> Ashish >> >> -Original Message- >> From: 张铎(Duo Zha

Re: Moving 2.0 forward

2017-07-31 Thread Stack
s, > Ashish > > -Original Message- > From: 张铎(Duo Zhang) [mailto:palomino...@gmail.com] > Sent: 26 July 2017 12:10 > To: dev@hbase.apache.org > Subject: Re: Moving 2.0 forward > > I set HBASE-18169 as a Blocker because I found some critial problems on > our cur

Re: Moving 2.0 forward

2017-07-31 Thread Stack
On Wed, Jul 26, 2017 at 1:39 AM, 张铎(Duo Zhang) wrote: > I set HBASE-18169 as a Blocker because I found some critial problems on our > current CPs. The semantics are broken. Although we are allowed to break CP > in a major release, I think we need to provide the same ability(in another > way). > >

Re: Moving 2.0 forward

2017-07-31 Thread Stack
On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser wrote: > ... > > I like the idea of this also hitting 2.0 as it would make the feature a > bit more "real", but am obviously a little nervous (I have no reason to be > nervous though). I am pretty happy with the feature in terms of how much it > is cov

Re: Moving 2.0 forward

2017-07-26 Thread karthikeyan thirunavukkarasu
Hi All, May be wrong forum, but how to unsubscribe my email id from this email list? ThanksKarthik From: ashish singhi To: "dev@hbase.apache.org" Sent: Wednesday, July 26, 2017 8:57 PM Subject: RE: Moving 2.0 forward One question regarding upgrade: Do we expect the us

RE: Moving 2.0 forward

2017-07-26 Thread ashish singhi
To: dev@hbase.apache.org Subject: Re: Moving 2.0 forward I set HBASE-18169 as a Blocker because I found some critial problems on our current CPs. The semantics are broken. Although we are allowed to break CP in a major release, I think we need to provide the same ability(in another way). 2017-07

Re: Moving 2.0 forward

2017-07-25 Thread Duo Zhang
I set HBASE-18169 as a Blocker because I found some critial problems on our current CPs. The semantics are broken. Although we are allowed to break CP in a major release, I think we need to provide the same ability(in another way). 2017-07-25 1:25 GMT+08:00 Josh Elser : > > > On 7/21/17 12:03 PM,

Re: Moving 2.0 forward

2017-07-24 Thread Josh Elser
On 7/21/17 12:03 PM, Stack wrote: Status update girls and boys! hbase-2.0.0-alpha1 went out June 22nd. alpha2 has been a bit slow to follow (holidays) though there has been steady progress closing out blockers and criticals by a bunch of you all. The plan is for a release in the first week or

Re: Moving 2.0 forward

2017-07-21 Thread Mike Drob
Yea, beta sounds good to me. Probably we'll find more things once we start running ITBLL and friends for long periods, but that can maybe come later. Existing tests should be cleaned up "soon". On Fri, Jul 21, 2017 at 3:45 PM, Andrew Purtell wrote: > Beta? > > > On Fri, Jul 21, 2017 at 1:16 PM,

Re: Moving 2.0 forward

2017-07-21 Thread Andrew Purtell
Beta? On Fri, Jul 21, 2017 at 1:16 PM, Stack wrote: > On Fri, Jul 21, 2017 at 5:30 PM, Mike Drob wrote: > > > > > One thing that is not clear to me from your summary and the linked > document > > is which release we can block on tests? Unit tests are in bad shape (but > > getting better) and I

Re: Moving 2.0 forward

2017-07-21 Thread Stack
On Fri, Jul 21, 2017 at 5:30 PM, Mike Drob wrote: > > One thing that is not clear to me from your summary and the linked document > is which release we can block on tests? Unit tests are in bad shape (but > getting better) and I recently started looking at ITs which also need some > care. > > Dun

Re: Moving 2.0 forward

2017-07-21 Thread Mike Drob
Thanks for tracking this, Stack! One thing that is not clear to me from your summary and the linked document is which release we can block on tests? Unit tests are in bad shape (but getting better) and I recently started looking at ITs which also need some care. Mike On Fri, Jul 21, 2017 at 11:0

Re: Moving 2.0 forward

2017-07-21 Thread Stack
Status update girls and boys! hbase-2.0.0-alpha1 went out June 22nd. alpha2 has been a bit slow to follow (holidays) though there has been steady progress closing out blockers and criticals by a bunch of you all. The plan is for a release in the first week or so of August. It should be fully up o

Re: Moving 2.0 forward

2017-05-15 Thread Stack
On Mon, May 15, 2017 at 1:46 AM, 张铎(Duo Zhang) wrote: > HBASE-18037 is a new blocker. I'm currently working on it, will be finished > soon I think. > > I made it a blocker then and added it to our hbase2 release doc [1] list as a blocker. Thanks, St.Ack 1. https://docs.google.com/document/d/1WC

Re: Moving 2.0 forward

2017-05-15 Thread Duo Zhang
HBASE-18037 is a new blocker. I'm currently working on it, will be finished soon I think. 2017-05-15 14:12 GMT+08:00 Stack : > A month on. Status. > > I've been working on the HBASE-14614 branch cluster testing. After a load > of fixing, the branch passes smaller test runs (an hour or so of ITBLL

Re: Moving 2.0 forward

2017-05-14 Thread Stack
A month on. Status. I've been working on the HBASE-14614 branch cluster testing. After a load of fixing, the branch passes smaller test runs (an hour or so of ITBLL up to 2B rows w/ killing monkeys). When I go larger, to a scale I've not done in a while, I start to run into other interesting issue

Re: Moving 2.0 forward

2017-04-13 Thread Stack
Some status: AMv2 (HBASE-14614) is near to passing all tests caveat my disabling of all to-do w/ fsck (fsck needs revamp) and tests that expect that they can move hbase;meta off master (AMv2 enforces this constraint; it is supposed to be enforced on AMv1 but meta-on-master is incompletely realized

Re: Moving 2.0 forward

2017-03-31 Thread Andrew Purtell
+1 on branching (yay!) I have EC2 resources for running ITBLL etc. On Thu, Mar 30, 2017 at 5:07 PM, Stack wrote: > Some notes on progress toward hbase2. > > Given that stability and performance are NOT emergent behaviors but rather > projects unto themselves, my thought is that we commit all t

Re: Moving 2.0 forward

2017-03-31 Thread Stack
On Thu, Mar 30, 2017 at 6:22 PM, Enis Söztutar wrote: > Thanks Stack for the update. +1 on branching as soon as possible. For > getting aforementioned stability, we need to start rejecting patches/ > features from 2.0.0. Branching early gives us the option of gradually > working towards that, but

Re: Moving 2.0 forward

2017-03-31 Thread Yu Li
+1 on early branch, and count me in for the coming stability/perf projects (smile). Best Regards, Yu On 31 March 2017 at 09:22, Enis Söztutar wrote: > Thanks Stack for the update. +1 on branching as soon as possible. For > getting aforementioned stability, we need to start rejecting patches/ >

Re: Moving 2.0 forward

2017-03-30 Thread Enis Söztutar
Thanks Stack for the update. +1 on branching as soon as possible. For getting aforementioned stability, we need to start rejecting patches/ features from 2.0.0. Branching early gives us the option of gradually working towards that, but also does not block new development to happen on master. I thin

Re: Moving 2.0 forward

2017-03-30 Thread Stack
Some notes on progress toward hbase2. Given that stability and performance are NOT emergent behaviors but rather projects unto themselves, my thought is that we commit all that we've agreed as core for hbase2 (see [1]), branch, and then work on stabilizing and perf rather than do stabilize, commit

Re: Moving 2.0 forward

2017-03-11 Thread Josh Elser
Stack wrote: On Tue, Mar 7, 2017 at 1:51 PM, Josh Elser wrote: Thanks for pulling in the FS Quotas work, Stack. I'm trying to cross the last T's and dot the last I's. The biggest thing I know I need to do still is to write a new chapter to the book. After that, I'd start entertaining larger

Re: Moving 2.0 forward

2017-03-10 Thread Andrew Purtell
I have started testing rsgroups. Unfortunately I can't release from master into our production so have had to backport onto ~1.3.0, including all subsequent fixes applied in-situ to trunk. Thanks so much for the work separating the rsgroup code into a separate maven module, that helps tremendously

Re: Moving 2.0 forward

2017-03-10 Thread Stack
On Fri, Mar 10, 2017 at 11:10 AM, Andrew Purtell wrote: > I would also like us to reexamine our branch RM model. > > Prior to 1.0 , branch RMs curated branches that lead to minor releases. > > Post 1.0, branch RMs curate branches that only lead to patch releases. > > This seems like a poorer use

Re: Moving 2.0 forward

2017-03-10 Thread Andrew Purtell
I would also like us to reexamine our branch RM model. Prior to 1.0 , branch RMs curated branches that lead to minor releases. Post 1.0, branch RMs curate branches that only lead to patch releases. This seems like a poorer use of precious resource (RM bandwidth and time), one we can scarcely aff

Re: Moving 2.0 forward

2017-03-10 Thread Andrew Purtell
I agree AMv2/Pv2 is almost finished and this makes sense as 2.0. Agreed, upgrade issues need to be addressed, but I have been assuming this will be done after branching during the stabilization and polishing part. Need the branch-2 first to stabilize and polish. Most of the rest justifies a 3.0,

Re: Moving 2.0 forward

2017-03-10 Thread Stack
On Mon, Mar 6, 2017 at 6:15 PM, Enis Söztutar wrote: > Thanks Stack for the nice writeup. > > I think we should shoot for an alpha release sooner than 2 months. It gives > a test target, and will be a great way to test-drive and push for the > release vehicles (packing, hadoop3, license issues, e

Re: Moving 2.0 forward

2017-03-10 Thread Stack
On Thu, Mar 9, 2017 at 10:25 PM, Stack wrote: > On Tue, Mar 7, 2017 at 1:51 PM, Josh Elser wrote: > >> Thanks for pulling in the FS Quotas work, Stack. I'm trying to cross the >> last T's and dot the last I's. >> >> The biggest thing I know I need to do still is to write a new chapter to >> the

Re: Moving 2.0 forward

2017-03-09 Thread Stack
On Wed, Mar 8, 2017 at 4:06 PM, Jerry He wrote: > Thanks for the write-up, Stack. > > I take the liberty to make the Spark item as: > > 4.3 hbase-spark > ktczrlKHK8N4SZzs/edit#heading=h.24l014zh4tmp> > > 4.3.1 Status= >

Re: Moving 2.0 forward

2017-03-09 Thread Stack
On Tue, Mar 7, 2017 at 1:51 PM, Josh Elser wrote: > Thanks for pulling in the FS Quotas work, Stack. I'm trying to cross the > last T's and dot the last I's. > > The biggest thing I know I need to do still is to write a new chapter to > the book. After that, I'd start entertaining larger reviews/

Re: Moving 2.0 forward

2017-03-08 Thread Jerry He
Thanks for the write-up, Stack. I take the liberty to make the Spark item as: 4.3 hbase-spark 4.3.1 Status=

Re: Moving 2.0 forward

2017-03-07 Thread Josh Elser
Thanks for pulling in the FS Quotas work, Stack. I'm trying to cross the last T's and dot the last I's. The biggest thing I know I need to do still is to write a new chapter to the book. After that, I'd start entertaining larger reviews/discussions to merge the feature into master. Anyone with

Re: Moving 2.0 forward

2017-03-06 Thread Enis Söztutar
Thanks Stack for the nice writeup. I think we should shoot for an alpha release sooner than 2 months. It gives a test target, and will be a great way to test-drive and push for the release vehicles (packing, hadoop3, license issues, etc) and also create some well-deserved excitement. I can help wi

Re: Moving 2.0 forward

2017-03-06 Thread Stack
On Mon, Mar 6, 2017 at 2:50 PM, Stack wrote: > ... > + No recent work on core decision tasks (clean-up narrative around RPC > timeout, hbase:meta on master or not, batch vs partial semantic, etc.) > > Correction. batch vs partial semantic is making goodprogress (HBASE-15484). S > Non-critical

Re: Moving 2.0 forward

2017-03-06 Thread Stack
Just a quick update (Please feel free to amend your own status in our hbase-2.0 doc [1]). On the criticals: + Little progress on the blocker AMv2. A month or more necessary still before code complete (Estimate) + Accordion (inmemory compaction) progressing nicely as is the offheap read/write path

Re: Moving 2.0 forward

2017-01-29 Thread Stack
On Thu, Jan 26, 2017 at 11:49 PM, ramkrishna vasudevan < ramkrishna.s.vasude...@gmail.com> wrote: > Hi All > > Thanks Stack. The doc looks great. The offheap write path/read path- I > think from the read path perspective we have some good feedback from > Alibaba folks. > Agree. > The write path

Re: Moving 2.0 forward

2017-01-26 Thread ramkrishna vasudevan
Hi All Thanks Stack. The doc looks great. The offheap write path/read path- I think from the read path perspective we have some good feedback from Alibaba folks. The write path subtasks are all done. We are currently working on some perf results that would help us to come up with some docs that su

Re: Moving 2.0 forward

2017-01-18 Thread Andrew Purtell
I'm interested in both split meta and rsgroups. Good news. I'd like to help test. > On Jan 18, 2017, at 2:53 PM, Stack wrote: > >> On Wed, Jan 18, 2017 at 2:26 PM, Francis Liu wrote: >> >> Hi Stack, >> I'd like to get split meta (HBASE-112288) in 2.x as well. I can have a 2.x >> draft up ne

Re: Moving 2.0 forward

2017-01-18 Thread Stack
On Wed, Jan 18, 2017 at 2:26 PM, Francis Liu wrote: > Hi Stack, > I'd like to get split meta (HBASE-112288) in 2.x as well. I can have a 2.x > draft up next week. Was working on the 1.x version internally. > Also if you'd like I can be the owner for rsgroups as well. > Thanks,Francis > > > > I ad

Re: Moving 2.0 forward

2017-01-18 Thread Francis Liu
Hi Stack, I'd like to get split meta (HBASE-112288) in 2.x as well. I can have a 2.x draft up next week. Was working on the 1.x version internally. Also if you'd like I can be the owner for rsgroups as well.  Thanks,Francis On Wednesday, January 18, 2017 11:29 AM, Stack wrote: Done

Re: Moving 2.0 forward

2017-01-18 Thread Stack
Done Thiruvel (And thanks Guanghao for adding hbase-replication). St.Ack On Tue, Jan 17, 2017 at 6:11 PM, Thiruvel Thirumoolan < thiru...@yahoo-inc.com.invalid> wrote: > Hi Stack, > I would like to add Favored Nodes to the ancillary section. > HBASE-15531: Favored Nodes EnhancementsStatus: Active

Re: Moving 2.0 forward

2017-01-17 Thread Thiruvel Thirumoolan
Hi Stack, I would like to add Favored Nodes to the ancillary section. HBASE-15531: Favored Nodes EnhancementsStatus: Active development.Owner: Thiruvel Thanks!-Thiruvel On Monday, January 16, 2017 2:10 PM, Stack wrote: On Mon, Jan 16, 2017 at 3:01 AM, Guanghao Zhang wrote: > For 6. Si

Re: Moving 2.0 forward

2017-01-16 Thread Stack
On Mon, Jan 16, 2017 at 3:01 AM, Guanghao Zhang wrote: > For 6. Significant contirbs in master only, there are some issues about > replication operations routed through master. They are sub-task > of HBASE-10504. And there are other umbrella issue for replication, like > HBase-14379 Replication V

Re: Moving 2.0 forward

2017-01-16 Thread Guanghao Zhang
For 6. Significant contirbs in master only, there are some issues about replication operations routed through master. They are sub-task of HBASE-10504. And there are other umbrella issue for replication, like HBase-14379 Replication V2 and HBASE-15867 Moving HBase Replication tracking from Zookeepe

Re: Moving 2.0 forward

2017-01-14 Thread Stack
Hey Eric! See this thread where it is suggested we remove the module for want of a sponsor [1]. The hbase-spark story is also muddled by there being different options available neither of which is complete instead of just 'the one' (Do a google search). St.Ack 1. http://apache-hbase.679495.n3.nab

Re: Moving 2.0 forward

2017-01-14 Thread Stack
On Sat, Jan 14, 2017 at 11:10 AM, Andrew Purtell wrote: > Thanks for putting that document together Stack, that was really helpful. > > > 1.1 New Assignment Manager, AMv2 > > ​Can we get a virtual show of hands who is working on this and plans to > finish it? It was Stephen and Matteo originally,

Re: Moving 2.0 forward

2017-01-14 Thread Ted Yu
After HBASE-16179 gets in, we can get wider feedback from interested users in using hbase-spark module. We would then be able to find missing pieces. > On Jan 14, 2017, at 12:30 PM, Eric Charles wrote: > > I read "3.3 hbase-spark STATUS: Needs work. No one on it at mo. Doc. is just > wrong.

Re: Moving 2.0 forward

2017-01-14 Thread Eric Charles
I read "3.3 hbase-spark STATUS: Needs work. No one on it at mo. Doc. is just wrong. What is there is dodgy. Could get punted." Unit tests are working and base functionality is there. Besides the doc and compilation against spark-2 (and scala-2.11), what else do you want to see? On 14/01/17

Re: Moving 2.0 forward

2017-01-14 Thread Andrew Purtell
Thanks for putting that document together Stack, that was really helpful. > 1.1 New Assignment Manager, AMv2 ​Can we get a virtual show of hands who is working on this and plans to finish it? It was Stephen and Matteo originally, right? Matteo seems temporarily sidelined, is that correct? > 1.3

Re: Moving 2.0 forward

2017-01-14 Thread Ted Yu
For 3.3, hbase-spark module, there is HBASE-16179 which enables support for Spark 2.0 It needs some review. Cheers > On Jan 13, 2017, at 11:25 PM, Stack wrote: > > On Sat, Dec 31, 2016 at 12:16 PM, Stephen Jiang > wrote: > >> Hello, Andrew, I was a helper on Matteo so that we can help eac

Re: Moving 2.0 forward

2017-01-13 Thread Andrew Purtell
While I don't disagree that half finished features are undesirable, I'm not suggesting that as a strategy so much as we kick out stuff that just doesn't seem to be getting done. Pushing 2.0 out another three months is fine if there's a good chance this is realistic and we won't be having this di

Re: Moving 2.0 forward

2017-01-13 Thread Stack
On Sat, Dec 31, 2016 at 12:16 PM, Stephen Jiang wrote: > Hello, Andrew, I was a helper on Matteo so that we can help each other > while we are focusing on the new Assignment Manager work. Now he is not > available (at least in the next few months). I have to be more focused on > the new AM work

Re: Moving 2.0 forward

2017-01-06 Thread Jerry He
I agree we need a long and stable 1.x release. Branch-1 is a good fit for that role. It has the stability and compatibility of 1.x, and it has still been quite open for flow of improvements and commits. +1 Jerry On Fri, Jan 6, 2017 at 1:01 PM, Mikhail Antonov wrote: > I support that idea of c

Re: Moving 2.0 forward

2017-01-06 Thread Mikhail Antonov
I support that idea of cutting branch-2 early. Yes it will create some burden for the RM and committers to port things between the branches, but until the branch is cut we won't have that sense of imminense of approaching release, and more importantly, until branch is cut _all_ commits will continu

Re: Moving 2.0 forward

2017-01-06 Thread Andrew Purtell
Considerations for a new branch-2 and branch-1 are orthogonal in my opinion. I intend to volunteer to be the RM for branch-1 itself (we've not had one before) as necessary for it to become a stable source of incremental releases for a long time, similar to how we had 0.98 active for almost thre

Re: Moving 2.0 forward

2017-01-05 Thread Phil Yang
Hi all After cutting branch-2, what will we do for branch-1? If I am not wrong, 1.4 may be the last 1.x release branch? Should 1.4.0 release before 2.0.0? If not, will it confuse users? Thanks, Phil 2017-01-01 5:20 GMT+08:00 Andrew Purtell : > On the other hand branching will force the issue. T

Re: Moving 2.0 forward

2016-12-31 Thread Andrew Purtell
On the other hand branching will force the issue. There will always be lists of issues to get in. How long have we been talking about 2.0? At least a year and a half. At some point it's time to stop talking and take action. Let me revisit progress at the end of January and bring this up again. A

Re: Moving 2.0 forward

2016-12-31 Thread Ted Yu
I agree with Stephen on not branching too early. When people come back from vacation, we can poll relevant parties on estimate of respective project to get a sense of when would be proper time for branching. On Sat, Dec 31, 2016 at 12:16 PM, Stephen Jiang wrote: > Hello, Andrew, I was a helper

Re: Moving 2.0 forward

2016-12-31 Thread Stephen Jiang
Hello, Andrew, I was a helper on Matteo so that we can help each other while we are focusing on the new Assignment Manager work. Now he is not available (at least in the next few months). I have to be more focused on the new AM work; plus other work in my company; it would be too much for me to 2