On Mon, May 15, 2017 at 1:46 AM, 张铎(Duo Zhang) <palomino...@gmail.com> wrote:
> HBASE-18037 is a new blocker. I'm currently working on it, will be finished > soon I think. > > I made it a blocker then and added it to our hbase2 release doc [1] list as a blocker. Thanks, St.Ack 1. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit# > 2017-05-15 14:12 GMT+08:00 Stack <st...@duboce.net>: > > > A month on. Status. > > > > I've been working on the HBASE-14614 branch cluster testing. After a load > > of fixing, the branch passes smaller test runs (an hour or so of ITBLL up > > to 2B rows w/ killing monkeys). When I go larger, to a scale I've not > done > > in a while, I start to run into other interesting issues -- some of which > > are related to AMv2 (I'm fixing), but others are not (100G WALs that take > > ten minutes to split makes for interesting cascades when monkeys kill > > inside the ten minutes...). I intend to keep on with this larger scale > > testing since it is uncovering good stuff (especially when HDFS is dog > slow > > because of background replications) but my thinking is that I should be > > large scale testing branch-2, not just HBASE-14614. I think HBASE-14614, > > the new AMv2, is good enough to merge to master these times. Given it is > > the last blocker, once in, I'll cut the hbase2 branch. > > > > I'll start up a 'Merge HBASE-14614' DISCUSSION thread in the next day or > so > > (I need to fix some unit tests...). > > > > The AMv2 doc is still a work in progress but should give a gist on where > we > > are currently[1]. There is a bunch of todo still but seems tractable; > e.g. > > rolling upgrade, finish doc., and we don't have an HBCK since it needs to > > be recast in light of how stuff now works but a redo on HBCK is premature > > given we don't know failure types as yet (we just fix the problems as > they > > come up). > > > > St.Ack > > 1. > > https://docs.google.com/document/d/1eVKa7FHdeoJ1- > > 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit# > > > > > > > > On Thu, Apr 13, 2017 at 1:43 PM, Stack <st...@duboce.net> wrote: > > > > > Some status: > > > > > > AMv2 (HBASE-14614) is near to passing all tests caveat my disabling of > > all > > > to-do w/ fsck (fsck needs revamp) and tests that expect that they can > > move > > > hbase;meta off master (AMv2 enforces this constraint; it is supposed to > > be > > > enforced on AMv1 but meta-on-master is incompletely realized in AMv1 > and > > > AMv2). A few other tests have been disabled for various reasons. See > [1] > > > for full list. > > > > > > There is a hefty list of TODOs still (Again see the messy doc [1]) but > > the > > > only 'blocker', IMO, is community confidence in AMv2. Currently, > cluster > > > tests with chaos fail (new form of 'stuck' regions). Takes time > > > investigating. > > > > > > Will keep you all posted. > > > St.Ack > > > > > > > > > > > > 1. https://docs.google.com/document/d/1eVKa7FHdeoJ1- > > > 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.92vclum0bvod > > > > > > > > > > > > On Fri, Mar 31, 2017 at 1:14 PM, Andrew Purtell <apurt...@apache.org> > > > wrote: > > > > > >> +1 on branching (yay!) > > >> > > >> I have EC2 resources for running ITBLL etc. > > >> > > >> > > >> On Thu, Mar 30, 2017 at 5:07 PM, Stack <st...@duboce.net> wrote: > > >> > > >> > Some notes on progress toward hbase2. > > >> > > > >> > Given that stability and performance are NOT emergent behaviors but > > >> rather > > >> > projects unto themselves, my thought is that we commit all that > we've > > >> > agreed as core for hbase2 (see [1]), branch, and then work on > > >> stabilizing > > >> > and perf rather than do stabilize, commit, and then branch. What > this > > >> means > > >> > in practice is that for features like Inmemory Compaction, we commit > > it > > >> > defaulted 'on' ("BASIC" mode) which is what we want in hbase2. > Should > > it > > >> > prove problematic under test, we disable it before release. > > >> > > > >> > Are folks good w/ this mode? I ask because, in a few issues there > are > > >> > requests for proof that a master feature is 'stable' before commit. > > >> This is > > >> > normally a healthy request only in master's case, it is hard to > > >> demonstrate > > >> > stability given its current state. > > >> > > > >> > Other outstanding issues such as decisions about whether master > hosts > > >> > system tables only (by default), I'm thinking, we can work out post > > >> branch > > >> > in alpha/betas before release. > > >> > > > >> > The awkward item is the long-pole Assignment Manager. This is an > > >> > all-or-nothing affair. Here we are switching in a new Master core. > > >> While I > > >> > think it fine that AMv2 is incomplete come branch time, those of us > > >> working > > >> > on the new AM still need to demonstrate to you all that it basically > > >> > viable. > > >> > > > >> > The point-of-no-return is commit of the patch in HBASE-14614. > > >> HBASE-14614 > > >> > (AMv2) is coming close to passing all unit tests. We'll spend some > > time > > >> > running it on a cluster to make sure it fundamentally sound and will > > >> report > > >> > back on our experience. There has been an ask for some dev doc and > > >> > low-levels on how it works (in progress). Let satisfaction of these > > >> > requests be blockers on commit. We'll put the HBASE-14614 commit up > > for > > >> a > > >> > vote on dev list given its import. > > >> > > > >> > Branch will happen after HBASE-14614 goes in (or its rejection) with > > our > > >> > first alpha soon after. Its looking like a week or two at least > given > > >> how > > >> > things have been going up to this. > > >> > > > >> > I intend to start in on hbase2 stability/perf projects after we > > branch. > > >> > > > >> > Interested in any thoughts you all might have on the above (Would > also > > >> > appreciate updates on state in [1] if you are a feature owner). > > >> > > > >> > Thanks, > > >> > St.Ack > > >> > > > >> > 1. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4 > > >> > z9iEu_ktczrlKHK8N4SZzs/edit# > > >> > > > >> > > > >> > > > >> > On Sat, Mar 11, 2017 at 5:32 PM, Josh Elser <els...@apache.org> > > wrote: > > >> > > > >> > > > > >> > > Stack wrote: > > >> > > > > >> > >> On Tue, Mar 7, 2017 at 1:51 PM, Josh Elser<els...@apache.org> > > >> wrote: > > >> > >> > > >> > >> Thanks for pulling in the FS Quotas work, Stack. I'm trying to > > cross > > >> the > > >> > >>> last T's and dot the last I's. > > >> > >>> > > >> > >>> The biggest thing I know I need to do still is to write a new > > >> chapter > > >> > to > > >> > >>> the book. After that, I'd start entertaining larger > > >> reviews/discussions > > >> > >>> to > > >> > >>> merge the feature into master. Anyone with free time (giggles) > is > > >> more > > >> > >>> than > > >> > >>> welcome to start perusing :) > > >> > >>> > > >> > >>> > > >> > >>> Out of interest, this could come in after 2.0 Josh? Any 2.0 > > specific > > >> > >> needs > > >> > >> to make this work? > > >> > >> > > >> > >> Meantime, updated the 2.0 doc 1. > > >> > >> > > >> > >> Thanks Josh, > > >> > >> St.Ack > > >> > >> > > >> > >> 1. > > >> > >> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i > > >> > >> Eu_ktczrlKHK8N4SZzs/edit# > > >> > >> > > >> > >> > > >> > > Nope, no need to block 2.0 on this one (given the other, related > > >> > chatter). > > >> > > Would be nice to get it in, but I completely understand if it > slips > > :) > > >> > > > > >> > > Thanks for updating the doc for me! > > >> > > > > >> > > > >> > > >> > > >> > > >> -- > > >> Best regards, > > >> > > >> - Andy > > >> > > >> If you are given a choice, you believe you have acted freely. - > Raymond > > >> Teller (via Peter Watts) > > >> > > > > > > > > >
