Heads up on where we are with branch-1.3. Lots of stuff we had outstanding 10 days ago got committed and now we are down to just 3 bugfixes and 2 features which I'm looking to get in, all have patches in review. I hope we could get them in in next few days and then I can start preparing RCs.
Bugfixes: HBASE-15691 Port HBASE-10205 (ConcurrentModificationException in BucketAllocator) to branch-1 (Andrew/Stack let me know if I can help here? Any more performance testing we wait for?) HBASE-15615 Wrong sleep time when RegionServerCallable need retry (almost there, just some more tests around would be great) HBASE-15593 Time limit of scanning should be offered by client (Stack - should we get it in or you're looking for more tests?) Features: HBASE-15454 Archive store files older than max age (Duo / Heng / Enis do we wait for more reviews/tests?) HBASE-15773 CellCounter improvements (should be pretty straightforward to get in) Let me know if I missed anything. -Mikhail On Wed, Apr 27, 2016 at 10:20 PM, Mikhail Antonov <[email protected]> wrote: > Thanks! > > -Mikhail > > On 4/27/16, ramkrishna vasudevan <[email protected]> wrote: > > I saw Andy's comment. Will create a patch for trunk also and will commit > > both together. > > > > Regards > > Ram > > > > On Thu, Apr 28, 2016 at 10:39 AM, Mikhail Antonov <[email protected]> > > wrote: > > > >> Thanks Ram! It seems like that's fixed now (the problem when we don't > >> compact often enough in TestHRegion). > >> > >> On Wed, Apr 27, 2016 at 4:19 AM, ramkrishna vasudevan < > >> [email protected]> wrote: > >> > >> > >>Also need to figure out situation around HBASE-14970 - HBASE-13082 > >> > - HBASE-15697, > >> > Regarding this - HBASE-14970 is found in both branch-1 and branch-1.3. > >> The > >> > patch for HBASE-15697 solves the ulimit issue with respect number of > >> > open > >> > files. Once Andrew verifies it we can commit the patch and you can > have > >> > that in the branch-1.3 release IMHO. > >> > > >> > Thanks Mikhail for the heads up. > >> > > >> > Regards > >> > Ram > >> > > >> > On Wed, Apr 27, 2016 at 2:40 PM, Mikhail Antonov <[email protected]> > >> > wrote: > >> > > >> > > Ok, I think it's time to bring up this thread again. Let's see where > >> > > we > >> > > are. > >> > > > >> > > So, outstanding - > >> > > > >> > > Significant bugs (looked through jira filters): > >> > > - Several bugfixes/improvements in HBase client, often related to > >> > > rpc/retries, went in recently or about to go in. > >> > > * HBASE-15645, when we don't use rpc timeouts properly, went in > >> > (pending > >> > > addendum for nits) > >> > > * HBASE-15658, when we unnecessarily clear MetaCache on retries, > >> > > went > >> > in > >> > > on 1.3+, > >> > > probably should go in 1.2 as well, see discussion there. > >> > > * HBASE-15593, and HBASE-15615 - those should be addressed soon I > >> guess > >> > > > >> > > - Bucket cache fixes improvements, HBASE-15240 and subtasks, would > >> > > be > >> > good > >> > > to pull in as much as possible. > >> > > - HBASE-15703, bug in deadline rpc scheduler, I'd need to fix it > >> before > >> > > the release > >> > > > >> > > Any other very-nice-to-get-in fixes we want? > >> > > > >> > > Also need to figure out situation around HBASE-14970 - HBASE-13082 > >> > > - HBASE-15697, > >> > > I kind of lost track what was committed/reverted/re-aplied on which > >> > branch. > >> > > @Andrew, Stack, Ram > >> > > > >> > > Features: > >> > > * Date-tiered compactions. I'm looking at umbrella jira for d-t > >> > > compactions, HBASE-15339 < > >> > > https://issues.apache.org/jira/browse/HBASE-15339> > >> > > , > >> > > looks like there's one big item left on the list (HBASE-15454, > >> > > archive StoreFile older than max age), > >> > > and it's being actively worked on / reviewed, and another one > - > >> > > documentation task, so I assume it's all good > >> > > and we get all date-tiers compactions improvements for 1.3. > >> > > * Spark connector. Haven't seen much activity on it lately. Sean - > >> what > >> > > do you think about HBASE-14160? > >> > > I'm inclined to push it to 1.4 (or whatever next release will > >> > > be) > >> if > >> > > there're no takers at this point. > >> > > > >> > > > >> > > Thanks! > >> > > Mikhail > >> > > > >> > > > >> > > > >> > > On Fri, Apr 22, 2016 at 2:22 PM, Ted Yu <[email protected]> > wrote: > >> > > > >> > > > bq. There are several patches > >> > > > > >> > > > Let us know the JIRA numbers. > >> > > > I can help with reviewing if needed. > >> > > > > >> > > > Cheers > >> > > > > >> > > > On Fri, Apr 22, 2016 at 2:18 PM, Mikhail Antonov < > >> [email protected] > >> > > > >> > > > wrote: > >> > > > > >> > > > > Last time I've tried to run 1.3 builds there were issues with > >> > balancer, > >> > > > > which are fixed now. There are several patches I definitely > would > >> > like > >> > > to > >> > > > > pull in, other than that I feel we are pretty close. I'll start > >> > > spinning > >> > > > > internal builds in a few days and if things look good will start > >> > > > preparing > >> > > > > RC's next week or so. > >> > > > > > >> > > > > I guess we are getting to feature-complete state, I'll walk > >> > > > > through > >> > the > >> > > > > jiras and send detailed email over weekend. > >> > > > > > >> > > > > Thanks! > >> > > > > Mikhail > >> > > > > > >> > > > > > On Apr 22, 2016, at 8:13 AM, Ted Yu <[email protected]> > >> > > > > > wrote: > >> > > > > > > >> > > > > > Mikhail: > >> > > > > > Any plan when to spin 1.3 RC0 ? > >> > > > > > > >> > > > > > HBaseCon is not very far. > >> > > > > > > >> > > > > > I was wondering if 1.3 release can be done before HBaseCon. > >> > > > > > > >> > > > > > Cheers > >> > > > > > > >> > > > > > On Thu, Feb 25, 2016 at 12:54 PM, Mikhail Antonov < > >> > > [email protected]> > >> > > > > > wrote: > >> > > > > > > >> > > > > >> To me it's not really about individual big features (besides, > >> big > >> > > > > features > >> > > > > >> might be hard to accommodate in a minor release), but enough > >> good > >> > > > > things to > >> > > > > >> justify minor release. > >> > > > > >> > >> > > > > >> What we can have (unless I'm missing something): > >> > > > > >> > >> > > > > >> [Already done or to be further improved] > >> > > > > >> - HBASE-15177 - more GC-friendly allocations in RPC services > >> > > > > >> - HBASE-14457 - multi WAL improvements > >> > > > > >> - HBASE-15222 - optimizations in metrics system, some more > >> metrics > >> > > > > >> (like HBASE-15135, HBASE-15068) > >> > > > > >> - HBASE-15306, HBASE-15136 - improving call queues handling > >> > > > > >> > >> > > > > >> [To be reviewed?): > >> > > > > >> - HBASE-15181 - date based tiered compactions (?) > >> > > > > >> - HBASE-11290 - unlock RegionStates. There was a patch update > >> > > > relatively > >> > > > > >> recently to it based on comments. > >> > > > > >> > >> > > > > >> [Possible?] > >> > > > > >> - HBASE-13557 - special handling for system tables WALs > >> > > > > >> - HBASE-13017 - keep table state in meta > >> > > > > >> > >> > > > > >> 1.2 was cut off mid-June 2015.. Should be enough time since > >> > > > > >> then > >> > > for a > >> > > > > >> minor release. > >> > > > > >> > >> > > > > >> Mikhail > >> > > > > >> > >> > > > > >> On Thu, Feb 25, 2016 at 10:56 AM, Enis Söztutar < > >> > [email protected] > >> > > > > >> > > > > >> wrote: > >> > > > > >> > >> > > > > >>> What are the "features" in current branch-1 that is not > there > >> in > >> > > 1.2? > >> > > > > If > >> > > > > >>> there is none, it is not worth branching yet. > >> > > > > >>> > >> > > > > >>> Enis > >> > > > > >>> > >> > > > > >>> On Wed, Feb 24, 2016 at 7:57 PM, Andrew Purtell < > >> > > > > >> [email protected]> > >> > > > > >>> wrote: > >> > > > > >>> > >> > > > > >>>> No, each 0.94.x/0.96.x/98.x was or is a minor release. :-) > >> > > Sometimes > >> > > > > >> the > >> > > > > >>>> changes in those releases could all be considered "point" > in > >> > scope > >> > > > or > >> > > > > >>>> effect but not always. Further supporting this point of > >> > > > > >>>> view, > >> > when > >> > > > we > >> > > > > >>> went > >> > > > > >>>> from 0.94 to 0.96 it was a major increment, in effect, due > >> > > > > >>>> to > >> > 'the > >> > > > > >>>> singularity'. > >> > > > > >>>> > >> > > > > >>>> Doing a new minor every month would be more like a return > to > >> > past > >> > > > > state > >> > > > > >>> of > >> > > > > >>>> affairs, for better or worse, in my humble opinion. > >> > > > > >>>> > >> > > > > >>>>>> On Feb 24, 2016, at 7:46 PM, Stack <[email protected]> > >> wrote: > >> > > > > >>>>>> > >> > > > > >>>>>> On Wed, Feb 24, 2016 at 11:50 AM, Elliott Clark < > >> > > > [email protected]> > >> > > > > >>>> wrote: > >> > > > > >>>>>> > >> > > > > >>>>>> Is it time to branch for 1.3 ? > >> > > > > >>>>> > >> > > > > >>>>> > >> > > > > >>>>>> Sean did a great job getting 1.2 out. However it was a > >> > > > > >>>>>> hard > >> > > > > >> difficult > >> > > > > >>>>>> process that I wouldn't wish on anyone. Is it time to > >> > > > > >>>>>> branch > >> > for > >> > > > 1.3 > >> > > > > >>> and > >> > > > > >>>>>> start the process of stabilizing again so that we can get > >> > > > > >>>>>> a > >> > > > monthly > >> > > > > >>>> cadence > >> > > > > >>>>>> for minor releases going? > >> > > > > >>>>> > >> > > > > >>>>> > >> > > > > >>>>> Monthly cadence for minors is upping the ante. We used to > >> > > > > >>>>> be > >> > > about > >> > > > > >>>>> monthly's for point releases. > >> > > > > >>>>> > >> > > > > >>>>> +1 for the mighty Mikhail as RM. Sean, please UPS him the > >> > special > >> > > > > >> robe > >> > > > > >>>> that > >> > > > > >>>>> he has to wear while performing his RMness duties. > >> > > > > >>>>> > >> > > > > >>>>> St.Ack > >> > > > > >> > >> > > > > > >> > > > > >> > > > >> > > >> > > > > > -- > Thanks, > Michael Antonov > -- Thanks, Michael Antonov
