You may want to track https://issues.apache.org/jira/browse/HBASE-15339 as a parent for date-tiered compaction improvements. Current compaction policy is useful in its own, but handling existing data, bulk loading etc will be improved with these subtasks. I think the patches can land before the 1.3 timeframe, but of course open for discussion for inclusion. My vote would be to include all the improvements since it would be easier to tell the story to users.
Enis On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <yuzhih...@gmail.com> wrote: > For Spark connector, we should start a separate discussion thread about > backporting to branch-1. > > Zhan has a bug fix coming this week which deals with how negative numbers > are handled in comparison. > > FYI > > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov <olorinb...@gmail.com> > wrote: > > > Hi folks, > > > > bringing this topic up again. I'm planning to start spinning 1.3 builds > and > > see if/where they break in a week or two, and (depending on how it does) > > start preparing RCs in a month or maybe two. So, let's see where we are. > > > > Big items first. There were long debates around three big items - MOBs, > > Spark connector and RS groups, whether we should have them or not. > > > > - MOBs > > I believe we decided that they aren't going to go in branch-1, and hence > > not in branch-1.3 for sure. We might get back to that debate and > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost certainly > they > > won't make it in 1.3 timeframe anyway as I feel. > > > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks open, one > > of which is big one (HBASE-14375) - define public API for Spark > > integration. From the Jira looks like active work is happening on other > > subtasks, but not on this one. So how's public API going? How stable it > is? > > Who would want to have Spark in 1.3 and willing to help with this? OTOH - > > who has objections about back-porting it? Has anyone been using it in > some > > real environment? > > > > - RS groups - there was recently a thread about them, I'd like to bring > it > > up again and get to some conclusion. > > > > Other features which we had in flight a month ago - > > > > - HBASE-15181 - date based tiered compactions has landed > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has moved > > forward quite a bit since the benchmark was run on this change :( - > Francis > > - are you using it now? If we could have some benchmarks on the latest > > rebase that I think would be great. > > > > > > - HBASE-13557 - special handling for system tables WALs - should we > still > > keep it targeted for 1.3? > > - HBASE-13017 - keep table state in meta, this one doesn't look like > it's > > going to make it in > > > > As a new item on my list - I'm looking forward to see more of HBASE-15492 > > (memory optimizations) subtasks to go in branch-1. > > > > Thanks! > > Mikhail > > > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell < > andrew.purt...@gmail.com > > > > > wrote: > > > > > I'm starting a push at work to get us up on 1.2. Assuming that happens > > > later this year I think that will be the end of my close attention to > > 0.98. > > > > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk <ndimi...@gmail.com> > wrote: > > > > > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell < > apurt...@apache.org> > > > > wrote: > > > > > > > >> In the meantime those of us running HBase in production would > benefit > > > from > > > >> fairly frequent minor releases. > > > > > > > > +1. Having to look back to 0.98 to get some new feature is > problematic. > > > > > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark <ecl...@apache.org> > > > wrote: > > > >> > > > >>> I disagree. We have agreed that 2.0 will have a new assignement > > > manager. > > > >>> There's a lot of work that has been done on getting that in, so far > > > there > > > >>> are no benefits to the end user from all that work. We should stick > > > with > > > >>> the plan and release 2.0 when it's ready. > > > >>> > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang < > > > syuanjiang...@gmail.com> > > > >>> wrote: > > > >>> > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like we > have a > > > >> lot > > > >>> of > > > >>>> new things in 1.3 release. > > > >>>> > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a lot of > > > >> efforts > > > >>>> to get a stable minor release out. From this, I have my own > 2-cents > > > on > > > >>> 1.4 > > > >>>> release. The plan is to have 2.0 release during summer time of > this > > > >> year > > > >>>> (yeah, *this year). * Given the limited time and resource, after > > 1.3 > > > >>>> release, instead of spending effort on 1.4 release, the community > > > >> should > > > >>>> focus on stabilizing master (or branch-2, not exist as of now) > > branch > > > >> and > > > >>>> make 2.0 release a priority. 2.0 release would bring more values > to > > > >>>> customer & move towards maturity of HBASE product. > > > >>>> > > > >>>> Thanks > > > >>>> Stephen > > > >>>> > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov < > > > olorinb...@gmail.com > > > >>> > > > >>>> wrote: > > > >>>> > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341 > > > >>>>> > > > >>>>> So it looks like we may have 1.4 release before 2.0 is out? I > tried > > > >> to > > > >>>> add > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms but I > > > >> couldn't - > > > >>>>> looks like I don't have permissions? > > > >>>>> > > > >>>>> -Mikhail > > > >>>>> > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell < > > apurt...@apache.org > > > >>> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> The guy we had looking at streaming replication moved on and > > > >> there's > > > >>> no > > > >>>>>> immediate plans to take on the work, FWIW > > > >>>>>> > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi < > > > >>>>> theo.berto...@gmail.com> > > > >>>>>> wrote: > > > >>>>>> > > > >>>>>>> I was shooting for summer for hbase 2.0, the main problem is > that > > > >>>> there > > > >>>>>> is > > > >>>>>>> still no code for the new AM or for fs changes, which are the > two > > > >>>> that > > > >>>>>> may > > > >>>>>>> impact compatibility (working slowly on that). Streaming > > > >>> replication > > > >>>>> and > > > >>>>>>> others seems compatible enough but no code there too. > > > >>>>>>> > > > >>>>>>> Matteo > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov < > > > >>>> olorinb...@gmail.com > > > >>>>>> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something affecting > > > >>>>>> decisions > > > >>>>>>> on > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is itself > > > >>>> separate > > > >>>>>>>> thread). > > > >>>>>>>> > > > >>>>>>>> -Mikhail > > > >>>>>>>> > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey < > > > >>> bus...@cloudera.com> > > > >>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov < > > > >>>>>> olorinb...@gmail.com > > > >>>>>>>> > > > >>>>>>>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out and have > > > >>>> mobs > > > >>>>>>>> there". > > > >>>>>>>>> - > > > >>>>>>>>>> So how far do we feel 2.0 release is? > > > >>>>>>>>> > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] thread, but > > > >>>> we're > > > >>>>>> now > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope it's > > > >> soon. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> -- > > > >>>>>>>>> Sean > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> -- > > > >>>>>>>> Thanks, > > > >>>>>>>> Michael Antonov > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> -- > > > >>>>>> Best regards, > > > >>>>>> > > > >>>>>> - Andy > > > >>>>>> > > > >>>>>> Problems worthy of attack prove their worth by hitting back. - > > Piet > > > >>>> Hein > > > >>>>>> (via Tom White) > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> Thanks, > > > >>>>> Michael Antonov > > > >> > > > >> > > > >> > > > >> -- > > > >> Best regards, > > > >> > > > >> - Andy > > > >> > > > >> Problems worthy of attack prove their worth by hitting back. - Piet > > Hein > > > >> (via Tom White) > > > >> > > > > > > > > > > > -- > > Thanks, > > Michael Antonov > > >
