FWIW, I'd like to see the Spark connector get in. We have users who will be interested in it.
On Mon, Mar 21, 2016 at 12:05 PM, Mikhail Antonov <[email protected]> wrote: > At this point branch-1.3 is very close (if different from at all) to > branch-1, so that's probably the same discussion. > > -Mikhail > > On Mon, Mar 21, 2016 at 11:47 AM, Sean Busbey <[email protected]> wrote: > > > Is the spark connector thread specifically about 1.3? or branch-1? > because > > we already had the branch-1 conversation. the specific gates were tracked > > in the umbrella jira. > > > > -Sean > > > > On Mon, Mar 21, 2016 at 1:33 PM, Mikhail Antonov <[email protected]> > > wrote: > > > > > Yeah, we probably should start discussion thread about Spark connector. > > > Anyone wants to start the thread and push it forward? > > > > > > Regarding date-tiered compactions - since first impl already went in > 1.3, > > > would be good to get any possible improvements in 1.3 as well, as long > as > > > they are stable, IMO. > > > > > > -Mikhail > > > > > > On Mon, Mar 21, 2016 at 11:03 AM, Enis Söztutar <[email protected]> > > > wrote: > > > > > > > You may want to track > > https://issues.apache.org/jira/browse/HBASE-15339 > > > as > > > > a parent for date-tiered compaction improvements. Current compaction > > > policy > > > > is useful in its own, but handling existing data, bulk loading etc > will > > > be > > > > improved with these subtasks. I think the patches can land before the > > 1.3 > > > > timeframe, but of course open for discussion for inclusion. My vote > > would > > > > be to include all the improvements since it would be easier to tell > the > > > > story to users. > > > > > > > > Enis > > > > > > > > On Mon, Mar 21, 2016 at 3:32 AM, Ted Yu <[email protected]> wrote: > > > > > > > > > For Spark connector, we should start a separate discussion thread > > about > > > > > backporting to branch-1. > > > > > > > > > > Zhan has a bug fix coming this week which deals with how negative > > > numbers > > > > > are handled in comparison. > > > > > > > > > > FYI > > > > > > > > > > On Mon, Mar 21, 2016 at 1:35 AM, Mikhail Antonov < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > bringing this topic up again. I'm planning to start spinning 1.3 > > > builds > > > > > and > > > > > > see if/where they break in a week or two, and (depending on how > it > > > > does) > > > > > > start preparing RCs in a month or maybe two. So, let's see where > we > > > > are. > > > > > > > > > > > > Big items first. There were long debates around three big items - > > > MOBs, > > > > > > Spark connector and RS groups, whether we should have them or > not. > > > > > > > > > > > > - MOBs > > > > > > I believe we decided that they aren't going to go in branch-1, > and > > > > hence > > > > > > not in branch-1.3 for sure. We might get back to that debate and > > > > > > re-consider MOBs for branch-1 if 2.0 is delayed, but almost > > certainly > > > > > they > > > > > > won't make it in 1.3 timeframe anyway as I feel. > > > > > > > > > > > > - Spark connector - HBASE-14160 it looks like it has 3 subtasks > > open, > > > > one > > > > > > of which is big one (HBASE-14375) - define public API for Spark > > > > > > integration. From the Jira looks like active work is happening on > > > other > > > > > > subtasks, but not on this one. So how's public API going? How > > stable > > > it > > > > > is? > > > > > > Who would want to have Spark in 1.3 and willing to help with > this? > > > > OTOH - > > > > > > who has objections about back-porting it? Has anyone been using > it > > in > > > > > some > > > > > > real environment? > > > > > > > > > > > > - RS groups - there was recently a thread about them, I'd like > to > > > > bring > > > > > it > > > > > > up again and get to some conclusion. > > > > > > > > > > > > Other features which we had in flight a month ago - > > > > > > > > > > > > - HBASE-15181 - date based tiered compactions has landed > > > > > > - HBASE-11290 - unlock RegionStates. I'm afraid the codebase has > > > moved > > > > > > forward quite a bit since the benchmark was run on this change > :( - > > > > > Francis > > > > > > - are you using it now? If we could have some benchmarks on the > > > latest > > > > > > rebase that I think would be great. > > > > > > > > > > > > > > > > > > - HBASE-13557 - special handling for system tables WALs - should > > we > > > > > still > > > > > > keep it targeted for 1.3? > > > > > > - HBASE-13017 - keep table state in meta, this one doesn't look > > like > > > > > it's > > > > > > going to make it in > > > > > > > > > > > > As a new item on my list - I'm looking forward to see more of > > > > HBASE-15492 > > > > > > (memory optimizations) subtasks to go in branch-1. > > > > > > > > > > > > Thanks! > > > > > > Mikhail > > > > > > > > > > > > On Sun, Feb 28, 2016 at 10:53 AM, Andrew Purtell < > > > > > [email protected] > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > I'm starting a push at work to get us up on 1.2. Assuming that > > > > happens > > > > > > > later this year I think that will be the end of my close > > attention > > > to > > > > > > 0.98. > > > > > > > > > > > > > > > On Feb 26, 2016, at 1:54 PM, Nick Dimiduk < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > On Fri, Feb 26, 2016 at 12:23 PM, Andrew Purtell < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> In the meantime those of us running HBase in production > would > > > > > benefit > > > > > > > from > > > > > > > >> fairly frequent minor releases. > > > > > > > > > > > > > > > > +1. Having to look back to 0.98 to get some new feature is > > > > > problematic. > > > > > > > > > > > > > > > >> On Fri, Feb 26, 2016 at 9:50 AM, Elliott Clark < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > >> > > > > > > > >>> I disagree. We have agreed that 2.0 will have a new > > assignement > > > > > > > manager. > > > > > > > >>> There's a lot of work that has been done on getting that > in, > > so > > > > far > > > > > > > there > > > > > > > >>> are no benefits to the end user from all that work. We > should > > > > stick > > > > > > > with > > > > > > > >>> the plan and release 2.0 when it's ready. > > > > > > > >>> > > > > > > > >>> On Fri, Feb 26, 2016 at 9:39 AM, Stephen Jiang < > > > > > > > [email protected]> > > > > > > > >>> wrote: > > > > > > > >>> > > > > > > > >>>> Thanks for Mikhail for taking the 1.3 RM role. Looks like > > we > > > > > have a > > > > > > > >> lot > > > > > > > >>> of > > > > > > > >>>> new things in 1.3 release. > > > > > > > >>>> > > > > > > > >>>> Based on the experience of 1.1 and 1.2 release, it takes a > > lot > > > > of > > > > > > > >> efforts > > > > > > > >>>> to get a stable minor release out. From this, I have my > own > > > > > 2-cents > > > > > > > on > > > > > > > >>> 1.4 > > > > > > > >>>> release. The plan is to have 2.0 release during summer > time > > > of > > > > > this > > > > > > > >> year > > > > > > > >>>> (yeah, *this year). * Given the limited time and > resource, > > > > after > > > > > > 1.3 > > > > > > > >>>> release, instead of spending effort on 1.4 release, the > > > > community > > > > > > > >> should > > > > > > > >>>> focus on stabilizing master (or branch-2, not exist as of > > now) > > > > > > branch > > > > > > > >> and > > > > > > > >>>> make 2.0 release a priority. 2.0 release would bring more > > > > values > > > > > to > > > > > > > >>>> customer & move towards maturity of HBASE product. > > > > > > > >>>> > > > > > > > >>>> Thanks > > > > > > > >>>> Stephen > > > > > > > >>>> > > > > > > > >>>> On Fri, Feb 26, 2016 at 1:49 AM, Mikhail Antonov < > > > > > > > [email protected] > > > > > > > >>> > > > > > > > >>>> wrote: > > > > > > > >>>> > > > > > > > >>>>> Created an umbrella jira for 1.3 release - HBASE-15341 > > > > > > > >>>>> > > > > > > > >>>>> So it looks like we may have 1.4 release before 2.0 is > > out? I > > > > > tried > > > > > > > >> to > > > > > > > >>>> add > > > > > > > >>>>> 1.4 version in jira so we can keep it in branch-1 poms > but > > I > > > > > > > >> couldn't - > > > > > > > >>>>> looks like I don't have permissions? > > > > > > > >>>>> > > > > > > > >>>>> -Mikhail > > > > > > > >>>>> > > > > > > > >>>>> On Thu, Feb 25, 2016 at 3:52 PM, Andrew Purtell < > > > > > > [email protected] > > > > > > > >>> > > > > > > > >>>>> wrote: > > > > > > > >>>>> > > > > > > > >>>>>> The guy we had looking at streaming replication moved on > > and > > > > > > > >> there's > > > > > > > >>> no > > > > > > > >>>>>> immediate plans to take on the work, FWIW > > > > > > > >>>>>> > > > > > > > >>>>>> On Thu, Feb 25, 2016 at 3:33 PM, Matteo Bertozzi < > > > > > > > >>>>> [email protected]> > > > > > > > >>>>>> wrote: > > > > > > > >>>>>> > > > > > > > >>>>>>> I was shooting for summer for hbase 2.0, the main > problem > > > is > > > > > that > > > > > > > >>>> there > > > > > > > >>>>>> is > > > > > > > >>>>>>> still no code for the new AM or for fs changes, which > are > > > the > > > > > two > > > > > > > >>>> that > > > > > > > >>>>>> may > > > > > > > >>>>>>> impact compatibility (working slowly on that). > Streaming > > > > > > > >>> replication > > > > > > > >>>>> and > > > > > > > >>>>>>> others seems compatible enough but no code there too. > > > > > > > >>>>>>> > > > > > > > >>>>>>> Matteo > > > > > > > >>>>>>> > > > > > > > >>>>>>> > > > > > > > >>>>>>> On Thu, Feb 25, 2016 at 3:29 PM, Mikhail Antonov < > > > > > > > >>>> [email protected] > > > > > > > >>>>>> > > > > > > > >>>>>>> wrote: > > > > > > > >>>>>>> > > > > > > > >>>>>>>> Agreed. I just meant - readiness of 2.0 is something > > > > affecting > > > > > > > >>>>>> decisions > > > > > > > >>>>>>> on > > > > > > > >>>>>>>> whether or not to backport mobs to branch-1 (which is > > > itself > > > > > > > >>>> separate > > > > > > > >>>>>>>> thread). > > > > > > > >>>>>>>> > > > > > > > >>>>>>>> -Mikhail > > > > > > > >>>>>>>> > > > > > > > >>>>>>>> On Thu, Feb 25, 2016 at 3:27 PM, Sean Busbey < > > > > > > > >>> [email protected]> > > > > > > > >>>>>>> wrote: > > > > > > > >>>>>>>> > > > > > > > >>>>>>>>> On Thu, Feb 25, 2016 at 5:21 PM, Mikhail Antonov < > > > > > > > >>>>>> [email protected] > > > > > > > >>>>>>>> > > > > > > > >>>>>>>>> wrote: > > > > > > > >>>>>>>>> > > > > > > > >>>>>>>>>> > > > > > > > >>>>>>>>>> - "Shouldn't we rather try to get 2.0 release out > and > > > have > > > > > > > >>>> mobs > > > > > > > >>>>>>>> there". > > > > > > > >>>>>>>>> - > > > > > > > >>>>>>>>>> So how far do we feel 2.0 release is? > > > > > > > >>>>>>>>> > > > > > > > >>>>>>>>> 2.0 readiness probably deserves its own [DISCUSS] > > thread, > > > > but > > > > > > > >>>> we're > > > > > > > >>>>>> now > > > > > > > >>>>>>>>> past a year since the HBase 1.0.0 release, so I hope > > it's > > > > > > > >> soon. > > > > > > > >>>>>>>>> > > > > > > > >>>>>>>>> > > > > > > > >>>>>>>>> > > > > > > > >>>>>>>>> -- > > > > > > > >>>>>>>>> Sean > > > > > > > >>>>>>>> > > > > > > > >>>>>>>> > > > > > > > >>>>>>>> > > > > > > > >>>>>>>> -- > > > > > > > >>>>>>>> Thanks, > > > > > > > >>>>>>>> Michael Antonov > > > > > > > >>>>>> > > > > > > > >>>>>> > > > > > > > >>>>>> > > > > > > > >>>>>> -- > > > > > > > >>>>>> Best regards, > > > > > > > >>>>>> > > > > > > > >>>>>> - Andy > > > > > > > >>>>>> > > > > > > > >>>>>> Problems worthy of attack prove their worth by hitting > > > back. - > > > > > > Piet > > > > > > > >>>> Hein > > > > > > > >>>>>> (via Tom White) > > > > > > > >>>>> > > > > > > > >>>>> > > > > > > > >>>>> > > > > > > > >>>>> -- > > > > > > > >>>>> Thanks, > > > > > > > >>>>> Michael Antonov > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> Best regards, > > > > > > > >> > > > > > > > >> - Andy > > > > > > > >> > > > > > > > >> Problems worthy of attack prove their worth by hitting > back. - > > > > Piet > > > > > > Hein > > > > > > > >> (via Tom White) > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Thanks, > > > > > > Michael Antonov > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Thanks, > > > Michael Antonov > > > > > > > > > > > -- > > busbey > > > > > > -- > Thanks, > Michael Antonov > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
