Great, I think there is some consensus on: 1. Doing 4.8.1 release. 2. Dropping HBase-1.0 for 4.9+ 3. Lars being RM for 4.9 (and 4.8.1?)
I just created 4.8.1 as a release version. Let's use that to track issues to backport. Committers, once 4.8.0 RC is cut, please start committing issues to 4.8 branches as well, and don't forget to set the fixVersion with 4.8.1 as well. Only bug fixes should be committed unless we need to do an exception. Enis On Sun, Jul 10, 2016 at 9:39 AM, Andrew Purtell <andrew.purt...@gmail.com> wrote: > It's the monthly sweep of commits on later branches, backporting, and > testing each change. So yeah it's the other end of the spectrum and good as > another a data point. Where a Phoenix RM would fall on that spectrum will > correlate with how long the long term support for a branch has been in > effect. > > I am also planning to run ITBLL with active chaos agents for 24 hours for > every HBase RC once I figure out Dima's clusterdock stuff. Speaking of > which: We should have something like this for Phoenix. For the most part we > can reuse HBase's chaos stuff. We'd want to write and verify the same type > of linked data, with all interesting features enabled like transactions and > local indexes. If we had this then doing it for Phoenix RCs would likely > fall to the RM. > > > > On Jul 10, 2016, at 2:42 AM, <la...@apache.org> <la...@apache.org> > wrote: > > > > 0.94 never took that much time from me.Sure some days it's a lot of > work, but on average I found it a task that can be easily done on the side. > > I relied a lot on the Apache Jenkins, triggering runs and checking back > the next day, etc.I also think you're more diligent in your RM duties :) > > > > -- Lars > > > > From: Andrew Purtell <andrew.purt...@gmail.com> > > To: dev@phoenix.apache.org > > Sent: Saturday, July 9, 2016 9:02 AM > > Subject: Re: [DISCUSS] RM for next release > > > > Huh. RMing HBase 0.98 consumes whole days with testing and patching. YMMV > > > >> On Jul 9, 2016, at 6:55 AM, <la...@apache.org> <la...@apache.org> > wrote: > >> > >> +1 on 4.8.x should support HBase 1.0. (although it would safe us > maintenance of > >> For 4.9 we can drop that (but likely should pull in support for HBase > 1.3).We'll need to continue with 0.98 support until 0.98 is EOL'd, I think. > >> > >> I'm happy to do 4.8.1 and 4.9 for one release, to confirm Stack's > nickname for me: "Iron Hand" :)After the first release more folks should > tag team. RM'ing is actually not that much work, maybe 15-30 mins/day. > >> > >> I'm actually quire exited about the potential of 4.9. We'll bring more > of the power of knowing the schema together with the proven stability of > HBase: > >> PHOENIX-1598 Encode column names to save space and improve performance > >> PHOENIX-2565 Store data for immutable tables in single KeyValue > >> The trick will be to add these in an absolutely backwards compatible > way (where an old 4.8 client will be able to run against a 4.9 server > indefinitely).We may have to start putting a schema version number into the > SYSTEM.CATALOG if we do not have this already. > >> > >> I'll propose merging those into 4.9 early (assuming they are ready and > stable - so that's something to ensure now in their resp branches).Bug > fixes would go into both 4.9 and 4.8.1. > >> > >> As soon as 4.8 is out (hopefully soon now), I'll propose the branches > and create them once we all agree. > >> > >> -- Lars > >> > >> ps. There's no 4.8-HBase-1.0, yet, right?! At least I don't see it. > >> > >> From: James Taylor <jamestay...@apache.org> > >> To: "dev@phoenix.apache.org" <dev@phoenix.apache.org> > >> Sent: Friday, July 8, 2016 3:01 PM > >> Subject: Re: [DISCUSS] RM for next release > >> > >> I think we should go ahead with the 4.8 release for HBase 1.0, but not > do > >> one for 4.9. As far as I recall, there were some objections about not > doing > >> an HBase 1.0 release because CDH was based off of 1.0. Seems like CDH > has > >> moved past this, though, so it's likely not necessary to continue. > Probably > >> something we should propose on the user list. > >> > >>> On Fri, Jul 8, 2016 at 10:51 PM, Enis Söztutar <e...@apache.org> > wrote: > >>> > >>> Let's start with planning for 4.8.1 then. It should be independent of > 4.9.0 > >>> plans. > >>> > >>> The burden is on committers, that they have to commit all bug fixes to > the > >>> 4.8 branches as well (not just master and 4.9 branches as previously). > The > >>> RM for 4.8.1 can spend a couple of extra cycles each day to gently > remind > >>> the committers to do the backport or RM can do the cherry-pick. In > HBase > >>> for example, it is a combination of both that RM can actively look for > bug > >>> fixes to backport, but committers also backport themselves if they > think > >>> that it is a good fix. Usually pinging the RM in the issue helps. > >>> > >>> hbase-1.0 is EOL'ed, and I have proposed in two @dev threads to drop > 1.0 > >>> branches. It was lazy consensus, but we kept the branches without > deleting. > >>> Shall I go ahead and delete the branches for 4.8-HBase-1.0 and> > 4.x-HBase-1.0? If we do it now, 4.8-HBase-1.0 will not be released. > >>> Alternatively, we can do it after the 4.8 release so that 4.9+ will be > >>> 1.1+. > >>> > >>> Enis > >>> > >>>> On Thu, Jul 7, 2016 at 7:59 PM, Nick Dimiduk <ndimi...@gmail.com> > wrote: > >>>> > >>>> +1 for patch releases on a more frequent cadence. I volunteered to do > a > >>>> 4.7.x line in support of my own requirements. I think folks are > settling > >>> on > >>>> a Phoenix minor release for longer periods, and they'll benefit from > >>>> receiving bug fixes along the way. Defining and enforcing > compatibility > >>> is > >>>> a part of that. > >>>> > >>>> On the branch issue, I think we should re-consider the compatibility > >>> shim. > >>>> This is working well for HBase on HDFS, it seems like a reasonable > >>> approach > >>>> for Phoenix on HBase. > >>>> > >>>>> For 4.9, can we drop support for HBase 1.0 (and perhaps 1.1), since > >>> those > >>>> HBase releases are EOL'd? > >>>> > >>>> 1.1 isn't EOL'd yet, but we're working in that direction. Maybe for > 4.10? > >>>> How long does Phoenix want to support users who are sticking to 1.1? > >>> Maybe > >>>> Phoenix moving on is a good forcing function to get folks upgrading to > >>>> 1.2+. > >>>> > >>>> On Thu, Jul 7, 2016 at 10:59 AM, Jan Fernando < > jferna...@salesforce.com> > >>>> wrote: > >>>> > >>>>> As a consumer of Phoenix, moving to where we have regular patch > >>> releases > >>>> on > >>>>> a predicable cadence that contain bug fixes would be incredibly > >>>> beneficial. > >>>>> For the most recent few releases we have only needed bug fixes and > were > >>>> not > >>>>> dependent on any of the new features. Therefore having to wait until > a > >>>>> release with large feature changes stabilizes adds a lot of risk for > >>> us. > >>>>> > >>>>> So I am +1 on Lars' proposal. > >>>>> > >>>>> --Jan > >>>>> > >>>>>> On Thu, Jul 7, 2016 at 2:44 AM, <la...@apache.org> wrote: > >>>>>> > >>>>>> Agreed. On all counts. > >>>>>> There are some bigger changes in the pipeline (column remapping and > >>>>>> "dense" column storage).Those are powerful and combine the power of > >>> SQL > >>>>> and > >>>>>> HBase quite nicely. I propose we get those soon after 4.8 and then > >>>> spin a > >>>>>> 4.9 from those. > >>>>>> > >>>>>> A 4.8.1 would be of great value as well. I think Phoenix has reached > >>>> that > >>>>>> level of maturity now (it's part of Hortonwork, and now finally in > >>>>>> Cloudera).To drive adoption and "satisfaction" now I think we need > to > >>>>>> provide a stable release. > >>>>>> > >>>>>> Questions: > >>>>>> - Can we do both a 4.8.1 and 4.9?- For 4.9, can we drop support for > >>>> HBase > >>>>>> 1.0 (and perhaps 1.1), since those HBase releases are EOL'd?- Maybe > >>> for > >>>>> 4.9 > >>>>>> we only support 0.98 and 1.2? That would reduce the number of > >>> branches > >>>> to > >>>>>> maintain. > >>>>>> - Support HBase 1.3? If we have a release in time.- If the two main > >>>>>> features above are the only "major" changes in 4.9, do we need a > >>> 4.8.1? > >>>>>> > >>>>>> If so we need to cover the > >>>>>> following:4.8.1-0.984.8.1-1.04.8.1-1.14.8.1-1.24.9-0.984.9-1.1 > >>>> (perhaps) > >>>>>> 4.9-1.2 > >>>>>> 4.9-1.3 (perhaps) > >>>>>> With 4.8.1 we could EOL 4.8.I can start RM'ing 4.8.1 as well (the > >>> patch > >>>>>> releases are usually little effort for the RM, it's mostly the > >>>> committers > >>>>>> who have to be diligent backporting bugfixes).But at some point I > >>> think > >>>>>> we'd need 2 RMs at any given time. > >>>>>> > >>>>>> -- Lars > >>>>>> > >>>>>> From: Enis Söztutar <e...@apache.org> > >>>>>> To: "dev@phoenix.apache.org" <dev@phoenix.apache.org> > >>>>>> Sent: Wednesday, July 6, 2016 2:57 PM > >>>>>> Subject: Re: [DISCUSS] RM for next release > >>>>>> > >>>>>> I would really like to get maintenance releases. Release cadence for > >>>>> minor > >>>>>> releases and patch releases should be orthogonal in theory, but we > >>> are > >>>>> all > >>>>>> human and there are only so many hours in a day. I would opt for > >>> doing > >>>>>> actual maintenance releases rather than more frequent minor > releases. > >>>>>> Having smaller changes is definitely good to get a release out the > >>>> door, > >>>>>> but hbase/phoenix is a database. Nobody on their right mind updates > >>>> their > >>>>>> database every month. > >>>>>> > >>>>>> +1 for Lars as always. > >>>>>> > >>>>>> Enis > >>>>>> > >>>>>> On Tue, Jul 5, 2016 at 6:51 PM, Andrew Purtell < > >>>> andrew.purt...@gmail.com > >>>>>> > >>>>>> wrote: > >>>>>> > >>>>>>> Stating it another way: As far as I know, all bugs found in the > >>> 4.7.0 > >>>>>>> release are going to be fixed with 4.8.0, not a 4.7.1, and there's > >>>>> nobody > >>>>>>> planning to maintain a 4.7.x line of releases. It was this way with > >>>>> 4.6.0 > >>>>>>> as well, all bug fixes for problems in 4.6.0 were put in the 4.7.0 > >>>>>> release, > >>>>>>> not a 4.6.1 patch release. I don't think there will ever be a 4.6.x > >>>>> patch > >>>>>>> release. > >>>>>>> > >>>>>>> Like Nick asked, with a monthly release cadence do we see this > >>>>> changing? > >>>>>>> > >>>>>>> Let me put forward this thought: It will be easy to hit a monthly > >>>>> release > >>>>>>> cadence if we treat bug fixes and bigger works like transactions > >>>> (4.7) > >>>>>> and > >>>>>>> local index reimplementations (4.8) differently. We branch > >>>>> appropriately > >>>>>>> for making patch releases but don't take advantage of them. That's > >>>> easy > >>>>>> to > >>>>>>> change. Commit bug fixes to development heads and > >>> maintenance/release > >>>>>>> branches both. Cut releases from the maintenance branches monthly. > >>>>>> Simple. > >>>>>>> When the time comes, just do it. Meanwhile as the bigger things > >>> fully > >>>>>> bake > >>>>>>> do a new minor or even major rev to release them. Bug fixes will > >>> not > >>>>> have > >>>>>>> been held up no matter how long it may have taken for next new > >>>> feature > >>>>> X > >>>>>> to > >>>>>>> bake. In exchange, there will be point releases to make. The HBase > >>>>>> "branch > >>>>>>> RM" model could be helpful for distributing that work. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Tue, Jul 5, 2016 at 1:27 PM, Andrew Purtell < > >>>>> andrew.purt...@gmail.com > >>>>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> But I don't see patch version releases generally. Right? So if > >>> you > >>>>> look > >>>>>>> at > >>>>>>>> release history you'd expect new minors not new patches. > >>>>>>>> > >>>>>>>>> On Jul 5, 2016, at 11:32 AM, Nick Dimiduk <ndimi...@apache.org > >>>> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>> We already support multiple release code lines via > >>>> branch-per-hbase > >>>>>>>> version. > >>>>>>>>> > >>>>>>>>>> On Tue, Jul 5, 2016 at 10:05 AM, Andrew Purtell < > >>>>>> apurt...@apache.org> > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Will under a new monthly cadence the project still produce new > >>>>> minor > >>>>>>>>>> versions at every release until the community decides to do a > >>>>> major > >>>>>>>>>> increment, then continue with minors again? > >>>>>>>>>> > >>>>>>>>>> Or is the plan as Nick wonders to support released minor > >>>> versions > >>>>>>>> longer, > >>>>>>>>>> via patch versions? If so, I suppose this would mean active > >>>>>>>> maintenance of > >>>>>>>>>> multiple code lines, and, if so, are we considering or should > >>> we > >>>>>>>> consider > >>>>>>>>>> the HBase "branch RM" style management for that? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On Tue, Jul 5, 2016 at 9:53 AM, Nick Dimiduk < > >>>>> ndimi...@apache.org> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Is this thread to discuss Lars for RM, for moving to a > >>> monthly > >>>>>>> release > >>>>>>>>>>> cadence, or propose specific JIRAs for the next release? One > >>>> the > >>>>>>> above: > >>>>>>>>>>> > >>>>>>>>>>> +1 for Lars, he knows how to make releases happen :) > >>>>>>>>>>> > >>>>>>>>>>> Is this monthly cadence for patch releases? So far this > >>>> community > >>>>>>>> hasn't > >>>>>>>>>>> seen fit to make patch releases, so I'm wondering what's > >>>> changed > >>>>>> now. > >>>>>>>> Are > >>>>>>>>>>> we thinking the rate of change in the product has > >>>>> slowed/stabilized > >>>>>>> and > >>>>>>>>>> now > >>>>>>>>>>> we're going to support released versions longer? Have we > >>>> decided > >>>>> on > >>>>>>>>>> policy > >>>>>>>>>>> re: what makes a change suitable for a patch release vs. the > >>>> next > >>>>>>>> minor? > >>>>>>>>>>> > >>>>>>>>>>> Re: these tickets, those all look like good improvements and > >>>>> fixes > >>>>>> to > >>>>>>>> get > >>>>>>>>>>> shipped. Hopefully the last two would qualify as patch > >>> release > >>>>>>>> material. > >>>>>>>>>>> > >>>>>>>>>>> On Sat, Jul 2, 2016 at 12:16 AM, James Taylor < > >>>>>>> jamestay...@apache.org> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Lars has bravely volunteered to be our RM for our next > >>> release > >>>>>> with > >>>>>>> an > >>>>>>>>>>> aim > >>>>>>>>>>>> to help us get on a monthly release cadence. Big +1 from me. > >>>> We > >>>>>>> have a > >>>>>>>>>>> few > >>>>>>>>>>>> features teed up on the encodecolumns branch: > >>>>>>>>>>>> > >>>>>>>>>>>> PHOENIX-1598 Encode column names to save space and improve > >>>>>>> performance > >>>>>>>>>>>> PHOENIX-2565 Store data for immutable tables in single > >>>> KeyValue > >>>>>>>>>>>> > >>>>>>>>>>>> These have both been implemented in a b/w compatible manner. > >>>>>>> Existing > >>>>>>>>>>>> tables would continue to work as-is and new tables would > >>> take > >>>>>>>> advantage > >>>>>>>>>>> of > >>>>>>>>>>>> these new formats. > >>>>>>>>>>>> > >>>>>>>>>>>> A couple of other important JIRAs that I know about are: > >>>>>>>>>>>> > >>>>>>>>>>>> PHOENIX-2995 Write performance severely degrades with large > >>>>> number > >>>>>>> of > >>>>>>>>>>> views > >>>>>>>>>>>> PHOENIX-2724 Query with large number of guideposts is slower > >>>>>>> compared > >>>>>>>>>> to > >>>>>>>>>>> no > >>>>>>>>>>>> stats > >>>>>>>>>>>> > >>>>>>>>>>>> Hopefully these can make it in, but it'd be up to the > >>>> digression > >>>>>> of > >>>>>>>> the > >>>>>>>>>>> RM, > >>>>>>>>>>>> of course. > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> James > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Best regards, > >>>>>>>>>> > >>>>>>>>>> - Andy > >>>>>>>>>> > >>>>>>>>>> Problems worthy of attack prove their worth by hitting back. - > >>>>> Piet > >>>>>>> Hein > >>>>>>>>>> (via Tom White) > > >
