I want to push an alpha-4 today. A few items didn't make it (HBASE-19092). They need more time. We'll pull them in for beta-1. CP API is basically done. There may be some changes for beta-1 but hopefully only changes informed by experience trying to port an existing Coprocessor to hbase2.
Shout if there is anything that needs to make alpha-4. Thanks, St.Ack On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <[email protected]> wrote: > Yup, that was going to be my plan, Mike! > > Making a pass now, and will check back later tonight again. I see others > have already done some work today on this front. > > > On 10/27/17 11:38 PM, Mike Drob wrote: > >> Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do >> it directly on the jenkins job, so you don't have to bother with JIRA >> uploads) >> >> If you're busy, then I can make time tomorrow or Sunday to kick off jobs, >> but I want to make sure we're not duplicating effort and jenkins cycles. >> >> On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <[email protected]> wrote: >> >> My turn to bump ;) >>> >>> By my take: HBASE-18770 and HBASE-19092 are the only issues that remain >>> needing some more work. The rest are just awaiting a good QA run. >>> >>> Unless I hear otherwise, I'll try to keep an eye on things over the >>> weekend, bump them along as necessary, and get them committed. Would be >>> great to be able get a vote up on Monday. >>> >>> >>> On 10/24/17 6:03 PM, Stack wrote: >>> >>> Chatting with my coworker Mr. Mike Drob, we were batting back and forth >>>> what remains to be done. Surfacing our thoughts here so you all clued >>>> in....Or if you think otherwise, please speak up. >>>> >>>> We have ~13 issues to land: >>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About >>>> two >>>> are meta-issues that are about process which leaves 11. >>>> >>>> Duo and Zheng Hu are to merge the FilterList fixes improvements >>>> (HBASE-19057, HBASE-18410 et al.). These are blocker because some >>>> changed >>>> API/semantic that we need to get out earlier rather than later. >>>> >>>> Once the above is merged, HBASE-13346, change of Filter method names to >>>> mention Cell instead of KeyValue can land. >>>> >>>> HBASE-199048 needs a review (Anoop will probably do it), removing >>>> IA.Private objects as params to MasterObserver... Hopefully this goes in >>>> soon. >>>> >>>> Duo is hard at work on trackers for flush and compaction for CPs >>>> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? >>>> >>>> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo >>>> is >>>> done w/ his work above. >>>> >>>> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all >>>> the >>>> purges allowing CPs do direct calls against Regions in same Host. >>>> >>>> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. >>>> >>>> Another day or two? >>>> >>>> St.Ack >>>> >>>> >>>> >>>> On Mon, Oct 23, 2017 at 2:52 PM, Stack <[email protected]> wrote: >>>> >>>> >>>> On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <[email protected]> wrote: >>>>> >>>>> +1 >>>>> >>>>>> >>>>>> I was trying to work on helping out on the outstanding alpha-4 stuff >>>>>> last >>>>>> week -- will be continuing to try to do the same this week. >>>>>> >>>>>> If you need any help, Stack, or if others need reviews where I haven't >>>>>> noticed on my own: feel free to @mention me. >>>>>> >>>>>> >>>>>> Thanks for the offer Josh. All items seem assigned and are being >>>>>> >>>>> actively >>>>> worked on. If you get a moment, reviews by you (or anyone else) helps >>>>> move >>>>> the process along. >>>>> >>>>> We need to merge in HBASE-18410 branch to pick up Filter improvements. >>>>> Then HBASE-13346 can go in. >>>>> >>>>> You are already helping out on HBASE-18906, thanks. Looks like that >>>>> will >>>>> be addressed by other alpha-4s about to land. >>>>> >>>>> St.Ack >>>>> TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 10/23/17 12:53 PM, Stack wrote: >>>>> >>>>>> >>>>>> (Reviving this thread) >>>>>> >>>>>>> >>>>>>> Lets push out alpha-4 this week. Alpha-4 is the release that has the >>>>>>> refactor of the Coprocessor API shutting down access to internals >>>>>>> marked >>>>>>> InterfaceAudience.Private. >>>>>>> >>>>>>> The outstanding list is here: >>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 >>>>>>> >>>>>>> Please push in anything marked alpha-4 that belongs to you. >>>>>>> >>>>>>> If issue, talk out loud on this thread. If you need a review to land >>>>>>> an >>>>>>> item, shout on the issue and here; we'll help you out. >>>>>>> >>>>>>> As is, items are coming along nicely I'd say. We need to merge the >>>>>>> filter >>>>>>> branch -- HBASE-18410 -- so APIs are finished for hbase2. >>>>>>> >>>>>>> Post alpha-4, we'll have to hunt down our downstreamers and help them >>>>>>> test >>>>>>> on top of alpha-4 so rolling into beta-1, we have confidence our >>>>>>> downstreamers know what to expect (or we discover what we missed >>>>>>> BEFORE >>>>>>> we >>>>>>> beta-1). >>>>>>> >>>>>>> Thanks for time, >>>>>>> S >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <[email protected]> wrote: >>>>>>> >>>>>>> I'll put up an alpha3 RC Monday, probably Monday night. That should >>>>>>> be >>>>>>> >>>>>>> time, if we all sprint, for the public-facing API fixes to be done. >>>>>>>> >>>>>>>> I had a bunch of Coprocessor refactor and fixup scheduled for alpha3 >>>>>>>> but >>>>>>>> it is plain that more time is needed (in spite of valiant effort so >>>>>>>> far >>>>>>>> by >>>>>>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose >>>>>>>> theme is >>>>>>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the >>>>>>>> following >>>>>>>> week. >>>>>>>> >>>>>>>> We should then be ready for beta (beta == no new features, no API >>>>>>>> changes, >>>>>>>> just fixes). >>>>>>>> >>>>>>>> Thanks, >>>>>>>> St.Ack >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <[email protected]> wrote: >>>>>>>> >>>>>>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on >>>>>>>> it. >>>>>>>> >>>>>>>> >>>>>>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to >>>>>>>>> get a >>>>>>>>> release out in the next week or so. >>>>>>>>> >>>>>>>>> I did a weeding of 2.0.0 issues over the last day. If folks are >>>>>>>>> interested in helping out, below are the items I think we need done >>>>>>>>> for >>>>>>>>> alpha3 (below are at least 'Critical' status, are API possibly >>>>>>>>> altering >>>>>>>>> items, and are absent those JIRAs that are making active progress, >>>>>>>>> i.e. the >>>>>>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs >>>>>>>>> doing is >>>>>>>>> what Andrew did comparing 1.3. and 1.4 APIs >>>>>>>>> >>>>>>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 and >>>>>>>>> branch-2 >>>>>>>>> This is to do what Andrew did between 1.3 and 1.4 branches only do >>>>>>>>> it >>>>>>>>> between branch-1 and branch-2. >>>>>>>>> >>>>>>>>> * HBASE-10462 Recategorize some of the client facing Public / >>>>>>>>> Private >>>>>>>>> interfaces >>>>>>>>> This one is almost done. It could do with a finish, attention to >>>>>>>>> the >>>>>>>>> items in last comment, and then our codebase could do with another >>>>>>>>> sweep >>>>>>>>> after the spirit of this issue since a bunch has gone in since the >>>>>>>>> pass >>>>>>>>> that was the basis of this issue. >>>>>>>>> >>>>>>>>> * HBASE-10504 Define Replication Interface >>>>>>>>> I was going to take a crack at this as part of the revamp forced by >>>>>>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service' >>>>>>>>> but if >>>>>>>>> anyone else is interested, be my guest. >>>>>>>>> >>>>>>>>> * HBASE-14996 Some more API cleanup for 2.0 >>>>>>>>> Has a bunch of subtasks, some of which are being worked on. Needs >>>>>>>>> finishing. >>>>>>>>> >>>>>>>>> * HBASE-14998 Unify synchronous and asynchronous methods in Admin >>>>>>>>> and >>>>>>>>> cleanup >>>>>>>>> Needs a pass. Small issue I think. Could also look at new >>>>>>>>> AsyncClient >>>>>>>>> and >>>>>>>>> make sure symmetry. >>>>>>>>> >>>>>>>>> * HBASE-15607 Remove PB references from Admin for 2.0 >>>>>>>>> Predicated on result of an ongoing DISCUSSION thread but needs to >>>>>>>>> be >>>>>>>>> done. >>>>>>>>> >>>>>>>>> Rolling upgrade will have implications for our API. Would be good >>>>>>>>> to >>>>>>>>> try >>>>>>>>> it and figure what needs fixup (as said above, according to trial >>>>>>>>> by >>>>>>>>> Sean, >>>>>>>>> we might not be too bad here): >>>>>>>>> * HBASE-16060 1.x clients cannot access table state talking to 2.0 >>>>>>>>> cluster >>>>>>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master >>>>>>>>> and >>>>>>>>> 1.x >>>>>>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2. >>>>>>>>> >>>>>>>>> * HBASE-17442 Move most of the replication related classes to >>>>>>>>> hbase-server package >>>>>>>>> The above would be good to do generally but it may make for ripples >>>>>>>>> in >>>>>>>>> API so would be good to do now. >>>>>>>>> >>>>>>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo >>>>>>>>> Balazs is working on this. The idea is that we avoid adding two new >>>>>>>>> types >>>>>>>>> to our API, two types that are nought but curtailed, read-only >>>>>>>>> views >>>>>>>>> on >>>>>>>>> internals. Input if you have time appreciated. >>>>>>>>> >>>>>>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a >>>>>>>>> hbase2 >>>>>>>>> cluster; verify >>>>>>>>> Esteban is looking at this one >>>>>>>>> >>>>>>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core >>>>>>>>> * HBASE-17143 Scan improvement >>>>>>>>> >>>>>>>>> Our Coprocessor Interface needs a tough edit. It exposes >>>>>>>>> implementations >>>>>>>>> marked audience Private and returns implementations rather than >>>>>>>>> Interfaces. >>>>>>>>> In a few locations, we allow returning an alternate implementation >>>>>>>>> altogether which is probably something we don't want a CP doing. To >>>>>>>>> that >>>>>>>>> end, the following issues started by Duo and Anoop need to be taken >>>>>>>>> to >>>>>>>>> the >>>>>>>>> finish line; ideally they'd have an owner: >>>>>>>>> >>>>>>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <= >>>>>>>>> The >>>>>>>>> umbrella issue. >>>>>>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP expose >>>>>>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver >>>>>>>>> and >>>>>>>>> RegionServerObserver >>>>>>>>> >>>>>>>>> >>>>>>>>> Nice-to-haves: >>>>>>>>> >>>>>>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove >>>>>>>>> unused >>>>>>>>> TimeRange constructors >>>>>>>>> >>>>>>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references >>>>>>>>> existing in the code >>>>>>>>> This is the end of an old long-running project moving up on to Cell >>>>>>>>> Interface. We think it is done but for a few little items >>>>>>>>> (deprecate >>>>>>>>> KV >>>>>>>>> methods in MR and provide Cell versions instead...) >>>>>>>>> >>>>>>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate; >>>>>>>>> needs >>>>>>>>> fixing >>>>>>>>> >>>>>>>>> * HBASE-13346 Clean up Filter package for post 1.0 >>>>>>>>> >>>>>>>>> * HBASE-14255 Simplify Cell creation post 1.0 >>>>>>>>> * HBASE-14997 >>>>>>>>> Move compareOp and Comparators out of filter to client package >>>>>>>>> >>>>>>>>> * HBASE-13740 Stop using Hadoop private interfaces >>>>>>>>> >>>>>>>>> What about: >>>>>>>>> >>>>>>>>> * HBASE-18601 Remove Htrace 3.2 >>>>>>>>> As has been noted, the HTrace API is our 'trace' API. >>>>>>>>> >>>>>>>>> If interested in any of the above and you need a legup, just ask in >>>>>>>>> the >>>>>>>>> issue and I'll be by.... >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> St.Ack >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <[email protected]> wrote: >>>>>>>>> >>>>>>>>> Heads-up: >>>>>>>>> >>>>>>>>> >>>>>>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme >>>>>>>>>> is >>>>>>>>>> updated dependencies, reliance on relocated popular libs (guava, >>>>>>>>>> netty, >>>>>>>>>> protobuf), purge of checked-in generated src, and >>>>>>>>>> master-carries-no-regions >>>>>>>>>> by default. >>>>>>>>>> >>>>>>>>>> alpha3 I hope will follow soon after (end-of-August?). Its theme >>>>>>>>>> will >>>>>>>>>> be >>>>>>>>>> settling the APIs and compatibility (At first blush, we are not >>>>>>>>>> looking too >>>>>>>>>> bad; our Sean ran some tests over weekend that have hbase-1 client >>>>>>>>>> running >>>>>>>>>> against an hbase-2 cluster....). The Coprocessor Interface revamp >>>>>>>>>> should be >>>>>>>>>> done by alpha3 (i.e. returning Interfaces rather than >>>>>>>>>> Implementations, and >>>>>>>>>> our shutdown of CPs accessing classes in hbase marked >>>>>>>>>> InterfaceAudience). >>>>>>>>>> We'll also have purged thirdparty classes from our API; e.g. guava >>>>>>>>>> 0.12 >>>>>>>>>> Service showing through in our replication API and protobufs in >>>>>>>>>> Admin >>>>>>>>>> Interface. On alpha3, we will have to do a bunch of outreach to >>>>>>>>>> make >>>>>>>>>> sure >>>>>>>>>> our downstreamers are up on what is coming down the pipe. >>>>>>>>>> >>>>>>>>>> Beta1 in mid-September? >>>>>>>>>> >>>>>>>>>> I encourage you to check out the items marked for hbase2: >>>>>>>>>> https://issues.apache.org/jira/projects/HBASE/versions/12327188 >>>>>>>>>> Edit >>>>>>>>>> as >>>>>>>>>> you see appropriate. Punt if you know the JIRA will not get any >>>>>>>>>> attention >>>>>>>>>> in next month or so. >>>>>>>>>> >>>>>>>>>> A bunch of issues marked blocker are unassigned. I'll leave them >>>>>>>>>> as >>>>>>>>>> is >>>>>>>>>> another while but I'll boot them soon. >>>>>>>>>> >>>>>>>>>> While I have your attention: >>>>>>>>>> >>>>>>>>>> + I think we should leave thrift version at 0.9.3. Moving hbase >>>>>>>>>> thrift >>>>>>>>>> to 0.10.0 will break existing clients. The change is easy enough >>>>>>>>>> if >>>>>>>>>> folks >>>>>>>>>> need to upgrade their hbase thrift. See HBASE-18591. >>>>>>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first >>>>>>>>>> (0.98?). >>>>>>>>>> >>>>>>>>>> St.Ack >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 7/31/17 9:00 AM, Stack wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<[email protected]> >>>>>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> ... >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> I like the idea of this also hitting 2.0 as it would make the >>>>>>>>>>>>>> feature a >>>>>>>>>>>>>> bit more "real", but am obviously a little nervous (I have no >>>>>>>>>>>>>> reason >>>>>>>>>>>>>> to be >>>>>>>>>>>>>> nervous though). I am pretty happy with the feature in terms >>>>>>>>>>>>>> of >>>>>>>>>>>>>> how >>>>>>>>>>>>>> much it >>>>>>>>>>>>>> is covered via testing. >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1 Josh. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Last think on space quota says that need doc too. See 'Space >>>>>>>>>>>>> Quota' in >>>>>>>>>>>>> here: >>>>>>>>>>>>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i >>>>>>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5 >>>>>>>>>>>>> Does this little section need an update Josh? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> S >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and some >>>>>>>>>>>>> >>>>>>>>>>>> docs >>>>>>>>>>>> already included for 17748. Happy to put that on my plate if >>>>>>>>>>>> you're good >>>>>>>>>>>> with it. I can reasonably assume that no one is against it :) >>>>>>>>>>>> >>>>>>>>>>>> I think I had knocked out docs for the "phase 1" stuff before we >>>>>>>>>>>> merged it in from the original feature branch. I'll double check >>>>>>>>>>>> and update >>>>>>>>>>>> the gdoc. Perhaps this was just a timing thing. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks Josh, >>>>>>>>>>>> >>>>>>>>>>> S >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>
