Alpha4 aims to freeze the CP related API, not the public API. We break everything for CP so we need to push out a stable(I hope) version to let CP users learn how to implement their project on top of the new APIs.
For public API we have rules on how to keep compatible so I think it is less hurt for users, beta1 is fine. 2017-10-26 17:27 GMT+08:00 Guanghao Zhang <zghao...@gmail.com>: > I saw beta == no new features, no API changes, just fixes. And I am working > on HBASE-18805 to unify Admin and AsyncAdmin methods. The fix version was > 2.0-beta-1. But I thought this will introduce API change(deprecate some API > and introduce new one). So should I change the fix versions to 2.0-alpha-4 > and finish it before we release 2.0-alpha-4? > > Issue HBASE-18978 (Align the methods in Table and AsyncTable) may have same > problem. Thanks. > > 2017-10-26 9:51 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > > > OK, skimmed, we are in trouble! The in memory compaction just use the > same > > constructor with normal compaction to construct a StoreScanner, and use > it > > to do compaction... > > > > We have to provide several preXXX and postXXX for it, at least we should > > allow user reset TTL and max versions, and also do filtering on the > > scanner. > > > > Thanks. > > > > > > > > 2017-10-26 9:41 GMT+08:00 张铎(Duo Zhang) <palomino...@gmail.com>: > > > > > When adding back the pre(Flush/Compact/Store)ScannerOpen methods in > > > HBASE-19033, I found that there maybe a hole in our CP hools. It is the > > in > > > memory compaction. > > > > > > As Anoop suggested in HBASE-19001, we still need to give user the > ability > > > to extend the max versions and TTL config, so I plan to add back the > > hooks > > > above to let CP users can change the max versions and TTL of a ScanInfo > > > object. But I'm not sure whether in memory compaction will also discard > > > expired cells, if so then we are in trouble... > > > > > > Thanks. > > > > > > 2017-10-25 6:03 GMT+08:00 Stack <st...@duboce.net>: > > > > > >> Chatting with my coworker Mr. Mike Drob, we were batting back and > forth > > >> what remains to be done. Surfacing our thoughts here so you all clued > > >> in....Or if you think otherwise, please speak up. > > >> > > >> We have ~13 issues to land: > > >> https://issues.apache.org/jira/projects/HBASE/versions/12341594 About > > two > > >> are meta-issues that are about process which leaves 11. > > >> > > >> Duo and Zheng Hu are to merge the FilterList fixes improvements > > >> (HBASE-19057, HBASE-18410 et al.). These are blocker because some > > changed > > >> API/semantic that we need to get out earlier rather than later. > > >> > > >> Once the above is merged, HBASE-13346, change of Filter method names > to > > >> mention Cell instead of KeyValue can land. > > >> > > >> HBASE-199048 needs a review (Anoop will probably do it), removing > > >> IA.Private objects as params to MasterObserver... Hopefully this goes > in > > >> soon. > > >> > > >> Duo is hard at work on trackers for flush and compaction for CPs > > >> (HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)? > > >> > > >> I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after > Duo > > >> is > > >> done w/ his work above. > > >> > > >> I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after > all > > >> the > > >> purges allowing CPs do direct calls against Regions in same Host. > > >> > > >> Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil. > > >> > > >> Another day or two? > > >> > > >> St.Ack > > >> > > >> > > >> > > >> On Mon, Oct 23, 2017 at 2:52 PM, Stack <st...@duboce.net> wrote: > > >> > > >> > > > >> > On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <els...@apache.org> > > wrote: > > >> > > > >> >> +1 > > >> >> > > >> >> I was trying to work on helping out on the outstanding alpha-4 > stuff > > >> last > > >> >> week -- will be continuing to try to do the same this week. > > >> >> > > >> >> If you need any help, Stack, or if others need reviews where I > > haven't > > >> >> noticed on my own: feel free to @mention me. > > >> >> > > >> >> > > >> > Thanks for the offer Josh. All items seem assigned and are being > > >> actively > > >> > worked on. If you get a moment, reviews by you (or anyone else) > helps > > >> move > > >> > the process along. > > >> > > > >> > We need to merge in HBASE-18410 branch to pick up Filter > improvements. > > >> > Then HBASE-13346 can go in. > > >> > > > >> > You are already helping out on HBASE-18906, thanks. Looks like that > > will > > >> > be addressed by other alpha-4s about to land. > > >> > > > >> > St.Ack > > >> > TODOs: https://issues.apache.org/jira/projects/HBASE/versions/ > > 12341594 > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> >> On 10/23/17 12:53 PM, Stack wrote: > > >> >> > > >> >>> (Reviving this thread) > > >> >>> > > >> >>> Lets push out alpha-4 this week. Alpha-4 is the release that has > the > > >> >>> refactor of the Coprocessor API shutting down access to internals > > >> marked > > >> >>> InterfaceAudience.Private. > > >> >>> > > >> >>> The outstanding list is here: > > >> >>> https://issues.apache.org/jira/projects/HBASE/versions/12341594 > > >> >>> > > >> >>> Please push in anything marked alpha-4 that belongs to you. > > >> >>> > > >> >>> If issue, talk out loud on this thread. If you need a review to > land > > >> an > > >> >>> item, shout on the issue and here; we'll help you out. > > >> >>> > > >> >>> As is, items are coming along nicely I'd say. We need to merge the > > >> filter > > >> >>> branch -- HBASE-18410 -- so APIs are finished for hbase2. > > >> >>> > > >> >>> Post alpha-4, we'll have to hunt down our downstreamers and help > > them > > >> >>> test > > >> >>> on top of alpha-4 so rolling into beta-1, we have confidence our > > >> >>> downstreamers know what to expect (or we discover what we missed > > >> BEFORE > > >> >>> we > > >> >>> beta-1). > > >> >>> > > >> >>> Thanks for time, > > >> >>> S > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> On Fri, Sep 8, 2017 at 2:04 PM, Stack <st...@duboce.net> wrote: > > >> >>> > > >> >>> I'll put up an alpha3 RC Monday, probably Monday night. That > should > > be > > >> >>>> time, if we all sprint, for the public-facing API fixes to be > done. > > >> >>>> > > >> >>>> I had a bunch of Coprocessor refactor and fixup scheduled for > > alpha3 > > >> but > > >> >>>> it is plain that more time is needed (in spite of valiant effort > so > > >> far > > >> >>>> by > > >> >>>> Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 > whose > > >> >>>> theme is > > >> >>>> "Coprocessor Fixup". Hopefully we can put an alpha-4 up by the > > >> following > > >> >>>> week. > > >> >>>> > > >> >>>> We should then be ready for beta (beta == no new features, no API > > >> >>>> changes, > > >> >>>> just fixes). > > >> >>>> > > >> >>>> Thanks, > > >> >>>> St.Ack > > >> >>>> > > >> >>>> > > >> >>>> On Thu, Aug 17, 2017 at 12:35 PM, Stack <st...@duboce.net> > wrote: > > >> >>>> > > >> >>>> I put up the hbase-2.0.0-alpha2 release candidate. Please vote on > > it. > > >> >>>>> > > >> >>>>> For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to > > get > > >> a > > >> >>>>> release out in the next week or so. > > >> >>>>> > > >> >>>>> I did a weeding of 2.0.0 issues over the last day. If folks are > > >> >>>>> interested in helping out, below are the items I think we need > > done > > >> for > > >> >>>>> alpha3 (below are at least 'Critical' status, are API possibly > > >> altering > > >> >>>>> items, and are absent those JIRAs that are making active > progress, > > >> >>>>> i.e. the > > >> >>>>> HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that > needs > > >> >>>>> doing is > > >> >>>>> what Andrew did comparing 1.3. and 1.4 APIs > > >> >>>>> > > >> >>>>> * HBASE-18622 Mitigate compatibility concerns between branch-1 > and > > >> >>>>> branch-2 > > >> >>>>> This is to do what Andrew did between 1.3 and 1.4 branches only > do > > >> it > > >> >>>>> between branch-1 and branch-2. > > >> >>>>> > > >> >>>>> * HBASE-10462 Recategorize some of the client facing Public / > > >> Private > > >> >>>>> interfaces > > >> >>>>> This one is almost done. It could do with a finish, attention to > > the > > >> >>>>> items in last comment, and then our codebase could do with > another > > >> >>>>> sweep > > >> >>>>> after the spirit of this issue since a bunch has gone in since > the > > >> pass > > >> >>>>> that was the basis of this issue. > > >> >>>>> > > >> >>>>> * HBASE-10504 Define Replication Interface > > >> >>>>> I was going to take a crack at this as part of the revamp forced > > by > > >> >>>>> 'HBASE-15982 Interface ReplicationEndpoint extends Guava's > > Service' > > >> >>>>> but if > > >> >>>>> anyone else is interested, be my guest. > > >> >>>>> > > >> >>>>> * HBASE-14996 Some more API cleanup for 2.0 > > >> >>>>> Has a bunch of subtasks, some of which are being worked on. > Needs > > >> >>>>> finishing. > > >> >>>>> > > >> >>>>> * HBASE-14998 Unify synchronous and asynchronous methods in > Admin > > >> and > > >> >>>>> cleanup > > >> >>>>> Needs a pass. Small issue I think. Could also look at new > > >> AsyncClient > > >> >>>>> and > > >> >>>>> make sure symmetry. > > >> >>>>> > > >> >>>>> * HBASE-15607 Remove PB references from Admin for 2.0 > > >> >>>>> Predicated on result of an ongoing DISCUSSION thread but needs > to > > be > > >> >>>>> done. > > >> >>>>> > > >> >>>>> Rolling upgrade will have implications for our API. Would be > good > > to > > >> >>>>> try > > >> >>>>> it and figure what needs fixup (as said above, according to > trial > > by > > >> >>>>> Sean, > > >> >>>>> we might not be too bad here): > > >> >>>>> * HBASE-16060 1.x clients cannot access table state talking to > 2.0 > > >> >>>>> cluster > > >> >>>>> * HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master > > and > > >> >>>>> 1.x > > >> >>>>> RSs; i.e. support Rolling Upgrade from hbase-1 to -2. > > >> >>>>> > > >> >>>>> * HBASE-17442 Move most of the replication related classes to > > >> >>>>> hbase-server package > > >> >>>>> The above would be good to do generally but it may make for > > ripples > > >> in > > >> >>>>> API so would be good to do now. > > >> >>>>> > > >> >>>>> * HBASE-18106 Redo ProcedureInfo and LockInfo > > >> >>>>> Balazs is working on this. The idea is that we avoid adding two > > new > > >> >>>>> types > > >> >>>>> to our API, two types that are nought but curtailed, read-only > > >> views on > > >> >>>>> internals. Input if you have time appreciated. > > >> >>>>> > > >> >>>>> * HBASE-18596 A hbase1 cluster should be able to replicate to a > > >> hbase2 > > >> >>>>> cluster; verify > > >> >>>>> Esteban is looking at this one > > >> >>>>> > > >> >>>>> * HBASE-9417 SecureBulkLoadEndpoint should be folded in core > > >> >>>>> * HBASE-17143 Scan improvement > > >> >>>>> > > >> >>>>> Our Coprocessor Interface needs a tough edit. It exposes > > >> >>>>> implementations > > >> >>>>> marked audience Private and returns implementations rather than > > >> >>>>> Interfaces. > > >> >>>>> In a few locations, we allow returning an alternate > implementation > > >> >>>>> altogether which is probably something we don't want a CP doing. > > To > > >> >>>>> that > > >> >>>>> end, the following issues started by Duo and Anoop need to be > > taken > > >> to > > >> >>>>> the > > >> >>>>> finish line; ideally they'd have an owner: > > >> >>>>> > > >> >>>>> * HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release > <= > > >> The > > >> >>>>> umbrella issue. > > >> >>>>> * HBASE-18298 RegionServerServices Interface cleanup for CP > expose > > >> >>>>> * HBASE-16769 Deprecate/remove PB references from MasterObserver > > and > > >> >>>>> RegionServerObserver > > >> >>>>> > > >> >>>>> > > >> >>>>> Nice-to-haves: > > >> >>>>> > > >> >>>>> * HBASE-15284 Make TimeRange constructors IA.Private and remove > > >> unused > > >> >>>>> TimeRange constructors > > >> >>>>> > > >> >>>>> * HBASE-10944 Remove all kv.getBuffer() and kv.getRow() > references > > >> >>>>> existing in the code > > >> >>>>> This is the end of an old long-running project moving up on to > > Cell > > >> >>>>> Interface. We think it is done but for a few little items > > >> (deprecate KV > > >> >>>>> methods in MR and provide Cell versions instead...) > > >> >>>>> > > >> >>>>> * HBASE-13271 Table#puts(List<Put>) operation is indeterminate; > > >> needs > > >> >>>>> fixing > > >> >>>>> > > >> >>>>> * HBASE-13346 Clean up Filter package for post 1.0 > > >> >>>>> > > >> >>>>> * HBASE-14255 Simplify Cell creation post 1.0 > > >> >>>>> * HBASE-14997 > > >> >>>>> Move compareOp and Comparators out of filter to client package > > >> >>>>> > > >> >>>>> * HBASE-13740 Stop using Hadoop private interfaces > > >> >>>>> > > >> >>>>> What about: > > >> >>>>> > > >> >>>>> * HBASE-18601 Remove Htrace 3.2 > > >> >>>>> As has been noted, the HTrace API is our 'trace' API. > > >> >>>>> > > >> >>>>> If interested in any of the above and you need a legup, just ask > > in > > >> the > > >> >>>>> issue and I'll be by.... > > >> >>>>> > > >> >>>>> Thanks, > > >> >>>>> St.Ack > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> On Mon, Aug 14, 2017 at 10:54 AM, Stack <st...@duboce.net> > wrote: > > >> >>>>> > > >> >>>>> Heads-up: > > >> >>>>>> > > >> >>>>>> I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. > > Theme > > >> is > > >> >>>>>> updated dependencies, reliance on relocated popular libs > (guava, > > >> >>>>>> netty, > > >> >>>>>> protobuf), purge of checked-in generated src, and > > >> >>>>>> master-carries-no-regions > > >> >>>>>> by default. > > >> >>>>>> > > >> >>>>>> alpha3 I hope will follow soon after (end-of-August?). Its > theme > > >> will > > >> >>>>>> be > > >> >>>>>> settling the APIs and compatibility (At first blush, we are not > > >> >>>>>> looking too > > >> >>>>>> bad; our Sean ran some tests over weekend that have hbase-1 > > client > > >> >>>>>> running > > >> >>>>>> against an hbase-2 cluster....). The Coprocessor Interface > revamp > > >> >>>>>> should be > > >> >>>>>> done by alpha3 (i.e. returning Interfaces rather than > > >> >>>>>> Implementations, and > > >> >>>>>> our shutdown of CPs accessing classes in hbase marked > > >> >>>>>> InterfaceAudience). > > >> >>>>>> We'll also have purged thirdparty classes from our API; e.g. > > guava > > >> >>>>>> 0.12 > > >> >>>>>> Service showing through in our replication API and protobufs in > > >> Admin > > >> >>>>>> Interface. On alpha3, we will have to do a bunch of outreach to > > >> make > > >> >>>>>> sure > > >> >>>>>> our downstreamers are up on what is coming down the pipe. > > >> >>>>>> > > >> >>>>>> Beta1 in mid-September? > > >> >>>>>> > > >> >>>>>> I encourage you to check out the items marked for hbase2: > > >> >>>>>> https://issues.apache.org/jira/projects/HBASE/versions/ > 12327188 > > >> Edit > > >> >>>>>> as > > >> >>>>>> you see appropriate. Punt if you know the JIRA will not get any > > >> >>>>>> attention > > >> >>>>>> in next month or so. > > >> >>>>>> > > >> >>>>>> A bunch of issues marked blocker are unassigned. I'll leave > them > > >> as is > > >> >>>>>> another while but I'll boot them soon. > > >> >>>>>> > > >> >>>>>> While I have your attention: > > >> >>>>>> > > >> >>>>>> + I think we should leave thrift version at 0.9.3. Moving hbase > > >> thrift > > >> >>>>>> to 0.10.0 will break existing clients. The change is easy > enough > > if > > >> >>>>>> folks > > >> >>>>>> need to upgrade their hbase thrift. See HBASE-18591. > > >> >>>>>> + Upgrade from 0.94 is disallowed. You have to get to 1.0 first > > >> >>>>>> (0.98?). > > >> >>>>>> > > >> >>>>>> St.Ack > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> On Wed, Aug 2, 2017 at 9:43 AM, Stack <st...@duboce.net> > wrote: > > >> >>>>>> > > >> >>>>>> > > >> >>>>>>> > > >> >>>>>>> On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <els...@apache.org > > > > >> >>>>>>> wrote: > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> On 7/31/17 9:00 AM, Stack wrote: > > >> >>>>>>>> > > >> >>>>>>>> On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser< > els...@apache.org > > > > > >> >>>>>>>>> wrote: > > >> >>>>>>>>> > > >> >>>>>>>>> ... > > >> >>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> I like the idea of this also hitting 2.0 as it would make > the > > >> >>>>>>>>>> feature a > > >> >>>>>>>>>> bit more "real", but am obviously a little nervous (I have > no > > >> >>>>>>>>>> reason > > >> >>>>>>>>>> to be > > >> >>>>>>>>>> nervous though). I am pretty happy with the feature in > terms > > of > > >> >>>>>>>>>> how > > >> >>>>>>>>>> much it > > >> >>>>>>>>>> is covered via testing. > > >> >>>>>>>>>> > > >> >>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-17748 > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> Sounds good to me. Whats involved? Backport? If so, +1 > Josh. > > >> >>>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>>> Last think on space quota says that need doc too. See 'Space > > >> >>>>>>>>> Quota' in > > >> >>>>>>>>> here: > > >> >>>>>>>>> https://docs.google.com/document/d/ > 1WCsVlnHjJeKUcl7wHwqb4z9i > > >> >>>>>>>>> Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5 > > >> >>>>>>>>> Does this little section need an update Josh? > > >> >>>>>>>>> > > >> >>>>>>>>> Thanks, > > >> >>>>>>>>> S > > >> >>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>> Yep, just a couple of cherry-picks. Good test coverage and > some > > >> docs > > >> >>>>>>>> already included for 17748. Happy to put that on my plate if > > >> >>>>>>>> you're good > > >> >>>>>>>> with it. I can reasonably assume that no one is against it :) > > >> >>>>>>>> > > >> >>>>>>>> I think I had knocked out docs for the "phase 1" stuff before > > we > > >> >>>>>>>> merged it in from the original feature branch. I'll double > > check > > >> >>>>>>>> and update > > >> >>>>>>>> the gdoc. Perhaps this was just a timing thing. > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> Thanks Josh, > > >> >>>>>>> S > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> > > > >> > > > > > > > > >
