Sure. I have assigned it to myself. Will look into it. Last time I checked, I did not find any failed tests and it was not hadoop3
--- Mallikarjun On Fri, May 14, 2021 at 2:34 AM Nick Dimiduk <ndimi...@apache.org> wrote: > Hi Mallikarjun, > > I just saw a bunch of backup tests fail on an unrelated PR build. I filed > HBASE-25888 and uploaded some logs. I have the full test-logs.zip file, but > it's too big to upload to jira. I linked it from the Jira, but the archive > will disappear when the PR is eventually closed. I would ping you from > Jira, but I didn't find any Jira user that seemed likely to be your > account. Would you mind taking a look? > > Thanks, > Nick > > On Wed, Dec 23, 2020 at 8:20 PM Mallikarjun <mallik.v.ar...@gmail.com> > wrote: > > > Yea. I have noticed that. > > > > I have some patches ready and have to add unit tests. Will start raising > in > > a couple of weeks time. > > --- > > Mallikarjun > > > > > > On Thu, Dec 24, 2020 at 7:48 AM 张铎(Duo Zhang) <palomino...@gmail.com> > > wrote: > > > > > The UTs in the backup module are easy to fail with NPE, I've seen this > in > > > several pre commit results. > > > > > > Any progress here? > > > > > > mallik.v.ar...@gmail.com <mallik.v.ar...@gmail.com> 于2020年12月3日周四 > > > 上午9:58写道： > > > > > > > On Tue, Dec 1, 2020 at 11:26 PM Stack <st...@duboce.net> wrote: > > > > > > > > > On Tue, Dec 1, 2020 at 6:38 AM mallik.v.ar...@gmail.com < > > > > > mallik.v.ar...@gmail.com> wrote: > > > > > > > > > > > On Tue, Dec 1, 2020 at 7:34 PM Sean Busbey <bus...@apache.org> > > > wrote: > > > > > > > > > > > > > One reason for moving it out of core is that at some point we > > will > > > > have > > > > > > > hbase 3 releases. Right now hbase 3 targeted things are in the > > > master > > > > > > > branch. If the feature is not sufficiently ready to be in a > > release > > > > > then > > > > > > > when the time for HBase 3 releases comes we'll have to do work > to > > > > pull > > > > > it > > > > > > > out of the release branch. If the feature needs to stay in the > > core > > > > > repo > > > > > > > that will necessitate that hbase 3 have a branch distinct from > > the > > > > > master > > > > > > > branch (which may or may not happen anyways). At that point we > > risk > > > > > > having > > > > > > > to do the work to remove the feature from a release branch > again > > > come > > > > > > hbase > > > > > > > 4. > > > > > > > > > > > > > > > > > > > > > I think a lot of this is moot if you'd like to start providing > > > > patches > > > > > > for > > > > > > > the things you've needed to use it. If there are gaps that you > > > think > > > > > can > > > > > > > trip folks up maybe we could label it "experimental" to provide > > > > better > > > > > > > context for others. > > > > > > > > > > > > > > > > > > > I will start putting effort in maintaining this feature. > > > > > > > > > > > > > > > > > FYI, seems like backup has a bunch of flakey tests. Might be worth > > > > looking > > > > > at. > > > > > > > > > > > > > Any reference I can get. They seem fine when I run tests. > > > > > > > > > > > > > Thanks, > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Dec 1, 2020, 07:48 mallik.v.ar...@gmail.com < > > > > > > > mallik.v.ar...@gmail.com> wrote: > > > > > > > > > > > > > > > On Tue, Dec 1, 2020 at 12:14 PM Stack <st...@duboce.net> > > wrote: > > > > > > > > > > > > > > > > > On Mon, Nov 30, 2020 at 9:30 PM mallik.v.ar...@gmail.com < > > > > > > > > > mallik.v.ar...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > Inline > > > > > > > > > > > > > > > > > > > > On Tue, Dec 1, 2020, 10:10 AM Andrew Purtell < > > > > > > > andrew.purt...@gmail.com > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > We are allowed to debate if this should be in the tree > or > > > > not. > > > > > > > Given > > > > > > > > > the > > > > > > > > > > > lack of interest, as evidenced by incomplete state, > lack > > of > > > > > > > release, > > > > > > > > > and > > > > > > > > > > > lack of contribution, it is more than fair to discuss > > > > removal. > > > > > > > > > > > > > > > > > > > > > > Here is my take: First of all, it is not released. > There > > is > > > > no > > > > > > > > implied > > > > > > > > > > > roadmap or support. Second, there do not seem to be any > > > > active > > > > > > > > > > maintainers > > > > > > > > > > > or volunteers as such. Third, unless someone shows up > > with > > > > more > > > > > > > > patches > > > > > > > > > > for > > > > > > > > > > > it there will be no polish or maturing, there can be no > > > > > > > expectations > > > > > > > > of > > > > > > > > > > > further improvement. > > > > > > > > > > > > > > > > > > > > > > That said, this is open source. New code contribution > > will > > > > > change > > > > > > > the > > > > > > > > > > > facts as they stand. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If there is an agreement upon the direction I am willing > to > > > > > provide > > > > > > > > > patches > > > > > > > > > > to take this feature forward. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What about the changes you made to make it work? > > > > > > > > > > > > > > > > > > > > > > > > > I'd be interested in patches that take this feature out of > > > core. > > > > > How > > > > > > > hard > > > > > > > > > do you think that would be? See the items Duo list above > for > > a > > > > > start > > > > > > on > > > > > > > > > obstacles to tackle? > > > > > > > > > > > > > > > > > > > > > > > > > I am yet to be very familiar with the code to comment on the > > > effort > > > > > > > > required to take it out of the core. Backport and bug fixes > > > > required > > > > > > very > > > > > > > > shallow understanding of the code and only some parts of it. > > > > > > > > > > > > > > > > I am not very clear on why some strong opinions on moving > > backup > > > > out > > > > > of > > > > > > > the > > > > > > > > core? Is there something defined on what should be operator > > > tools? > > > > > > > > > > > > > > > > Quoting what Duo Zhang said > > > > > > > > > > > > > > > > You can see the code under the > > > > org.apache.hadoop.hbase.backup.master > > > > > > > > > package, we need to set up log cleaner at master side, and > > > also, > > > > > > > > > the LogRollMasterProcedureManager class needs > > > MasterServices(not > > > > > > > > > MasterRpcServices), which means it must be used in the same > > > > process > > > > > > > with > > > > > > > > > HMaster. > > > > > > > > > > > > > > > > . > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 30, 2020, at 5:52 PM, > mallik.v.ar...@gmail.com > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 30, 2020 at 9:59 PM Stack < > > st...@duboce.net > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > >>> On Mon, Nov 30, 2020 at 5:55 AM > > > mallik.v.ar...@gmail.com > > > > < > > > > > > > > > > > >>> mallik.v.ar...@gmail.com> wrote: > > > > > > > > > > > >>> > > > > > > > > > > > >>> Inline. > > > > > > > > > > > >>> > > > > > > > > > > > >>> On Sun, Nov 29, 2020, 9:18 AM 张铎(Duo Zhang) < > > > > > > > > palomino...@gmail.com > > > > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > > >>> > > > > > > > > > > > >>>> I'm afraid it is not easy to be moved to > > > > > > hbase-operator-tools. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> You can see the code under the > > > > > > > > > org.apache.hadoop.hbase.backup.master > > > > > > > > > > > >>>> package, we need to set up log cleaner at master > > side, > > > > and > > > > > > > also, > > > > > > > > > > > >>>> the LogRollMasterProcedureManager class needs > > > > > > > MasterServices(not > > > > > > > > > > > >>>> MasterRpcServices), which means it must be used in > > the > > > > > same > > > > > > > > > process > > > > > > > > > > > >> with > > > > > > > > > > > >>>> HMaster. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> And I'm OK with purging this feature, especially > if > > > > there > > > > > is > > > > > > > no > > > > > > > > > > > >> developer > > > > > > > > > > > >>>> who wants to maintain it. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>> Is the sole reason to purge this feature is no > > > developer > > > > > > > > > volunteering > > > > > > > > > > > to > > > > > > > > > > > >>> maintain this? > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >> I'd suggest this feature does not belong in core at > > all. > > > > > > > > > > > >> > > > > > > > > > > > >> And some of us have spent time reviewing the feature > > in > > > > the > > > > > > past > > > > > > > > and > > > > > > > > > > > back > > > > > > > > > > > >> then determined it needed lots of finishing work, > > test, > > > > and > > > > > > > polish > > > > > > > > > -- > > > > > > > > > > > none > > > > > > > > > > > >> of which it seems to have gotten since commit. > > > > > > > > > > > >> > > > > > > > > > > > >> You are using it Mallik? Tell us more please. How is > > it > > > > > > working > > > > > > > > for > > > > > > > > > > you? > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > It has been a decent v1 release for us. As pointed > out > > by > > > > > > others, > > > > > > > > > > feature > > > > > > > > > > > > should be polished and matured without debating if > > backup > > > > > > should > > > > > > > be > > > > > > > > > > part > > > > > > > > > > > of > > > > > > > > > > > > core or not. Some of notable points are > > > > > > > > > > > > > > > > > > > > > > > > - Limitations like serial backup set runs per > > > deployment > > > > > > > bothered > > > > > > > > > us. > > > > > > > > > > > > - Minor logging enhancements for debugging, some > > NPE's, > > > > etc > > > > > > > > > > > > - Bandwidth limits, incremental backup did not work > > > > > outright. > > > > > > > > > > > > > > > > > > > > > > > > Since it is only a couple of months, we might not > have > > > > seen a > > > > > > lot > > > > > > > > of > > > > > > > > > > > corner > > > > > > > > > > > > cases. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> S > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >>> > > > > > > > > > > > >>>> For me, I suspect that the backup feature could be > > > done > > > > > more > > > > > > > > > > > separately > > > > > > > > > > > >>>> with the main cluster. We could use replication to > > > > backup > > > > > > the > > > > > > > > > WALs, > > > > > > > > > > > and > > > > > > > > > > > >>> use > > > > > > > > > > > >>>> Snapshot and ExportSnapshot to backup the HFiles. > > The > > > > > > feature > > > > > > > > > could > > > > > > > > > > be > > > > > > > > > > > >>> done > > > > > > > > > > > >>>> as a separated project. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Thanks. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Stack <st...@duboce.net> 于2020年11月20日周五 > 上午10:18写道： > > > > > > > > > > > >>>> > > > > > > > > > > > >>>>> It strikes me as work that has been abandoned > with > > no > > > > > > > > supporting > > > > > > > > > > > >>>> developer. > > > > > > > > > > > >>>>> It has had no improvement and few commits other > > than > > > > > > > adjustment > > > > > > > > > > > >>> because a > > > > > > > > > > > >>>>> backing dependency has changed since original > > > > > contribution. > > > > > > > It > > > > > > > > > has > > > > > > > > > > > >> not > > > > > > > > > > > >>>> been > > > > > > > > > > > >>>>> included in a release so has no users as yet. > Does > > > > anyone > > > > > > use > > > > > > > > it > > > > > > > > > or > > > > > > > > > > > >>> want > > > > > > > > > > > >>>>> it? > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> We have back ported this feature to 2.1.x and has > > been > > > > > using > > > > > > > for > > > > > > > > > last > > > > > > > > > > > few > > > > > > > > > > > >>> months in few of our deployments. > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>>> If not, I suggest we remove it. I could file an > > issue > > > > for > > > > > > it > > > > > > > to > > > > > > > > > be > > > > > > > > > > > >>> > > > > > > > > > > > >>>> added to hbase-operator-tools for some gallant dev > > to > > > > pick > > > > > > up > > > > > > > if > > > > > > > > > > they > > > > > > > > > > > >>>>> wanted to use this backup work? (I could help w/ > > the > > > > > > > > migration). > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> What do others think? > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> S > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
