Could we use Procedure V2 framework to do split? It seems to be a great work...
2016-01-07 4:35 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > bq. is splitting and subsequent merging a necessarily bad thing? > > Please note that merging may happen before the size of R1 (R2) drops to > zero. Meaning, there may be churn in normalization activity. > I would say that the normalization in the scenario below is premature. > > Cheers > > On Wed, Jan 6, 2016 at 11:41 AM, Mikhail Antonov <olorinb...@gmail.com> > wrote: > > > Yeah, I see. Btw, in scenario you described, is splitting and subsequent > > merging a necessarily bad thing? If data expiration happens not that > often, > > having more evenly sized regions in between major collections isn't worth > > it? > > > > On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > bq. collect statistics to blacklist some RSs with high failure rate > > > > > > The metadata would help pinpoint the regions which consistently fail > > split > > > (merge) in the recent past. The failure could be due to corrupt > HFile(s) > > or > > > other reason. > > > Having the statistics would also help normalizer avoid the following > > > scenario: > > > > > > region R gets split into R1 and R2 > > > size of R1 and R2 decreases due to expiration of data > > > R1 and R2 get merged into R' > > > more data comes into R', resulting in split, again > > > > > > Cheers > > > > > > On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinb...@gmail.com > > > > > wrote: > > > > > > > Adding this tracing information to know who initiated split is in > > general > > > > useful thing. Right now though I'm not sure I see how that would help > > to > > > > make better normalization decisions? Region split failure implies > > > > underlying FS issue? Any examples/ideas? > > > > > > > > Kind of..collect statistics to blacklist some RSs with high failure > > rate > > > > and don't attempt to split regions hosted there in future? > > > > > > > > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > > > Hi, > > > > > I recently worked on improving region normalization feature. > > > > > > > > > > If region split request triggered by the execution of > > > > > SplitNormalizationPlan fails, there is no way of knowing whether > the > > > > failed > > > > > split originated from region normalization. > > > > > Such association would give RegionNormalizer information so that it > > can > > > > > make better normalization decisions in the subsequent invocations. > > > > > > > > > > One enhancement I can think of is to embed metadata in SplitRequest > > > which > > > > > gets passed through RegionStateTransitionContext when > > > > > RegionServerServices#reportRegionStateTransition() is called. > > > > > This way, RegionStateListener can be notified with the metadata (id > > of > > > > the > > > > > requester). > > > > > > > > > > Comment is welcome. > > > > > > > > > > > > > > > > > > > > > -- > > > > Thanks, > > > > Michael Antonov > > > > > > > > > > > > > > > -- > > Thanks, > > Michael Antonov > > >
