Yeah, I see. Btw, in scenario you described, is splitting and subsequent merging a necessarily bad thing? If data expiration happens not that often, having more evenly sized regions in between major collections isn't worth it?
On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhih...@gmail.com> wrote: > bq. collect statistics to blacklist some RSs with high failure rate > > The metadata would help pinpoint the regions which consistently fail split > (merge) in the recent past. The failure could be due to corrupt HFile(s) or > other reason. > Having the statistics would also help normalizer avoid the following > scenario: > > region R gets split into R1 and R2 > size of R1 and R2 decreases due to expiration of data > R1 and R2 get merged into R' > more data comes into R', resulting in split, again > > Cheers > > On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinb...@gmail.com> > wrote: > > > Adding this tracing information to know who initiated split is in general > > useful thing. Right now though I'm not sure I see how that would help to > > make better normalization decisions? Region split failure implies > > underlying FS issue? Any examples/ideas? > > > > Kind of..collect statistics to blacklist some RSs with high failure rate > > and don't attempt to split regions hosted there in future? > > > > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > Hi, > > > I recently worked on improving region normalization feature. > > > > > > If region split request triggered by the execution of > > > SplitNormalizationPlan fails, there is no way of knowing whether the > > failed > > > split originated from region normalization. > > > Such association would give RegionNormalizer information so that it can > > > make better normalization decisions in the subsequent invocations. > > > > > > One enhancement I can think of is to embed metadata in SplitRequest > which > > > gets passed through RegionStateTransitionContext when > > > RegionServerServices#reportRegionStateTransition() is called. > > > This way, RegionStateListener can be notified with the metadata (id of > > the > > > requester). > > > > > > Comment is welcome. > > > > > > > > > > > -- > > Thanks, > > Michael Antonov > > > -- Thanks, Michael Antonov
