[DISCUSS] Normalizer and pre-split tables

2020-06-26 Thread Nick Dimiduk
Heya,

I've seen a lot of use-cases where the normalizer would be a nice solution
for operators and application developers. I've been trying to beef it up a
bit to handle these cases. However, some of these considerations are at
odds, so I want to vet the ideas here.

The normalizer is a background chore in the HMaster that attempts to
converge region sizes within a table toward the average region size. It has
a pretty wide error bar, but that's the overall goal.

Early on, it was observed that an operator needs to pre-split a table, so
special considerations were included, by way of
`hbase.normalizer.min.region.count`,
`hbase.normalizer.merge.min_region_age.days`, and
`hbase.normalizer.merge.min_region_size.mb`. All these nobs are designed to
give an operator means of controlling this behavior.

We have (what I see as) a competing objective: doing away with empty, or
nearly-empty regions. The use-case is pretty common when there's a TTL
applied to a table, especially if there's also a timestamp component in the
rowkey. In this case, we want the normalizer to "merge away" these empty
regions.

The trouble is we ship defaults for all of the `*min*` configs, and right
now there's no way to "unset" them, disable the functionality. Which means
there still isn't a way to support the empty regions use-case without
awkward special-case checks. This is where I'm looking for suggestions from
the community. There's some discussion under way over on the PR for
HBASE-24583. Please take a look.

Thanks in advance,
Nick


Re: [DISCUSS] Removing problematic terms from our project

2020-06-26 Thread Andrew Purtell
Circling back after more inputs, if we use this as a description of the
proposals:

1. Replace "master"/"hmaster" with ???, this one has by far the most
significant impact and both opinion and interpretation on this one is mixed.

2. Replace "slave" with "follower", seems to impact the cross cluster
replication subsystem only.

3. Replace "black list" with "deny list".

4. Replace "white list" with "accept list".

Then by my read of the responses we have consensus to do #2, #3, and #4.
They were not controversial. JIRAs and patches will be welcome. Seems
pretty clear committers and PMC will approve and do what is needed to
complete any necessary deprecation cycle.

Regarding #1, opinion is mixed. By my read I also think committers and PMC
will approve patches and do what is needed to complete any necessary
deprecation cycle for this one too. Enough PMC members expressed support to
successfully vote on a release (although not if there were to be opposing
votes). If a contributor were to open a JIRA and provide patches for this,
there would be more discussion. There is no consensus, yet, on what
replacement term is best. Personally, I can accept Zheng's recent
suggestion of "controller". I can see how syllable count matters.

I don't mean this summary to close the conversation. It is only a
checkpoint.

If anyone reading this has an opinion they do not wish to express
publically, you are welcome to write to priv...@hbase.apache.org to state
your opinion and the PMC will of course respectfully listen to it.



On Thu, Jun 25, 2020 at 7:47 PM zheng wang <18031...@qq.com> wrote:

> I like the controller.
>
>
> Coordinator is a bit long for me to write and speak.
> Manager and Admin is used somewhere yet in HBase.
>
>
>
>
> -- 原始邮件 --
> 发件人: "Andrew Purtell" 发送时间: 2020年6月26日(星期五) 上午9:08
> 收件人: "Hbase-User" 抄送: "dev" 主题: Re: [DISCUSS] Removing problematic terms from our project
>
>
>
> > - AdminServer (as you already have AdminClient to talk to it).
>
> Oh... I like AdminServer. AdminServer (serving admin functions) and
> RegionServer (serving region data).
>
> On Thu, Jun 25, 2020 at 4:46 PM Andrey Elenskiy
> 
> > > Is there a word that's not "master" and not "coordinator" that
> is clear
> > and
> > suitable for (diverse, polyglot) community?
> >
> > There are also:
> > - captain (sounds pretty close to "master" without the negative side
> and it
> > should be relatable around the world)
> > - conductor (as in orchestra)
> > - controller (in kafka controller assigns partitions)
> > - RegionDriver (more relevant to what it's actually doing in hbase and
> > borrowed from PlacementDrive of TiKV)
> > - AdminServer (as you already have AdminClient to talk to it).
> >
> > On Thu, Jun 25, 2020 at 3:49 PM Sean Busbey  wrote:
> >
> > > How about "manager"?
> > >
> > > (It would help me if folks could explain what is lacking in
> > "coordinator".)
> > >
> > > On Thu, Jun 25, 2020, 13:32 Nick Dimiduk  wrote:
> > >
> > > > On Wed, Jun 24, 2020 at 10:14 PM 张铎(Duo Zhang) <
> palomino...@gmail.com>
> > > > wrote:
> > > >
> > > > > -0/+1/+1/+1
> > > > >
> > > > > I’m the one who asked whether ‘master’ is safe to use
> without ‘slave’
> > > in
> > > > > the private list.
> > > > >
> > > > > I’m still not convinced that it is really necessary
> and I do not
> > think
> > > > > other words like ‘coordinator’ can fully describe the
> role of HMaster
> > > in
> > > > > HBase. HBase is more than 10 years old. In the context
> of HBase, the
> > > word
> > > > > ‘HMaster’ has its own meaning. Changing the name will
> hurt our users
> > > and
> > > > > make them confusing, especially for us non native
> English speakers...
> > > > >
> > > >
> > > > Is there a word that's not "master" and not "coordinator"
> that is clear
> > > and
> > > > suitable for (diverse, polyglot) community?
> > > >
> > > > Stack  > > > >
> > > > > > +1/+1/+1/+1 where hbase3 adds the deprecation and
> hbase4 follows
> > > hbase3
> > > > > > soon after sounds good to me. I'm up for working
> on this.
> > > > > > S
> > > > > >
> > > > > > On Wed, Jun 24, 2020 at 2:26 PM Xu Cang <
> xuc...@apache.org> wrote:
> > > > > >
> > > > > > > Strongly agree with what Nick said here:
> > > > > > >
> > > > > > >  " From my perspective, we gain nothing
> as a project or as a
> > > > community
> > > > > be
> > > > > > > willfully retaining use of language that is
> well understood to be
> > > > > > > problematic or hurtful, On the contrary,
> we have much to gain
> > > by
> > > > > > > encouraging
> > > > > > > contributions from as many people as
> possible."
> > > > > > >
> > > > > > > +1 to Andrew's proposal.
> > > > > > >
> > > > > > > It might be good to have a source of truth
> web page or README
> > file
> > > > for
> > > > > > > developers and users to refer to regarding
> all naming
> > transitions.
> > > > It's
> > > > > > > going to help both developers changing the
> code and users looking
> > > for
> > > > > > some
> > > > > 

JMX Issues

2020-06-26 Thread Govindhan S
Hello Team,
I am getting the below error flooding on my hmaster logs.. Any suggestions or 
way forwards might help me out. 
Version -> 1.1.2
ERROR [1955596624@qtp-58762-113898] jmx.JMXJsonServlet: Caught an exception 
while processing JMX requestjavax.management.MalformedObjectNameException: 
Invalid character ':' in key part of property 
 WARN  [1955596624@qtp-58762-113898] mortbay.log: Committed before 400 null

ERROR [1955596624@qtp-58762-113898] mortbay.log: 
/jmxjava.lang.IllegalStateException: Committed


~ Govins.