[jira] [Created] (HBASE-24659) Calcuate FIXED_OVERHEAD automatically

2020-06-29 Thread Duo Zhang (Jira)
Duo Zhang created HBASE-24659:
-

 Summary: Calcuate FIXED_OVERHEAD automatically
 Key: HBASE-24659
 URL: https://issues.apache.org/jira/browse/HBASE-24659
 Project: HBase
  Issue Type: Improvement
Reporter: Duo Zhang


Now the FIXED_OVERHEAD in some classes are maintained manually, an we have a 
method to TestHeapSizes to confirm that the value is correct.

But it is really hard for developers to count the fields in a complicated class 
like HRegion. Since we have the ability to calcuate the accurate size in UT, I 
think we it is also possible to calcuate it when loading the class.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HBASE-24658) Update PolicyBasedChaosMonkey to use a thread pool

2020-06-29 Thread Nick Dimiduk (Jira)
Nick Dimiduk created HBASE-24658:


 Summary: Update PolicyBasedChaosMonkey to use a thread pool
 Key: HBASE-24658
 URL: https://issues.apache.org/jira/browse/HBASE-24658
 Project: HBase
  Issue Type: Task
  Components: integration tests
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk


Running {{ServerKillingChaosMonkey}} via {{RESTApiClusterManager}} for any 
duration of time slowly leaks region servers. I see failures on the RESTApi 
side go unreported on the ChaosMonkey side. It seems like {{RuntimeExceptions}} 
are being thrown and lost.

{{PolicyBasedChaosMonkey}} uses a primitive means of thread management anyway. 
Update to use a thread pool, thread groups, and an uncaughtExceptionHandler.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (HBASE-24648) Remove the legacy 'forceSplit' related code at region server side

2020-06-29 Thread Michael Stack (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-24648.
---
Resolution: Fixed

Re-resolving after pushing small addendum.

> Remove the legacy 'forceSplit' related code at region server side
> -
>
> Key: HBASE-24648
> URL: https://issues.apache.org/jira/browse/HBASE-24648
> Project: HBase
>  Issue Type: Task
>  Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> I believe it is useless after we move split operation to master. Let's purge 
> the uesless code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Reopened] (HBASE-24648) Remove the legacy 'forceSplit' related code at region server side

2020-06-29 Thread Michael Stack (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack reopened HBASE-24648:
---

Reopen to apply addendum to fix heap size calcuation in HRegion.

> Remove the legacy 'forceSplit' related code at region server side
> -
>
> Key: HBASE-24648
> URL: https://issues.apache.org/jira/browse/HBASE-24648
> Project: HBase
>  Issue Type: Task
>  Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> I believe it is useless after we move split operation to master. Let's purge 
> the uesless code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HBASE-24657) JsonBean representation of metrics at /jmx endpoint now quotes all numbers

2020-06-29 Thread David Manning (Jira)
David Manning created HBASE-24657:
-

 Summary: JsonBean representation of metrics at /jmx endpoint now 
quotes all numbers
 Key: HBASE-24657
 URL: https://issues.apache.org/jira/browse/HBASE-24657
 Project: HBase
  Issue Type: Bug
  Components: metrics
Affects Versions: 1.4.11, 1.3.6, 1.6.0, 1.5.0
Reporter: David Manning
Assignee: David Manning


HBASE-20571 had a fix to look for NaN or Infinity in numbers, and to quote 
those as strings. The order of the `if-else` block is different in branch-1 
(https://github.com/apache/hbase/commit/2d493556f3c8ae87fb92422b525bf7c9345e6ccd)
 and branch-2 
(https://github.com/apache/hbase/commit/39ea1efa885e2f27f41af59228e0a12c4ded08f8)

HBASE-23015 changed the JsonBean.java code in a meaningful way, and the order 
of the changes were consistent between branch-1 
([https://github.com/apache/hbase/commit/f77c14d18150f55ee892f8d24a5ee231c1ae7e20#diff-87e9e2722b9210eebfd8c820c5d72a46L319-L324])
 and branch-2 
([https://github.com/apache/hbase/commit/761aef6d9d0b8a455842de4d5eac7d9486f00633#diff-2c8f5dd222141c69112c5c5b5f70cf55R319-R324])
 Unfortunately, they need to be reversed since the order is different between 
branch-1 and branch-2.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[VOTE] Second release candidate for HBase 2.3.0 (RC1) is available

2020-06-29 Thread Nick Dimiduk
Please vote on this Apache hbase release candidate,
hbase-2.3.0RC1

The VOTE will remain open for at least 72 hours.

[ ] +1 Release this package as Apache hbase 2.3.0
[ ] -1 Do not release this package because ...

The tag to be voted on is 2.3.0RC1:

  https://github.com/apache/hbase/tree/2.3.0RC1

The release files, including signatures, digests, as well as CHANGES.md
and RELEASENOTES.md included in this RC can be found at:

  https://dist.apache.org/repos/dist/dev/hbase/2.3.0RC1/

Maven artifacts are available in a staging repository at:

  https://repository.apache.org/content/repositories/orgapachehbase-1394/

Artifacts were signed with the ndimi...@apache.org key which can be found
in:

  https://dist.apache.org/repos/dist/release/hbase/KEYS

To learn more about Apache hbase, please see

  http://hbase.apache.org/

Thanks,
Your HBase Release Manager


[jira] [Created] (HBASE-24656) [Flakey Tests] branch-2 TestMasterNoCluster.testStopDuringStart

2020-06-29 Thread Michael Stack (Jira)
Michael Stack created HBASE-24656:
-

 Summary: [Flakey Tests] branch-2 
TestMasterNoCluster.testStopDuringStart
 Key: HBASE-24656
 URL: https://issues.apache.org/jira/browse/HBASE-24656
 Project: HBase
  Issue Type: Bug
Reporter: Michael Stack


org.apache.hadoop.hbase.master.TestMasterNoCluster.testStopDuringStart is 
(only) flakey on branch-2 currently. Fails here:

Error Message

KeeperErrorCode = Directory not empty for /hbase/backup-masters

Stacktrace

org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = 
Directory not empty for /hbase/backup-masters
at 
org.apache.hadoop.hbase.master.TestMasterNoCluster.tearDown(TestMasterNoCluster.java:121)



I can see the zk events in teardown as we purge children as part of cleanup. 
Can also see that the backup master registers later. Other than that, log is 
opaque on why the teardown is failing. This is just clean up so adding in retry 
to see if that helps.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HBASE-24655) Address asciidoc warnings about images

2020-06-29 Thread Nick Dimiduk (Jira)
Nick Dimiduk created HBASE-24655:


 Summary: Address asciidoc warnings about images
 Key: HBASE-24655
 URL: https://issues.apache.org/jira/browse/HBASE-24655
 Project: HBase
  Issue Type: Task
  Components: build, community
Reporter: Nick Dimiduk


Noticed this while tailing the log for 2.3.0rc1. I assume this is related to 
generating the PDF version of the book. Should verify these images do indeed 
make it into the pdf.

{noformat}
17:15:36 [INFO] asciidoctor: WARN: image to embed not found or not readable: 
/home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/WAL_splitting.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: 
/home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/master-snapshot.png
 
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: 
/home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/1-snapshot.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: 
/home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/2-snapshots.png
17:16:37 [INFO] asciidoctor: WARN: image to embed not found or not readable: 
/home/vagrant/hbase-rm/output/hbase/src/main/asciidoc/images/empty-snapshots.png
17:17:03 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/top_screen.gif to PNG.
  
17:17:03 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/scrolling_metric_records.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/changing_mode.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/changing_refresh_delay.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/changing_displayed_fields.gif to PNG. 
  
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/changing_sort_field.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/changing_order_of_fields.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/adding_filters.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/showing_and_clearing_filters.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/driling_down.gif to PNG.
17:17:04 [INFO] asciidoctor: WARN: GIF image format not supported. Install the 
prawn-gmagick gem or convert 
https://hbase.apache.org/hbtop-images/help_screen.gif to PNG.
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (HBASE-24552) Replica region needs to check if primary region directory exists at file system in TransitRegionStateProcedure

2020-06-29 Thread Huaxiang Sun (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Huaxiang Sun resolved HBASE-24552.
--
Fix Version/s: 2.3.0
   3.0.0-alpha-1
   Resolution: Fixed

> Replica region needs to check if primary region directory exists at file 
> system  in TransitRegionStateProcedure 
> 
>
> Key: HBASE-24552
> URL: https://issues.apache.org/jira/browse/HBASE-24552
> Project: HBase
>  Issue Type: Bug
>  Components: read replicas
>Affects Versions: 2.3.0
>Reporter: Huaxiang Sun
>Assignee: Huaxiang Sun
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
>
> In hbase-1, it always runs into the situation that primary region has been 
> closed/removed and replica region still stays in master's in-memory db and 
> open at one of the region servers. Balancer can move this replica region to a 
> new region server. During the region open, replica region does not check if 
> primary region has been removed and moves forward. During store open, it will 
> recreates primary region directory at hdfs and caused inconsistency.
>  
> In hbase-2, things get much better. To prevent the above inconsistency from 
> happening, it adds more checks for a replica region, i.e, if primary regions' 
> directory exists and there is a .regioninfo under. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (HBASE-24144) Update docs from master

2020-06-29 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-24144.
--
Resolution: Fixed

> Update docs from master
> ---
>
> Key: HBASE-24144
> URL: https://issues.apache.org/jira/browse/HBASE-24144
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.3.0
>
>
> Take a pass updating the docs. Have a look at what's on branch-2.2 and add 
> whatever updates we need from master. Consider refreshing branch-2 as well, 
> since it's been a while.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (HBASE-24648) Remove the legacy 'forceSplit' related code at region server side

2020-06-29 Thread Duo Zhang (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-24648.
---
Resolution: Fixed

Pushed to branch-2+.

Thanks [~vjasani] for reviewing.

> Remove the legacy 'forceSplit' related code at region server side
> -
>
> Key: HBASE-24648
> URL: https://issues.apache.org/jira/browse/HBASE-24648
> Project: HBase
>  Issue Type: Task
>  Components: regionserver
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> I believe it is useless after we move split operation to master. Let's purge 
> the uesless code.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (HBASE-24117) Shutdown AssignmentManager before ProcedureExecutor may cause SCP to accidentally skip assigning a region

2020-06-29 Thread Duo Zhang (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-24117.
---
Hadoop Flags: Reviewed
  Resolution: Fixed

Thanks [~ndimiduk]. Resolved.

> Shutdown AssignmentManager before ProcedureExecutor may cause SCP to 
> accidentally skip assigning a region
> -
>
> Key: HBASE-24117
> URL: https://issues.apache.org/jira/browse/HBASE-24117
> Project: HBase
>  Issue Type: Bug
>  Components: proc-v2
>Reporter: Michael Stack
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 2.3.0, 2.2.6
>
> Attachments: 
> org.apache.hadoop.hbase.master.assignment.TestCloseRegionWhileRSCrash-output.txt
>
>
> I saw this on TestCloseRegionWithRSCrash. The Region 
> 788a516d1f86af98e0a16bcc1afe4fa1 was being moved to RS  
> example.com,62652,1586032098445 just after it was killed. The Move Close 
> fails because the RS has no node in the Master. The Move then tries to 
> 'confirm' the close but it fails because no remote RS. We are then to wait in 
> this state until operator or some other procedure intervenes to 'fix' the 
> state. Normally a ServerCrashProcedure would do the job but in this test the 
> Master is restarted after the RS is killed, a condition we do not accommodate.
> Let me attach the test log.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSS] Normalizer and pre-split tables

2020-06-29 Thread Nick Dimiduk
On Mon, Jun 29, 2020 at 7:13 AM Whitney Jackson  wrote:

> > The trouble is we ship defaults for all of the `*min*` configs, and right
> now there's no way to "unset" them, disable the functionality.
>
> Why is that the case? Can I not just set
> hbase.normalizer.merge.min_region_size.mb to 0? Do I risk blowing away
> regions from pre-splits or something?
>

Yes, the idea was to guard against merging away intentional pre-splits.

On Mon, Jun 29, 2020 at 2:42 AM Wellington Chevreuil <
> wellington.chevre...@gmail.com> wrote:
>
> > >
> > > The trouble is we ship defaults for all of the `*min*` configs, and
> right
> > > now there's no way to "unset" them, disable the functionality. Which
> > means
> > > there still isn't a way to support the empty regions use-case without
> > > awkward special-case checks.
> > >
> >
> > HBASE-23562  added a RegionsMerger tool to hbase-operators-tools project,
> > as a mean to allow multiple merges without checking minimum size. Of
> course
> > it's not as convenient as normalizer, but at least gives an alternative
> for
> > such edge cases where users ended with lots of empty regions.
> >
> > Em sex., 26 de jun. de 2020 às 22:30, Nick Dimiduk 
> > escreveu:
> >
> > > Heya,
> > >
> > > I've seen a lot of use-cases where the normalizer would be a nice
> > solution
> > > for operators and application developers. I've been trying to beef it
> up
> > a
> > > bit to handle these cases. However, some of these considerations are at
> > > odds, so I want to vet the ideas here.
> > >
> > > The normalizer is a background chore in the HMaster that attempts to
> > > converge region sizes within a table toward the average region size. It
> > has
> > > a pretty wide error bar, but that's the overall goal.
> > >
> > > Early on, it was observed that an operator needs to pre-split a table,
> so
> > > special considerations were included, by way of
> > > `hbase.normalizer.min.region.count`,
> > > `hbase.normalizer.merge.min_region_age.days`, and
> > > `hbase.normalizer.merge.min_region_size.mb`. All these nobs are
> designed
> > to
> > > give an operator means of controlling this behavior.
> > >
> > > We have (what I see as) a competing objective: doing away with empty,
> or
> > > nearly-empty regions. The use-case is pretty common when there's a TTL
> > > applied to a table, especially if there's also a timestamp component in
> > the
> > > rowkey. In this case, we want the normalizer to "merge away" these
> empty
> > > regions.
> > >
> > > The trouble is we ship defaults for all of the `*min*` configs, and
> right
> > > now there's no way to "unset" them, disable the functionality. Which
> > means
> > > there still isn't a way to support the empty regions use-case without
> > > awkward special-case checks. This is where I'm looking for suggestions
> > from
> > > the community. There's some discussion under way over on the PR for
> > > HBASE-24583. Please take a look.
> > >
> > > Thanks in advance,
> > > Nick
> > >
> >
>


[jira] [Resolved] (HBASE-24644) Add a clause to the book noting that sometimes we short-circuit the deprecation cycle

2020-06-29 Thread Nick Dimiduk (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-24644.
--
Fix Version/s: 3.0.0-alpha-1
   Resolution: Fixed

> Add a clause to the book noting that sometimes we short-circuit the 
> deprecation cycle
> -
>
> Key: HBASE-24644
> URL: https://issues.apache.org/jira/browse/HBASE-24644
> Project: HBase
>  Issue Type: Task
>  Components: community, documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
>
> Let's add a note to the book that describes the circumstances around 
> HBASE-21782 and how that can result in code not following our stated 
> deprecation cycle guidelines.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSS] Normalizer and pre-split tables

2020-06-29 Thread Whitney Jackson
I was just facing the many-zero-sized-regions issue last week and pondering
how best to approach it. So ++ for this work!

> The trouble is we ship defaults for all of the `*min*` configs, and right
now there's no way to "unset" them, disable the functionality.

Why is that the case? Can I not just set
hbase.normalizer.merge.min_region_size.mb to 0? Do I risk blowing away
regions from pre-splits or something?

> HBASE-23562  added a RegionsMerger tool to hbase-operators-tools

Nice. I didn't know about this. I see the tool wants you to specify a
desired number of regions. In my particular case I don't have a view on the
number of regions I want. I just know that all the post compaction 0 sized
regions should go. Can I still make use of this tool?

Whitney


On Mon, Jun 29, 2020 at 2:42 AM Wellington Chevreuil <
wellington.chevre...@gmail.com> wrote:

> >
> > The trouble is we ship defaults for all of the `*min*` configs, and right
> > now there's no way to "unset" them, disable the functionality. Which
> means
> > there still isn't a way to support the empty regions use-case without
> > awkward special-case checks.
> >
>
> HBASE-23562  added a RegionsMerger tool to hbase-operators-tools project,
> as a mean to allow multiple merges without checking minimum size. Of course
> it's not as convenient as normalizer, but at least gives an alternative for
> such edge cases where users ended with lots of empty regions.
>
> Em sex., 26 de jun. de 2020 às 22:30, Nick Dimiduk 
> escreveu:
>
> > Heya,
> >
> > I've seen a lot of use-cases where the normalizer would be a nice
> solution
> > for operators and application developers. I've been trying to beef it up
> a
> > bit to handle these cases. However, some of these considerations are at
> > odds, so I want to vet the ideas here.
> >
> > The normalizer is a background chore in the HMaster that attempts to
> > converge region sizes within a table toward the average region size. It
> has
> > a pretty wide error bar, but that's the overall goal.
> >
> > Early on, it was observed that an operator needs to pre-split a table, so
> > special considerations were included, by way of
> > `hbase.normalizer.min.region.count`,
> > `hbase.normalizer.merge.min_region_age.days`, and
> > `hbase.normalizer.merge.min_region_size.mb`. All these nobs are designed
> to
> > give an operator means of controlling this behavior.
> >
> > We have (what I see as) a competing objective: doing away with empty, or
> > nearly-empty regions. The use-case is pretty common when there's a TTL
> > applied to a table, especially if there's also a timestamp component in
> the
> > rowkey. In this case, we want the normalizer to "merge away" these empty
> > regions.
> >
> > The trouble is we ship defaults for all of the `*min*` configs, and right
> > now there's no way to "unset" them, disable the functionality. Which
> means
> > there still isn't a way to support the empty regions use-case without
> > awkward special-case checks. This is where I'm looking for suggestions
> from
> > the community. There's some discussion under way over on the PR for
> > HBASE-24583. Please take a look.
> >
> > Thanks in advance,
> > Nick
> >
>


Announcing ApacheCon @Home 2020

2020-06-29 Thread Rich Bowen

Hi, Apache enthusiast!

(You’re receiving this because you’re subscribed to one or more dev or 
user mailing lists for an Apache Software Foundation project.)


The ApacheCon Planners and the Apache Software Foundation are pleased to 
announce that ApacheCon @Home will be held online, September 29th 
through October 1st, 2020. We’ll be featuring content from dozens of our 
projects, as well as content about community, how Apache works, business 
models around Apache software, the legal aspects of open source, and 
many other topics.


Full details about the event, and registration, is available at 
https://apachecon.com/acah2020


Due to the confusion around how and where this event was going to be 
held, and in order to open up to presenters from around the world who 
may previously have been unable or unwilling to travel, we’ve reopened 
the Call For Presentations until July 13th. Submit your talks today at 
https://acna2020.jamhosted.net/


We hope to see you at the event!
Rich Bowen, VP Conferences, The Apache Software Foundation


Re: [DISCUSS] Normalizer and pre-split tables

2020-06-29 Thread Wellington Chevreuil
>
> The trouble is we ship defaults for all of the `*min*` configs, and right
> now there's no way to "unset" them, disable the functionality. Which means
> there still isn't a way to support the empty regions use-case without
> awkward special-case checks.
>

HBASE-23562  added a RegionsMerger tool to hbase-operators-tools project,
as a mean to allow multiple merges without checking minimum size. Of course
it's not as convenient as normalizer, but at least gives an alternative for
such edge cases where users ended with lots of empty regions.

Em sex., 26 de jun. de 2020 às 22:30, Nick Dimiduk 
escreveu:

> Heya,
>
> I've seen a lot of use-cases where the normalizer would be a nice solution
> for operators and application developers. I've been trying to beef it up a
> bit to handle these cases. However, some of these considerations are at
> odds, so I want to vet the ideas here.
>
> The normalizer is a background chore in the HMaster that attempts to
> converge region sizes within a table toward the average region size. It has
> a pretty wide error bar, but that's the overall goal.
>
> Early on, it was observed that an operator needs to pre-split a table, so
> special considerations were included, by way of
> `hbase.normalizer.min.region.count`,
> `hbase.normalizer.merge.min_region_age.days`, and
> `hbase.normalizer.merge.min_region_size.mb`. All these nobs are designed to
> give an operator means of controlling this behavior.
>
> We have (what I see as) a competing objective: doing away with empty, or
> nearly-empty regions. The use-case is pretty common when there's a TTL
> applied to a table, especially if there's also a timestamp component in the
> rowkey. In this case, we want the normalizer to "merge away" these empty
> regions.
>
> The trouble is we ship defaults for all of the `*min*` configs, and right
> now there's no way to "unset" them, disable the functionality. Which means
> there still isn't a way to support the empty regions use-case without
> awkward special-case checks. This is where I'm looking for suggestions from
> the community. There's some discussion under way over on the PR for
> HBASE-24583. Please take a look.
>
> Thanks in advance,
> Nick
>


[jira] [Created] (HBASE-24654) Allow unset table's rsgroup

2020-06-29 Thread Sun Xin (Jira)
Sun Xin created HBASE-24654:
---

 Summary: Allow unset table's rsgroup
 Key: HBASE-24654
 URL: https://issues.apache.org/jira/browse/HBASE-24654
 Project: HBase
  Issue Type: New Feature
  Components: rsgroup
Affects Versions: 3.0.0-alpha-1
Reporter: Sun Xin
Assignee: Sun Xin
 Fix For: 3.0.0-alpha-1


In TableDescriptorBuilder, we have only one method to set rsgroup, but have no 
one to unset it. this unset method is necessary In some cases.

If the table had rsgroup config before, but now I want to use the namespace 
config. It doesn't work that I set table rsgroup config to default rsgroup, 
must remove rsgroup config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)