So, I tried decommission that RS and recommission it. No change. Server still idle.
Tried decommission another server and see if HBASE sets itself right. Now I have two RS that are idle. ba-hbase18.jumbo.hq.com,16020,1604413480001 Tue Nov 03 15:24:40 CET 2020 1 s 2.1.0-cdh6.3.2 13 471 ba-hbase19.jumbo.hq.com,16020,1610095488001 Fri Jan 08 09:44:48 CET 2021 0 s 2.1.0-cdh6.3.2 0 6 ba-hbase20.jumbo.hq.com,16020,1610096850259 Fri Jan 08 10:07:30 CET 2021 0 s 2.1.0-cdh6.3.2 0 0 ba-hbase21.jumbo.hq.com,16020,1604414101652 Tue Nov 03 15:35:01 CET 2020 1 s 2.1.0-cdh6.3.2 15 447 From the logs: 2021-01-08 10:25:36,875 ERROR org.apache.hadoop.hbase.ScheduledChore: Caught error java.lang.IllegalStateException: Expected only one of namespace and tablename to be null This is reappearing in hbase master log M -----Original Message----- From: Sean Busbey <[email protected]> Sent: Thursday, January 7, 2021 7:30 PM To: Hbase-User <[email protected]> Subject: Re: Region server idle EXTERNAL Sounds like https://issues.apache.org/jira/browse/HBASE-24139 The description of that jira has a workaround. On Thu, Jan 7, 2021, 05:23 Marc Hoppins <[email protected]> wrote: > Hi all, > > I have a setup with 67 region servers. 29 Dec one system had to be > shut down to have EMM module swapped out - which took one work day. > Host was back online 30 Dec. > > My HBASE is very basic so I appreciate your patience. > > My understanding of the defaults that are setup is that a major > compaction should occur every 7 days. Moreover, do I assume that more > extensive balancing may occur after this happens? > > When I check (via hbase master UI) the status of HBASE, I see the > following: > > ServerName > > Start time > > Last contact > > Version > > Requests Per Second > > Num. Regions > > ba-hbase16.jumbo.hq.com,16020,1604413068640< > http://ba-hbase16.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:17:48 CET 2020 > > 3 s > > 2.1.0-cdh6.3.2 > > 46 > > 462 > > ba-hbase17.jumbo.hq.com,16020,1604413274393< > http://ba-hbase17.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:21:14 CET 2020 > > 1 s > > 2.1.0-cdh6.3.2 > > 19 > > 462 > > ba-hbase18.jumbo.hq.com,16020,1604413480001< > http://ba-hbase18.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:24:40 CET 2020 > > 2 s > > 2.1.0-cdh6.3.2 > > 62 > > 461 > > ba-hbase19.jumbo.hq.com,16020,1609326754985< > http://ba-hbase19.jumbo.hq.eset.com:16030/rs-status> > > Wed Dec 30 12:12:34 CET 2020 > > 2 s > > 2.1.0-cdh6.3.2 > > 0 > > 0 > > ba-hbase20.jumbo.hq.com,16020,1604413895967< > http://ba-hbase20.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:31:35 CET 2020 > > 2 s > > 2.1.0-cdh6.3.2 > > 62 > > 503 > > ba-hbase21.jumbo.hq.com,16020,1604414101652< > http://ba-hbase21.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:35:01 CET 2020 > > 3 s > > 2.1.0-cdh6.3.2 > > 59 > > 442 > > ba-hbase22.jumbo.hq.com,16020,1604414308289< > http://ba-hbase22.jumbo.hq.eset.com:16030/rs-status> > > Tue Nov 03 15:38:28 CET 2020 > > 0 s > > 2.1.0-cdh6.3.2 > > 40 > > 438 > > > Why, after more than 7 days, is this host not hosting more (any) regions? > Should I initiate some kind of rebalancing? > > Thanks in advance. > > M >
