> I'm also curious, is there no way to limit quota enforcement to just > the type of quotas that are desired? In this case, We only care about > throttling requests and so I'm surprised to see that namespace limits > are being checked. +1 to limit quota enforcement to only the enabled quotas. Clusters with only request quotas don't need NamespaceStateManager checks (and associated issues with NamespaceStateManager), and those with only namespace quotas don't need QuotaCache / QuotaRefresherChore running on all region servers.
If your quota region split failed due to NPE in SPLIT_TABLE_REGION_PRE_OPERATION you may want to check if you were impacted by https://issues.apache.org/jira/browse/HBASE-28533 From: dev@hbase.apache.org At: 11/28/24 09:21:08 UTC-5:00To: dev@hbase.apache.org Subject: Re: Quotas on System tables? This implies to me that the original intention was to not perform quota enforcement on system tables. https://github.com/apache/hbase/blob/63cdd026f08cdde6ac0fde1342ffd050e8e02441/hb ase-server/src/main/java/org/apache/hadoop/hbase/namespace/NamespaceStateManager .java#L214 On Thu, Nov 28, 2024 at 2:55 PM Nick Dimiduk <ndimi...@apache.org> wrote: > > Heya, > > We recently tried to split the hbase:quota table and were surprised to > hit an NPE. I was able to reproduce this with a simple test, I've > pasted the stack trace below. > > Looking into the nature of this failure, I'm left scratching my head. > It appears that hbase has some quota enforcement system around the > number of regions permitted in a namespace. When the split request > comes in, the NamespaceAuditor is checked and I get this failure, > apparently because there's no entry for the hbase:quotas table in the > NamespaceTableAndRegionInfo object. > > I can pursue a fix of the bug via some basic null-handling or ensure > this structure is populated on initialization. However, I'm wondering > if it is intentional that we enforce any kind of quotas on the hbase > system namespace at all. This seems like a bug to me. > > I'm also curious, is there no way to limit quota enforcement to just > the type of quotas that are desired? In this case, We only care about > throttling requests and so I'm surprised to see that namespace limits > are being checked. > > Thanks, > Nick > > 2024-11-28T14:11:38,672 ERROR [PEWorker-2 {}] > procedure2.ProcedureExecutor(1844): CODE-BUG: Uncaught runtime > exception: pid=7, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, > hasLock=true; SplitTableRegionProcedure table=hbase:quota, > parent=3c1dbd551845ed2e8033b76570c6d8f6, > daughterA=a8f5e70728d93859d7c3d95aa5a0f63b, > daughterB=c71f49ad7e689a57742a9fe65b918d08 > java.lang.NullPointerException: Cannot invoke > "java.util.concurrent.atomic.AtomicInteger.addAndGet(int)" because the > return value of "java.util.Map.get(Object)" is null > at org.apache.hadoop.hbase.namespace.NamespaceTableAndRegionInfo.incRegionCountForT able(NamespaceTableAndRegionInfo.java:101) > ~[classes/:?] > at org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceR egionCount(NamespaceStateManager.java:95) > ~[classes/:?] > at org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToSplitRegion(Names paceAuditor.java:116) > ~[classes/:?] > at org.apache.hadoop.hbase.quotas.MasterQuotaManager.onRegionSplit(MasterQuotaManag er.java:544) > ~[classes/:?] > at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.preSplitRegi on(SplitTableRegionProcedure.java:607) > ~[classes/:?] > at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromS tate(SplitTableRegionProcedure.java:298) > ~[classes/:?] > at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromS tate(SplitTableRegionProcedure.java:98) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachinePro cedure.java:188) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:941) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExec utor.java:1825) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureE xecutor.java:1503) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(P rocedureExecutor.java:2154) > ~[classes/:?] > at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) > ~[classes/:?] > at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureE xecutor.java:2181) > ~[classes/:?]
