Heya,
We recently tried to split the hbase:quota table and were surprised to
hit an NPE. I was able to reproduce this with a simple test, I've
pasted the stack trace below.
Looking into the nature of this failure, I'm left scratching my head.
It appears that hbase has some quota enforcement system around the
number of regions permitted in a namespace. When the split request
comes in, the NamespaceAuditor is checked and I get this failure,
apparently because there's no entry for the hbase:quotas table in the
NamespaceTableAndRegionInfo object.
I can pursue a fix of the bug via some basic null-handling or ensure
this structure is populated on initialization. However, I'm wondering
if it is intentional that we enforce any kind of quotas on the hbase
system namespace at all. This seems like a bug to me.
I'm also curious, is there no way to limit quota enforcement to just
the type of quotas that are desired? In this case, We only care about
throttling requests and so I'm surprised to see that namespace limits
are being checked.
Thanks,
Nick
2024-11-28T14:11:38,672 ERROR [PEWorker-2 {}]
procedure2.ProcedureExecutor(1844): CODE-BUG: Uncaught runtime
exception: pid=7, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION,
hasLock=true; SplitTableRegionProcedure table=hbase:quota,
parent=3c1dbd551845ed2e8033b76570c6d8f6,
daughterA=a8f5e70728d93859d7c3d95aa5a0f63b,
daughterB=c71f49ad7e689a57742a9fe65b918d08
java.lang.NullPointerException: Cannot invoke
"java.util.concurrent.atomic.AtomicInteger.addAndGet(int)" because the
return value of "java.util.Map.get(Object)" is null
at
org.apache.hadoop.hbase.namespace.NamespaceTableAndRegionInfo.incRegionCountForTable(NamespaceTableAndRegionInfo.java:101)
~[classes/:?]
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceRegionCount(NamespaceStateManager.java:95)
~[classes/:?]
at
org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToSplitRegion(NamespaceAuditor.java:116)
~[classes/:?]
at
org.apache.hadoop.hbase.quotas.MasterQuotaManager.onRegionSplit(MasterQuotaManager.java:544)
~[classes/:?]
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.preSplitRegion(SplitTableRegionProcedure.java:607)
~[classes/:?]
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298)
~[classes/:?]
at
org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:98)
~[classes/:?]
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188)
~[classes/:?]
at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:941)
~[classes/:?]
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1825)
~[classes/:?]
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1503)
~[classes/:?]
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:2154)
~[classes/:?]
at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
~[classes/:?]
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:2181)
~[classes/:?]
