Peiming Wu created HBASE-28984:
----------------------------------
Summary: Alter table causes AccessDeniedException at the client
side
Key: HBASE-28984
URL: https://issues.apache.org/jira/browse/HBASE-28984
Project: HBase
Issue Type: Bug
Affects Versions: 2.4.16
Reporter: Peiming Wu
Assignee: Peiming Wu
When altering a table there is a chance that the client catches some
AccessDeniedExceptions:
{code:java}
#444021290, table=***, attempt=1/5 failed=1ops, last exception:
org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions (user=***, scope=default:***, family=***,
params=[table=***,family=***],action=WRITE)
at
org.apache.hadoop.hbase.security.access.AccessController.prePut(AccessController.java:1388)
at
org.apache.hadoop.hbase.coprocessor.RegionObserver.prePut(RegionObserver.java:413)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$23.call(RegionCoprocessorHost.java:907)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$23.call(RegionCoprocessorHost.java:904)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:904)
at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.callPreMutateCPHook(HRegion.java:4160)
at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.access$800(HRegion.java:3610)
at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation$1.visit(HRegion.java:3685)
at
org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations(HRegion.java:3218)
at
org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.checkAndPrepare(HRegion.java:3674)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4447)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:4380)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:1009)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp(RSRpcServices.java:927)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:892)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2855)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45008)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) {code}
This is because HBase updates the table owner when a table is altered. By
updating the table, ZKPermissionWatcher receives the permission changes and
tries to update the permission cache in AuthManager:
[https://github.com/apache/hbase/blob/rel/2.4.16/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AuthManager.java#L185-L192]
{code:java}
private void updateTableCache(TableName table, ListMultimap<String,
Permission> tablePerms) {
PermissionCache<TablePermission> cacheToUpdate =
tableCache.getOrDefault(table, new PermissionCache<>());
clearCache(cacheToUpdate);
updateCache(tablePerms, cacheToUpdate);
tableCache.put(table, cacheToUpdate);
mtime.incrementAndGet();
} {code}
At the same time, AccessController is accessing the same reference of the
permission cache in AuthManager:
[https://github.com/apache/hbase/blob/rel/2.4.16/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AuthManager.java#L298-L319]
{code:java}
public boolean accessUserTable(User user, TableName table, Permission.Action
action) {
if (user == null) {
return false;
}
if (table == null) {
table = PermissionStorage.ACL_TABLE_NAME;
}
if (authorizeUserNamespace(user, table.getNamespaceAsString(), action)) {
return true;
}
PermissionCache<TablePermission> tblPermissions =
tableCache.getOrDefault(table, TBL_NO_PERMISSION);
if (hasAccessTable(tblPermissions.get(user.getShortName()), action)) {
return true;
}
for (String group : user.getGroupNames()) {
if (hasAccessTable(tblPermissions.get(AuthUtil.toGroupEntry(group)),
action)) {
return true;
}
}
return false;
} {code}
I think this is a bug introduced in the previous refactoring:
https://issues.apache.org/jira/browse/HBASE-21255
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
