[
https://issues.apache.org/jira/browse/HBASE-13375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484112#comment-14484112
]
Mikhail Antonov commented on HBASE-13375:
-----------------------------------------
Hm, what I found debugging TestHBaseFsck#testHBaseFsck.
Test itself runs just fine. If I go into
AnnotationReadingPriorityFunction#getPriority, and add there at first line:
bq. if (true) return HConstants.ADMIN_QOS;
things fail with exception:
{code}
2015-04-07 14:35:49,843 WARN [Thread-313]
client.ConnectionImplementation(1740): Checking master connection
com.google.protobuf.ServiceException:
org.apache.hadoop.hbase.ipc.CallTimeoutException: callId: 2 methodName:
IsMasterRunning param {TODO: class
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest}
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:223)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:50206)
at
org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceState.isMasterRunning(ConnectionImplementation.java:1028)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.isKeepAliveMasterConnectedAndRunning(ConnectionImplementation.java:1733)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1385)
at
org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
at
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:115)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3704)
at
org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptorsByTableName(HBaseAdmin.java:2646)
at
org.apache.hadoop.hbase.util.HBaseFsck.getHTableDescriptors(HBaseFsck.java:3167)
at org.apache.hadoop.hbase.util.HBaseFsck.getTables(HBaseFsck.java:3158)
at
org.apache.hadoop.hbase.util.HBaseFsck.reportTablesInFlux(HBaseFsck.java:1040)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:636)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:671)
at
org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.doFsck(HbckTestingUtil.java:69)
at
org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.doFsck(HbckTestingUtil.java:43)
at
org.apache.hadoop.hbase.util.hbck.HbckTestingUtil.doFsck(HbckTestingUtil.java:38)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testHBaseFsck(TestHBaseFsck.java:205)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: callId: 2
methodName: IsMasterRunning param {TODO: class
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$IsMasterRunningRequest}
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:235)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
... 27 more
{code}
That's not right. Mentioned change might have affected latency/throughput, but
not correctness I'd say.
> Provide HBase superuser higher priority over other users in the RPC handling
> ----------------------------------------------------------------------------
>
> Key: HBASE-13375
> URL: https://issues.apache.org/jira/browse/HBASE-13375
> Project: HBase
> Issue Type: Improvement
> Components: rpc
> Reporter: Devaraj Das
> Assignee: Mikhail Antonov
> Fix For: 1.1.0
>
> Attachments: HBASE-13375-v0.patch
>
>
> HBASE-13351 annotates Master RPCs so that RegionServer RPCs are treated with
> a higher priority compared to user RPCs (and they are handled by a separate
> set of handlers, etc.). It may be good to stretch this to users too - hbase
> superuser (configured via hbase.superuser) gets higher priority over other
> users in the RPC handling. That way the superuser can always perform
> administrative operations on the cluster even if all the normal priority
> handlers are occupied (for example, we had a situation where all the master's
> handlers were tied up with many simultaneous createTable RPC calls from
> multiple users and the master wasn't able to perform any operations initiated
> by the admin). (Discussed this some with [~enis] and [~elserj]).
> Does this make sense to others?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
