[
https://issues.apache.org/jira/browse/HBASE-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372046#comment-15372046
]
Enis Soztutar commented on HBASE-16095:
---------------------------------------
bq. In general we are approaching this in an ad hoc manner with static pools,
predefined and limited QoS levels, and magic constants.
Agreed, this is not the cleanest approach. I've thought about doing an
unlimited thread pool cache, but there is no out-of-the-box thread pool that
does what we want in the executor service. We would like something like keep a
thread pool cache with max size N, and have a unbounded linked queue to queue
up events.
bq. Use the same approach for dispatch to META: META handlers would become just
a pool allocated at the highest priority level. The downside is more work at
dispatch than simple test-and-branch with precompiled constants. Just a thought.
This is similar, except for one-per-priority, there is three groups of fixed
priority.
> Add priority to TableDescriptor and priority region open thread pool
> --------------------------------------------------------------------
>
> Key: HBASE-16095
> URL: https://issues.apache.org/jira/browse/HBASE-16095
> Project: HBase
> Issue Type: Bug
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
> Attachments: hbase-16095_v0.patch, hbase-16095_v1.patch,
> hbase-16095_v2.patch
>
>
> This is in the similar area with HBASE-15816, and also required with the
> current secondary indexing for Phoenix.
> The problem with P secondary indexes is that data table regions depend on
> index regions to be able to make progress. Possible distributed deadlocks can
> be prevented via custom RpcScheduler + RpcController configuration via
> HBASE-11048 and PHOENIX-938. However, region opening also has the same
> deadlock situation, because data region open has to replay the WAL edits to
> the index regions. There is only 1 thread pool to open regions with 3 workers
> by default. So if the cluster is recovering / restarting from scratch, the
> deadlock happens because some index regions cannot be opened due to them
> being in the same queue waiting for data regions to open (which waits for
> RPC'ing to index regions which is not open). This is reproduced in almost all
> Phoenix secondary index clusters (mutable table w/o transactions) that we
> see.
> The proposal is to have a "high priority" region opening thread pool, and
> have the HTD carry the relative priority of a table. This maybe useful for
> other "framework" level tables from Phoenix, Tephra, Trafodian, etc if they
> want some specific tables to become online faster.
> As a follow up patch, we can also take a look at how this priority
> information can be used by the rpc scheduler on the server side or rpc
> controller on the client side, so that we do not have to set priorities
> manually per-operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
