[
https://issues.apache.org/jira/browse/HBASE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882158#action_12882158
]
Todd Lipcon commented on HBASE-2782:
------------------------------------
Ehh, I'm still unconvinced. Moving META to ZK means extra work for things like
snapshots, backups, etc, where currently we can use the same mechanisms for
user tables as for meta tables. Plus these issues that we see on META are
issues that we'll also see on user tables. For example, right now it's very
easy for a MR job to completely monopolize the capacity of a cluster to the
point that interactive queries start having 30sec+ latencies. Really good QoS
is hard, but I think a simple solution like above can get us a lot of benefit
for not much work. Especially if we can make the QoS policy pluggable, maybe
someone will just write a really good one and contribute it back.
Multi-tenancy is a huge problem we haven't even begun to tackle, and QoS is
just a tiny bit of it, but my point is that we need to solve this problem
regardless of what happens to META.
> QOS for META table access
> -------------------------
>
> Key: HBASE-2782
> URL: https://issues.apache.org/jira/browse/HBASE-2782
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Todd Lipcon
>
> I'd like to brainstorm some ideas on how we can prioritize reads and writes
> to META above reads and writes to other tables. I've noticed that if the
> regionserver hosting META is under heavy load, then lots of other operations
> take much longer than they should. For example, I'm currently running 120
> threads of YCSB across 3 client nodes hitting a 5-node cluster. Doing a full
> scan of META (only 600 rows) takes upwards of 30 seconds in the shell, since
> all of the handler threads are tied up and there's a long RPC queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
