[
https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185909#comment-13185909
]
Jean-Daniel Cryans commented on HBASE-5190:
-------------------------------------------
About #3.
I like the elegance of that solution since we don't have to keep track of the
calls in flight but I see 2 big issues:
- if you set a max call size you need to keep both clients and servers in sink
and also decide who's going to do the check.
- if you plan for big calls by default, you may end up with a tiny size for
the queue. For example, let's say you cap calls at 10% of the heap and set
their max individual size at 10MB, it means that you can only allow 10 items in
the queue (and you don't account listeners).
> Limit the IPC queue size based on calls' payload size
> -----------------------------------------------------
>
> Key: HBASE-5190
> URL: https://issues.apache.org/jira/browse/HBASE-5190
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Fix For: 0.94.0
>
>
> Currently we limit the number of calls in the IPC queue only on their count.
> It used to be really high and was dropped down recently to num_handlers * 10
> (so 100 by default) because it was easy to OOME yourself when huge calls were
> being queued. It's still possible to hit this problem if you use really big
> values and/or a lot of handlers, so the idea is that we should take into
> account the payload size. I can see 3 solutions:
> - Do the accounting outside of the queue itself for all calls coming in and
> out and when a call doesn't fit, throw a retryable exception.
> - Same accounting but instead block the call when it comes in until space is
> made available.
> - Add a new parameter for the maximum size (in bytes) of a Call and then set
> the size the IPC queue (in terms of the number of items) so that it could
> only contain as many items as some predefined maximum size (in bytes) for the
> whole queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
