Limit the IPC queue size based on calls' payload size
-----------------------------------------------------
Key: HBASE-5190
URL: https://issues.apache.org/jira/browse/HBASE-5190
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Fix For: 0.94.0
Currently we limit the number of calls in the IPC queue only on their count. It
used to be really high and was dropped down recently to num_handlers * 10 (so
100 by default) because it was easy to OOME yourself when huge calls were being
queued. It's still possible to hit this problem if you use really big values
and/or a lot of handlers, so the idea is that we should take into account the
payload size. I can see 3 solutions:
- Do the accounting outside of the queue itself for all calls coming in and
out and when a call doesn't fit, throw a retryable exception.
- Same accounting but instead block the call when it comes in until space is
made available.
- Add a new parameter for the maximum size (in bytes) of a Call and then set
the size the IPC queue (in terms of the number of items) so that it could only
contain as many items as some predefined maximum size (in bytes) for the whole
queue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
