UNSUBSCRIBE On Wed, Mar 28, 2018 at 7:31 PM, Chance Li (JIRA) <j...@apache.org> wrote:
> Chance Li created HBASE-20303: > --------------------------------- > > Summary: RS RPC server should not allow the response queue > size to be too large > Key: HBASE-20303 > URL: https://issues.apache.org/jira/browse/HBASE-20303 > Project: HBase > Issue Type: Improvement > Components: rpc > Environment: 2000+ Region Servers > Reporter: Chance Li > Assignee: Chance Li > Fix For: 3.0.0 > > > With async clients, in some scenarios RS RPC server will occur Full GC > because of the large reponse queue. > Netty provides a WriteBufferHighWaterMark on channel, But this doesn't > meet RS's needs(Consider 5k ~ 10k sockets in one RS server, it will need > 50G ~ 100G heap for 10M per channel). > > RS rpc server will add a gloabl response buffer water mark(2G by default). > when reaching the throttle, RS will not serve any request. > And RS rpc server will add a water mark for channel (100M by default), > because it's mostly possible that this client is abnomal. > > We created a unit test to simulate abnormal case: a client that has only 1 > socket can lead RS server to occupy heap up to 100M. > > Notes: > 1. For client compability, we still use existed > exception(CALL_QUEUE_TOO_BIG_EXCEPTION) but error message is different. > 2. Not for SimpleRpcServer, because It's rarely used. > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005) > -- k A r T h I k__________
