[ https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833023#comment-13833023 ]
Nicolas Liochon commented on HBASE-9535: ---------------------------------------- bq. What do you mean when you say "rpc doesn't work anymore"? The server gets stuck somewhere. I don't really know where nor why. bq. On patch, you pull in classes just for your test? I plan to keep them. The pool is only in hadoop 2, so we need it for hadoop 1 compat. The ByteBufferInputStream is a technical class. It's public in ZK, but not really part of their API... And I've done some changes already. > Try a pool of direct byte buffers handling incoming ipc requests > ---------------------------------------------------------------- > > Key: HBASE-9535 > URL: https://issues.apache.org/jira/browse/HBASE-9535 > Project: HBase > Issue Type: Brainstorming > Reporter: stack > Assignee: stack > Attachments: 9535.v1.patch > > > ipc takes in a query by allocating a ByteBuffer of the size of the request > and then reading off the socket into this on-heap BB. > Experiment with keeping a pool of BBs so we have some buffer reuse to cut on > garbage generated. Could checkout from pool in RpcServer#Reader. Could > check back into the pool when Handler is done just before it queues the > response on the Responder's queue. We should be good since, at least for > now, kvs get copied up into MSLAB (not references) when data gets stuffed > into MemStore; this should make it so no references left over when we check > the BB back into the pool for use next time around. > If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes > time so if already allocated, should be good. GC of DBBs is a pain but if in > a pool, we shouldn't be wanting this to happen). The copy from socket to the > DBB will be off-heap (should be fast). > Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by > size (we might want to bypass the pool if an object is > size N). > DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when > short-circuit reading. It'd be a bummer if we had to allocate big objects > on-heap. Would still be an improvement. -- This message was sent by Atlassian JIRA (v6.1#6144)