[ https://issues.apache.org/jira/browse/HBASE-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834707#comment-13834707 ]
Nicolas Liochon commented on HBASE-9535: ---------------------------------------- bq. It would be great to add some mem used counters arond ElasticByteBufferPool, that will be useful while we encourntered a OOM Yeah, I agree. I'm actually wondering if I have to integrate it to the metrics from day one or if I can wait. Thanks for looking at the patch. It's still early stage, yes. If you can try it from a performance point of view, I'm interested. I would like to have a "good enough" (i.e. fully implemented with the right configuration parameters available) version for tomorrow. > Try a pool of direct byte buffers handling incoming ipc requests > ---------------------------------------------------------------- > > Key: HBASE-9535 > URL: https://issues.apache.org/jira/browse/HBASE-9535 > Project: HBase > Issue Type: Brainstorming > Reporter: stack > Assignee: stack > Attachments: 9535.v1.patch, 9535.v2.patch > > > ipc takes in a query by allocating a ByteBuffer of the size of the request > and then reading off the socket into this on-heap BB. > Experiment with keeping a pool of BBs so we have some buffer reuse to cut on > garbage generated. Could checkout from pool in RpcServer#Reader. Could > check back into the pool when Handler is done just before it queues the > response on the Responder's queue. We should be good since, at least for > now, kvs get copied up into MSLAB (not references) when data gets stuffed > into MemStore; this should make it so no references left over when we check > the BB back into the pool for use next time around. > If on-heap BBs work, we could then try direct BBs (Allocation of DBBs takes > time so if already allocated, should be good. GC of DBBs is a pain but if in > a pool, we shouldn't be wanting this to happen). The copy from socket to the > DBB will be off-heap (should be fast). > Could start w/ the HDFS DirectBufferPool. It is unbounded and keeps items by > size (we might want to bypass the pool if an object is > size N). > DBBs for this task would contend w/ offheap BBs used in BlockReadLocal when > short-circuit reading. It'd be a bummer if we had to allocate big objects > on-heap. Would still be an improvement. -- This message was sent by Atlassian JIRA (v6.1#6144)