[
https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209749#comment-15209749
]
deepankar commented on HBASE-13819:
-----------------------------------
I totally agree with your idea, but I think practically it would be hard to
exhaust out the offheap stuff if the allocation is proportional to the expected
response size, one reason why we did not decrease the buffer size from
BoundedByteBufferPool is that these large size requests are not that uncommon,
they do occur occasionally and we did not want them having to allocate a full
size buffer at that time with our initial assumption being with
BoundedByteBufferPool there might not be any more allocations at all. May be we
should try out internally by restricting max size from BBBP to a much lower
value, may be its ok for large RPCs to have to create their own buffers.
bq. And one more thing for GC is that the full GC only can clean the off heap
area?
I think young could also clear them, the problem is that Bits class when there
is not enough offheap space calls the System.GC which tr iggers full GC I think
but any way I agree this GC is wastefull
bq. We need to make pool such that we will give a BB back if it is having a
free one. When it is not having a free one and capacity is not reached, it
makes a new DBB and return
This would be nice, we had a hot patch internally which just fails the request
when you see the Bits is going to call System.GC(), this was just temporarily
to stop the RegionServer from crashing.
> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
> Key: HBASE-13819
> URL: https://issues.apache.org/jira/browse/HBASE-13819
> Project: HBase
> Issue Type: Sub-task
> Components: Scanners
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch,
> HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch, q.png
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an
> on heap byte buffer (via BoundedByteBufferPool). The pool will keep upto
> certain number of buffers. This jira aims at testing possibility for making
> this buffers off heap ones. (DBB) The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are
> not using unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap
> writes will be better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer
> to the socket channel, it will create a temp DBB and copy data to there and
> only DBBs will be moved to Sockets. If we make DBB 1st hand itself, we can
> avoid this one more level of copying.
> Will do different perf testing with changed and report back.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
