[ https://issues.apache.org/jira/browse/HBASE-13142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349510#comment-14349510 ]
stack commented on HBASE-13142: ------------------------------- Hmm... I love testing. My key implementation is actually broken. Am allocating every time. Even with this breakage, the fact that we are getting an average size makes it so we skip a bunch of resizing and this alone is making for increased throughput (in spite of upped GC). Let me fix or rather, do the @eclark suggestion > [PERF] Reuse the IPCUtil#buildCellBlock buffer > ---------------------------------------------- > > Key: HBASE-13142 > URL: https://issues.apache.org/jira/browse/HBASE-13142 > Project: HBase > Issue Type: Improvement > Components: Performance > Reporter: stack > Assignee: stack > Labels: beginner > Fix For: 2.0.0, 1.1.0 > > Attachments: 13142.txt, 13142v2.txt, 13142v3.txt, traces.2.svg, > traces.svg > > > Running some scan profiling, flight recorder was mildly fingering resize of > the buffer allocated in IPCUtil#buildCellBlock as a point of contention. It > was half-hearted blaming it for a few hundreds of ms over a five minute > sampling with a few tens of instances showing. > I tried then w/ flamegraph/lightweight profiler and this reported the buffer > allocations taking 22% of our total CPU. See attachment trace.svg. > I enabled TRACE-level logging on org.apache.hadoop.hbase.ipc.IPCUtil and > indeed every allocation was doing a resize from initial allocation of 16k -- > the default up to 220k (this test returns ten randomly sized rows zipfian > sized between 0 and 8k). > Upping the allocation to 220k meant we now avoided the resize but the initial > allocation was now blamed for 10% of allocations (see trace.2.svg attached). > Lets do buffer reuse. Will save a bunch of allocation and CPU. -- This message was sent by Atlassian JIRA (v6.3.4#6332)