[
https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136667#comment-15136667
]
Anoop Sam John commented on HBASE-15204:
----------------------------------------
{code}
2779 public void addtoCellCountFromCP(int count) {
2780 this.cellCountFromCP += count;
2781 }
2782
2783 public int getCellCountFromCP() {
2784 return this.cellCountFromCP;
2785 }
{code}
We don'e even need this.
{code}
int cellCount = batchOp.getCellCountFromCP();
3128 for (int i = firstIndex; !isInReplay && i < lastIndexExclusive;
i++) {
....
for (List<Cell> cells : familyMaps[i].values()) {
3143 cellCount += cells.size();
3144 }
{code}
Within the outer for loop, we can get the cell count in wal edit from cp
associated with the corresponding Mutation.
batchOp.walEditsFromCoprocessors will give the WALEdit from which u can know.
That will be the exact count need also. Because getCellCountFromCP() will
give all Mutation's CP returned WALEdit cell count. We will be splitting these
mutations into mini batches as per row lock avail.
> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>
> Key: HBASE-15204
> URL: https://issues.apache.org/jira/browse/HBASE-15204
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-15204.patch, HBASE-15204_1.patch,
> HBASE-15204_1.patch, HBASE-15204_2.patch, WAlEdit_add_allocation.jpg,
> WAlEdit_add_allocation_after_patch.jpg
>
>
> The write path profiling shows that when we try to add Cells to WALEdits we
> try to do a lot of Array copy inorder to grow the Arraylist backing the
> WALEdits. In a simple one min profiling of the write path with 50 YCSB
> threads shows around 261MB of allocation done for the Array copy to happen.
> We can try to avoid that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
