[
https://issues.apache.org/jira/browse/HBASE-30390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116061#comment-18116061
]
huginn commented on HBASE-30390:
--------------------------------
PR: https://github.com/apache/hbase/pull/8662
> Avoid copying previous cell values during data block encoding
> -------------------------------------------------------------
>
> Key: HBASE-30390
> URL: https://issues.apache.org/jira/browse/HBASE-30390
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 2.4.11
> Reporter: huginn
> Priority: Major
> Labels: pull-request-available
>
> ## What happens
> Data block encoders retain previous cells while writing HFile blocks. When a
> block is shipped, the current implementation materializes a complete copy of
> the previous cell, including its value bytes, even though FAST_DIFF only
> needs the previous value for comparison.
> ## When it happens
> When HFile blocks are written with buffered data block encoding, especially
> FAST_DIFF or DIFF, and writer-backed cells are shipped or reused.
> ## Impact
> This creates unnecessary value-array allocations and byte copies on the write
> path, increasing allocation and garbage-collection overhead for encoded HFile
> writes.
> ## Root cause
> On master, EncodingState.beforeShipped() copies the previous cell with
> KeyValueUtil.copyToNewKeyValue(). FastDiffDeltaEncoder then reads the copied
> cell for previous-value length and content comparison. The previous key
> metadata must remain stable, but copying the complete value is unnecessary
> because the encoded value bytes are already present in the writer buffer.
> Upstream code pointers:
> -
> hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/EncodingState.java:43-49
> -
> hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/FastDiffDeltaEncoder.java:217-269
> -
> hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java:822-827
> ## Proposed fix
> Keep a key-only materialization for general encoder state, track the previous
> value length separately, and make FAST_DIFF compare against the value bytes
> already present in the encoded writer buffer after shipping. Pass the writer
> buffer range to the encoding state and add focused regression coverage for
> shipped cells, reused buffers, and encoding failures.
> ## Reproduction
> A focused regression test will be included with the proposed change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
