[
https://issues.apache.org/jira/browse/HIVE-30133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30133:
----------------------------------
Description:
VectorReduceSinkObjectHashOperator extracts and hashes the partition columns
for every row. When every partition column repeats across the batch, every row
gets the same hashCode, so it is now computed once per batch. The bucket
number and _bucket_number are still computed per row.
Iceberg V3 merge-on-read DELETE distributes by FILE__PATH, which repeats within
a batch. In a microbenchmark of the operator: 107.7 -> 53.4 ns/row for a
repeating 150-byte string key, -24% for a repeating bigint, unchanged when the
key does not repeat.
> Vectorized ReduceSink hashes a repeating partition key for every row
> --------------------------------------------------------------------
>
> Key: HIVE-30133
> URL: https://issues.apache.org/jira/browse/HIVE-30133
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration
> Reporter: Denys Kuzmenko
> Priority: Major
>
> VectorReduceSinkObjectHashOperator extracts and hashes the partition columns
> for every row. When every partition column repeats across the batch, every
> row gets the same hashCode, so it is now computed once per batch. The bucket
> number and _bucket_number are still computed per row.
> Iceberg V3 merge-on-read DELETE distributes by FILE__PATH, which repeats
> within a batch. In a microbenchmark of the operator: 107.7 -> 53.4 ns/row for
> a repeating 150-byte string key, -24% for a repeating bigint, unchanged when
> the key does not repeat.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
