Priyesh K created HDDS-15076:
--------------------------------
Summary: Incorrect pending deletion size for EC blocks in DN
Key: HDDS-15076
URL: https://issues.apache.org/jira/browse/HDDS-15076
Project: Apache Ozone
Issue Type: Sub-task
Components: Ozone Recon
Reporter: Priyesh K
Assignee: Priyesh K
Recon's {{totalPendingDeletionSize}} aggregated across DataNodes is inflated by
a factor of {{k+m}} (the number of EC shards) for Erasure Coded keys. For
EC(6,3), the reported value is 9× the expected total physical size, while OM
and SCM report correct values.
In {{{}DeletedBlockLogImpl.constructNewTransaction(){}}}, {{totalBlockSize}} is
currently set using the logical block size.
This works for Ratis replication since each DataNode stores a full copy.
However, for Erasure Coding (EC), the same logical size is assigned to all (k +
m) DataNodes.
Each DataNode adds this full value to {{{}blockPendingDeletionBytes{}}}, and
Recon sums it across nodes, leading to inflated values.
Example:
* Logical size = 64 MB
* EC nodes = 9
* Reported = 9 × 64 MB = 576 MB
* Actual data ≈ 100 MB
*Root Cause:*
Logical size is used per DataNode instead of actual per-replica (per-shard)
size.
*Proposed Fix:*
Set {{totalBlockSize}} as:
perReplicaSize = totalReplicatedSize / numberOfReplicas
*Expected Result:*
Recon reports correct values.
Example:
9 × (100 MB / 9) = 100 MB
*Notes:*
No impact to Ratis. Fix applies only to EC.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]