Priyesh K created HDDS-15076:
--------------------------------

             Summary: Incorrect pending deletion size for EC blocks in DN
                 Key: HDDS-15076
                 URL: https://issues.apache.org/jira/browse/HDDS-15076
             Project: Apache Ozone
          Issue Type: Sub-task
          Components: Ozone Recon
            Reporter: Priyesh K
            Assignee: Priyesh K


Recon's {{totalPendingDeletionSize}} aggregated across DataNodes is inflated by 
a factor of {{k+m}} (the number of EC shards) for Erasure Coded keys. For 
EC(6,3), the reported value is 9× the expected total physical size, while OM 
and SCM report correct values.

In {{{}DeletedBlockLogImpl.constructNewTransaction(){}}}, {{totalBlockSize}} is 
currently set using the logical block size.

This works for Ratis replication since each DataNode stores a full copy. 
However, for Erasure Coding (EC), the same logical size is assigned to all (k + 
m) DataNodes.

Each DataNode adds this full value to {{{}blockPendingDeletionBytes{}}}, and 
Recon sums it across nodes, leading to inflated values.

Example:
 * Logical size = 64 MB

 * EC nodes = 9

 * Reported = 9 × 64 MB = 576 MB

 * Actual data ≈ 100 MB

*Root Cause:*

Logical size is used per DataNode instead of actual per-replica (per-shard) 
size.

*Proposed Fix:*

Set {{totalBlockSize}} as:

perReplicaSize = totalReplicatedSize / numberOfReplicas

*Expected Result:*

Recon reports correct values.

Example:
9 × (100 MB / 9) = 100 MB

*Notes:*

No impact to Ratis. Fix applies only to EC.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to