[ 
https://issues.apache.org/jira/browse/HDDS-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HDDS-15076:
----------------------------------
    Labels: pull-request-available  (was: )

> Incorrect pending deletion size for EC blocks in DN
> ---------------------------------------------------
>
>                 Key: HDDS-15076
>                 URL: https://issues.apache.org/jira/browse/HDDS-15076
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: Ozone Recon
>            Reporter: Priyesh K
>            Assignee: Priyesh K
>            Priority: Major
>              Labels: pull-request-available
>
> Recon's {{totalPendingDeletionSize}} aggregated across DataNodes is inflated 
> by a factor of {{k+m}} (the number of EC shards) for Erasure Coded keys. For 
> EC(6,3), the reported value is 9× the expected total physical size, while OM 
> and SCM report correct values.
> In {{{}DeletedBlockLogImpl.constructNewTransaction(){}}}, {{totalBlockSize}} 
> is currently set using the logical block size.
> This works for Ratis replication since each DataNode stores a full copy. 
> However, for Erasure Coding (EC), the same logical size is assigned to all (k 
> + m) DataNodes.
> Each DataNode adds this full value to {{{}blockPendingDeletionBytes{}}}, and 
> Recon sums it across nodes, leading to inflated values.
> Example:
>  * Logical size = 64 MB
>  * EC nodes = 9
>  * Reported = 9 × 64 MB = 576 MB
>  * Actual data ≈ 100 MB
> *Root Cause:*
> Logical size is used per DataNode instead of actual per-replica (per-shard) 
> size.
> *Proposed Fix:*
> Set {{totalBlockSize}} as:
> perReplicaSize = totalReplicatedSize / numberOfReplicas
> *Expected Result:*
> Recon reports correct values.
> Example:
> 9 × (100 MB / 9) = 100 MB
> *Notes:*
> No impact to Ratis. Fix applies only to EC.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to