[
https://issues.apache.org/jira/browse/HDDS-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-15581:
----------------------------------
Labels: pull-request-available (was: )
> ozone debug replicas chunk-info reports wrong block size for EC keys
> --------------------------------------------------------------------
>
> Key: HDDS-15581
> URL: https://issues.apache.org/jira/browse/HDDS-15581
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ren Koike
> Assignee: Ren Koike
> Priority: Major
> Labels: pull-request-available
>
> {{ozone debug replicas chunk-info}} returns incorrect {{blockData.size}} (and
> chunk metadata) for EC-replicated keys. Every internal block/replica is
> reported with the same size of a block in a specific datanode, instead of the
> expected per-replica sizes.
> *Steps to reproduce*
> # Create an EC key with {{rs-3-2-1024k}} (or similar), e.g. size 1,148,576
> bytes (between 1 MiB and 2 MiB).
> # Run:
> ozone debug replicas chunk-info o3://<volume>/<bucket>/<key>
> # Inspect {{blockData.size}} for each entry in {{{}keyLocations{}}}.
> Expected (EC 3+2, 1,148,576 bytes):
> ||Replica||Expected size||
> |Data 1|1,048,576|
> |Data 2|100,000|
> |Data 3|0|
> |Parity 4, 5|1,048,576 each|
> Actual: all replicas show 1,048,576.
> *Root cause*
> This is a regression introduced when HDDS-13445 replaced
> {{getBlockFromAllNodes()}} with a per-datanode loop that calls
> {{{}ContainerProtocolCalls.getBlock(){}}}.
> {{getBlock()}} uses {{{}tryEachDatanode(){}}}, which always queries the same
> datanode (the pipeline’s “closest” / first node), not the datanode from the
> loop variable. Each iteration then writes that datanode’s block metadata
> under a different hostname/file path, duplicating the same replica’s data 5×
> (for EC 3+2).
> *Proposed fix*
> * Add {{ContainerProtocolCalls.getBlockFromDatanode(..., datanode,
> replicaIndexes)}} that uses the existing private {{getBlock(..., datanode,
> ...)}} without {{{}tryEachDatanode{}}}.
> * Use it in {{ChunkKeyHandler}} with the loop’s {{{}datanodeDetails{}}}.
> Avoid restoring {{getBlockFromAllNodes()}} to prevent holding all block
> metadata in memory for large keys, which was original intention of
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
