Layam Pavan Venkat created HDDS-17832:
-----------------------------------------
Summary: Recon MPU size totals are not reduced for split-layout
MPUs completed or aborted after a Recon restart
Key: HDDS-17832
URL: https://issues.apache.org/jira/browse/HDDS-17832
Project: Apache Ozone
Issue Type: Bug
Reporter: Layam Pavan Venkat
{\{MultipartInfoInsightHandler}} (added in HDDS-16363) computes the replicated
size of \{{multipartPartsTable}} events using the parent MPU's replication
config, because \{{OmMultipartPartInfo}} does not carry one. The config is kept
in an in-memory cache keyed by uploadId, filled only from
\{{multipartInfoTable}} PUT and UPDATE events.
After a Recon restart the cache is empty. When an MPU initiated before the
restart is completed or aborted, OM deletes the \{{multipartInfoTable}} row
before its \{{multipartPartsTable}} rows in the same batch. The DELETE event
does not fill the cache, so every part delete logs \{{No cached replication
config for upload ...; skipping part size update}}, and the upload's bytes stay
in \{{/keys/open/mpu/summary}} until the next full reprocess.
*Steps to reproduce*
# Initiate an MPU on a bucket that uses the split parts table layout, and
commit a few parts.
# Restart Recon.
# Complete or abort the MPU.
# \{{/keys/open/mpu/summary}} still reports the upload's size.
*Possible fix*
Also cache the replication config on the \{{multipartInfoTable}} DELETE event.
*To review as part of this issue*
* The cache is bounded at 10,000 uploads (least recently used). On a miss, the
part size update is skipped, so the totals drift until the next reprocess.
Check whether eviction can drop configs still needed by active uploads, and
whether a miss should fall back to looking up the config, or flag the totals
for recompute, instead of skipping.
* Check that every add and remove path (part commit, part re-upload, complete,
abort, expired MPU cleanup) finds the config it needs.
Found during the review of HDDS-16364
(https://github.com/apache/ozone/pull/11685).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]