[
https://issues.apache.org/jira/browse/HDDS-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18126945#comment-18126945
]
Layam Pavan Venkat edited comment on HDDS-16364 at 10/14/26 6:47 AM:
---------------------------------------------------------------------
incomplete MPU uploads are counted as open keys of size 0
When an MPU upload is initiated, OM writes a multipart open key to the open
key/file table with {{dataSize}} 0 and no blocks
(S3InitiateMultipartUploadRequest). It stays there until the upload is
completed or aborted. Committed parts are moved to the multipart parts table
and are reported by {{/api/v1/keys/open/mpu/summary}}.
Recon's OpenKeysInsightHandler and {{/api/v1/keys/open}} count these multipart
open keys as regular open keys. Repro on compose with 2 incomplete uploads (2 x
5 MiB parts each):
* {{/api/v1/keys/open/summary}} -> totalOpenKeys: 2, totalReplicatedDataSize:
0, totalUnreplicatedDataSize: 0
* {{/api/v1/keys/open/mpu/summary}} -> totalOpenMPUKeys: 2, totalDataSize:
20971520
*Proposed fix (Recon only):*
* Exclude multipart open keys (latest location group has isMultipartKey=true)
from {{/open}} and {{/open/summary}}. MPU uploads stay reported by
{{/open/mpu/summary}}.
* Parts that are still uploading are normal open keys and remain visible.
* Apply the filter inside the table scan used by {{/open}}, so a page of only
multipart open keys does not return 204 and stop pagination.
* Recon needs a reinit after upgrade so the stored open-key totals are
recomputed.
was (Author: JIRAUSER314517):
incomplete MPU uploads are counted as open keys of size 0*
When an MPU upload is initiated, OM writes a multipart open key to the open
key/file table with {{dataSize}} 0 and no blocks
(S3InitiateMultipartUploadRequest). It stays there until the upload is
completed or aborted. Committed parts are moved to the multipart parts table
and are reported by {{/api/v1/keys/open/mpu/summary}}.
Recon's OpenKeysInsightHandler and {{/api/v1/keys/open}} count these multipart
open keys as regular open keys. Repro on compose with 2 incomplete uploads (2 x
5 MiB parts each):
* {{/api/v1/keys/open/summary}} -> totalOpenKeys: 2, totalReplicatedDataSize:
0, totalUnreplicatedDataSize: 0
* {{/api/v1/keys/open/mpu/summary}} -> totalOpenMPUKeys: 2, totalDataSize:
20971520
*Proposed fix (Recon only):*
* Exclude multipart open keys (latest location group has isMultipartKey=true)
from {{/open}} and {{/open/summary}}. MPU uploads stay reported by
{{/open/mpu/summary}}.
* Parts that are still uploading are normal open keys and remain visible.
* Apply the filter inside the table scan used by {{/open}}, so a page of only
multipart open keys does not return 204 and stop pagination.
* Recon needs a reinit after upgrade so the stored open-key totals are
recomputed.
> OpenKey size is not consistent across different endpoints in Recon
> ------------------------------------------------------------------
>
> Key: HDDS-16364
> URL: https://issues.apache.org/jira/browse/HDDS-16364
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Recon
> Reporter: Priyesh Karatha
> Assignee: Layam Pavan Venkat
> Priority: Major
>
> We have multiple endpoints for showing openkey information.
> {code:java}
> /api/v1/keys/open/summary
> /api/v1/keys/open/mpu/summary
> /api/v1/keys/open?includeFso=true&includeNonFso=false&limit=1000{code}
> The output of `/api/v1/keys/open/summary` and
> `/api/v1/keys/open?includeFso=true&includeNonFso=false&limit=1000` and shows
> keys count but size is always 0. So we have to define these APIs properly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
