Huang Kuan Hao created HDDS-16311:
-------------------------------------
Summary: Avoid the O(n^2) part-number lookup in S3 multipart
complete
Key: HDDS-16311
URL: https://issues.apache.org/jira/browse/HDDS-16311
Project: Apache Ozone
Issue Type: Improvement
Reporter: Huang Kuan Hao
Assignee: Huang Kuan Hao
S3MultipartUploadCompleteRequest collects the requested part numbers into an
ArrayList, then for every uploaded part calls partNumbers.contains(...) to find
the unused parts. S3 allows up to 10000 parts, so this is O(n*m) with
autoboxing, and it runs while the bucket write lock is held.
Current:
List<Integer> partNumbers = new ArrayList<>();
...
for (PartKeyInfo partKeyInfo : partKeyInfoMap) {
if (!partNumbers.contains(partKeyInfo.getPartNumber())) {
partNumbers is only used for add/contains, and the strict-increasing order is
validated independently in getPartsListSize, so switching to a HashSet is safe.
Fix: use Set<Integer>/HashSet (and the matching parameter type). Behavior
unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
