chihsuan commented on code in PR #10795:
URL: https://github.com/apache/ozone/pull/10795#discussion_r3959951161
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/QuotaRepairTask.java:
##########
@@ -86,12 +91,19 @@ public class QuotaRepairTask {
QuotaRepairTask.class);
@VisibleForTesting
static final int BATCH_SIZE = 5000;
+ /**
+ * A legacy schema multipart upload holds every committed part inline, so
its rows are far
+ * larger than a key or file row: measured at roughly 320 bytes per part, a
1000 part upload
+ * decodes to about 320KB. Batch fewer of them to keep the in-flight batches
bounded.
+ */
+ @VisibleForTesting
+ static final int MPU_BATCH_SIZE = 100;
Review Comment:
> I feel like it might be better to just process the legacy rows
individually instead of batching them, because adding the parts inline is
easier on the CPU. We can use batching only when schemaVersion = 1 i.e. new MPU
format.
Good catch @spacemonkd this makes sense. I'll update this to process legacy
rows inline and keep batching for split-parts uploads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
