spacemonkd commented on code in PR #10795:
URL: https://github.com/apache/ozone/pull/10795#discussion_r3956081552
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/QuotaRepairTask.java:
##########
@@ -86,12 +91,19 @@ public class QuotaRepairTask {
QuotaRepairTask.class);
@VisibleForTesting
static final int BATCH_SIZE = 5000;
+ /**
+ * A legacy schema multipart upload holds every committed part inline, so
its rows are far
+ * larger than a key or file row: measured at roughly 320 bytes per part, a
1000 part upload
+ * decodes to about 320KB. Batch fewer of them to keep the in-flight batches
bounded.
+ */
+ @VisibleForTesting
+ static final int MPU_BATCH_SIZE = 100;
Review Comment:
I just asked Claude for inputs, here is what it gave me:
"
Decoding happens single-threaded in the producer, while the workers for the
legacy path only sum longs. So the batch/worker parallelism buys almost nothing
for legacy rows — it only helps the split-parts path (the scanParts DB seeks).
That argues for handling legacy rows inline/streamed and reserving workers for
split-parts.
"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
