github-actions[bot] commented on code in PR #65854:
URL: https://github.com/apache/doris/pull/65854#discussion_r3627509994
##########
be/src/cloud/cloud_cumulative_compaction.cpp:
##########
@@ -554,6 +592,91 @@ Status
CloudCumulativeCompaction::pick_rowsets_to_compact() {
return Status::OK();
}
+Status
CloudCumulativeCompaction::prepare_merge_input_rowsets(MergeInputRowsetsResult*
result) {
+ if (!cloud::should_use_single_rowset_grouped_compaction(
Review Comment:
[P1] Preserve the grouped-mode decision while the task is queued
Selection can use the mutable enable/minimum-segment settings to replace the
policy-selected range with one large singleton, after which the task is
registered and may wait in the compaction pool. If either setting is validly
changed before this recheck (for example, the feature is disabled or the
minimum is raised), this returns without setting grouped mode and the
already-truncated task falls back to an ordinary whole-rowset merge. That
restores the memory spike this path is meant to bound; it also leaves
`_is_single_rowset_grouped_compaction` false, so `modify_rowsets()` may apply a
normal cumulative-point transition to an input chosen under grouped semantics.
Please snapshot the grouped decision and group size during preparation, or
cancel and re-pick when the queued task no longer matches it.
##########
be/src/cloud/config.cpp:
##########
@@ -54,6 +54,9 @@ DEFINE_mInt32(max_base_compaction_task_num_per_disk, "2");
DEFINE_mBool(prioritize_query_perf_in_compaction, "false");
DEFINE_mInt32(compaction_max_rowset_count, "10000");
DEFINE_mInt64(compaction_txn_max_size_bytes, "7340032"); // 7MB
+DEFINE_mBool(enable_cloud_single_rowset_compaction, "false");
+DEFINE_mInt32(cloud_single_rowset_compaction_min_segments, "512");
+DEFINE_mInt32(cloud_single_rowset_compaction_segment_group_size, "64");
Review Comment:
[P2] Reject non-positive group sizes before scheduling
This mutable setting has no positive-domain validation, while rowset
selection does not consult it. With grouped compaction enabled, `0` or a
negative value therefore passes `prepare_compact()`, gets registered as a
pending task, and acquires the meta-service tablet job; only
`prepare_merge_input_rowsets()` rejects it during execution. Every eligible
tablet can repeat that failed scheduler/job cycle until the setting is
repaired. Please reject invalid startup/live values, or at minimum validate and
snapshot the size during preparation before task registration; the new unit
test currently proves only the late execution failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]