TangSiyang2001 commented on code in PR #48399:
URL: https://github.com/apache/doris/pull/48399#discussion_r1981264011
##########
cloud/src/meta-service/meta_service_job.cpp:
##########
@@ -192,11 +194,27 @@ void start_compaction_job(MetaServiceCode& code,
std::string& msg, std::stringst
}), compactions.end());
// clang-format on
// Check conflict job
+ if (std::ranges::any_of(compactions, [](const auto& c) {
+ return c.type() == TabletCompactionJobPB::STOP_TOKEN;
+ })) {
+ auto it = std::ranges::find_if(compactions, [](const auto& c) {
+ return c.type() == TabletCompactionJobPB::STOP_TOKEN;
+ });
+ msg = fmt::format(
+ "compactions are not allowed on tablet_id={} currently,
blocked by schema "
+ "change job delete_bitmap_initiator={}",
+ tablet_id, it->delete_bitmap_lock_initiator());
+ code = MetaServiceCode::JOB_TABLET_BUSY;
+ return;
+ }
if (compaction.type() == TabletCompactionJobPB::FULL) {
// Full compaction is generally used for data correctness repair
// for MOW table, so priority should be given to performing full
// compaction operations and canceling other types of compaction.
compactions.Clear();
+ } else if (compaction.type() == TabletCompactionJobPB::STOP_TOKEN) {
+ // fail all existing compactions
+ compactions.Clear();
Review Comment:
It is better to wait until all ongoing compactions have completed. If any
compaction failed, the allocated resources will be wasted, and retrying after
the compaction stop token is removed will add unnecessary overhead to BE.
##########
cloud/src/meta-service/meta_service_job.cpp:
##########
@@ -1111,6 +1129,16 @@ void process_schema_change_job(MetaServiceCode& code,
std::string& msg, std::str
auto job_val = recorded_job.SerializeAsString();
txn->put(job_key, job_val);
if (!new_tablet_job_val.empty()) {
+ auto& compactions = *new_recorded_job.mutable_compaction();
+ compactions.erase(
+ std::remove_if(
+ compactions.begin(), compactions.end(),
+ [&](auto& c) {
+ return
c.has_delete_bitmap_lock_initiator() &&
+ c.delete_bitmap_lock_initiator() ==
+
schema_change.delete_bitmap_lock_initiator();
+ }),
+ compactions.end());
Review Comment:
Unfortunately, there is a known issue in the schema change process due to
missing initial parameters. As a result, abort requests will consistently fail
before entering this code branch. Therefore, if the schema change is cancelled,
the compaction stop token will never be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
