ninsmiracle opened a new pull request, #1556: URL: https://github.com/apache/incubator-pegasus/pull/1556
### What problem does this PR solve? <!--add issue link with summary if exists--> #1479 ### What is changed and how does it work? In the existing logic, if the number of replicas for a particular table undergoing manual_compact on the current node exceeds the limit, the LPC_MANUAL_COMPACT task will be discarded once it enters the task queue. In the new logic, the task will be delayed by 60 seconds before being reinserted into the queue. In the existing logic, the logic that filters whether a replica should be added to the queue can prevent tasks that have already undergone compaction on the same day from repeatedly entering the queue. Therefore, a simple modification to the task dequeue logic can ensure that replicas that should undergo compaction will not unexpectedly skip compaction due to the calculation at zero o'clock. ### Checklist <!--REMOVE the items that are not applicable--> ##### Tests <!-- At least one of them must be included. --> - Manual test (add detailed scripts or steps below) I create a pegasus app,and set manual compact time to 23:59. So that some replica have to do manual compact after zero o'clock. I got result like this: `33872:D2023-07-03 23:59:15.411 (1688399955411071211 15114) replica.compact0.04040001000001b4: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34071:D2023-07-04 00:00:15.411 (1688400015411226976 15118) replica.compact3.0407000100000001: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34249:D2023-07-04 00:01:15.411 (1688400075411318261 15117) replica.compact2.0407000400000002: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34459:D2023-07-04 00:02:15.411 (1688400135411507719 15120) replica.compact5.0407000500000003: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34609:D2023-07-04 00:03:15.411 (1688400195411521826 15117) replica.compact2.0407000600000004: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34803:D2023-07-04 00:04:15.411 (1688400255411667156 15118) replica.compact3.0407000000000004: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 34982:D2023-07-04 00:05:15.411 (1688400315411802093 15122) replica.compact7.0407000500000004: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 35157:D2023-07-04 00:06:15.411 (1688400375411913107 15116) replica.compact1.0407000200000003: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 35362:D2023-07-04 00:07:15.412 (1688400435412059719 15118) replica.compact3.0407000000000006: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction 35515:D2023-07-04 00:08:15.412 (1688400495412150909 15119) replica.compact4.0407000200000004: pegasus_manual_compact_service.cpp:320:begin_manual_compact(): [[email protected]:37801] start to execute manual compaction -bash-4.4$ grep -rn "end_manual_compact" * 33896:D2023-07-03 23:59:15.481 (1688399955481189237 15114) replica.compact0.04040001000001b4: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 45ms 34094:D2023-07-04 00:00:15.498 (1688400015498819014 15118) replica.compact3.0407000100000001: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 60ms 34272:D2023-07-04 00:01:15.488 (1688400075488255418 15117) replica.compact2.0407000400000002: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 45ms 34479:D2023-07-04 00:02:15.503 (1688400135503458132 15120) replica.compact5.0407000500000003: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 57ms 34631:D2023-07-04 00:03:15.483 (1688400195483022456 15117) replica.compact2.0407000600000004: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 45ms 34824:D2023-07-04 00:04:15.494 (1688400255494586453 15118) replica.compact3.0407000000000004: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 56ms 35002:D2023-07-04 00:05:15.515 (1688400315515958178 15122) replica.compact7.0407000500000004: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 76ms 35176:D2023-07-04 00:06:15.486 (1688400375486139793 15116) replica.compact1.0407000200000003: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 50ms 35380:D2023-07-04 00:07:15.487 (1688400435487399087 15118) replica.compact3.0407000000000006: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 48ms 35532:D2023-07-04 00:08:15.508 (1688400495508893878 15119) replica.compact4.0407000200000004: pegasus_manual_compact_service.cpp:328:end_manual_compact(): [[email protected]:37801] finish to execute manual compaction, time_used = 67ms` As shown in the above log, the adjusted compaction results are as expected. Replicas that had compact operations scheduled after midnight were successfully processed. Furthermore, there were no replicas that underwent compaction multiple times, ensuring a non-repetitive compaction process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
