stdpain opened a new pull request, #3384:
URL: https://github.com/apache/brpc/pull/3384
### What problem does this PR solve?
Issue Number: #3383
Problem Summary:
### What is changed and the side effects?
bthread::TimerThread reclaims a task's pooled Task slot only when the task
is popped from the internal min-heap at its run_time. unschedule() just marks
the task removed and does not free the slot. So a
cancelled timer keeps its slot until run_time, and the live Task count
grows to ~ qps * timeout even though almost all of those tasks were unscheduled
long ago. Two retention paths:
1. Heap retention — a task pulled into the heap is only checked for
cancellation once (via try_delete() at consume time). If unscheduled
afterwards, it lingers in the heap until run_time.
2. Bucket accumulation — the timer thread only consumes buckets when it
wakes, and schedule() wakes it only for an earlier task. If every pending task
is far in the future, scheduled-then-unscheduled
tasks pile up in the buckets, unconsumed and unreclaimed, until that far
deadline.
This is significant for RPC deadline timers with large timeouts (e.g. 100k
qps × 300s ≈ ~1.9 GB held mostly by already-cancelled timers).
Changed:
- Heap sweep: in TimerThread::run(), after consuming buckets, drop
unscheduled tasks from the heap. Triggered by a timer-thread-local heuristic
(heap ≥ brpc_timer_heap_sweep_min_size, default 4096, and
roughly doubled since the last sweep), so the unschedule() hot path is
untouched (no new shared atomics). Amortized O(1) per task.
- Periodic wakeup: cap the sleep at brpc_timer_max_wakeup_interval_ms
(default 0 = disabled, legacy behavior) so the thread periodically wakes to
drain buckets / sweep the heap even when all deadlines
are far-future. An empty heap still sleeps until woken by schedule().
- Added pending_task_count() and allocated_task_count() for observability.
- Added unit tests for both the heap-retention and bucket-accumulation
cases (each verified to fail with the corresponding fix disabled).
Side effects:
- Performance effects:
- Breaking backward compatibility:
---
### Check List:
- Please make sure your changes are compilable.
- When providing us with a new feature, it is best to add related tests.
- Please follow [Contributor Covenant Code of
Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
