allthingssecurity opened a new pull request, #30304: URL: https://github.com/apache/camel/pull/30304
# Description [CAMEL-27088](https://issues.apache.org/jira/browse/CAMEL-27088) The Delay EIP, the Throttle EIP with `asyncDelayed` and the Threads EIP hand the exchange to a task on their own thread pool and return `false`; the task calls the callback later. When the route is removed, `doShutdown` shuts that pool down with `shutdownNow`, and the tasks still queued in it are dropped. Nothing calls their callback. That happens when a route is stopped with a timeout that runs out (the default graceful timeout of 45 s, or `stopRoute(id, timeout, unit)`) and then removed, which is also what a route reload does. A delay longer than the timeout, a throttle that has queued exchanges, or a busy Threads pool is enough. Then: - a synchronous caller (a `ProducerTemplate` request, a `direct` caller in another route) waits for ever; only a CamelContext stop releases it, through the await manager; - the exchange stays in the in-flight registry, its unit of work is never done (no `onCompletion`, no commit or rollback of the consumer). A route stop alone is not affected: the pool keeps running, the task runs later and fails the exchange with a `RouteStoppingException` (CAMEL-25502). Fix: - `DelayProcessorSupport` (Delay) and `AbstractThrottler` (both throttle modes) keep the tasks they scheduled and that have not started yet in a concurrent set. The task and `doShutdown` each claim a task by removing it from the set, and only the one that removed it acts: the task routes the exchange on; `doShutdown`, after it shut down the EIP's own pool with `shutdownNow`, fails it with a `RouteStoppingException` and calls `callback.done(false)`. A task rejected by the pool is removed from the set before the existing rejected path runs. The throttler task is still a `Callable` (the lambda it replaces was one), so the pool is called with the same `submit` / `schedule` overload as before; the throttler tests of camel-management use a pool that rejects only `schedule(Callable)`. - `ThreadsProcessor` rejects the tasks that `shutdownNow` returns (its `ProcessCall` is already `Rejectable` and fails the exchange with a `RouteStoppingException` once the processor is stopped). - The rule is the same in the three EIPs (and in the parallel EIPs of CAMEL-27089): exchanges are failed only when the EIP shut down its own pool, as only then are the tasks dropped. A pool of the user (`executorService` with an instance) is not shut down by the EIP; its tasks still run later and fail the exchange with a `RouteStoppingException`, as on main. Why the claim: a task can already be taken by a pool thread when `shutdownNow` runs, and still be about to run. I checked the protocol with a small TLA+ model (task scheduled, pool thread runs it, route stop with graceful wait or timeout, route removal with `shutdownNow`, 2 exchanges). On main the invariant "once the route is removed and nothing is left to run, every exchange that entered the route was completed" is violated (5-step trace: send, stop, timeout, remove, the exchange is never completed; 35 distinct states). Failing every scheduled exchange at shutdown without the claim violates "the callback is called at most once" (77 states). With the claim both hold (79 states), and a forced removal where every exchange is completed is still reachable. Tests: `AsyncDelayedRemoveRouteTest` sends an exchange that waits in the pool (Delay of 60 s; total requests throttle of 1 per 60 s after a first exchange; concurrent requests throttle of 1 on a pool of 1 thread while the permit is held; Threads with 1 thread kept busy), stops the route with a 100 ms timeout, removes it, and expects the exchange to complete with a `RouteStoppingException` and nothing left in flight. A fifth test uses a Delay on a single-thread pool of the user whose thread is kept busy: the removal must not fail the exchange, and it completes with a `RouteStoppingException` once the task runs. On main the first four time out (the caller waits for ever) and the fifth passes. With the change all five pass, the delay, throttle, threads, thread pool, shutdown and remove route tests of camel-core pass (275 run, 0 failures, 4 skipped by their own conditions) and so do the throttler tests of camel-management (30 run, 0 failures), and the new test class passes 20 times in one JVM (100/100). Related: CAMEL-27089 (parallel Multicast, Split and Recipient List) fixes the same drop for the parts of a parallel EIP. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-27088](https://issues.apache.org/jira/browse/CAMEL-27088). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built the core modules with the formatter and import-sort plugins; no option, doc or generated file changes. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
