allthingssecurity opened a new pull request, #30243: URL: https://github.com/apache/camel/pull/30243
# Description [CAMEL-26966](https://issues.apache.org/jira/browse/CAMEL-26966), part of [CAMEL-26630](https://issues.apache.org/jira/browse/CAMEL-26630) (exchange sanitizer), after R2 ([CAMEL-26768](https://issues.apache.org/jira/browse/CAMEL-26768), #29664). It builds on the dev console and `camel get sanitizer` of [CAMEL-26769](https://issues.apache.org/jira/browse/CAMEL-26769) (#29901) and needs [CAMEL-26965](https://issues.apache.org/jira/browse/CAMEL-26965) (#30129, `interceptSendToEndpoint`), both merged: the camel-core run below found that one processor of Camel broke the rule, and it got its own fix first, as the design asks, since R3 is on by default in tests (see "Validation"). Step 4 of the "Order of work" of `design/exchange-sanitizer.adoc`, first part: the `AsyncProcessor` contract of rule R3 ("one thread at a time"). The second part of R3, the sampled watch windows, is not in this PR. R1 and R2 are unchanged. ## What it adds The rule, as the Javadoc of `AsyncProcessor` states it: the value the callback of a call is called with (`doneSync`) is the value `process(exchange, callback)` returns, and `true` means the callback was called already, on the calling thread. A caller that uses the return value continues the exchange on `true` and leaves it to the callback on `false`, so a processor that returns the other value lets two threads continue the exchange, or none. Three new kinds of `Violation` (`getRule()` returns `R3`), checked at the call of each step, below its error handler, as R1 is: | Kind | When | |---|---| | `RETURNED_TRUE_CALLBACK_ON_OTHER_THREAD` | `process` returned `true` and its callback had been called on another thread (such as a processor that waits for its worker) | | `RETURNED_TRUE_BEFORE_CALLBACK` | `process` returned `true`, and the callback came later on another thread, or with `true` | | `RETURNED_FALSE_CALLBACK_TRUE` | `process` returned `false` and its callback was called with `true` (either order, any thread) | ```text Exchange sanitizer: process returned true, and its callback was called on another thread (R3) exchange 1FF90A20897EAF2-0000000000000000, route prices, step lookup, processor org.apache.camel.sanitizer.BrokenUserCode$ReturnsTrueAfterAsyncDone the callback was called with false: thread user-worker at org.apache.camel.sanitizer.BrokenUserCode$ReturnsTrueAfterAsyncDone.lambda$process$0(BrokenUserCode.java:180) at java.base/java.lang.Thread.run(Thread.java:1583) then process returned true: thread Camel (camel-61) thread #91 - ProducerTemplate at org.apache.camel.support.StepHelper.process(StepHelper.java:63) at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler.processOutput(RedeliveryErrorHandler.java:266) ... process(exchange, callback) must return the value its callback is called with (see AsyncProcessor). Usual fix: when the work goes on another thread, return false from process and let that thread call the callback with false; return true only after the callback was called with true on the calling thread. ``` - **`SanitizerDoneSyncListener`**, a second `StepListener` with a small record per call (`DoneSyncCall`). It needs no table and keeps no exchange or thread (the id of the calling thread only). The return and the first callback are applied one at a time under the lock of the record, and whichever comes second compares, once. A stack is taken only for a callback on another thread before the return, or when a violation is reported. - **Option.** `camel.sanitizer.checks` gets `threads`. The default becomes `callback,threads`, and `callback,commit,threads` in a test, as the options table of the design lists `threads` in the default and R3a costs little (see Cost). camel-main option, generated metadata, `main.adoc`, catalog copies. `checks=threads` alone installs only this listener, which then counts the calls. - **Console and CLI.** The `sanitizer` dev console gets a `checks` field (the checks that run) and its descriptions name R3; `camel get sanitizer --report` says "R3 not checked" when `threads` is not in the checks, and does not suggest `threads` to an integration of an older version (it would not start with an unknown check). The rest of the console and the command show the new kinds as they are. - **Docs.** An R3 section in `sanitizer.adoc` (the kinds, the case that is not reported and why, a report); the design (R3 narrowed, step 4 partly built, the defaults); the `--sanitizer` text of `camel run`, `camel dev` and `camel debug` and the jbang managing page say `callback,commit,threads` where they told how to turn R2 on, so following them does not turn R3 off. ## Where this differs from the design **Validation of R3 is done for this part on camel-core only.** The design asks, before R3 is complete, for the runs on the async component suites (netty, netty-http, vertx-http, JMS request/reply), a TLA+ model of the record, and the JMH run. They come with the sampled overlaps; the design says so now. The record of this part is two values compared under one lock, and the tests drive every order and race it. The cost below is a quick loop, not JMH. **Narrowed for the case the Javadoc allows.** A processor that hands its callback to the reactive executor returns the value of the processor it calls, and the reactive executor calls the callback with `false` on the calling thread, at once or after the work that thread is running. The channel of a step without an error handler does this, so a step that calls such a channel directly sees it, such as a `choice` whose branch has one step in a route with `noErrorHandler()` (with the default error handler the channel returns `false`). So a callback with `false` on the thread that called `process` is never reported, whatever `process` returns and whether it comes before or after the return. This relies on the default reactive executor of camel-core, which runs a callback on the thread that scheduled it. The design text says so. A side effect: a processor that hands its work to a pool that the calling thread also belongs to, and gets the same thread back for the callback, is not seen. ## Validation **1. User code.** `SanitizerDoneSyncTest` (camel-core, `@CamelSanitizer`): a processor that waits for its worker and returns `true` while the worker called the callback with `false` (the planted "two threads continue the exchange"); one that returns `true` and lets a worker call back around the return; one that calls back with `true` from a worker and returns `false`; one that calls back with `true` and then returns `false`. Each fails its test with a report that names its step, route and class, unless it says `@ExpectSanitizerViolation`. 200 broken calls at once give exactly 200 reports. Correct code is quiet: an asynchronous processor on a pool, `to("direct:...")`, a `choice` in a route with `noErrorHandler()` (where a recording listener checks that the allowed case really happened: returned `true`, callback with `false` on the calling thread), `threads()`, a parallel `split`, `delay().asyncDelayed()` and `to("seda:...")`; the asynchronous processor, the `direct` route on a pool and the EIPs also 200 times at once. **2. The record.** `SanitizerDoneSyncCallTest` drives `DoneSyncCall` in every order of the return, the throw and the first and second callbacks, on the calling thread and on another one, and races the return against a callback on another thread 2000 times for three pairs of values (returned `true` with a callback with `false`, `true` with `true`, `false` with `true`) and for the correct pair (`false` with `false`): one report for each broken call, none for the correct one. Negative controls, each a mutant of the listener run against `SanitizerDoneSyncTest` and `SanitizerDoneSyncCallTest` (18 tests): no exception for the reactive executor, 2 tests fail (also the route with the `noErrorHandler()` choice); no report at the return of a callback that came first on another thread, 5 fail; no compare at the callback, 6 fail; no check of a throw, 1 fails. Without the lock nothing failed in 10 runs of the race test: the two threads never lost a compare, so the tests cannot show what the lo ck is for. **3. Camel on camel-core.** The whole camel-core suite with the CI switch (`-Psanitizer`): 9368 tests, 26 failures, 12 errors. R3 reports nothing in Camel's EIPs except one processor: **`InterceptSendToEndpointProcessor`** returns the value of the `before` of the interceptor (a route, so `false`) and calls the callback with the value of the producer after it (`true` for `mock`), and with `mockEndpoints()` and an asynchronous producer the other way round. That fails 16 test classes that use `interceptSendToEndpoint` or advice with `mockEndpoints` (37 reports, all at that processor). It is a real break of the contract: a caller that uses the return value would continue the exchange twice or not at all (Camel's own callers go on through the callback, so the routes work today). It is fixed in CAMEL-26965 (#30129); with that fix applied the same run (9371 tests) has no R3 report outside the tests that break the rule on purpose; what fails there is the list below, plus one case of the f ix's own new test that I corrected and ran again on its own. The other failures of the run are not R3: `AggregateOnCompletionThrowsTest` (2, an R1 report of a test added by CAMEL-26881 whose strategy throws an `Error` on purpose; it needs `@ExpectSanitizerViolation`, as #29672 did for similar tests), the `DumpModel*` tests (a stale `camel-yaml-io` in my local repository) and `ValidatorExternalResourceTest` (network), which fail the same way without the sanitizer. **4. Tests.** The sanitizer tests of camel-core, camel-management, camel-main and camel-test-main-junit6, `StepListener*Test` (camel-core 178, camel-management 1, camel-main 10, camel-test-main-junit6 4), `ChaosWithSanitizerTest` 6, the cli-connector sanitizer tests 3, `ProjectRunnerTest` 21, and in camel-jbang-core `ListSanitizerTest`, `ListSanitizerActionTest`, `ListChaosTest`, `RunTest`, `CommandInputErrorsTest` and the CLI help and parse goldens, 96, all pass. `RunExportGoldenTest` fails 18 of 49 cases here: those cases resolve dependencies that my offline build does not have, and they do not touch the sanitizer. 20 times in one JVM, `SanitizerDoneSyncTest`, `SanitizerDoneSyncCallTest`, `SanitizerActivationTest` and `SanitizerUserCodeTest` and `SanitizerDevConsoleTest`: 1180 of 1180; with the CI switch, `SanitizerDoneSync*`, `StepListenerTest`, `ChoiceTest` and `AsyncEndpointTest`: 700 of 700. ## Cost Off (no `threads` in the checks, or no sanitizer): nothing is installed and nothing runs. On: one small object per call of a step, two short uncontended locks, two reads of the current thread, and a stack only for a callback on another thread before the return. A quick loop (not JMH; the 6 steps of the R2 benchmark, `pendingTimeout=0`, one sending thread, 5 rounds of 200 000 messages after a warm-up), per message: sanitizer off 1.00 us; R1 1.21 to 1.28 us; R1 and R3 1.25 to 1.30 us; R3 alone 1.21 to 1.27 us (`stackDepth=0`). With the default 32 frames, R1 is 53.5 to 54.2 us with or without R3: R1 takes a stack at every callback, and R3 is in the noise. The harness is outside this PR; I can add a case to the sanitizer JMH benchmark of camel-performance-tests if wanted. ## Follow-ups (not in this PR) - The second part of R3, the sampled watch windows, and the runs of R3 on the test suites of netty, netty-http, vertx-http and JMS request/reply, as the design asks before R3 is complete. This PR ran camel-core only. - `AggregateOnCompletionThrowsTest` with `@ExpectSanitizerViolation`. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26966](https://issues.apache.org/jira/browse/CAMEL-26966). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built the core modules, the base components, camel-sanitizer, camel-management, camel-main, camel-test-main-junit6, camel-cli-connector and camel-jbang-core with the formatter and import-sort plugins and the package plugin; the camel-main metadata and `main.adoc`, the dev console JSON and the camel-jbang command metadata and pages are generated, and the catalog copies match them. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
