davsclaus opened a new pull request, #29279: URL: https://github.com/apache/camel/pull/29279
_Claude Code on behalf of Claus Ibsen (davsclaus)_ [CAMEL-26543](https://issues.apache.org/jira/browse/CAMEL-26543): follow-up of CAMEL-26219 (#29066), where the review found this race. ## Problem The chat task read the history of a memory ID in `prepareInput`, before the model call and outside any lock. In `processOutput` it wrote the whole returned history back. When two turns of one conversation overlapped (a concurrent consumer, or callers without a `CamelChatMemoryId` header, who all share `default`): - each turn sent the model a history without the other turn - the last write replaced the history, so the other turn was lost - a `CamelChatClearHistory` sent while a turn ran was undone when that turn wrote its old history back This also happened with the default `pooling=true`, because that lock only covers the model call. ## Fix - `ChatPredictor.predict` holds a lock per memory ID for the whole turn: read, model call, write. A clear now waits for the running turn and clears after it. - The locks are in a `ConcurrentHashMap` with a reference count that changes inside `compute`. An entry is removed when no turn of its memory ID is in flight, so the map needs no bound. An LRU map of locks could evict a lock while a thread holds it. - Turns of different memory IDs do not wait for each other. With `pooling=true` the model calls were already serialized, so throughput does not change. - `AbstractTaskPredictor.predict` now calls a new `protected Output infer(Input)` for the model call. `processOutput` runs after the pooling lock is released; the other predictors' `processOutput` only parse the output. ## Tests `ChatHistoryTest` plays the model in a subclass that overrides `infer` and holds every turn until the test releases it: - two turns of one conversation: the second waits, and is sent the first turn - a clear sent while a turn runs is not undone - turns of two conversations are with the model at the same time, and no lock entry is left behind With the lock removed, the first test failed on every run and the clear test on 1 of 3 runs. The clear test can still pass without the lock, because which turn writes last is not fixed. ## Docs One sentence in the chat history section of `huggingface-component.adoc`, and the catalog copy of the doc. No option or default changes, so there is no upgrade-guide entry. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
