oscerd opened a new pull request, #29066: URL: https://github.com/apache/camel/pull/29066
Fixes [CAMEL-26219](https://issues.apache.org/jira/browse/CAMEL-26219), found while adding the catalog body types (CAMEL-24961). ## 1. The chat history map grows without bound **Was:** `ChatPredictor` kept the history of every memory ID (the `CamelChatMemoryId` header) in a `ConcurrentHashMap` for the life of the endpoint, and an entry was removed only with `CamelChatClearHistory`. With one memory ID per user or session, the map and each history grew without bound. **Now:** The histories are kept in an LRU cache from `LRUCacheFactory` with room for `maxChatHistories` entries (new option, default 1000). Once there are more memory IDs, the least recently used history is evicted, and the next message for that memory ID starts a new conversation. `0` or less keeps every history, as before. The new `maxHistoryMessages` option (default 0, no limit) limits the messages of each history: the system prompt is kept, the oldest messages are dropped first, and the kept messages start with a user message, so chat templates that require alternating user and assistant roles still accept the history. ## 2. Any output containing the quoted text "error" is treated as a failure **Was:** Every predictor failed with "Python inference failed" when its output contained the text `"error"`. A generated text, a summary, an answer, a transcript, a chat reply or a class label with that word in quotes failed the exchange, and so did image bytes that happened to read as it. **Now:** `AbstractTaskPredictor.failOnPythonError` fails only on the error payload of the Python handlers, `json.dumps({"error": str(e)})`: a JSON object whose single field is `error`. The tasks that answer with plain text (text-generation, summarization, question-answering, zero-shot with autoSelect, and the bytes of text-to-image) read the output as JSON only when the whole output is a JSON object. The JSON tasks check the JSON they already parse. The exception message is `Python inference failed: ` followed by the error text, instead of the whole payload (automatic-speech-recognition said `Python inference error: `). ## 3. text-to-speech sets Content-Type application/octet-stream on a POJO body **Was:** `TextToSpeechPredictor` set `Content-Type: application/octet-stream` while the body is a DJL `Audio` object, not bytes. **Now:** It sets no `Content-Type`. The body stays the DJL `Audio`, as documented, and so does the `CamelHuggingFaceOutput` header. The Javadoc of the output contract says `Audio` instead of a JSON string. ## Tests The predictors need Python and DJL at runtime, so the tests call `prepareInput` and `processOutput` directly, as the issue did. - `ChatHistoryTest` plays the model, answering each turn with the whole conversation as the text-generation pipeline does, and checks the messages sent to it. With the default, the history of the least recently used of 1001 memory IDs is evicted and a recent one is kept. `maxChatHistories=2` evicts the oldest of three histories, `maxChatHistories=0` keeps them all, `maxHistoryMessages=3` keeps the system prompt and the last messages from a user message, and by default a history is not trimmed. - `PredictorErrorHandlingTest`: the error payload still fails text-classification, zero-shot-classification, sentence-embeddings, automatic-speech-recognition, text-to-speech and chat, with the error text in the message (text-generation, summarization, question-answering and text-to-image were already covered). Outputs that have the word error are results: a text with `"error"` in it, a JSON object with an `error` field and other fields, and a text that starts with an error object (text-generation), a summary, the answer `"error"`, the class label `error` (text-classification, and zero-shot with autoSelect=false), the transcript `error`, the chat reply `error`, and image bytes that read as `"error"`. - `RevisionAndOutputHeaderTest`: text-to-speech gives the `Audio` (samples, sampling rate, mono) in the body and the output header, and no `Content-Type`. Revert check: with the src/main change reverted (with the generated configurers, which call the new setters), 15 of 22 new tests fail. The default bound, the 10 outputs with the word error and the text-to-speech `Content-Type` fail by assertion or with the old "Python inference failed" or "Python inference error" exception. The 3 tests that set `maxChatHistories` or `maxHistoryMessages` fail because the options do not exist. The 7 that pass are guards: the 6 error-payload tests (the old check also failed on the payload; they cover the predictors that now parse first and check after) and the test that a history is not trimmed by default. Module: 34 tests, 0 failures (22 new, 12 existing). `HuggingFaceIT` needs Python and the models and was not run. ## Docs and generated files - New options `maxChatHistories` (default 1000) and `maxHistoryMessages` (default 0) on the component and the endpoint: regenerated configurers, URI factory and `huggingface.json`, and the catalog copy of the JSON. The endpoint DSL needs to be regenerated. - The component page says how the chat histories are kept, bounded and cleared, that only the error payload of the Python handler fails a task, and that text-to-speech sets no `Content-Type`. Catalog copy of the page. - `@BodyTypes` is unchanged, as the bodies are the same. - Upgrade guide entry for 4.24: the oldest conversations are evicted once there are more than 1000 memory IDs, an output with the word error no longer fails and the exception message changed, and text-to-speech no longer sets `Content-Type`. ## Not changed - text-to-speech keeps the DJL `Audio` body rather than giving encoded WAV bytes: `Audio` is the documented and tested output, and the DJL api reads audio but has no encoder. A route that needs WAV can encode the samples of the `Audio`. - The Python handlers still answer with plain text on success and with the error object on failure, so a generated text that is exactly `{"error": "..."}`, with no other field, is still read as a failure. Wrapping every result in JSON would change the protocol of all the handlers. - The bound counts memory IDs, not bytes or idle time. An idle-time eviction or a pluggable chat memory store would be a new feature. - The `CamelChatMemoryId` and `CamelChatClearHistory` headers are still not declared in `HuggingFaceConstants`, so the catalog does not list them. Main only, as the other fixes of the CAMEL-24961 follow-ups. _Claude Code on behalf of Andrea Cosentino (oscerd)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
