JiriOndrusek opened a new issue, #9039: URL: https://github.com/apache/camel-quarkus/issues/9039
The langchain4j-ingest extension currently forgets what it has ingested: every application restart re-reads and re-embeds the whole source, wasting embedding calls and piling duplicate content into the store. Pipelines fed by a component (Kafka, S3, ...) have no duplicate protection at all - a redelivered message is ingested again. Until the full synchronising engine arrives, a pipeline should be able to remember what it already ingested - across restarts and, where the backing store allows it, across instances - and skip duplicate deliveries. The register backing this should be pluggable, following what Camel users already know: the idempotent repository. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
