JiriOndrusek opened a new pull request, #9018: URL: https://github.com/apache/camel-quarkus/pull/9018
fixes https://github.com/apache/camel-quarkus/issues/9015 Declares the RAG ingestion pipeline instead of coding it: documents are read, split, embedded in batches and written to an EmbeddingStore, each segment carrying camel_quarkus_pipeline and camel_quarkus_document_id so retrieval can cite its source. With the RAG bridge (#9013), an @RegisterAiService answers from the store a pipeline filled — no glue code. Declaration. Properties or Java, as strict twins: @Ingest methods (discovered via Jandex, validated at build time, invoked once at startup) convert into the configuration model, so both paths share one route-building implementation. Sources. A directory (file consumer with guarded defaults: noop, sized idempotent register, changed read lock, UTF-8), or any of the ~300 Camel consumers — source.uri in properties, Source.endpoint(dsl -> dsl.aws2S3(...)) in Java, where dsl. completes every component's typed options. No hand-curated source types. Identity is the consumer's business: document-id names the header (CamelAwsS3Key, CamelKafkaKey) or gives an expression. Config split. source.uri is build-time-fixed — a runtime-overridable consumer URI would be arbitrary component invocation. Per-deployment values (directory, document-id, enabled) stay runtime; placeholders in the URI resolve at startup, so credentials do too. Logged URIs are sanitized. Engine. Deliberately naive and shipped as preview: it writes what it is given and keeps no record, so re-ingestion duplicates and restarts re-read — documented, with the synchronising engine (on tracker #9003) following separately behind the same API. Validation. Everything decidable at build time fails there with the fix in the message: unknown component scheme (plus the add-extension command), malformed @Ingest methods, splitter bounds, conflicting source declarations — produced as ValidationErrorBuildItem so dev/test mode check too. Tests. 15 deployment tests lock in every validation message; 6 integration tests cover directory, pushed endpoint, builder pipeline, S3 (MinIO) and Kafka (container), both scoped per class; native CI category added. Out of scope: synchronisation, document parsing (text only), retry/DLC hooks, automatic pipeline→augmentor pairing. <!-- Uncomment and fill this section if your PR is not trivial [ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes. [ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #. [ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough. [ ] Phrases like Fix #<issueNumber> or Fixes #<issueNumber> will auto-close the named issue upon merging the pull request. Using them is typically a good idea. [ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request. [ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
