[
https://issues.apache.org/jira/browse/CAMEL-25027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25027.
---------------------------------
Resolution: Fixed
> camel-langchain4j-ingest - Add media modality to embed images, audio, video
> and PDF whole
> --------------------------------------------------------------------------------------------
>
> Key: CAMEL-25027
> URL: https://issues.apache.org/jira/browse/CAMEL-25027
> Project: Camel
> Issue Type: New Feature
> Components: camel-langchain4j
> Affects Versions: 4.23.0
> Reporter: Jiri Ondrusek
> Assignee: Jiri Ondrusek
> Priority: Major
> Fix For: 4.23.0
>
>
> The langchain4j-ingest producer reads the body as text, splits it and embeds
> the segments. Media files cannot be ingested as media embeddings; they are
> decoded as garbled text and embedded anyway.
>
> LangChain4j 1.20 added an experimental multimodal EmbeddingRequest API: an
> input can be image, audio, video or PDF content, and a model declares what it
> accepts in supportedContentTypes().
> * Add a modality endpoint option (text, the default, or media). With
> modality=media: the body is read as bytes and embedded whole, as one vector
> * the vector is stored under a placeholder TextSegment whose text is the
> document id, with the same identity metadata as text segments
> * the MIME type comes from a new contentType option or from the document id's
> extension; headers are not consulted
> * the endpoint refuses to start with a model that declares no media type,
> with a documentSplitter, or with contentType under modality=text; a document
> whose medium the model lacks fails before its dedup claim and body read
> * maxDocumentSize and minDocumentSize count bytes; splitter and batch options
> do not apply
> * filtering, deduplication and results are unchanged
> Image is the only medium a LangChain4j provider embeds today (Jina, Voyage
> AI, Cohere, Gemini, Bedrock Titan). Audio, video and PDF need a custom
> EmbeddingModel.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
