[
https://issues.apache.org/jira/browse/CAMEL-26250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrea Cosentino reassigned CAMEL-26250:
----------------------------------------
Assignee: Andrea Cosentino
> camel-google-vertexai - generateMultimodal silently drops a stream or File
> image, and rawPredict rejects byte[], stream and File JSON bodies
> --------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26250
> URL: https://issues.apache.org/jira/browse/CAMEL-26250
> Project: Camel
> Issue Type: Bug
> Components: camel-google
> Reporter: Claus Ibsen
> Assignee: Andrea Cosentino
> Priority: Major
> Fix For: 4.24.0
>
>
> Found while adding the catalog body types of google-vertexai (CAMEL-24961, PR
> #28404).
> h4. 1. generateMultimodal silently drops a stream, File or byte[] image
> {{GoogleVertexAIProducer.generateMultimodal}} (main 2a465aa4814c, lines
> 315-322) uses the body only when it is a {{byte[]}} with the
> {{CamelGoogleVertexAIMediaMimeType}} header set, or a String without a
> prompt. An {{InputStream}}, {{File}} or {{GenericFile}} body (a file or HTTP
> consumer), or a byte[] without the mime type header, is ignored without an
> error, so only the prompt is sent and the model answers without the image.
> Checked on 4.24.0-SNAPSHOT with a JDK {{HttpServer}} fake of the Gemini API.
> Proposed: convert the body to byte[] with the type converter, and fail when
> there is media without a mime type (or guess it from the file name).
> h4. 2. rawPredict and streamRawPredict reject byte[], stream and File bodies
> {{buildRawPredictRequestBody}} (lines 491-523) accepts a String or a Map only
> and throws "Request body must be a JSON String, Map, or plain text prompt"
> for anything else, so JSON coming from a file or HTTP consumer fails. Checked
> on 4.24.0-SNAPSHOT with a mocked {{PredictionServiceClient}}.
> Proposed: convert other bodies to String with the type converter before the
> checks.
> h4. 3. Six declared headers are never used
> {{GoogleVertexAIConstants}} declares {{CamelGoogleVertexAIChatMessages}},
> {{SystemInstruction}}, {{SafetySettings}} (lines 62-69), {{SafetyRatings}},
> {{ContentBlocked}} (lines 85-88) and {{AnthropicVersion}} (line 151), and the
> docs list them, but the producer neither reads nor sets them. Read in the
> code, not run.
> Proposed: implement them or remove them from the constants and the docs.
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
