[
https://issues.apache.org/jira/browse/CAMEL-26069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen reassigned CAMEL-26069:
-----------------------------------
Assignee: Claus Ibsen
> camel-aws-bedrock - the invoke operations send the body as its toString(),
> and the image and embeddings streaming operations give an empty text
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26069
> URL: https://issues.apache.org/jira/browse/CAMEL-26069
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2
> Affects Versions: 4.18.0, 4.22.0, 4.23.0
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Priority: Major
> Fix For: 4.24.0
>
>
> Found while adding the catalog body types of aws-bedrock (CAMEL-24961).
> h4. 1. The invoke operations send the body as its toString()
> invokeTextModel, invokeImageModel, invokeEmbeddingsModel and the three
> streaming operations build the request of the model with
> {{SdkBytes.fromUtf8String(String.valueOf(exchange.getMessage().getBody()))}}
> ({{BedrockProducer}}, main ed3d719b379c, lines 170, 217, 269 and 567), so no
> type conversion is done, and only a String body works. Checked on
> 4.24.0-SNAPSHOT with {{camel run}} against a fake of the AWS API that logs
> the requests it gets, with the content type headers set:
> * a byte[] of the JSON request is sent as {{[B@4d774bb6}};
> * an InputStream is sent as
> {{org.apache.camel.converter.stream.ByteArrayInputStreamCache@620106cb}};
> * a java.io.File is sent as its path (and a GenericFile from a file consumer
> as {{GenericFile[...]}}, read in the code);
> * a Map is sent as {{\{inputText=Hello\}}};
> * no body is sent as the text {{null}}.
> *Proposed change*: read the body with {{getMandatoryBody(String.class)}}, or
> as bytes with {{SdkBytes.fromByteArray}}.
> h4. 2. invokeImageModelStreaming and invokeEmbeddingsModelStreaming give an
> empty text
> Both call {{invokeTextModelStreaming}} (lines 574-584), whose stream parsers
> read the generated text of each chunk ({{outputText}} for Titan). A chunk of
> an image or of an embedding has no such text. Checked on 4.24.0-SNAPSHOT with
> {{camel run}} against a fake of the AWS API that logs the requests it gets,
> streaming one chunk with {{images}} for {{amazon.titan-image-generator-v1}}
> and one with {{embedding}} for {{amazon.titan-embed-text-v1}}: the body is an
> empty String, with {{CamelAwsBedrockChunkCount}} 1.
> *Proposed change*: parse the image and embedding chunks, or remove these two
> operations if the models do not stream.
> h4. 3. pojoRequest=true without the modelId option fails after the call
> With pojoRequest=true the body is an {{InvokeModelRequest}} that carries its
> model id, but the response is read with the model of the endpoint: {{switch
> (modelId)}} at line 292 fails with {{NullPointerException: Cannot invoke
> "String.hashCode()"}} after the model was called. Checked on 4.24.0-SNAPSHOT
> with {{camel run}} against a fake of the AWS API that logs the requests it
> gets. The streaming operations fail with "Model ID cannot be null"
> ({{StreamParserFactory}}, lines 35-37), although the request carries the
> model (read in the code).
> *Proposed change*: take the model id of the request in POJO mode.
> h4. 4. amazon.titan-embed-text-v2:0 is accepted by invokeTextModel
> The docs say that an embedding model fails fast with invokeTextModel, and
> {{amazon.titan-embed-text-v1}} does, but {{amazon.titan-embed-text-v2:0}} is
> listed with the Titan text models (line 297), so invokeTextModel gives the
> JSON of the embedding as text. Checked on 4.24.0-SNAPSHOT with {{camel run}}
> against a fake of the AWS API that logs the requests it gets.
> h4. 5. No operation fails with a NullPointerException
> The operation option has no default, and with no CamelAwsBedrockOperation
> header, {{switch (determineOperation(exchange))}} at line 80 fails with
> {{NullPointerException: Cannot invoke "...BedrockOperations.ordinal()"
> because the return value of "...determineOperation(...)" is null}}. Checked
> on 4.24.0-SNAPSHOT with {{camel run}} against the AWS emulator of {{camel
> infra run aws}}.
> *Proposed change*: fail with a message that says to set the operation option
> or header.
> h4. Also read in the code, not run
> * The streaming operations do not resolve an inference profile id or an ARN
> to the model ({{StreamParserFactory.getParser}} gets the raw model id, lines
> 610 and 626), so {{us.anthropic.claude-3-haiku-20240307-v1:0}} fails with
> "Unsupported model for streaming". CAMEL-24224 fixed this for invokeTextModel
> only.
> * converseStream never sets the {{CamelAwsBedrockGuardrailTrace}} header:
> {{ConverseStreamHandler.setGuardrailTrace}} is never called, so lines 914-916
> never run. It does not set {{CamelAwsBedrockConverseOutputMessage}} either,
> which the docs list.
> * Header types in the catalog: {{CamelAwsBedrockGuardrailOutput}} is a
> {{List<GuardrailOutputContent>}}, not a GuardrailAssessment;
> {{CamelAwsBedrockGuardrailTrace}} is a {{GuardrailTraceAssessment}}, not the
> enum GuardrailTrace; converseStream reads {{CamelAwsBedrockGuardrailConfig}}
> as a {{GuardrailStreamConfiguration}}, so the GuardrailConfiguration the docs
> name is ignored there.
> * From the documented response formats, not run: AI21 Jamba gives a null body
> with invokeTextModel (line 509 reads {{completions}}, Jamba answers with
> {{choices}}), and the streaming of Jamba and Nova gives an empty text, as
> their chunks do not match the Titan and Claude parsers
> ({{StreamParserFactory}}, lines 65-72).
> _Claude Code on behalf of davsclaus_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
