[
https://issues.apache.org/jira/browse/CAMEL-26411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137894#comment-18137894
]
Claus Ibsen commented on CAMEL-26411:
-------------------------------------
[~fmariani] two findings for camel_eval_expression from the local-model
benchmark (qwen3.6:35b on Ollama, run r26 on Nov 3-4, 7 rounds):
# *A SpEL filter inside Simple evaluates to "ok, false".* rest-code-first step
1, run r26n-4: the model sent ${bodyAs(java.util.List).?[#orderId ==
'ORD-1001'].size()} > 0 eleven times. Each answer was "status ok, result
false", so it never learned that the syntax is wrong, and the step ended at the
call cap with no write. The write-time validator refuses the same expression
(CAMEL-26484: SpEL or JsonPath syntax inside Simple). The eval answer could
give the same message.
# *${header.CamelRedeliveryCounter} is null at eval time.*
fail-well-error-handling step 2, run r26l-2 (and r25l-1, r24l-7): the eval
answered nullResult, then the model made 11 Exchange API lookups and wrote
nothing. A note that the header is set only while the error handler redelivers,
so it is null in an eval, would have ended the loop.
The eval tool also has no note for repeated identical calls. The catalog tools
answer "already called" from the third one.
> camel-jbang - AI tools: camel_eval_expression evaluates a JSON body as parsed
> data, and its answers name a missing body or an unset header
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26411
> URL: https://issues.apache.org/jira/browse/CAMEL-26411
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang, camel-jbang-mcp
> Reporter: Claus Ibsen
> Priority: Minor
> Fix For: 4.24.0
>
>
> A follow-up to CAMEL-26400, from the local-model benchmark (qwen3.6:35b on
> Ollama, run r24 on Oct 29-30 2026).
> The model used camel_eval_expression more than before: 63 calls, against 24
> in the run before. No step of the series failed because of CAMEL-26400. But
> some answers sent the model the wrong way:
> * *A JSON body is text.* With the JSON passed as body,
> {{$\{body[lines].size()\}}} fails with "the body is text (a String):
> unmarshal it first (unmarshal: json) to read its fields". That is right for a
> route, but the model cannot unmarshal inside the tool, so it cannot test the
> expression the way the route will see the body after unmarshal: json. In
> route-filter-and-multicast step 2, it sent the same call 10 times in a row.
> * *body.lines on text.* {{$\{body.lines.size()\}}} on a JSON text body calls
> String.lines(), and the answer is "Method with name: size() not found on
> bean: java.util.stream.ReferencePipeline$Head".
> * *jsonpath with no body.* The answer is "java.lang.IllegalArgumentException:
> json string can not be null or empty" (3 calls). It does not say that the
> JSON goes in body.
> * *No body is empty text, not null.* {{$\{body} == null}} answers false, also
> with "body": null. A route before any step has a null body.
> * *An unset Camel header.* {{$\{header.CamelRedeliveryCounter\}}} gives
> nullResult with the note "a header, property or variable that is not set".
> The model then tried {{$\{exchangeProperty.CamelRedeliveryCounter\}}}, which
> is also null. The note could say that the header is not set in the test
> exchange, and that the headers argument sets it.
> Proposal:
> * A way to evaluate against a JSON body as parsed data (a Map or a List, as
> after unmarshal: json), for example when the expression reads fields of a
> body that is JSON text. The answer says that it did so.
> * Name the missing body for jsonpath (and the other languages that need one).
> * Evaluate with a null body when no body is given, or say in the note that
> the body is empty text.
> * For an unset header, point to the headers argument.
> * Note an identical repeated call ("same call as before, same answer").
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
