Federico Mariani created CAMEL-26401:
----------------------------------------
Summary: camel-jbang-mcp - camel_eval_expression: an elvis
operator written inside ${ } fails with no hint, or without its space evaluates
to null and is reported as ok with no result
Key: CAMEL-26401
URL: https://issues.apache.org/jira/browse/CAMEL-26401
Project: Camel
Issue Type: Improvement
Components: camel-jbang, camel-jbang-mcp
Affects Versions: 4.23.0
Reporter: Federico Mariani
Assignee: Federico Mariani
Found on main (4.24.0-SNAPSHOT, commit ffef175654d1); the line numbers below
are from that commit. The same code is in the 4.23.0 release.
h3. Reproduction
Since CAMEL-24921 a ternary can go inside a function ({{${header.n == 1 ? 'a' :
'b'}}}), and models write the elvis operator in the same place:
{code}
camel_eval_expression
{"language":"simple","body":"x","expression":"${header.missing ?: 10}"}
-> ${ cannot accept ?: at location 17
camel_eval_expression
{"language":"simple","body":"x","expression":"${header.missing ?:10}"}
-> {"language":"simple","expression":"${header.missing
?:10}","evaluatedIn":"the running integration (pid 3562)","status":"ok"}
camel_eval_expression
{"language":"simple","body":"x","expression":"${header.missing} ?: '10'"}
-> {"language":"simple",...,"status":"ok","result":"10"}
{code}
The second call is the costly one. It reports {{"status":"ok"}}, and since the
value is null the answer has no {{result}} key at all. The model read that as
"this form works": it rewrote its SQL file to {{LIMIT :#${header.offset ?:0},
:#${header.limit ?:10}}}, restarted the app, and got {{BadSqlGrammarException}}
on the next request. It then spent the next 18 calls reading camel-sql sources
from GitHub and trying more forms, and finally landed on {{${header.limit} ?:
10}}, which the tool description gives as its example ({{e.g. ${random(1,10)}
or ${body} ?: 'none'}}).
Eval evidence (200 trials on 4.24.0-SNAPSHOT): 5 failed calls with {{?:}}
inside {{${ }}} (GLM 5.3: 4, DeepSeek V4.1 Flash: 1), plus 3 calls of the
no-space form answered {{status: ok}} with no result (GLM 5.3, one trial; the
detour described above).
h3. Cause
* The simple parser rejects {{?:}} inside a function
({{core/camel-core-languages/.../BaseSimpleParser.java:180}}: "${ cannot accept
?:"). Without the space after it, the text is not read as an operator and the
expression evaluates to null without an error.
* {{ExpressionEvaluator}} passes the parser's message on as it is, and adds no
hint in the in-process path
({{camel-jbang-core/.../commands/ai/ExpressionEvaluator.java:122-153}};
{{syntaxCheck}} runs only for local evaluation, {{:153-187}}).
* When the value is null, the result has no {{result}} key
({{ExpressionEvaluator.java:146}}: {{result.put("result",
out.get("result"))}}), so "null" and "nothing to say" look the same.
h3. Fix
* In {{ExpressionEvaluator}} (both paths) and in the Simple checks of the
validator ({{SimpleChecks}}): when a {{${...}}} function contains {{?:}},
answer with the expression rewritten, as CAMEL-26316 does for the ternary:
"{{${header.missing ?: 10}}} -> {{${header.missing} ?: 10}}: the elvis operator
goes after the function". Do this on error and also when the evaluation returns
null.
* Always write {{"result": null}} for a null value, with a note ("evaluated to
null: a header, property or variable that is not set, or a function name that
includes the rest of the text"), so a model does not take a null for success.
* Possibly, in camel-core: since a ternary works inside a function
(CAMEL-24921), accept the elvis operator there too, which would make the hint
unnecessary. Up to the simple language maintainers.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
