[
https://issues.apache.org/jira/browse/CAMEL-25541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Mariani updated CAMEL-25541:
-------------------------------------
Description:
Three findings from an evaluation of coding agents (Claude Code and IBM Bob) on
Mule-to-Camel migration tasks with camel-jbang-mcp 4.22.1 concern the same
answer, the component branch of {{camel_catalog_doc}}
({{dsl/camel-jbang/camel-jbang-core/.../commands/ai/CatalogDocs.java}}), and
are fixed together here. CAMEL-25540 and CAMEL-25542 were merged into this
issue.
h3. 1. The option list has no bound (15-35 K characters by default)
{{putOptions}} ({{CatalogDocs.java:1753-1781}}) adds every option the
{{OptionScope}} accepts; {{common}} only drops deprecated and advanced options,
so activemq still has 88 options (about 38 K of JSON), file 75 (35 K), smtp 76
(24 K), http 60 (19 K). Recorded sizes with 4.22.1:
{noformat}
camel_catalog_component_doc activemq 32 959 chars (includeOptions=all: 84
212, rejected by the client)
camel_catalog_component_doc file 31 423
camel_catalog_component_doc smtp 20 611
camel_component_properties activemq 123 893 "exceeds maximum allowed tokens"
camel_component_properties platform-http 14 996, called 35 times to find the
HTTP port, which is camel.server.* (CAMEL-25533)
{noformat}
121 of 192 sessions got at least one result over 20 000 characters; Claude Code
rejected 4 and wrote 48 to a file instead of showing them to the model.
h3. 2. Nothing matches, and the answer says nothing
25 of 81 calls with {{optionsFilter}} came back with {{"matchedOptions": 0}}
and empty lists, and the agents read that as "unsupported": on smtp with
{{optionsFilter: tls}} (15 sessions) several decided "no TLS support"; on xslt
with {{optionsFilter: param}} the agents concluded XSLT takes no parameters and
one moved to FreeMarker; on activemq {{redeliver}} found nothing. On main the
filter also searches descriptions, groups and labels, which does not close it:
smtp + {{tls}} returns only {{useJavaMailSessionPropertiesFromHeaders}}, the
option whose own description warns against this use, while the supported way
({{mail.smtp.starttls.enable=true}} as a URI parameter, which the catalog knows
as the multi-value prefix option {{additionalJavaMailProperties}}) is not
shown; xslt + {{param}} still matches nothing. (A third case,
{{schemaValidation}} with {{includeOptions: common}} finding nothing because
the option is {{advanced}}, is fixed on main: the filter now searches all
options.)
Names that still do not resolve on main: the page names {{camel_catalog_docs}}
returns ({{wireTap-eip}}, {{jackson-dataformat}}) end in "Artifact not found"
because the {{-eip}}/{{-component}}/{{-dataformat}}/{{-language}} suffix is not
stripped ({{catalogDoc}}, {{:187-300}}); {{retry}} resolves to
{{optimisticLockRetryPolicy}} by a title word ({{eipByTerm}}, {{:431-440}},
{{suggestEipNames(term, 1)}}) instead of pointing at the error handler;
{{camel_catalog_example_file}} with an unknown example answers "Example not
found: X" without the names ({{ExampleTools.java:113}}).
h3. 3. The page sections that decide correctness are never in the answer
{{docExcerpt}} ({{CatalogDocs.java:138}}, {{DOC_EXCERPT_BUDGET = 1400}})
carries the first {{==}} section only. Three behaviours that decide whether a
route is correct live further down the page and were never seen by any session:
* xslt: "By default, all headers and variables are added as parameters"
(section "Getting Usable Parameters into the XSLT", line 129 of 442). One
session rewrote the transformation in FreeMarker.
* mail: the Subject/From/To headers are ignored unless
{{useHeaderSubject}}/{{useHeaderFrom}}/{{useHeaderRecipients}} is true (section
"Opting out of header overrides", line ~300). 11 sessions sent a mail with an
empty subject. The option description itself is CAMEL-25534.
* platform-http under {{camel run}}: {{camel.server.port}} needs
{{camel.server.enabled=true}}. Only {{platform-http-main.adoc}} says so and
{{platform-http-component.adoc}} has no {{xref}} to it, so {{docPages}}
({{componentDocPages}}, {{:1432}}) is empty. 5 of 10 sessions on one task
listened on 8080 after 15-47 calls each.
h3. Fix
# *Bound the default answer*: without {{optionsFilter}}, return the required
options plus the first N common options (N about 25), with {{truncated: true}},
the total count and a hint ("optionsFilter=<word> finds one; includeOptions=all
with a limit pages the rest"). Alternatively names only (name, type, default)
when no filter is given, descriptions for filtered calls. Optionally a hard cap
on any catalog tool result (16 000 characters) with a truncation marker.
# *Say when nothing matches*: {{"note": "no option matches 'tls'"}} plus the
closest option names by edit distance (the catalog has the suggester since
CAMEL-24666) with their group; the multi-value prefix options of the component
({{additionalJavaMailProperties}}: "any mail.* URI parameter is passed to
JavaMail"), also when the filter matched something.
# *Make the page discoverable*: return the page's section titles ({{"sections":
[...]}}, with the ones whose text contains the filter term first) and a way to
fetch one ({{docPage=<section>}}), so a model that sees an empty filter can
still find the section that answers it. Optionally a short curated {{notes}}
list per component for defaults that silently change behaviour (mail
useHeader*, xslt header parameters), returned regardless of {{optionsFilter}}.
# *Resolve the names the tools themselves hand out*: strip a {{-component}},
{{-dataformat}}, {{-language}} or {{-eip}} suffix and look up again
(case-insensitive); do not resolve a bare word such as {{retry}} to an EIP by a
title word alone, return it as a suggestion with the error handler entries (or
add {{retry}}/{{redelivery}} as aliases of the error handler); list the closest
example names on "Example not found".
# *platform-http-component.adoc*: add a Camel Main / {{camel run}} entry to
"Platform HTTP Provider" (the first section, so it lands in the excerpt) with
{{camel.server.enabled=true}} + {{camel.server.port}}, and
{{xref:others:platform-http-main.adoc[...]}}, which {{componentDocPages}}
already turns into {{docPage=main}}.
# *Drop {{camel_component_properties}}* rather than add {{filter}}/{{limit}} to
it: its description says endpoint options "can be set as component-level
defaults via the same prefix", which is false
({{AbstractCamelCatalog.validateConfigurationProperty}} checks
{{camel.component.<scheme>.*}} against the component options only, {{:1221}}),
so the tool emits keys that {{camel_configuration_validate}} and {{camel run}}
reject. A {{configKey}} on each component option in {{camel_catalog_doc}}
covers the valid half.
Related: CAMEL-25539 (EIP/language answer: placement and YAML shapes),
CAMEL-25534, CAMEL-25533, CAMEL-25040 (the excerpt), CAMEL-25283
({{docPages}}), CAMEL-23475 (the scopes). Merged here: CAMEL-25540, CAMEL-25542.
_Claude Code on behalf of Croway_
was:
The catalog tools return the full option list of a component with no size
bound. For the components a migration typically needs, the default answer
(common options only) is 15-35 K characters, and {{camel_component_properties}}
returns 15-124 K. Agents ask these tools for one option name.
In an evaluation of coding agents (Claude Code and IBM Bob) on migration tasks
with camel-jbang-mcp 4.22.1, 121 of 192 sessions that used
{{camel_catalog_component_doc}}, {{camel_component_properties}} or
{{camel_catalog_language_doc}} got at least one result over 20 000 characters
(211 of 772 calls). Claude Code rejected 4 results outright and wrote 48 more
to a file instead of showing them to the model.
h3. Reproduction
Sizes of the recorded results (component name only; runtime main, camelVersion
4.22.1):
{noformat}
camel_catalog_component_doc activemq 32 959 chars
(includeOptions=all: 84 212, rejected by the client)
camel_catalog_component_doc file 31 423
camel_catalog_component_doc smtp 20 611
(includeOptions=all: 37 786)
camel_catalog_component_doc http / https 14 857 / 14 869
camel_component_properties activemq 123 893 "Error:
result (123,893 characters across 1 line) exceeds maximum allowed tokens"
camel_component_properties netty-http 76 218 rejected
the same way
camel_component_properties file / smtp / http 62 K / 61 K / 47 K
camel_component_properties platform-http 14 996
camel_catalog_language_doc simple about 70 K (152
functions with parameters; 7 options)
{noformat}
{{camel_component_properties platform-http}} was called 35 times (20 Claude
Code, 15 IBM Bob) to find the HTTP server port; each answer was 15 K of options
and none contained it (those are {{camel.server.*}}, see CAMEL-25533).
h3. Cause
* {{CatalogTools.toComponentDetailResult}} (4.22.1) and, on main where
{{camel_catalog_component_doc}} was folded into {{camel_catalog_doc}} by
CAMEL-24712, the component branch of {{CatalogDocs}} with {{putOptions}}
(camel-jbang-core, {{commands/ai/CatalogDocs.java}}) add every option the
{{OptionScope}} accepts. {{common}} only drops deprecated and advanced options:
on main activemq still has 88 common options (about 38 K of JSON), file 75 (35
K), smtp 76 (24 K), http 60 (19 K). CAMEL-23475 introduced the scopes with the
goal of a lean default (Kafka under 2K tokens); for these components the
default is still an order of magnitude larger. Nothing limits the count;
{{omittedOptions}} only reports what the scope dropped.
* {{ComponentPropertiesTools.camel_component_properties}} has no filter at all:
it returns every component option and every endpoint option, with descriptions,
javaType and enum lists. It is unchanged on main apart from the optional
arguments of #27511.
* {{CatalogTools.toLanguageDetailResult}} (4.22.1) returns all non-deprecated
functions of simple with their parameters. On main {{camel_catalog_doc}}
already returns the simple functions as names by group unless {{optionsFilter}}
is given, so this part is solved there; it is listed for the 4.22.x line only.
h3. Fix
* {{camel_catalog_doc}} (component, data format, language options): without
{{optionsFilter}}, return the required options plus the first N common options
(N around 25), and add {{truncated: true}} and the total count with a hint
("optionsFilter=<word> finds one; includeOptions=all with a limit pages the
rest"). Alternatively return option names only (name, type, default) when no
filter is given and keep descriptions for filtered calls.
* {{camel_component_properties}}: add {{filter}} and {{limit}} arguments with
the same default bound, and leave out the endpoint options unless asked
({{includeEndpointOptions}}), since only the component options are
{{camel.component.<scheme>.*}} keys. Or drop the tool and add the
component-level keys to {{camel_catalog_doc}} (for example
{{includeProperties=true}}).
* Optionally, a hard cap on any catalog tool result (for example 16 000
characters) with a truncation marker, so a client never rejects or silently
cuts the answer.
Summary: camel-jbang-mcp - camel_catalog_doc component answer: bounded
option lists, an explicit answer when nothing matches, and the page sections
that decide correctness (was: camel-jbang-mcp - catalog tools return unbounded
option lists: 15-35 K for a component by default, up to 124 K from
camel_component_properties)
> camel-jbang-mcp - camel_catalog_doc component answer: bounded option lists,
> an explicit answer when nothing matches, and the page sections that decide
> correctness
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25541
> URL: https://issues.apache.org/jira/browse/CAMEL-25541
> Project: Camel
> Issue Type: Improvement
> Components: camel-jbang-mcp
> Affects Versions: 4.22.1
> Reporter: Federico Mariani
> Priority: Major
>
> Three findings from an evaluation of coding agents (Claude Code and IBM Bob)
> on Mule-to-Camel migration tasks with camel-jbang-mcp 4.22.1 concern the same
> answer, the component branch of {{camel_catalog_doc}}
> ({{dsl/camel-jbang/camel-jbang-core/.../commands/ai/CatalogDocs.java}}), and
> are fixed together here. CAMEL-25540 and CAMEL-25542 were merged into this
> issue.
> h3. 1. The option list has no bound (15-35 K characters by default)
> {{putOptions}} ({{CatalogDocs.java:1753-1781}}) adds every option the
> {{OptionScope}} accepts; {{common}} only drops deprecated and advanced
> options, so activemq still has 88 options (about 38 K of JSON), file 75 (35
> K), smtp 76 (24 K), http 60 (19 K). Recorded sizes with 4.22.1:
> {noformat}
> camel_catalog_component_doc activemq 32 959 chars (includeOptions=all: 84
> 212, rejected by the client)
> camel_catalog_component_doc file 31 423
> camel_catalog_component_doc smtp 20 611
> camel_component_properties activemq 123 893 "exceeds maximum allowed
> tokens"
> camel_component_properties platform-http 14 996, called 35 times to find
> the HTTP port, which is camel.server.* (CAMEL-25533)
> {noformat}
> 121 of 192 sessions got at least one result over 20 000 characters; Claude
> Code rejected 4 and wrote 48 to a file instead of showing them to the model.
> h3. 2. Nothing matches, and the answer says nothing
> 25 of 81 calls with {{optionsFilter}} came back with {{"matchedOptions": 0}}
> and empty lists, and the agents read that as "unsupported": on smtp with
> {{optionsFilter: tls}} (15 sessions) several decided "no TLS support"; on
> xslt with {{optionsFilter: param}} the agents concluded XSLT takes no
> parameters and one moved to FreeMarker; on activemq {{redeliver}} found
> nothing. On main the filter also searches descriptions, groups and labels,
> which does not close it: smtp + {{tls}} returns only
> {{useJavaMailSessionPropertiesFromHeaders}}, the option whose own description
> warns against this use, while the supported way
> ({{mail.smtp.starttls.enable=true}} as a URI parameter, which the catalog
> knows as the multi-value prefix option {{additionalJavaMailProperties}}) is
> not shown; xslt + {{param}} still matches nothing. (A third case,
> {{schemaValidation}} with {{includeOptions: common}} finding nothing because
> the option is {{advanced}}, is fixed on main: the filter now searches all
> options.)
> Names that still do not resolve on main: the page names
> {{camel_catalog_docs}} returns ({{wireTap-eip}}, {{jackson-dataformat}}) end
> in "Artifact not found" because the
> {{-eip}}/{{-component}}/{{-dataformat}}/{{-language}} suffix is not stripped
> ({{catalogDoc}}, {{:187-300}}); {{retry}} resolves to
> {{optimisticLockRetryPolicy}} by a title word ({{eipByTerm}}, {{:431-440}},
> {{suggestEipNames(term, 1)}}) instead of pointing at the error handler;
> {{camel_catalog_example_file}} with an unknown example answers "Example not
> found: X" without the names ({{ExampleTools.java:113}}).
> h3. 3. The page sections that decide correctness are never in the answer
> {{docExcerpt}} ({{CatalogDocs.java:138}}, {{DOC_EXCERPT_BUDGET = 1400}})
> carries the first {{==}} section only. Three behaviours that decide whether a
> route is correct live further down the page and were never seen by any
> session:
> * xslt: "By default, all headers and variables are added as parameters"
> (section "Getting Usable Parameters into the XSLT", line 129 of 442). One
> session rewrote the transformation in FreeMarker.
> * mail: the Subject/From/To headers are ignored unless
> {{useHeaderSubject}}/{{useHeaderFrom}}/{{useHeaderRecipients}} is true
> (section "Opting out of header overrides", line ~300). 11 sessions sent a
> mail with an empty subject. The option description itself is CAMEL-25534.
> * platform-http under {{camel run}}: {{camel.server.port}} needs
> {{camel.server.enabled=true}}. Only {{platform-http-main.adoc}} says so and
> {{platform-http-component.adoc}} has no {{xref}} to it, so {{docPages}}
> ({{componentDocPages}}, {{:1432}}) is empty. 5 of 10 sessions on one task
> listened on 8080 after 15-47 calls each.
> h3. Fix
> # *Bound the default answer*: without {{optionsFilter}}, return the required
> options plus the first N common options (N about 25), with {{truncated:
> true}}, the total count and a hint ("optionsFilter=<word> finds one;
> includeOptions=all with a limit pages the rest"). Alternatively names only
> (name, type, default) when no filter is given, descriptions for filtered
> calls. Optionally a hard cap on any catalog tool result (16 000 characters)
> with a truncation marker.
> # *Say when nothing matches*: {{"note": "no option matches 'tls'"}} plus the
> closest option names by edit distance (the catalog has the suggester since
> CAMEL-24666) with their group; the multi-value prefix options of the
> component ({{additionalJavaMailProperties}}: "any mail.* URI parameter is
> passed to JavaMail"), also when the filter matched something.
> # *Make the page discoverable*: return the page's section titles
> ({{"sections": [...]}}, with the ones whose text contains the filter term
> first) and a way to fetch one ({{docPage=<section>}}), so a model that sees
> an empty filter can still find the section that answers it. Optionally a
> short curated {{notes}} list per component for defaults that silently change
> behaviour (mail useHeader*, xslt header parameters), returned regardless of
> {{optionsFilter}}.
> # *Resolve the names the tools themselves hand out*: strip a {{-component}},
> {{-dataformat}}, {{-language}} or {{-eip}} suffix and look up again
> (case-insensitive); do not resolve a bare word such as {{retry}} to an EIP by
> a title word alone, return it as a suggestion with the error handler entries
> (or add {{retry}}/{{redelivery}} as aliases of the error handler); list the
> closest example names on "Example not found".
> # *platform-http-component.adoc*: add a Camel Main / {{camel run}} entry to
> "Platform HTTP Provider" (the first section, so it lands in the excerpt) with
> {{camel.server.enabled=true}} + {{camel.server.port}}, and
> {{xref:others:platform-http-main.adoc[...]}}, which {{componentDocPages}}
> already turns into {{docPage=main}}.
> # *Drop {{camel_component_properties}}* rather than add {{filter}}/{{limit}}
> to it: its description says endpoint options "can be set as component-level
> defaults via the same prefix", which is false
> ({{AbstractCamelCatalog.validateConfigurationProperty}} checks
> {{camel.component.<scheme>.*}} against the component options only,
> {{:1221}}), so the tool emits keys that {{camel_configuration_validate}} and
> {{camel run}} reject. A {{configKey}} on each component option in
> {{camel_catalog_doc}} covers the valid half.
> Related: CAMEL-25539 (EIP/language answer: placement and YAML shapes),
> CAMEL-25534, CAMEL-25533, CAMEL-25040 (the excerpt), CAMEL-25283
> ({{docPages}}), CAMEL-23475 (the scopes). Merged here: CAMEL-25540,
> CAMEL-25542.
> _Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
