davsclaus opened a new issue, #1746: URL: https://github.com/apache/camel-website/issues/1746
## Summary `llms.txt` promises that every documentation page is available as Markdown by swapping `.html` for `.md`, but 35 documentation pages on the live site have no `.md` twin and return 404. Several of them are linked directly from `llms.txt`, e.g. https://camel.apache.org/manual/camel-jbang-mcp.md. The HTML-to-Markdown step (`gulp/tasks/generate-markdown.js`) silently skips these pages. The last Jenkins build log shows: ``` Found 5523 HTML files to convert Successfully generated 5485 Markdown files ``` with no error lines. 3 of the 38 skipped files are expected non-doc pages (`index.html`, `schema/HEADER.html`, the Google verification page). The other 35 are real documentation pages. ## Affected pages (published tree of `camel-website-pub`, `asf-site`, 2026-09-12) Manual: - `manual/camel-jbang-mcp` - `manual/camel-jbang-tui` - `manual/security-model` - `manual/camel-4x-upgrade-guide-4_14`, `4_18`, `4_19`, `4_21`, `4_22`, `4_23` Components and sub-projects: - `components/{next,4.22.x,4.18.x}/mail-component` - `components/{next,4.22.x}/others/yaml-dsl` - `components/{next,4.22.x}/ai-tool-component` - `components/next/opa-component` - `components/{next,4.22.x}/others/mcp-server` - `components/{next,4.22.x,4.18.x}/others/leveldb` - `components/{next,4.22.x}/others/mdc` - `camel-spring-boot/{next,4.22.x}/starters/micrometer` - `camel-quarkus/{next,3.33.x}/reference/extensions/jolokia` - `camel-quarkus/next/reference/extensions/micrometer` - `camel-quarkus/next/reference/extensions/mcp-server` - `camel-quarkus/next/migration-guide/3.39.0`, `camel-quarkus/3.33.x/migration-guide/3.33.3` - `camel-kamelets/{next,4.22.x}/security-model` - `camel-kafka-connector/next/security-model` - `camel-karaf/4.9.x/security-model` ## Root cause The affected pages contain mis-nested inline HTML produced by AsciiDoc. A `*` wildcard inside backticks is interpreted as a bold marker, so for example `security-model.adoc` renders: ```html <code>core/camel-<strong></code> module is judged against these ... </strong> ``` `camel-jbang-mcp.adoc` line 189 has a related glitch where the closing backtick inside `_"..."_` is not recognised and the `<code>` runs on into the next sentence. Browsers tolerate this. `node-html-parser` does not: it leaves the unmatched tags on its open-element stack, and at end of file it unwraps every unclosed ancestor, including `<article class="doc">`. The converter then hits the silent `if (!mainContent) continue;` branch and skips the page. Page size is a red herring; large pages simply have more chances to contain a wildcard. Minified HTML parses fine because `html-minifier` repairs the nesting, but `build:markdown` runs before `build:minify`, so the production build never benefits. Reproduction on a local Antora build: with the default parser 19 of 4762 pages lose their `article.doc`; with `parseNoneClosedTags: true` none do, and the resulting Markdown is sane. ```js const { parse } = require('node-html-parser'); const html = '<html><body><article class="doc"><p>A <code>core/camel-<strong></code> module</strong> x</p></article></body></html>'; parse(html).querySelector('article.doc'); // null parse(html, { parseNoneClosedTags: true }).querySelector('article.doc'); // found ``` ## Proposed fix 1. In `gulp/tasks/generate-markdown.js`, parse with `parseNoneClosedTags: true`: ```js const root = parse(htmlContent, { parseNoneClosedTags: true }); ``` 2. Replace the silent `continue` for pages without main content with a logged warning naming the file, so the next regression shows up in the Jenkins log instead of as 404s. 3. Optionally, in the source repos, escape the wildcards in the `.adoc` files (`+Camel*+` or `Camel\*`) since the rendered HTML is invalid either way. That is a separate change in `apache/camel` and the sub-project repos; the parser option is what makes the `.md` mirror reliable regardless of content. --- _Claude Code on behalf of davsclaus_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
