TomShawn opened a new pull request, #392: URL: https://github.com/apache/cloudberry-site/pull/392
<!--Thank you for contributing! --> <!--In case of an existing issue or discussions, please reference it--> closes: #ISSUE_Number <!--Remove this section if no corresponding issue.--> --- ## Change logs Appending `.md` to any page URL now returns clean Markdown, and both readers and crawlers have a way to find it. 1. **Build-time Markdown export** (`src/plugins/markdown-export/`) — every doc, blog post and Markdown page gets a plain-Markdown twin: `/docs/introduction/cbdb-overview` → `/docs/introduction/cbdb-overview.md`. 1146 files, 8.6 MB. Nothing about the rendered site changes. 2. **A "Copy page" menu** on every doc, PXF and blog page — copy the Markdown, view the source, or hand the page to Claude or ChatGPT as context. 3. **Discovery** — `<link rel="alternate" type="text/markdown">` in each page's head, plus the `.md` URLs in `sitemap.xml`. Without this nothing can find the files: the menu is behind an `open &&` guard, so its links never reach the server-rendered HTML. ### Why Readers increasingly paste a docs URL into an AI assistant, which today spends most of the context window on markup. Across the 503 released doc pages: | | HTML | Markdown | | |---|---|---|---| | Total | 38.4 MB | 3.5 MB | **9.1%** | | `sql-stmts/create-table` | 435 KB | 81 KB | 18.6% | | `sql-stmts/close` | 79 KB | 1.6 KB | 2.0% | The 50× gap on short reference pages is typical — a 79 KB page carries 1.6 KB of content. The whole `sql-stmts` set is now 1.73 MB, small enough to hand over at once instead of being truncated. The twins come from the Markdown source, not from scraping the rendered HTML, so tables, admonitions and code samples survive verbatim — including cases an HTML-to-Markdown converter mangles, like shell samples with `export VAR=...` or pg_filedump output containing literal `<Header>` markers. ### Cost - **Build output +8.6 MB** (the `docs` tree is already 141 MB). `asf-site` is content-addressed, so an unchanged `.md` produces no new object: roughly 1.2 MB in git once, then ~0.44 MB/year of churn. - **`sitemap.xml` grows 1456 → 2086 entries.** Every page appears twice, and ASF's static hosting gives us no way to send `X-Robots-Tag: noindex` on the Markdown half. Duplicate-content risk looks low (non-HTML `text/*` is generally not indexed) but is real — deleting the `sitemap` block in `docusaurus.config.ts` returns to HTML-only. - **A sanitiser to keep current with**: a new MDX component may need an entry in `STRUCTURAL_TAGS` / `OPAQUE_TAGS`. The plugin warns at build time when a component leaks into the output, deriving the list from each file's own imports so new components are covered automatically. ### Deliberate omissions - **`docs/1.x` skipped entirely** — legacy, not worth the weight on every `asf-site` commit. Its HTML is untouched. - **`docs/next` gets the menu but stays out of `sitemap.xml`** — a contributor reading the dev docs should get the dev docs; a crawler should not answer user questions out of an unreleased version. 491 of its 516 pages are byte-identical to 2.x anyway. - **No `llms.txt`** — Ahrefs' server-log study of 137,000 domains found 97% of `llms.txt` files get zero requests, AI bots never probe for the path, and no major vendor has committed to reading it. `sitemap.xml` is the one discovery file crawlers demonstrably fetch. ### Verification 1146 twins served as `text/markdown; charset=utf-8`, 0 failures, 0 residual component markup. `sitemap.xml` has 2086 entries, none pointing at a missing file and none from `docs/next` or `docs/1.x`. Menu and head link present on docs 2.x, docs/next, PXF, blog and Markdown pages; absent on docs/1.x, blog list pages and `.tsx` pages. Layout checked at 375–1920 px: the dropdown stays inside the viewport, and article and TOC widths are unchanged from `main`. ## Contributor's checklist Here are some reminders before you submit your pull request: * [x] Make sure that your Pull Request has a clear title and commit message. You can take the [Git commit template](https://github.com/apache/cloudberry/blob/main/.gitmessage) as a reference. * [x] Learn the [code contribution](https://cloudberry.apache.org/contribute/code) and [doc contribution](https://cloudberry.apache.org/contribute/doc) guides for better collaboration. * [x] Make sure that your changes deployment preview is successful. * [ ] List your communications in the [GitHub Issues](https://github.com/apache/cloudberry-site/issues) or [Discussions](https://github.com/apache/cloudberry/discussions) (if has or needed). * [ ] Feel free to ask for the [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers) or other people to help review and approve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
