timsaucer opened a new pull request, #22003:
URL: https://github.com/apache/datafusion/pull/22003
## Which issue does this PR close?
Tracked under apache/datafusion-python#1394 ("Make it easier for agents to
generate datafusion-python code") — this is **PR 6** in the implementation plan
([comment](https://github.com/apache/datafusion-python/issues/1394#issuecomment-4252413645)):
the upstream `apache/datafusion` `llms.txt` hub.
- Closes #.
## Rationale for this change
[llms.txt](https://llmstxt.org) is an emerging convention for exposing a
machine-readable, agent-facing entry point at a site's docs root. Subprojects
in the DataFusion ecosystem are starting to publish their own
(`apache/datafusion-python` PR #1505 added one). The main
`datafusion.apache.org` site is the natural top-level discovery point for the
whole ecosystem, so it should expose a hub `llms.txt` that points agents at:
- the core DataFusion (Rust) user / library / contributor guides and Rust
API docs,
- each subproject's docs root, where agents following the llmstxt.org
convention can probe `<docs root>/llms.txt` for project-specific guidance.
Net effect: an agent fetching `https://datafusion.apache.org/llms.txt` lands
in a categorized directory of the entire ecosystem's agent guidance.
## What changes are included in this PR?
- `docs/source/llms.txt` — new file, llmstxt.org schema. Sections: Core
DataFusion (Rust), Subprojects, Optional. The Subprojects section links to docs
roots (not pending `llms.txt` URLs) and includes a one-line note describing the
probe convention so the hub stays correct as subprojects ship their own files.
- `docs/source/conf.py` — `html_extra_path = ["llms.txt"]` so Sphinx copies
the file verbatim to the build output root, served at
`https://datafusion.apache.org/llms.txt`.
- `dev/release/rat_exclude_files.txt` — exclude `docs/source/llms.txt` from
the RAT license-header check (the file body is rendered markdown and cannot
carry the standard `..` comment header without breaking the format).
## Are these changes tested?
No automated tests. The change is a single static file plus a Sphinx config
line that mirrors a pattern already used in `apache/datafusion-python`
(`html_extra_path = ["llms.txt"]`, PR #1505). Verification will be done at
deploy time: confirm `https://datafusion.apache.org/llms.txt` resolves and
renders.
## Are there any user-facing changes?
Yes — adds a new public URL `https://datafusion.apache.org/llms.txt`. No
existing pages are modified. No API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
