Abhishekmishra2808 opened a new pull request, #63750:
URL: https://github.com/apache/airflow/pull/63750
## Summary
This PR adds a focused Proof of Concept under `agent_skills_poc/` for
generating AI-agent skills from executable documentation.
The goal is to keep contributor docs as the source of truth and produce
machine-readable `skills.json` that agents consume at runtime.
## Problem
Airflow contributor workflows are not generic Python workflows.
Agents often run incorrect commands like plain `pytest` instead of
Airflow-specific commands (`uv run --project ...`, `breeze ...`).
This PoC demonstrates a reliable pipeline from docs to runtime command
resolution.
## What This PR Adds
1. `.. agent-skill::` parsing from `docs/CONTRIBUTING_POC.rst` via
`docutils`.
2. Validated workflow model and `skills.json` generation.
3. Runtime context detection (`host` vs `breeze-container`).
4. Command resolution from generated `skills.json` using environment-aware
logic.
5. CI guard test to fail when docs and committed `skills.json` diverge.
6. Real benchmarks for parsing, generation, context detection, and command
resolution.
7. One realistic Airflow workflow example (`run_single_core_test`) in
executable docs.
## Architecture
```mermaid
flowchart TD
A[CONTRIBUTING_POC.rst] --> B[docutils parser]
B --> C[Validated Workflow model]
C --> D[skills.json generator]
D --> E[output/skills.json]
E --> F[Context detection]
F --> G[Command resolver]
G --> H[Agent executes resolved command]
```
## Runtime Command Selection
```mermaid
flowchart TD
A["resolve_command skill_id"] --> B["Load skills.json"]
B --> C["Detect context"]
C --> D{Environment}
D -->|host| E["Prefer local step"]
D -->|breeze-container| F["Use local step"]
E --> G{"Local step exists"}
G -->|yes| H["Return local command"]
G -->|no| I["Return fallback command"]
F --> H
```
## Validation Performed
### Tests
Command:
`uv run --project scripts --with docutils pytest agent_skills_poc/tests -xvs`
Result:
`20 passed`
### Docs/Skills Sync Guard
Command:
`uv run --project scripts --with docutils pytest
agent_skills_poc/tests/test_sync.py -xvs`
Result:
`test_docs_and_committed_skills_are_in_sync PASSED`
### Benchmarks (real local measurements)
Parser benchmark command:
`uv run --project scripts --with docutils python
agent_skills_poc/benchmarks/benchmark_parser.py`
Resolver benchmark command:
`uv run --project scripts --with docutils python
agent_skills_poc/benchmarks/benchmark_resolver.py`
Observed outputs:
- `Workflows: 100`
- `Parsing time: 159.624 ms`
- `Skill generation time: 0.276 ms`
- `Context detection time: 0.028 ms`
- `Command resolution time: 0.569 ms`
- `Resolver latency (dedicated): 0.571 ms`
## Screenshot Placeholders
### Screenshot 1: Full PoC test pass (`20 passed`)
<img width="1472" height="748" alt="image"
src="https://github.com/user-attachments/assets/3bf53e51-2ae6-4c69-9961-281e4fb5e1d1";
/>
### Screenshot 2: Sync guard test pass
<img width="1274" height="307" alt="image"
src="https://github.com/user-attachments/assets/1735d117-45c1-4600-be92-272dc027ebcf";
/>
### Screenshot 3: Generated `skills.json` includes `run_single_core_test`
<img width="1216" height="668" alt="image"
src="https://github.com/user-attachments/assets/d7aadab7-8da2-47a9-9aba-dc44c1a8c0e9";
/>
<img width="1041" height="371" alt="image"
src="https://github.com/user-attachments/assets/c67f098b-9712-44aa-9e4c-35430852137a";
/>
### Screenshot 4: Benchmarks output (parser + resolver)
<img width="1208" height="504" alt="image"
src="https://github.com/user-attachments/assets/306e5d4c-092e-431c-a94e-8e66cb8c0c8b";
/>
PARSER
<img width="1238" height="312" alt="image"
src="https://github.com/user-attachments/assets/019f8b0b-83c8-487d-905c-6335b34c9647";
/>
RESOLVER
## Scope
This PR is intentionally limited to:
`agent_skills_poc/`
No changes were made to core Airflow runtime paths.
## Generative AI Disclosure
- [X] Yes — GitHub Copilot (Claude Opus 4.6- thinking)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
