nsivabalan opened a new pull request, #20195: URL: https://github.com/apache/hudi/pull/20195
### Describe the issue this Pull Request addresses Part of #19262 (Hudi-aware agent skills and tools), under the Agentic Lakehouse umbrella #19256; overall plan in https://github.com/apache/hudi/issues/19262#issuecomment-5750570015. Hudi has a lot of configuration — this change finds 939 distinct `hoodie.*` keys across the source tree. The website's configuration page lists key, default and description for each, which answers "what is this?" but not the questions users actually get stuck on: *where is this read?*, *why did setting it change nothing?*, and *what else do I need to set alongside it?* That last class is the painful one. `hoodie.compact.inline.max.delta.commits` is only consulted under some values of `hoodie.compact.inline.trigger.strategy`; set it under `TIME_ELAPSED` and it is silently ignored. Nothing in the prose tells you which configs behave this way, and finding out means reading the source. ### Summary and Changelog **`scripts/generate_config_catalog.py`** (1421 lines, Python 3 standard library only) — walks the source tree, parses every `ConfigProperty` builder chain, and emits a JSON catalog. Per config it records the declared facts (key, default, type, documentation, valid values, alternatives, `sinceVersion`, `deprecatedAfter`, advanced flag, infer-function presence, declaring file and line) and then enriches them from the code: - **Read sites** — the accessor methods that consume the config and the call sites of those accessors, each as `class.method (file:line)`, so "where does this actually take effect" has an answer you can open. - **Gating conditions** — the `switch` cases and `if` guards enclosing a read. This is what surfaces "only applies when X". - **Co-configs** — other configs read in the same method, which is a decent proxy for "what do I need to set alongside this". Results against this tree: 1137 declarations, **939 distinct configs**, 881 fully parsed (93.8%), 4438 read sites across 915 configs, 774 configs with co-configs, 301 with gating conditions. The 58 configs with warnings are all enrichment gaps — every key and every alternative resolved to a literal string. The residue is concentrated in source and sync configs read reflectively, which no static pass will follow. Output is deterministic and sorted, so regenerating against a new release produces a readable diff. A worked check on the gating heuristic: for `hoodie.compact.inline.max.delta.commits` it recovers `NUM_COMMITS`, `NUM_COMMITS_AFTER_LAST_REQUEST`, `NUM_OR_TIME` and `NUM_AND_TIME` from the `switch` in `ScheduleCompactionActionExecutor.needCompact` — exactly the four strategies the config's own documentation names, derived independently from code. That cross-check is in the schema doc precisely because it is falsifiable. **`hudi-agent-gateway/skills/hudi-config-consultant/`** — a skill that answers config questions from the catalog: what a config does, what to set for a goal, why a config appears to do nothing, how two configs differ, and what must be set alongside one. It cites `file:line` so answers can be verified, states the Hudi version the catalog was generated from, and will say a key is unknown rather than invent one. The committed `config-catalog.json` is generated output. It is checked in so the skill works on arrival — a user running a released Hudi from Maven has no source tree to generate from, and a skill that answers nothing until someone runs a build is a worse skill. The generator ships alongside it for anyone on a custom build. If reviewers would rather not carry the artifact, generating at install time is a small change. ### Impact Purely additive. No Java, no configs, no behaviour change. The script is not wired into the build and nothing imports it. ### Risk Level none The generator is read-only over the source tree and is run by hand. ### Documentation Update The skill documents itself. The catalog is derived from the same `ConfigProperty` declarations the website's configuration page is generated from, so it stays consistent with the docs by construction. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
