nsivabalan opened a new pull request, #20575: URL: https://github.com/apache/hudi/pull/20575
### Describe the issue this Pull Request addresses Add the RFC-117 document, *Unified Packaged Lakehouse* (number claimed in #20574). Users want to connect LLM agents such as Claude and ChatGPT to their lakehouse, ask questions in natural language, and get correct answers computed from their own Hudi tables. Most of the building blocks already exist in the repo: the Hudi agent gateway (#19265), the Spark SQL Connect endpoint (#20243), the gateway's Spark SQL Connect backend (#20329), BI integrations (#20467), and quickstart validation (#20462). What's still missing is a semantic layer, a defined catalog interface, and an installation simple enough that users without a JDK, Maven, or Kubernetes can get a working lakehouse with one command. Related discussion: https://github.com/apache/hudi/discussions/19843 ### Summary and Changelog - Add `rfc/rfc-117/rfc-117.md`, which covers: - **Requirements:** a natural-language prompt goes through the data catalog and the semantic layer, becomes SQL, runs on Spark SQL Connect, and the answer comes back. Includes worked examples on a ride-hailing `trips` dataset. - **Architecture:** building blocks (storage layer; a gateway with Spark SQL Connect, data catalog, semantic layer, and a future governance layer), the request flow, and the gateway tool surface. - **Installation:** three tiers (one-command Docker Compose, local Kubernetes, production Helm), plus the packaging and release work needed so users never install a JVM or build from source. - **Data catalog:** the options (HMS, Glue, Gravitino, Unity Catalog OSS, Polaris, DataHub, OpenMetadata) with what each serves and doesn't, plus pros and cons. No option is chosen yet. - **Semantic layer:** consumes pre-built models only (generating models is out of scope); MetricFlow vs. Cube Core. - **Governance** (future) and **Spark job submission** (stretch). - Rollout phases, test plan, and open questions. - Link the RFC-117 entry in `rfc/README.md` to the document. ### Impact Documentation only. No runtime, API, storage format, or configuration changes. ### Risk Level none ### Documentation Update This PR adds the RFC document. Opened as a draft to gather early feedback on scope and the open questions. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable (not applicable: documentation-only change) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
