cshuo opened a new pull request, #20329: URL: https://github.com/apache/hudi/pull/20329
### Describe the issue this Pull Request addresses Implements the gateway integration portion of [apache/hudi#20232](https://github.com/apache/hudi/issues/20232). **Depends on [apache/hudi#20243](https://github.com/apache/hudi/pull/20243), which must merge first.** This branch currently contains two commits: the prerequisite endpoint commit and the gateway integration commit. Review gateway changes in the second commit, `4a53e69d0834`. The gateway uses the official PySpark Connect client instead of the issue's proposed ADBC client. Packaged Superset/Zeppelin examples remain outside this PR, so the issue stays open. ### Summary and Changelog - Add a selectable Spark Connect SQL backend with shared `list_tables`, `describe_table`, and `query_lakehouse` tools for Agent and MCP, preserving read-only guardrails. - Add bounded results, query timeouts, best-effort cancellation, session cleanup, chat deadlines, and UI progress/cancellation. - Add backend-aware metadata/readiness, pinned client dependencies, Helm settings, and regression coverage. ### Impact Spark Connect is opt-in through `GATEWAY_SQL_BACKEND=spark-connect` or Helm `sqlBackend`. Trino remains the default. Tool interfaces remain unchanged; Spark metadata uses Spark SQL type names. ### Risk Level Medium: shared SQL dispatch and chat lifecycle paths change. Validation: gateway tests **352 passed, 6 skipped**; lakehouse/chart tests **38 passed**. Live external integration tests were not rerun after rebase. Cancellation remains best-effort. ### Documentation Update Updated gateway and lakehouse READMEs with setup, Helm configuration, client requirements, and timeout/result-limit behavior. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
