cshuo opened a new issue, #20466: URL: https://github.com/apache/hudi/issues/20466
### Task Description Provide a reproducible Spark Connect integration for Superset and Zeppelin BI users, as part of the work discussed in #19843. ### Scope - Package the initial experimental Superset SQLAlchemy dialect and engine spec under `hudi-lakehouse`, using Foundry Spark ADBC; document pinned dependencies and connection setup. Evaluate upstreaming separately. - Support Superset SQL Lab queries and direct INSERT, UPDATE and DELETE, including statements without result sets, with readback. Document enabling Allow DDL and DML on a disposable connection. - Provide a Zeppelin 0.12 `%python` example using ADBC for queries, INSERT, UPDATE and DELETE, readback and result rendering. Native `%jdbc` support is outside this scope. - Provide application-level checks, a dashboard example and explicit manual steps to verify refresh after an independent write. - Document tested versions, installation, client connectivity, cleanup and limitations. Keep the gateway's read-only behavior distinct from direct BI client writes. ### Acceptance Criteria - [ ] Superset can connect to a Helm-deployed Spark Connect endpoint, query a Hudi table, perform direct INSERT/UPDATE/DELETE and read back the results. - [ ] A Superset chart or dashboard can be created and its results refreshed after a write. - [ ] Zeppelin can query and write through ADBC, read back the results, and render them in a notebook. - [ ] Examples provide reproducible client dependencies and configuration without modifying upstream driver binaries; test data is isolated and safely cleaned up. - [ ] Document ADBC cancellation, error and retry limitations without implying that a client error proves a server-side write was rolled back. ### Task Type Other — BI client integrations and examples. ### Related Issues - #20231 — Helm-deployable Spark Connect endpoint, a prerequisite. - #20232 — Agent and MCP integration through the read-only gateway. - #20233 — Quickstart, end-to-end validation, and operational readiness. Draft #20462 currently includes both BI and validation changes; its scope may be adjusted as the work is split. Production-scale concurrency, security, authentication and broader JDBC compatibility are follow-up work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
