voonhous opened a new pull request, #19217: URL: https://github.com/apache/hudi/pull/19217
Stacked on #19298 (-> #19295 -> #19288 -> #18837) -- only the E2E harness/pipeline commits are new here. Merges last: the pipeline needs the connector fixes below it to pass. ### Describe the issue this Pull Request addresses The RFC-105 connector (#18837) had no end-to-end coverage against a real Trino server. This adds a testcontainers E2E pipeline: the full demo stack (HDFS/Hive/Spark) plus a Trino 481 coordinator running the connector built at HEAD, with tables seeded by Spark using DEFAULT configs and queried through the Trino CLI. Seeding with defaults leaves the metadata table with uncompacted HFILE deltas, so every query here exercises MDT-on reads -- this pipeline is what surfaced #19279, and it now doubles as that fix's acceptance test (the catalog runs with `hudi.metadata-enabled` at its default `true`). ### Summary and Changelog - integ2 harness: `TrinoService` + three test classes (~30 tests) -- `ITTestTrinoSmoke` (plugin/metastore/CLI liveness), `ITTestTrinoStockTicks` (the classic COW + MOR `_ro` demo queries), `ITTestTrinoCustomType` (BLOB inline and out-of-line shapes, VECTOR, VARIANT on Spark 4.x composes). - `docker/trino`: Trino 481 image with the plugin assembled from the connector at HEAD via the in-repo shim, catalog/JVM configs; compose files gain a profile-gated `trinocoordinator`. - `.github/workflows/hudi_trino_e2e.yml`: builds Hudi artifacts (JDK 17) and the connector (JDK 25), bakes and smoke-boots the image, then runs `ITTestTrino*` under the `trino` compose profile. - One TEMP commit push-triggers the workflow on this dev branch for fork-side iteration; it gets dropped at merge time (the `pull_request` trigger stays). ### Impact Test/CI only -- no connector or runtime changes. ### Risk Level none ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
