geyanggang opened a new pull request, #13266:
URL: https://github.com/apache/gravitino/pull/13266
### What changes were proposed in this pull request?
Add in-tree Docker build definitions for the three engine connectors under
`dev/docker/`:
- `dev/docker/trino-connectors/`
- `dev/docker/flink-connectors/`
- `dev/docker/spark-connectors/`
Each directory contains:
- `Dockerfile` — Red Hat UBI 10 minimal base, required certification labels
(`name`, `vendor`, `version`, `release`, `summary`, `description`,
`maintainer`), `/licenses` with `LICENSE` and `NOTICE`, runs as non-root
(UID 1000, GID 0), OpenShift arbitrary-UID compatible.
- `*-connectors-dependency.sh` — builds the connector artifacts. The set of
version modules is discovered from the Gradle project rather than
hard-coded, so the same script tracks whatever versions the branch
supports:
- Trino: each `trino-connector-<band>` module via `assembleTrinoConnector`.
- Flink: each `flink-runtime-<ver>` module (Scala 2.12).
- Spark: each `spark-runtime-<major>` module, deriving Scala variants
(2.12 always; 2.13 for Spark 3.4+; Spark 4.0 pinned to 2.13, detected
from its build script).
- `copy-connector.sh` — entrypoint. Resolves the requested version from the
directories baked into the image at runtime and copies the jars into a
mounted `/target` volume; prints available versions when `/target` is not
mounted.
- `README.md` — install and configuration guide.
The Trino image ships the full plugin layout (connector jar plus its runtime
dependency jars, `LICENSE`, `NOTICE`, `README`) and no database JDBC drivers;
those come from Trino's own `mysql` / `postgresql` plugins. Flink and Spark
images ship the shaded runtime jars.
### Why are the changes needed?
There is no first-party connector container image in the repository for
installing the connectors into Trino, Flink or Spark on Kubernetes. These
images give operators a ready-to-use init container per engine, with contents
that stay purely Apache-2.0 and a version matrix that follows the branch
automatically.
Fix: #13265
### Does this PR introduce _any_ user-facing change?
Yes. It adds new connector container images
(`apache/gravitino-trino-connector`,
`apache/gravitino-flink-connector`, `apache/gravitino-spark-connector`) and
their build tooling. No API or property changes.
### How was this patch tested?
- Built the Flink connector image end to end: the dependency script
discovered
and built all Flink runtime modules (1.18/1.19/1.20), the image built on
UBI 10,
and the entrypoint copied the correct jar for the requested `FLINK_VERSION`
into a mounted `/target`.
- Verified the produced jars contain only open source connector code (no
proprietary JDBC adapters) and that `/licenses` contains only `LICENSE` and
`NOTICE`.
- Verified the required image labels and non-root user (UID 1000) are
present.
- Confirmed the Trino and Spark dependency scripts discover the correct
Gradle
modules and construct valid task lists (including the Spark Scala-variant
derivation for 3.5 / 4.0), and validated all shell scripts with `bash -n`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
