The GitHub Actions job "Required Checks" on texera.git/main has failed. Run started by GitHub user github-merge-queue[bot] (triggered by github-merge-queue[bot]).
Head commit for run: c4f5389ebd9182f55674eb329f6e41f3b2a0dc1c / Ryan Zhang <[email protected]> feat(notebook-migration, single-node): deploy the notebook migration service and JupyterLab (#7932) ### What changes were proposed in this PR? The notebook migration tool shipped its backend service and workspace UI, but nothing deployed them. This PR makes the tool work out of the box in the single-node Docker Compose stack: it publishes images for the service and for the customized JupyterLab server, runs both in the stack, and routes the tool's API traffic through the existing nginx proxy. **`StorageConfig`, `NotebookMigrationResource` (one Jupyter URL becomes two)** - `storage.jupyter.url` splits into `internal-url` and `public-url`. The service calls Jupyter at the internal address (health probes, notebook upload and delete); the iframe URL handed to the browser is built from the public one. A containerized deployment needs both at once, since the in-network service name does not resolve from the browser. - `public-url` defaults to `internal-url`, so native local development and every existing deployment are unchanged. The old key had a single consumer and appeared in no deployment configuration, so no compatibility alias is kept. - The existing note about this service still targeting one Jupyter per process (#7665) is updated to name which URL is which. **`bin/dockerfiles/notebook-migration-service.dockerfile` (new)** - Follows `config-service.dockerfile`, which copies the shared config resources the service reads at startup. Omits the `.git` copy that exists elsewhere for jgit calls from `OPVersion`: that lives in `workflow-core` and `workflow-operator`, which this service does not depend on. **`bin/dockerfiles/jupyter.dockerfile` (new)** - Builds the customized JupyterLab image from the files that already live with the service, so the image can be published rather than built by hand. - Replaces `notebook-migration-service/src/main/resources/Dockerfile` and the standalone `docker-compose.yml` beside it, both now deleted. Every compose file in the repo consumes published images, so the per-service compose file had no remaining role once the image is built by CI. **`build-and-push-images.yml`** - Adds image name mappings for both new dockerfiles. Without them the discovery step falls through to its default and publishes under bare names, missing the `texera-` prefix every other image uses. **`bin/single-node/`** - `docker-compose.yml` runs `notebook-migration-service` and `jupyter`, the latter publishing its port because the browser loads it directly rather than through nginx. - The Jupyter container deliberately does not receive `env_file`. Users execute arbitrary code there, so it gets only the two values it needs instead of the whole environment, which carries the database, S3, lakeFS and LiteLLM credentials. This also keeps `JUPYTER_PORT` out of that container, which the base image would otherwise read and use to move the server off its expected port. - `nginx.conf` routes `/api/notebook-migration/` to the service. Longest prefix matching gives it priority over the `/api/` catch all. - `.env` gains the published port, the Jupyter token, the internal Jupyter address, and the GUI toggle for the tool. Values needing interpolation are set in the compose file instead, matching how lakeFS composes its browser facing presigned endpoint. **Docs** - `bin/single-node/README.md` covers the new port in all three places ports are listed, how to use and disable the tool, and a warning that all users share one JupyterLab server and one token, which is acceptable on a single machine but must not be carried into a multi-user deployment. ### Any related issues, documentation, discussions? Closes #7931 Parent issue #4301 ### How was this PR tested? Added a test in `StorageConfigSpec.scala` pinning that the public Jupyter URL defaults to the internal one, so splitting them stays a no-op outside containerized deployments. The rest of this PR is deployment configuration, which the existing suites do not cover. Existing suites pass: `NotebookMigrationService/test` (42), `Config/testOnly StorageConfigSpec`, and `bin/single-node/tests/test_single_node_sh.sh`. Manually verified by running the full stack with `bin/single-node.sh up`: - All containers reach a healthy state, including both new ones. - Authenticated through nginx, then called `/api/notebook-migration/get-jupyter-url`, which returned the browser reachable address rather than the in-network one. That request also confirms the internal direction, since the endpoint only succeeds after its reachability probe to Jupyter passes. - `POST /api/notebook-migration/set-notebook` through nginx succeeded and the notebook landed inside the Jupyter container. The iframe URL then returned 200 with the expected `frame-ancestors` header naming the Texera origin. - Confirmed no credentials are present in the Jupyter container's environment. - The existing routes (`/api/dataset`, `/api/computing-unit`, `/api/models`, `/api/compile`, `/`) are unaffected. - With an API key configured, a live model call through the tool's LLM path returned successfully. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) --------- Co-authored-by: Meng Wang <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/32907130319 With regards, GitHub Actions via GitBox
