viirya opened a new pull request, #22: URL: https://github.com/apache/spark-connect-gateway/pull/22
### What changes were proposed in this pull request? The deploy/examples/e2e-trust-boundary walkthrough is what verifies the gateway-to-backend trust boundary: the backend enforces spark.connect.authenticate.token, the gateway presents it, and clients must not be able to read it back. None of that was covered by CI. The in-process tests cover the outbound-credential plumbing, but nothing verified the deployed arrangement or the Config-response withholding against a real Spark backend. This adds an e2e-trust-boundary job to the E2E workflow, on its own kind cluster, running the walkthrough's four assertions: (1) A direct connection to the backend, bypassing the gateway, is refused with UNAUTHENTICATED "No authentication token provided". This runs before the gateway is installed, so nothing could be presenting a token. (2) The same client succeeds through the gateway, whose startup log confirms "outbound backend authentication enabled". (3) Negative control: with backendToken.enabled=false, that same client through that same gateway is refused again with the identical error. Without this, (2) would prove nothing; it is what shows the token is the discriminator rather than the network path or some other setting. (4) A client asking the Config RPC for spark.connect.authenticate.token sees it unset, and the gateway records a config.redacted audit event naming the withheld key. Assertion (4) is why the token layer holds at all: Spark's Config RPC will otherwise hand back any config key the server holds, which would let any client read the credential and then dial the backend directly. The walkthrough's NetworkPolicy half is deliberately out of scope: kind's default CNI does not enforce NetworkPolicy, which is itself why the token layer matters, since it does not depend on the CNI. Verified by running all four assertions locally first: direct connection refused, through-gateway query returned 10 rows, tokenless gateway refused with the identical error, and the token read back as unset with one config.redacted audit event naming spark.connect.authenticate.token. ### Why are the changes needed? The trust boundary is a security property: without it, any pod that can reach the backend Service can run arbitrary Spark jobs as anyone. The deployed arrangement that enforces it — Secret, backend enforcement, gateway outbound credential, and Config withholding — had no CI coverage. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? All four assertions were run locally before writing the job (see above). This PR's own CI run exercises them on a hosted runner. ### Was this patch authored or co-authored using generative AI tooling? Yes, co-authored with Claude Code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
