viirya opened a new pull request, #19:
URL: https://github.com/apache/spark-connect-gateway/pull/19
### What changes were proposed in this pull request?
Two binaries under `crates/proxy/examples/` assert end-to-end invariants,
but were
only ever *compiled* by `cargo clippy --all-targets` and never executed — so
the
behaviour they cover had no CI verification at all:
- **`drain_smoke`** — exercises the two-step drain the gateway performs on
SIGTERM
(in-process, triggered through the same channel `shutdown_signal()` uses).
Asserts that `/readyz` flips to not-ready when the drain starts, that
in-flight
streams keep receiving messages through the drain, and that the gateway
shuts
down cleanly afterwards. **No external dependencies.**
- **`ha_smoke`** — spawns two real `SparkConnectProxy` gateways sharing one
Redis
affinity store and one static pool of two fake backends, then verifies
three
multi-replica HA properties: shared affinity state across replicas, that
failover preserves an existing session binding, and that the op-id reverse
index
resolves across replicas. Needs a Redis (`REDIS_URL`, default
`redis://127.0.0.1:6399`).
Both signal failure by panicking on an assert, so a plain `cargo run` is a
sufficient CI check. This adds them as two steps in the `integration` job,
plus a
Redis **service container** on `:6399` for `ha_smoke`.
The Redis *tests* in that job are unaffected — they start their own container
through `testcontainers` and do not use the service container.
The kind/Helm end-to-end walkthroughs under `deploy/examples/` remain out of
scope; they need a much heavier harness and are better suited to a separate
scheduled workflow.
### Why are the changes needed?
Graceful drain and multi-replica HA are both production behaviours — the
first is
what makes a rolling restart not drop in-flight queries, the second is what
makes
running more than one replica correct. Neither had any automated coverage,
even
though the code to verify them was already in the tree.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Locally, with a Redis on `:6399` standing in for the service container,
running
all four steps of the job in sequence: the testcontainers Redis tests still
pass
(4 + 5), the tracing test passes, and both smoke examples exit 0 — "all drain
invariants passed" and "all HA invariants passed". This PR's own CI run
exercises
the new steps and the service container.
### Was this patch authored or co-authored using generative AI tooling?
Yes, co-authored with Claude Code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
