aanogueira opened a new pull request, #14142: URL: https://github.com/apache/apisix/pull/14142
## Problem A freshly created health-check target defaults to `internal_health = healthy` with zero probes (`add_target`'s hardcoded `is_healthy=true`), so a pod that restarts while its backend is already unhealthy briefly routes real traffic to it until enough active probes correct the target's state. Separately, checker creation itself is entirely lazy — seeded only by `fetch_checker()` on the live request path — so an idle-but-critical upstream with no prior traffic would never get a checker built at all ahead of a readiness check. There is currently no way to tell "healthy" (a real check passed) apart from "healthy" (the zero-probe default) from outside the checker, so nothing can gate readiness on "has this actually been checked yet." ## Changes `apisix/healthcheck_manager.lua` adds two accessors for a readiness plugin to use: - `ensure_checker(resource_path)`: proactively seeds a checker for a resource even with zero prior traffic, reusing the existing `timer_create_checker` construction path. Also resolves domain-name upstream nodes via `parse_domain_in_up` up front — that resolution otherwise only happens on the live request path, so a checker built ahead of traffic would start probing under an unresolved identity and get silently rebuilt (wiping its probe count) the moment real traffic first resolved the domain. - `is_resource_probed(resource_path)`: true only once every target of the resource's checker has had enough real active-check attempts for its state to have actually converged — a single attempt is not always enough (e.g. with `unhealthy.http_failures = 2`, two consecutive attempts are required), so this computes the real threshold from the checker's own config (`max(unhealthy.http_failures, .tcp_failures, .timeouts, healthy.successes)`) rather than treating "attempted once" as sufficient. This depends on a companion library-side change to `api7/lua-resty-healthcheck` adding a per-target probe-attempt counter in shm and an `all_targets_probed(name, shm_name, min_attempts)` module function, so any worker can ask "has every target had at least `min_attempts` real checks yet?" ## Validation Validated end to end in a local kind cluster: pod-restart-while-already-unhealthy-backend scenario shows zero leaked requests before or after the readiness transition, down from dozens of leaked responses without this gate. `t/node/healthcheck-fresh-node-default-healthy.t` covers both the zero-probe-gate and the attempt-threshold behavior directly (a gate built on "attempted once" must stay false until the real convergence threshold is met). ## Note This supersedes #13896 and #13897, which I closed shortly after opening while still validating the fix. Since then it's been running in production (behind a feature flag) with no issues, and I've since found and fixed two gaps the earlier version had (probe-count threshold vs single-attempt, and domain-name resolution ordering) — both included here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
