AlinsRan opened a new pull request, #13666: URL: https://github.com/apache/apisix/pull/13666
### Description When `least_conn` proxies long-lived connections (e.g. WebSocket) and the upstream is scaled, load stays skewed on the original nodes and the newly added nodes are not preferred — `least_conn` effectively degrades to round-robin. **Root cause.** The picker keeps per-server scores in a binary heap that lives inside the picker instance. The picker is cached by upstream version and rebuilt whenever the upstream changes (scaling bumps the version). On rebuild every score is reset to the base weight `1/weight`, so the connections already established on the old nodes are forgotten. New connections are then spread evenly across all nodes instead of favoring the empty new node, so the imbalance persists for the whole lifetime of the long connections. **Fix.** Keep a per-worker in-flight connection count for each `(upstream, server)` outside the picker, keyed by the upstream `resource_key`/`resource_id` which is stable across scaling (unlike the picker version). The rebuilt heap seeds each score with `(count + 1) / weight`, so surviving nodes keep their load while freshly added nodes start empty and are preferred right after scaling. The count is incremented/decremented in exactly the same places the heap score already moves (`get` / `after_balance`), so it always equals the current in-flight count. Servers that leave the upstream and have drained are pruned to bound memory on node churn. No new config, no shared dict, no opt-in flag: this is the intended `least_conn` behavior, so it is on by default. Counting is per-worker, matching the existing per-worker heap — each worker balances its own connections. For short requests the count returns to zero as soon as each request completes, so steady-state behavior is unchanged. #### Which issue(s) this PR fixes: Fixes #12217 ### Notes for reviewers This is an alternative to #12261 for the same issue. Differences: - **Per-worker table instead of a shared dict.** Avoids cross-worker races and new nginx `lua_shared_dict` plumbing; a worker only ever balances its own connections, so a per-worker count is both simpler and more accurate than a global one. - **Stable key.** Keyed on `resource_key`/`resource_id`. #12261's fallback hashes the whole upstream config (which includes `nodes`), so the key changes on the very scaling event it targets and the persisted count is lost for id-less upstreams. - **On by default, no `persistent_conn_counting` flag.** The imbalance is surprising default behavior, so the fix should apply without users having to discover a switch. - **Single-node fast path kept**, so regular `least_conn` users are unaffected. - **Memory bounded** via pruning of drained, removed servers. ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
