vortegatorres commented on issue #2708: URL: https://github.com/apache/apisix-ingress-controller/issues/2708#issuecomment-3833975822
As mentioned earlier, the issue is intermittent and not deterministic. In almost all cases where we observed the replica count scaling up and down, everything worked as expected. Because of that, I can’t provide additional reproduction steps beyond what I already did and what you’ve already tried. Regarding the logs: during the incident, APISIX first reported a `(111: Connection refused)` while connecting to the upstream: ```text 2026-01-29 04:05:10.638 error 2026/01/29 03:05:10 [error] 49#49: *1392456 connect() failed (111: Connection refused) while connecting to upstream, client: XXXXXX, server: _, request: "POST /api HTTP/1.1", upstream: "http://YYYYYY:8000/api";, host: "url" ``` Right after that, the request was retried and succeeded. The key detail is that `upstream_addr` contains two IPs: it first tried a stale IP, then retried using the IP that worked: ```json 2026-01-29 04:05:11.430 { "ts": "2026-01-29T03:05:11+00:00", "service": "apisix", "resp_body_size": "0", "host": "url", "address": "XXXXX", "request_length": "595", "method": "POST", "uri": "/api", "status": "204", "user_agent": "Go-http-client/2.0", "resp_time": "0.512", "upstream_addr": "X:8000, Y:8000", "upstream_status": "502, 204", "traceparent": "00-0bc72f67dd2ee53ce6d7f03e4a4eb7d6-3a5171c169e8eb46-01", "trace_id": "0bc72f67dd2ee53ce6d7f03e4a4eb7d6", "span_id": "3a5171c169e8eb46", "org_slug": "", "matched_uri": "" } ``` Sorry — I don’t have more relevant logs or additional steps to reproduce. This seems like a corner case and may not be easy to reproduce reliably. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
