shreemaan-abhishek commented on PR #13778: URL: https://github.com/apache/apisix/pull/13778#issuecomment-5188028833
Pushed a follow-up removing the hop-by-hop header change from this PR. `ai-cache-streaming.t` TEST 37 and 38 failed in CI. It reproduces locally on this branch and not on `main`, and reverting only the blacklist hunk makes it go away, so the cause was dropping `Connection` from the forwarded headers, not the client cutover. The symptom is an `ai-proxy-multi` retry after a stream that dies before its first byte: it stops finding an instance to re-pick and returns 502 with `failed to pick new AI instance: all upstream servers tried`. Two things are worth following up separately, neither of which belongs in a PR about which HTTP client the transport builds: - Forwarding a downstream `Connection` header to a third-party LLM upstream is wrong per RFC 9110, and it feeds the client's keepalive decision. - The streaming read-error path in `parse_streaming_response` returns without closing `res._httpc`, so a timed-out upstream connection is leaked. That looks like the real reason the retry behaves differently once `Connection: close` stops reaching the upstream. This PR is now just the client selection, and the transport diff stays byte-identical with the paired PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
