ruanweihong-aaa opened a new pull request, #13801: URL: https://github.com/apache/apisix/pull/13801
### Description `ai-proxy` and `ai-proxy-multi` currently expose a single `timeout` value that is applied to the HTTP connect, send, and read phases. This makes it difficult to fail fast during connection establishment or request upload while allowing a longer timeout for LLM response generation. This PR adds three optional phase-specific timeout fields: - `connect_timeout`: timeout for establishing the upstream connection; - `send_timeout`: timeout for sending the request to the upstream; - `read_timeout`: timeout for each socket read from the upstream. All three fields are expressed in milliseconds and accept integer values from `1` to `600000`. The change preserves backward compatibility: - when none of the new fields is configured, the transport continues to call the existing numeric `set_timeout(timeout)` path; - when only some phase-specific fields are configured, every omitted phase falls back to the existing `timeout` value; - existing configurations therefore keep their previous behavior. Implementation details: - add the three fields to both the `ai-proxy` and `ai-proxy-multi` schemas; - build a complete connect/send/read timeout tuple at the `ai-proxy` transport call site; - use `lua-resty-http`'s `set_timeouts(connect, send, read)` for phase-specific configuration while retaining the original `set_timeout(timeout)` compatibility path; - document the new fields in the English and Chinese plugin documentation; - add schema boundary tests, fallback/call-site tests, transport API tests, and delayed-upstream read-timeout regression coverage. ### Compatibility The new fields are optional. Existing users who only configure `timeout`, or rely on its default value, continue to use the original code path. The upper bound of each new field is the same as the existing `timeout` upper bound. `read_timeout` is a per-socket-read timeout and is not a total wall-clock limit for streaming responses. Existing `max_stream_duration_ms` and `max_response_bytes` controls remain responsible for total streaming duration and response-size limits. ### Verification Targeted, source-matched Docker tests: - schema validation for `ai-proxy` and `ai-proxy-multi`, including valid boundaries and invalid values; - call-site fallback behavior for omitted and partially configured phase timeouts; - transport dispatch to `set_timeouts(connect, send, read)` and compatibility with the numeric `set_timeout(timeout)` path; - delayed-upstream read-timeout behavior; - result: `Files=4, Tests=66, Result: PASS`. Additional local dual-Gateway A/B verification used identical etcd, routes, Docker network, and controlled upstream fixtures for the pre-fix and fixed images: - read phase: pre-fix returned `200` after about `250 ms`; fixed returned `504` after about `50 ms` with `read_timeout=50`; - connect phase: pre-fix timed out after about `1 s`; fixed timed out after about `50 ms` with `connect_timeout=50`; - send phase: with a `60 MiB` request and an upstream that accepted headers but did not consume the body, the fixed image timed out significantly earlier with `send_timeout=50`; - Gateway logs were checked to associate each failure with the intended connect, send, or read phase. These results cover the tests related to this change; they are not a claim that the complete APISIX test suite was executed locally. #### Which issue(s) this PR fixes: Fixes #12072 ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [x] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
