jackiechan1981 commented on issue #13474:
URL: https://github.com/apache/apisix/issues/13474#issuecomment-4627798058
It should be noted that it is not timeout every time！rather, it occurs
randomly.
1. ai-proxy router configuration is provided below:
```json
# deepseek
{
"provider": "deepseek",
"auth": {
"header": {
"Authorization": "Bearer sk-xxxx"
}
},
"ssl_verify": false,
"options": {
"model": "deepseek-v4-flash"
},
"logging": {
"summaries": true,
"payloads": true
}
}
```
```json
# Qwen
{
"provider": "openai-compatible",
"override": {
"endpoint":
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
},
"options": {
"model": "qwen3.5-flash"
},
"auth": {
"header": {
"Authorization": "Bearer sk-xxxx"
}
},
"logging": {
"summaries": true,
"payloads": true
},
"timeout": 30000,
"keepalive": true,
"keepalive_timeout": 120000,
"keepalive_pool": 100
}
```
2. deployment mode：Docker - apache/apisix:3.16.0-ubuntu，host machine
VM：Ubuntu 24.04.3 LTS;
3. yes, conducting multiple tests in the same environment, with timeouts
occurring randomly；
4. logs is provided below:
deepseek
```text
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] trusted-addresses.lua:46:
is_trusted(): trusted_addresses_matcher is not initialized, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] radixtree_host_uri.lua:161:
match(): route match mode: radixtree_host_uri, client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] init.lua:749:
http_access_phase(): matched route:
{"value":{"plugins":{"ai-proxy":{"logging":{"summaries":true,"payloads":true},"_meta":{},"ssl_verify":false,"keepalive_pool":30,"provider":"deepseek","auth":{"header":{"Authorization":"Bearer
sk-xxxx"}},"timeout":30000,"keepalive":true,"options":{"model":"deepseek-v4-flash"},"keepalive_timeout":60000},"kafka-logger":{"include_resp_body":false,"include_req_body":false,"name":"kafka
logger","timeout":3,"meta_format":"default","buffer_duration":60,"batch_max_size":1000,"max_retry_count":0,"retry_delay":1,"brokers":[{"port":9094,"host":"10.32.10.24"}],"_meta":{},"max_req_body_bytes":524288,"cluster_name":1,"producer_batch_num":200,"producer_batch_size":1048576,"producer_max_buffering":50000,"producer_time_linger":1,"meta_refresh_interval":30,"max_resp_body_bytes":524288,"inactive_timeout":5,"kafka_topic":"ai-proxy-llm","producer_type":"async","required_acks":1},"key-auth":{"header":"Author
ization","hide_credentials":false,"_meta":{},"realm":"key","query":"apikey"}},"status":1,"methods":["POST"],"create_time":1779525995,"name":"deepseek-v4-flash","uri":"/v1/*","id":"ai-proxy-deepseek","vars":[["post_arg.model","==","deepseek-v4-flash"]],"update_time":1780622309,"priority":0},"clean_handlers":{},"has_domain":false,"key":"/apisix/routes/ai-proxy-deepseek","createdIndex":28,"modifiedIndex":343,"orig_modifiedIndex":343},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.xxx.com:9080", request_id:
"e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] key-auth.lua:120:
phase_func(): hit key-auth rewrite, client: 10.32.10.24, server: _, request:
"POST /v1/chat/completions HTTP/1.1", host: "ai-gateway-test.xxx.com:9080",
request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] plugin.lua:767:
merge_consumer_route(): route conf:
{"value":{"plugins":{"ai-proxy":{"logging":{"summaries":true,"payloads":true},"_meta":{},"ssl_verify":false,"keepalive_pool":30,"provider":"deepseek","keepalive_timeout":60000,"timeout":30000,"keepalive":true,"options":{"model":"deepseek-v4-flash"},"auth":{"header":{"Authorization":"Bearer
sk-xxxx"}}},"kafka-logger":{"include_resp_body":false,"include_req_body":false,"name":"kafka
logger","timeout":3,"meta_format":"default","max_req_body_bytes":524288,"batch_max_size":1000,"max_retry_count":0,"retry_delay":1,"brokers":[{"host":"10.32.10.24","port":9094}],"_meta":{},"required_acks":1,"cluster_name":1,"producer_batch_num":200,"producer_batch_size":1048576,"producer_max_buffering":50000,"producer_time_linger":1,"meta_refresh_interval":30,"max_resp_body_bytes":524288,"inactive_timeout":5,"producer_type":"async","kafka_topic":"ai-proxy-llm","buffer_duration":60},"key-auth":{"header":"Auth
orization","hide_credentials":false,"_meta":{},"realm":"key","query":"apikey"}},"status":1,"priority":0,"update_time":1780622309,"name":"deepseek-v4-flash","uri":"/v1/*","id":"ai-proxy-deepseek","vars":[["post_arg.model","==","deepseek-v4-flash"]],"create_time":1779525995,"methods":["POST"]},"has_domain":false,"orig_modifiedIndex":343,"key":"/apisix/routes/ai-proxy-deepseek","createdIndex":28,"modifiedIndex":343,"clean_handlers":{}},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.xxx.com:9080", request_id:
"e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] plugin.lua:768:
merge_consumer_route(): consumer group conf: null, client: 10.32.10.24, server:
_, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] init.lua:825:
http_access_phase(): find consumer model-test, config changed: true, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] openai-base.lua:285: request
extra_opts to LLM server: {"auth":{"header":{"Authorization":"Bearer
sk-xxxx"}},"model_options":{"model":"deepseek-v4-flash"},"conf":{}}, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] openai-base.lua:362: sending
request to LLM server:
{"method":"POST","path":"/chat/completions","query":{},"ssl_server_name":"api.deepseek.com","headers":{"authorization":"Bearer
sk-xxxx","x-forwarded-host":"ai-gateway-test.xxx.com","content-type":"application/json","x-forwarded-port":"9080","user-agent":"Java/17.0.17","x-user-id":"13662621035","Content-Type":"application/json","accept":"text/html,
image/gif, image/jpeg, */*;
q=0.2","X-Consumer-Username":"model-test","x-forwarded-proto":"http","connection":"keep-alive"},"port":443,"body":{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"Reply
with exactly one word:
OK"}],"max_tokens":16},"host":"api.deepseek.com","ssl_verify":false,"scheme":"https"},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.xxx.com:9080", request_id:
"e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] client.lua:123: dns_parse():
dns resolve api.deepseek.com, result:
{"section":3,"name":"api.deepseek.com","type":1,"address":"192.35.51.30","ttl":20,"class":1},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.xxx.com:9080", request_id:
"e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] resolver.lua:84:
parse_domain(): parse addr:
{"section":3,"name":"api.deepseek.com","type":1,"class":1,"ttl":20,"address":"192.35.51.30"},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.xxx.com:9080", request_id:
"e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] resolver.lua:85:
parse_domain(): resolver: ["127.0.0.11"], client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] resolver.lua:86:
parse_domain(): host: api.deepseek.com, client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:36:48 [info] 49#49: *67541 [lua] resolver.lua:88:
parse_domain(): dns resolver domain: api.deepseek.com to 192.35.51.30, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:37:18 [error] 49#49: *67541 [lua] openai-base.lua:366: failed
to connect to LLM server: timeout, client: 10.32.10.24, server: _, request:
"POST /v1/chat/completions HTTP/1.1", host: "ai-gateway-test.xxx.com:9080",
request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:37:18 [warn] 49#49: *67541 [lua] plugin.lua:1224:
common_phase(): ai-proxy exits with http status code 504, client: 10.32.10.24,
server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
2026/06/05 02:37:18 [info] 49#49: *67541 [lua] log-util.lua:297:
get_log_entry(): metadata: null while logging request, client: 10.32.10.24,
server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.xxx.com:9080", request_id: "e0977e33832f1b13f13eb6593d996d70"
```
Qwen
```text
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] trusted-addresses.lua:46:
is_trusted(): trusted_addresses_matcher is not initialized, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] radixtree_host_uri.lua:161:
match(): route match mode: radixtree_host_uri, client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] init.lua:749:
http_access_phase(): matched route:
{"value":{"plugins":{"ai-proxy":{"logging":{"summaries":true,"payloads":true},"_meta":{},"ssl_verify":true,"keepalive_pool":100,"provider":"openai-compatible","timeout":30000,"override":{"endpoint":"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"},"auth":{"header":{"Authorization":"Bearer
sk-xxxx"}},"keepalive":true,"keepalive_timeout":120000,"options":{"model":"qwen3.5-flash"}},"kafka-logger":{"include_resp_body":false,"include_req_body":false,"name":"kafka
logger","meta_format":"default","producer_type":"async","brokers":[{"port":9094,"host":"10.32.10.24"}],"batch_max_size":1000,"max_retry_count":0,"retry_delay":1,"buffer_duration":60,"_meta":{},"cluster_name":1,"producer_batch_num":200,"producer_batch_size":1048576,"max_req_body_bytes":524288,"producer_time_linger":1,"meta_refresh_interval":30,"producer_max_buffering":50000,"inactive_timeout":5,"max_resp_body_byt
es":524288,"timeout":3,"required_acks":1,"kafka_topic":"ai-proxy-llm"},"forward-auth":{"keepalive_timeout":60000,"_meta":{},"request_headers":["x-user-id","X-Consumer-Username"],"ssl_verify":true,"keepalive_pool":5,"upstream_headers":["x-user-id","X-Consumer-Username"],"request_method":"GET","client_headers":["x-user-id","X-Consumer-Username","X-error-message"],"uri":"http://10.32.10.24:8080/api/v1/quota/forward-auth","timeout":3000,"keepalive":true,"allow_degradation":false,"status_on_error":403},"key-auth":{"header":"Authorization","hide_credentials":false,"_meta":{},"realm":"key","query":"apikey"}},"status":1,"methods":["POST"],"create_time":1779716710,"name":"qwen3.5-flash","uri":"/v1/*","id":"ai-proxy-qwen3.5-flash","vars":[["post_arg.model","==","qwen3.5-flash"]],"update_time":1780623958,"priority":0},"clean_handlers":{},"has_domain":false,"key":"/apisix/routes/ai-proxy-qwen3.5-flash","createdIndex":68,"modifiedIndex":349,"orig_modifiedIndex":349},
client: 10.32.10.24, server:
_, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] key-auth.lua:120:
phase_func(): hit key-auth rewrite, client: 10.32.10.24, server: _, request:
"POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] plugin.lua:767:
merge_consumer_route(): route conf:
{"value":{"plugins":{"ai-proxy":{"logging":{"summaries":true,"payloads":true},"_meta":{},"ssl_verify":true,"options":{"model":"qwen3.5-flash"},"provider":"openai-compatible","keepalive_pool":100,"override":{"endpoint":"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"},"auth":{"header":{"Authorization":"Bearer
sk-xxxx"}},"keepalive":true,"keepalive_timeout":120000,"timeout":30000},"kafka-logger":{"include_resp_body":false,"include_req_body":false,"name":"kafka
logger","meta_format":"default","_meta":{},"max_req_body_bytes":524288,"batch_max_size":1000,"max_retry_count":0,"retry_delay":1,"buffer_duration":60,"required_acks":1,"cluster_name":1,"producer_batch_num":200,"producer_batch_size":1048576,"kafka_topic":"ai-proxy-llm","producer_time_linger":1,"meta_refresh_interval":30,"timeout":3,"max_resp_body_bytes":524288,"inactive_timeout":5,"producer_max_buffering":50000
,"brokers":[{"host":"10.32.10.24","port":9094}],"producer_type":"async"},"forward-auth":{"keepalive_timeout":60000,"_meta":{},"request_headers":["x-user-id","X-Consumer-Username"],"ssl_verify":true,"keepalive_pool":5,"upstream_headers":["x-user-id","X-Consumer-Username"],"uri":"http://10.32.10.24:8080/api/v1/quota/forward-auth","client_headers":["x-user-id","X-Consumer-Username","X-error-message"],"request_method":"GET","timeout":3000,"keepalive":true,"allow_degradation":false,"status_on_error":403},"key-auth":{"header":"Authorization","hide_credentials":false,"_meta":{},"realm":"key","query":"apikey"}},"status":1,"priority":0,"update_time":1780623958,"name":"qwen3.5-flash","uri":"/v1/*","id":"ai-proxy-qwen3.5-flash","vars":[["post_arg.model","==","qwen3.5-flash"]],"create_time":1779716710,"methods":["POST"]},"has_domain":false,"orig_modifiedIndex":349,"key":"/apisix/routes/ai-proxy-qwen3.5-flash","createdIndex":68,"modifiedIndex":349,"clean_handlers":{}},
client: 10.32.10.24, serve
r: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] plugin.lua:768:
merge_consumer_route(): consumer group conf: null, client: 10.32.10.24, server:
_, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] init.lua:825:
http_access_phase(): find consumer model-test, config changed: true, client:
10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] openai-base.lua:285: request
extra_opts to LLM server:
{"endpoint":"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions","model_options":{"model":"qwen3.5-flash"},"conf":{},"auth":{"header":{"Authorization":"Bearer
sk-xxxx"}}}, client: 10.32.10.24, server: _, request: "POST
/v1/chat/completions HTTP/1.1", host: "ai-gateway-test.iboxchain.com:9080",
request_id: "10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] openai-base.lua:362: sending
request to LLM server:
{"method":"POST","path":"/compatible-mode/v1/chat/completions","query":{},"ssl_server_name":"dashscope.aliyuncs.com","headers":{"authorization":"Bearer
sk-xxxx","connection":"keep-alive","user-agent":"Java/17.0.17","Content-Type":"application/json","x-user-id":"13662621035","x-forwarded-host":"ai-gateway-test.iboxchain.com","x-forwarded-proto":"http","accept":"text/html,
image/gif, image/jpeg, */*;
q=0.2","X-Consumer-Username":"model-test","content-type":"application/json","x-forwarded-port":"9080"},"port":443,"body":{"messages":[{"role":"user","content":"Reply
with exactly one word:
OK"}],"max_tokens":16,"model":"qwen3.5-flash"},"host":"dashscope.aliyuncs.com","ssl_verify":true,"scheme":"https"},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] client.lua:123: dns_parse():
dns resolve dashscope.aliyuncs.com, result:
{"section":3,"name":"dashscope.aliyuncs.com","type":1,"address":"192.12.94.30","ttl":160,"class":1},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] resolver.lua:84:
parse_domain(): parse addr:
{"section":3,"name":"dashscope.aliyuncs.com","type":1,"class":1,"ttl":160,"address":"192.12.94.30"},
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] resolver.lua:85:
parse_domain(): resolver: ["127.0.0.11"], client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] resolver.lua:86:
parse_domain(): host: dashscope.aliyuncs.com, client: 10.32.10.24, server: _,
request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:12 [info] 48#48: *80139 [lua] resolver.lua:88:
parse_domain(): dns resolver domain: dashscope.aliyuncs.com to 192.12.94.30,
client: 10.32.10.24, server: _, request: "POST /v1/chat/completions HTTP/1.1",
host: "ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:42 [error] 48#48: *80139 [lua] openai-base.lua:366: failed
to connect to LLM server: timeout, client: 10.32.10.24, server: _, request:
"POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:42 [warn] 48#48: *80139 [lua] plugin.lua:1224:
common_phase(): ai-proxy exits with http status code 504, client: 10.32.10.24,
server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
2026/06/05 02:46:42 [info] 48#48: *80139 [lua] log-util.lua:297:
get_log_entry(): metadata: null while logging request, client: 10.32.10.24,
server: _, request: "POST /v1/chat/completions HTTP/1.1", host:
"ai-gateway-test.iboxchain.com:9080", request_id:
"10844424ec4dfbf85333106e139254b9"
```
5. In a newly deployed Apisix environment, only one request is sent each
time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
