huanghuang0913 opened a new issue, #12776: URL: https://github.com/apache/apisix/issues/12776
### Description I created a route for our deepseek r1: curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ -H "X-API-KEY: ${admin_key}" \ -d '{ "id": "ai-proxy-route-DeepSeek-R1", "name": "DeepSeek-R1", "uri": "/deepseek-r1/chat/completions", "methods": ["POST"], "plugins": { "ai-proxy": { "provider": "openai-compatible", "auth": { "header": { "Authorization": "Bearer '"$DEEPSEEK_API_KEY"'" } }, "override": { "endpoint": "http://<our ip>:<our port>/v1/chat/completions" } }, "prometheus": {} } }' But when I send the request via apisix: [root@tmpl logs]# curl "http://10.103.43.14:9080/deepseek-r1/chat/completions"; -X POST \ > -H "Content-Type: application/json" \ > -d '{ > "model": "DeepSeek-R1", > "messages": [ > { "role": "system", "content": "You are a mathematician" }, > { "role": "user", "content": "What is 1+1?" } > ], > "stream": true > }' Here is the response: data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984277,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":"{think"}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984277,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":"\\n"}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984277,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":"First"}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984278,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":","}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984278,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":" the"}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984278,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":" user"}],"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984278,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":"1"}],"logprobs":null,"finish_reason":null}]} d8 da curl: (56) Malformed encoding found in chunked-encoding data: {"id":"endpoint_common_673","object":"chat.completion.chunk","created":1763984278,"model":"DeepSeek-R1","choices":[{"index":0,"delta":{"role":"assistant","content":"\""}],"logprobs":null,"finish_reason":nu[root@tmpl logs]# If I send the same request to our ai model directly, the response is fine. ### Environment APISIX version (run apisix version): 3.14 Operating system (run uname -a): x86_64 GNU/Linux OpenResty / Nginx version (run openresty -V or nginx -V):1.21.4.1 etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):bitnami/etcd:3.5.11 APISIX Dashboard version, if relevant:apache/apisix-dashboard:latest Plugin runner version, for issues related to plugin runners:prom/prometheus:v2.25.0 LuaRocks version, for installation issues (run luarocks --version): 3.12.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
