RGZingYang opened a new issue, #13501:
URL: https://github.com/apache/apisix/issues/13501
### Current Behavior
When the upstream LLM provider returns HTTP 429 or 5xx with a JSON error
body, ai-proxy currently returns only the status code and closes the HTTP
connection before reading res.body_reader.
As a result, the provider-side error details are discarded and cannot be
captured by logs or observability pipelines. This is especially problematic for
monitoring upstream 5xx errors, because users need the actual error body
returned by the LLM provider to investigate the root cause, distinguish between
provider failures, quota/rate-limit conditions, request validation errors, and
other provider-specific diagnostics.
### Expected Behavior
When the upstream LLM provider returns HTTP 429 or 5xx with an error
response body, ai-proxy should read the body up to a configurable maximum size
before closing the connection. The captured body should either be exposed
through logging/observability fields such as `llm_response_text`, or be
preserved and returned to the client together with the upstream status code,
while respecting response size limits and the upstream `Content-Type`.
### Error Logs
_No response_
### Steps to Reproduce
```
local res, err = httpc:request(params)
if not res then
core.log.warn("failed to send request to LLM server: ", err)
return handle_error(err)
end
-- handling this error separately is needed for retries
if res.status == 429 or (res.status >= 500 and res.status < 600 )then
return res.status
end
local code, body = read_response(extra_opts.conf, ctx, res,
self.response_filter)
if conf.keepalive then
local ok, err = httpc:set_keepalive(conf.keepalive_timeout,
conf.keepalive_pool)
if not ok then
core.log.warn("failed to keepalive connection: ", err)
end
end
```
### Environment
- APISIX version (run `apisix version`):
- Operating system (run `uname -a`):
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl
http://127.0.0.1:9090/v1/server_info`):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
