nic-6443 commented on PR #13876: URL: https://github.com/apache/apisix/pull/13876#issuecomment-5420509626
Good catch — confirmed and fixed in the follow-up commit. I traced the path you described and it holds exactly as written. `parse_streaming_response()` clears `ctx.ai_stream_aborted` per attempt but never `ctx.var.llm_request_done`, and the `[DONE]`-only case is reachable: `anthropic-messages-to-openai-chat.convert_sse_events()` returns nil for a `done` event when `state.is_first` is still true, so `output_sent` stays false while `llm_request_done` is already set, EOF returns 502, and `http_5xx` falls back inside the same context. One thing worth adding to your analysis: the inherited flag hurts before the read error too. On the retry the moderation plugins see `llm_request_done` true from the very first chunk, so `ai-aliyun-content-moderation` in `final_packet` mode appends a terminator to a stream that is still running — the truncated response is reported to the client as complete. That is what the regression test asserts on, since the two effects you named (`ai_stream_aborted` and the finalization pass) are otherwise invisible: `ai-cache` already refuses truncated streams independently via `stream.stream_completed()`, and an empty finalization pass produces nothing observable on its own. Did both things you asked for: - `ctx.var.llm_request_done = nil` at the start of every attempt, next to the existing abort-flag reset. - A new attempt-local `protocol_completed`, set only where this attempt parses a `done` / `usage_and_done` event, and used for both read-error decisions instead of the shared ctx var. `TEST 10` in `t/plugin/ai-proxy-stream-truncated.t` covers the scenario: attempt 1 serves a `[DONE]`-only stream through the Anthropic converter and produces no downstream output, the retry emits one content event plus a usage event and then truncates. It asserts the content is delivered and that no `message_stop` is synthesized. It fails without this commit (the response gains a `message_stop`) and passes with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
