shreemaan-abhishek opened a new pull request, #13307: URL: https://github.com/apache/apisix/pull/13307
## Summary Phase 2 of the Bedrock provider (Phase 1 was #13249). Routes requests with `body.stream = true` to `/model/<model>/converse-stream` and forwards the AWS EventStream binary response unchanged. - New `ai-transport.eventstream` codec parses/encodes EventStream frames (12-byte prelude + headers + payload + trailing CRC, validated via `ngx.crc32_long`); same API surface as `ai-transport.sse` so providers pick a framing module by name. - `ai-providers.base.parse_streaming_response` selects `sse` vs `aws-eventstream` framing via `provider.streaming_framing`. - Bedrock provider declares `streaming_framing = \"aws-eventstream\"` and routes `ctx.var.request_type == \"ai_stream\"` to `/converse-stream`. - `bedrock-converse` protocol decodes EventStream events: `contentBlockDelta` -> texts; `metadata` -> `usage_and_done`; `:message-type=exception` -> `done` with warning. The wire format is decoded in pure Lua against the spec at https://docs.aws.amazon.com/lexv2/latest/dg/event-stream-encoding.html. Streaming is opt-in (`body.stream = true`); existing non-streaming routes keep going to `/converse`. ## Test plan - [x] New streaming test cases (4 in `t/plugin/ai-proxy-bedrock.t`, 3 in `t/plugin/ai-proxy-bedrock-single.t`) cover: route hits `/converse-stream`, SigV4 still validates, Content-Type is `application/vnd.amazon.eventstream`, raw bytes contain expected event ordering, prompt/completion tokens extracted from `metadata`, and a non-streaming control on the same route. - [x] Test fixture (`t/fixtures/bedrock/bedrock-converse-streaming.bin`) is a real recorded `/converse-stream` response (6 frames: messageStart, contentBlockDelta×2, contentBlockStop, messageStop, metadata). - [x] All 114 existing Bedrock tests still pass (60 multi + 54 single). - [x] Live verification against real AWS Bedrock (`anthropic.claude-3-haiku-20240307-v1:0` in `ap-southeast-1`): streaming, system prompt, per-request `body.model`, `ai-proxy-multi`, non-streaming after streaming on the same route, and error paths (missing model, invalid JSON, wrong method) — all work. ## Docs - `docs/{en,zh}/latest/plugins/ai-proxy.md` and `ai-proxy-multi.md` — added `stream` to the Bedrock request format table; replaced the \"not yet supported\" note with a streaming usage section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
