kayx23 commented on code in PR #13731: URL: https://github.com/apache/apisix/pull/13731#discussion_r3711219840
########## docs/en/latest/plugins/ai-proxy-multi.md: ########## @@ -44,6 +44,22 @@ In addition, the Plugin also supports logging LLM request information in the acc ## Request Format +### Request Protocol Detection + +The Plugin detects the client request protocol before selecting a compatible upstream endpoint. It checks the following rules in order: + +| Client protocol | Detection | Route URI | +| --- | --- | --- | +| Bedrock Converse | The request body contains a `messages` array and the request URI ends in `/converse`. | The URI can have a custom prefix, but it must keep the `/converse` suffix. | +| Anthropic Messages | The request body is a JSON object and the request URI ends in `/v1/messages`. | The URI can have a custom prefix, but it must keep the `/v1/messages` suffix. | +| OpenAI Responses | The request body contains `input` and the request URI ends in `/v1/responses`. | The URI can have a custom prefix, but it must keep the `/v1/responses` suffix. | +| OpenAI Chat Completions | The request body contains a `messages` array. | Any URI matched by the Route. | +| OpenAI Embeddings | The request body contains `input`, and no earlier rule matched. | Any URI matched by the Route. | + +The URI-specific rules run before the body-only rules. This prevents Bedrock Converse and Anthropic Messages requests containing `messages` from being identified as Chat Completions. Responses and Embeddings requests both use `input`, so a request containing `input` but not `messages` is identified as Embeddings unless its URI ends in `/v1/responses`. If no earlier rule matches, the Plugin treats a non-empty JSON object as passthrough. Empty or invalid request bodies are rejected. Review Comment: Fixed in 3f78320: `anthropic-messages.matches` accepts any decoded Lua table, `passthrough.matches` accepts any non-empty table, and `patch_cjson` preserves decoded arrays as tables. Updated the wording to cover JSON objects and arrays. Validation: `git diff --check`. ########## docs/zh/latest/plugins/ai-proxy-multi.md: ########## @@ -44,6 +44,22 @@ import TabItem from '@theme/TabItem'; ## 请求格式 +### 请求协议检测 + +插件会先检测客户端请求协议，再选择兼容的上游端点。插件按以下顺序检查规则： + +| 客户端协议 | 检测条件 | 路由 URI | +| --- | --- | --- | +| Bedrock Converse | 请求体包含 `messages` 数组，并且请求 URI 以 `/converse` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/converse` 后缀。 | +| Anthropic Messages | 请求体为 JSON 对象，并且请求 URI 以 `/v1/messages` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/v1/messages` 后缀。 | +| OpenAI Responses | 请求体包含 `input`，并且请求 URI 以 `/v1/responses` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/v1/responses` 后缀。 | +| OpenAI Chat Completions | 请求体包含 `messages` 数组。 | 路由匹配的任意 URI。 | +| OpenAI Embeddings | 请求体包含 `input`，并且之前的规则均未匹配。 | 路由匹配的任意 URI。 | + +基于 URI 的规则在仅基于请求体的规则之前运行，以免包含 `messages` 的 Bedrock Converse 和 Anthropic Messages 请求被识别为 Chat Completions。Responses 和 Embeddings 请求都使用 `input`，因此包含 `input` 但不包含 `messages` 的请求会被识别为 Embeddings，除非其 URI 以 `/v1/responses` 结尾。如果之前的规则均未匹配，插件会将非空 JSON 对象视为透传请求。空请求体或无效请求体会被拒绝。 Review Comment: 已在 3f78320 中修复：`anthropic-messages.matches` 接受任意解码后的 Lua table，`passthrough.matches` 接受任意非空 table，且 `patch_cjson` 将解码后的数组保留为 table。文档已改为同时说明 JSON 对象和数组。验证：`git diff --check`。 ########## docs/zh/latest/plugins/ai-proxy.md: ########## @@ -44,6 +44,22 @@ import TabItem from '@theme/TabItem'; ## 请求格式 +### 请求协议检测 + +插件会先检测客户端请求协议，再选择兼容的上游端点。插件按以下顺序检查规则： + +| 客户端协议 | 检测条件 | 路由 URI | +| --- | --- | --- | +| Bedrock Converse | 请求体包含 `messages` 数组，并且请求 URI 以 `/converse` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/converse` 后缀。 | +| Anthropic Messages | 请求体为 JSON 对象，并且请求 URI 以 `/v1/messages` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/v1/messages` 后缀。 | +| OpenAI Responses | 请求体包含 `input`，并且请求 URI 以 `/v1/responses` 结尾。 | URI 可以包含自定义前缀，但必须保留 `/v1/responses` 后缀。 | +| OpenAI Chat Completions | 请求体包含 `messages` 数组。 | 路由匹配的任意 URI。 | +| OpenAI Embeddings | 请求体包含 `input`，并且之前的规则均未匹配。 | 路由匹配的任意 URI。 | + +基于 URI 的规则在仅基于请求体的规则之前运行，以免包含 `messages` 的 Bedrock Converse 和 Anthropic Messages 请求被识别为 Chat Completions。Responses 和 Embeddings 请求都使用 `input`，因此包含 `input` 但不包含 `messages` 的请求会被识别为 Embeddings，除非其 URI 以 `/v1/responses` 结尾。如果之前的规则均未匹配，插件会将非空 JSON 对象视为透传请求。空请求体或无效请求体会被拒绝。 Review Comment: 已在 3f78320 中修复：`anthropic-messages.matches` 接受任意解码后的 Lua table，`passthrough.matches` 接受任意非空 table，且 `patch_cjson` 将解码后的数组保留为 table。文档已改为同时说明 JSON 对象和数组。验证：`git diff --check`。 ########## docs/en/latest/plugins/ai-proxy.md: ########## @@ -44,6 +44,22 @@ In addition, the Plugin also supports logging LLM request information in the acc ## Request Format +### Request Protocol Detection + +The Plugin detects the client request protocol before selecting a compatible upstream endpoint. It checks the following rules in order: + +| Client protocol | Detection | Route URI | +| --- | --- | --- | +| Bedrock Converse | The request body contains a `messages` array and the request URI ends in `/converse`. | The URI can have a custom prefix, but it must keep the `/converse` suffix. | +| Anthropic Messages | The request body is a JSON object and the request URI ends in `/v1/messages`. | The URI can have a custom prefix, but it must keep the `/v1/messages` suffix. | +| OpenAI Responses | The request body contains `input` and the request URI ends in `/v1/responses`. | The URI can have a custom prefix, but it must keep the `/v1/responses` suffix. | +| OpenAI Chat Completions | The request body contains a `messages` array. | Any URI matched by the Route. | +| OpenAI Embeddings | The request body contains `input`, and no earlier rule matched. | Any URI matched by the Route. | + +The URI-specific rules run before the body-only rules. This prevents Bedrock Converse and Anthropic Messages requests containing `messages` from being identified as Chat Completions. Responses and Embeddings requests both use `input`, so a request containing `input` but not `messages` is identified as Embeddings unless its URI ends in `/v1/responses`. If no earlier rule matches, the Plugin treats a non-empty JSON object as passthrough. Empty or invalid request bodies are rejected. Review Comment: Fixed in 3f78320: `anthropic-messages.matches` accepts any decoded Lua table, `passthrough.matches` accepts any non-empty table, and `patch_cjson` preserves decoded arrays as tables. Updated the wording to cover JSON objects and arrays. Validation: `git diff --check`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
