Copilot commented on code in PR #13249:
URL: https://github.com/apache/apisix/pull/13249#discussion_r3108077216
##########
docs/en/latest/plugins/ai-proxy-multi.md:
##########
@@ -58,10 +70,11 @@ In addition, the Plugin also supports logging LLM request
information in the acc
| balancer.key | string | False |
| | Used when `type` is `chash`. When
`hash_on` is set to `header` or `cookie`, `key` is required. When `hash_on` is
set to `consumer`, `key` is not required as the consumer name will be used as
the key automatically. |
| instances | array[object] | True |
| | LLM instance configurations. |
| instances.name | string | True |
| | Name of the LLM service instance. |
-| instances.provider | string | True |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, openai-compatible] | LLM service provider. When
set to `openai`, the Plugin will proxy the request to `api.openai.com`. When
set to `deepseek`, the Plugin will proxy the request to `api.deepseek.com`.
When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver and proxies
the request to `api.aimlapi.com` by default. When set to `anthropic`, the
Plugin will proxy the request to `api.anthropic.com` by default. When set to
`openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the
request to `openrouter.ai` by default. When set to `gemini`, the Plugin uses
the OpenAI-compatible driver and proxies the request to
`generativelanguage.googleapis.com` by default. When set to `vertex-ai`, the
Plugin will proxy the request to `aiplatform.googleapis.com` by default and
requires `provide
r_conf` or `override`. When set to `openai-compatible`, the Plugin will proxy
the request to the custom endpoint configured in `override`. |
-| instances.provider_conf | object | False |
| | Configuration for the specific
provider. Required when `provider` is set to `vertex-ai` and `override` is not
configured. |
+| instances.provider | string | True |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM service
provider. When set to `openai`, the Plugin will proxy the request to
`api.openai.com`. When set to `deepseek`, the Plugin will proxy the request to
`api.deepseek.com`. When set to `aimlapi`, the Plugin uses the
OpenAI-compatible driver and proxies the request to `api.aimlapi.com` by
default. When set to `anthropic`, the Plugin will proxy the request to
`api.anthropic.com` by default. When set to `openrouter`, the Plugin uses the
OpenAI-compatible driver and proxies the request to `openrouter.ai` by default.
When set to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies
the request to `generativelanguage.googleapis.com` by default. When set to
`vertex-ai`, the Plugin will proxy the request to `aiplatform.googleapis.com`
by default and requires
`provider_conf` or `override`. When set to `bedrock`, the Plugin will proxy
the request to the AWS Bedrock Converse API
(`bedrock-runtime.<region>.amazonaws.com`) and signs the request with AWS
SigV4. When set to `openai-compatible`, the Plugin will proxy the request to
the custom endpoint configured in `override`. |
+| instances.provider_conf | object | False |
| | Configuration for the specific
provider. Required when `provider` is set to `vertex-ai` and `override` is not
configured. Required when `provider` is set to `bedrock`. |
| instances.provider_conf.project_id | string | True |
| | Google Cloud Project ID. |
| instances.provider_conf.region | string | True |
| | Google Cloud Region. |
+| instances.provider_conf.region | string | True |
| minLength = 1 | AWS region used to construct the
Bedrock endpoint and to sign the request with SigV4. Required when `provider`
is set to `bedrock`. |
| instances.priority | integer | False | 0
| | Priority of the LLM instance in load
balancing. `priority` takes precedence over `weight`. |
| instances.weight | string | True | 0
| greater or equal to 0 | Weight of the LLM instance in
load balancing. |
| instances.auth | object | True |
| | Authentication configurations. |
Review Comment:
The attributes table now lists `instances.provider_conf.region` twice
(Google Cloud Region vs AWS Region). Duplicate keys in the same table are
ambiguous and make it hard to tell which providers each row applies to. Please
consolidate into a single row with provider-specific semantics/requirements, or
split provider-specific config into separate subsections.
```suggestion
| instances.provider_conf.region | string | True |
| minLength = 1 | Provider-specific region. For
`vertex-ai`, this is the Google Cloud region. For `bedrock`, this is the AWS
region used to construct the Bedrock endpoint and sign the request with SigV4.
Required when `provider` is set to `bedrock`, and required for `vertex-ai` when
`provider_conf` is used. |
| instances.priority | integer | False | 0
| | Priority of the LLM instance in load
balancing. `priority` takes precedence over `weight`. |
| instances.weight | string | True | 0
| greater or equal to 0 | Weight of the LLM instance in
load balancing. |
| instances.auth | object | True |
| | Authentication configurations. |
```
##########
docs/en/latest/plugins/ai-proxy.md:
##########
@@ -47,25 +47,42 @@ In addition, the Plugin also supports logging LLM request
information in the acc
| `messages.role` | String | True | Role of the message (`system`,
`user`, `assistant`).|
| `messages.content` | String | True | Content of the message.
|
+### Bedrock Converse Request Format
+
+When `provider` is set to `bedrock`, the Plugin expects requests in the
[Bedrock Converse
API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html)
format. The request URI must end with `/converse` and the body must contain a
`messages` array.
+
+| Name | Type | Required | Description
|
+| ------------------ | ------ | -------- |
----------------------------------------------------------------------------------------------------
|
+| `messages` | Array | True | An array of message objects.
|
+| `messages.role` | String | True | Role of the message (`user`,
`assistant`). |
+| `messages.content` | Array | True | An array of content blocks. Each
block contains a `text` field (e.g., `[{"text": "What is 1+1?"}]`). |
+| `system` | Array | False | Optional system prompt blocks
(e.g., `[{"text": "You are a helpful assistant."}]`). |
+| `inferenceConfig` | Object | False | Optional inference parameters such
as `maxTokens`, `temperature`, `topP`, etc. |
+
## Attributes
| Name | Type | Required | Default | Valid values
| Description |
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | True | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai,
openai-compatible] | LLM service provider. When set to `openai`, the Plugin
will proxy the request to `https://api.openai.com/chat/completions`. When set
to `deepseek`, the Plugin will proxy the request to
`https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin
uses the OpenAI-compatible driver and proxies the request to
`https://api.aimlapi.com/v1/chat/completions` by default. When set to
`anthropic`, the Plugin will proxy the request to
`https://api.anthropic.com/v1/chat/completions` by default. When set to
`openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the
request to `https://openrouter.ai/api/v1/chat/completions` by default. When set
to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the
request to
`https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` by
defa
ult. When set to `vertex-ai`, the Plugin will proxy the request to
`https://aiplatform.googleapis.com` by default and requires `provider_conf` or
`override`. When set to `openai-compatible`, the Plugin will proxy the request
to the custom endpoint configured in `override`. |
-| provider_conf | object | False | |
| Configuration for the specific provider. Required when
`provider` is set to `vertex-ai` and `override` is not configured. |
+| provider | string | True | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM service provider. When set to `openai`, the Plugin
will proxy the request to `https://api.openai.com/chat/completions`. When set
to `deepseek`, the Plugin will proxy the request to
`https://api.deepseek.com/chat/completions`. When set to `aimlapi`, the Plugin
uses the OpenAI-compatible driver and proxies the request to
`https://api.aimlapi.com/v1/chat/completions` by default. When set to
`anthropic`, the Plugin will proxy the request to
`https://api.anthropic.com/v1/chat/completions` by default. When set to
`openrouter`, the Plugin uses the OpenAI-compatible driver and proxies the
request to `https://openrouter.ai/api/v1/chat/completions` by default. When set
to `gemini`, the Plugin uses the OpenAI-compatible driver and proxies the
request to
`https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
` by default. When set to `vertex-ai`, the Plugin will proxy the request to
`https://aiplatform.googleapis.com` by default and requires `provider_conf` or
`override`. When set to `bedrock`, the Plugin will proxy the request to the AWS
Bedrock Converse API (`https://bedrock-runtime.<region>.amazonaws.com`) and
signs the request with AWS SigV4. When set to `openai-compatible`, the Plugin
will proxy the request to the custom endpoint configured in `override`. |
+| provider_conf | object | False | |
| Configuration for the specific provider. Required when
`provider` is set to `vertex-ai` and `override` is not configured. Required
when `provider` is set to `bedrock`. |
| provider_conf.project_id | string | True | |
| Google Cloud Project ID. |
| provider_conf.region | string | True | |
| Google Cloud Region. |
+| provider_conf.region | string | True | | minLength = 1
| AWS region used to construct the Bedrock endpoint and to sign
the request with SigV4. Required when `provider` is set to `bedrock`. |
Review Comment:
The attributes table now lists `provider_conf.region` twice (once for Google
Cloud Region and once for AWS Region). This is ambiguous for readers and makes
it unclear which providers each row applies to. Please consolidate into a
single `provider_conf.region` row that explains the per-provider
meaning/requirements, or split provider-specific configuration into separate
subsections to avoid duplicate keys.
```suggestion
| provider_conf.region | string | True | | minLength = 1
| Provider-specific region. For `vertex-ai`, this is the
Google Cloud region. For `bedrock`, this is the AWS region used to construct
the Bedrock endpoint and sign the request with SigV4. Required when `provider`
is set to `bedrock`. |
```
##########
apisix/plugins/ai-providers/base.lua:
##########
@@ -187,6 +187,20 @@ function _M.build_request(self, ctx, conf, request_body,
opts)
request_body.model = nil
end
+ -- AWS SigV4 signing (must be last — signs the finalized body)
+ if auth.aws then
+ local auth_aws = require("apisix.plugins.ai-transport.auth-aws")
+ local region = opts.conf and opts.conf.region
+ if not region then
+ return nil, "missing region for AWS SigV4 signing "
+ .. "(provider_conf.region required for bedrock)"
+ end
+ local sign_err = auth_aws.sign_request(params, auth.aws, region)
+ if sign_err then
+ return nil, "failed to sign AWS request: " .. sign_err
+ end
+ end
Review Comment:
AWS SigV4 signing is triggered solely by the presence of `auth.aws`,
regardless of which provider is selected. Because the schema doesn’t currently
restrict `auth.aws` to Bedrock-only, a misconfigured instance could end up
SigV4-signing requests to non-AWS providers (adding/overwriting
`authorization`, `host`, etc.). Consider gating this behind `self`/provider
type (e.g., only sign when provider is `bedrock`) and/or tightening the schema
so `auth.aws` is only allowed/required for Bedrock.
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -120,9 +145,16 @@ local ai_instance_schema = {
properties = {
endpoint = {
type = "string",
- description = "To be specified to override the
endpoint of the AI Instance",
+ description = "Override the endpoint of the AI
Instance. "
+ .. "Typically used for custom hosts (e.g., AWS "
+ .. "PrivateLink, reverse proxies) — provide only
the "
+ .. "scheme + host so the plugin computes the path.
"
+ .. "If you include a path with reserved characters
"
+ .. "(e.g., Bedrock inference profile ARNs
containing "
+ .. "':' or '/'), they must be URL-encoded.",
Review Comment:
The schema description for `override.endpoint` says to “provide only the
scheme + host so the plugin computes the path”, but `ai-providers/base.lua`
explicitly uses `parsed_url.path` when a path is present (and the Bedrock
tests/examples set a full `/model/.../converse` path). This description is
misleading; please update it to reflect that `override.endpoint` may include a
path (and query), and clarify when the plugin computes vs honors the provided
path.
```suggestion
.. "PrivateLink, reverse proxies). You may
provide "
.. "only the scheme + host, in which case the
plugin "
.. "computes the provider-specific path, or
provide "
.. "a full endpoint including path and query, in
"
.. "which case the plugin uses the supplied
path/query. "
.. "If your custom path or query contains
reserved "
.. "characters (e.g., Bedrock inference profile
ARNs "
.. "containing ':' or '/'), they must be
URL-encoded.",
```
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -133,19 +165,51 @@ local ai_instance_schema = {
}
},
required = {"name", "provider", "auth", "weight"},
- ["if"] = {
- properties = { provider = { enum = { "vertex-ai" } } },
- },
- ["then"] = {
- properties = {
- provider_conf = provider_vertex_ai_schema,
+ allOf = {
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "vertex-ai" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_vertex_ai_schema,
+ },
+ anyOf = {
+ { required = { "provider_conf" } },
+ { required = { "override" } },
+ },
+ },
+ },
+ {
+ ["if"] = {
+ properties = { provider = { enum = { "bedrock" } } },
+ },
+ ["then"] = {
+ properties = {
+ provider_conf = provider_bedrock_schema,
+ },
+ required = { "provider_conf" },
Review Comment:
For `provider=bedrock`, the schema currently enforces `provider_conf.region`
and (optionally) `options.model`, but it does not require `auth.aws`. Given the
provider relies on SigV4 signing, it’s better to fail fast at config time by
requiring `auth.aws` (and potentially disallowing it for non-AWS providers) to
avoid unsigned requests reaching Bedrock and returning confusing upstream
errors.
```suggestion
auth = {
required = { "aws" },
},
},
required = { "provider_conf", "auth" },
```
##########
t/plugin/ai-proxy-bedrock.t:
##########
@@ -0,0 +1,553 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+use t::APISIX 'no_plan';
+
+log_level("info");
+repeat_each(1);
+no_long_string();
+no_shuffle();
+no_root_location();
+
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!defined $block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ my $main_config = $block->main_config // <<_EOC_;
+ env AWS_EC2_METADATA_DISABLED=true;
+_EOC_
+ $block->set_value("main_config", $main_config);
+
+ my $user_yaml_config = <<_EOC_;
+plugins:
+ - ai-proxy-multi
+ - prometheus
+_EOC_
+ $block->set_value("extra_yaml_config", $user_yaml_config);
+
+ my $http_config = $block->http_config // <<_EOC_;
+ server {
+ server_name bedrock;
+ listen 6724;
+
+ default_type 'application/json';
+
+ location ~ ^/model/.+/converse\$ {
+ content_by_lua_block {
+ local json = require("cjson.safe")
+
+ -- Log the raw request URI so tests can assert the exact
+ -- path shape (e.g., that ARN model IDs are URL-encoded
+ -- as a single path segment).
+ ngx.log(ngx.WARN, "[test] received uri: ",
ngx.var.request_uri)
+
+ if ngx.req.get_method() ~= "POST" then
+ ngx.status = 400
+ ngx.say("Unsupported request method: ",
ngx.req.get_method())
+ return
+ end
+
+ -- Check SigV4 auth headers
+ local auth_header = ngx.req.get_headers()["authorization"]
+ local amz_date = ngx.req.get_headers()["x-amz-date"]
+ if not auth_header or not amz_date then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Missing Authentication Token"
+ }))
+ return
+ end
+
+ -- Validate Authorization header structure (SigV4):
+ -- AWS4-HMAC-SHA256
Credential=<AK>/<DATE>/<REGION>/<SERVICE>/aws4_request,
+ -- SignedHeaders=<list>, Signature=<64 hex>
+ if not auth_header:match("^AWS4%-HMAC%-SHA256 ") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing
AWS4-HMAC-SHA256 algorithm prefix"
+ }))
+ return
+ end
+
+ if not auth_header:match("Credential=[^,]+") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing Credential
component"
+ }))
+ return
+ end
+
+ -- Strict credential scope:
access_key/<date>/us-east-1/bedrock/aws4_request
+ if not auth_header:match(
+
"Credential=AKIAIOSFODNN7EXAMPLE/%d%d%d%d%d%d%d%d/us%-east%-1/bedrock/aws4_request"
+ ) then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization Credential scope does not
match expected "
+ ..
"AKIAIOSFODNN7EXAMPLE/<DATE>/us-east-1/bedrock/aws4_request"
+ }))
+ return
+ end
+
+ if not auth_header:match("SignedHeaders=[^,]+") then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization header missing
SignedHeaders component"
+ }))
+ return
+ end
+
+ -- Lua patterns don't support {n} quantifiers, so match
+ -- exactly 64 hex chars by repeating %x sixty-four times.
+ local hex64 = string.rep("%x", 64)
+ if not auth_header:match("Signature=" .. hex64) then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "Authorization Signature is missing or
not 64 hex chars"
+ }))
+ return
+ end
+
+ -- Validate X-Amz-Date format: YYYYMMDDTHHMMSSZ
+ if not amz_date:match("^%d%d%d%d%d%d%d%dT%d%d%d%d%d%dZ\$")
then
+ ngx.status = 403
+ ngx.say(json.encode({
+ message = "X-Amz-Date header does not match
YYYYMMDDTHHMMSSZ format"
+ }))
+ return
+ end
+
+ -- Capture session token if provided so tests can assert
+ -- that auth.aws.session_token was propagated as
+ -- x-amz-security-token.
+ local session_token =
ngx.req.get_headers()["x-amz-security-token"]
+
+ ngx.req.read_body()
+ local body_data = ngx.req.get_body_data()
+ local body, err = json.decode(body_data)
+
+ if not body then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "Invalid JSON: " ..
(err or "") }))
+ return
+ end
+
+ -- Verify model is NOT in the body (remove_model = true)
+ if body.model then
+ ngx.status = 400
+ ngx.say(json.encode({
+ message = "model field should not be in request
body"
+ }))
+ return
+ end
+
+ -- Verify request has messages
+ if not body.messages or #body.messages < 1 then
+ ngx.status = 400
+ ngx.say(json.encode({ message = "messages is required"
}))
+ return
+ end
+
+ -- Return Bedrock Converse response
+ local assistant_text = "1 + 1 = 2."
+ if session_token then
+ assistant_text = assistant_text
+ .. " session_token_seen=" .. session_token
+ end
+ ngx.status = 200
+ ngx.say(json.encode({
+ output = {
+ message = {
+ role = "assistant",
+ content = {{text = assistant_text}}
+ }
+ },
+ stopReason = "end_turn",
+ usage = {
+ inputTokens = 10,
+ outputTokens = 8,
+ totalTokens = 18
+ }
+ }))
+ }
+ }
+ }
+_EOC_
+
+ $block->set_value("http_config", $http_config);
+});
+
+run_tests();
+
+__DATA__
+
+=== TEST 1: set route with bedrock provider
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "uri": "/ai/converse",
+ "plugins": {
+ "ai-proxy-multi": {
+ "instances": [
+ {
+ "name": "bedrock",
+ "provider": "bedrock",
Review Comment:
This test suite exercises Bedrock via `ai-proxy-multi` only. Since the PR
claims Bedrock support for both `ai-proxy` and `ai-proxy-multi`, it would be
good to add at least one analogous happy-path + schema-validation case for the
single-instance `ai-proxy` plugin (especially around required
`provider_conf.region` and `auth.aws`).
##########
docs/zh/latest/plugins/ai-proxy-multi.md:
##########
@@ -58,10 +70,11 @@ description: ai-proxy-multi 插件通过负载均衡、重试、故障转移和
| balancer.key | string | 否 |
| | 当 `type` 为 `chash` 时使用。当 `hash_on` 设置为
`header` 或 `cookie` 时，需要 `key`。当 `hash_on` 设置为 `consumer` 时，不需要
`key`，因为消费者名称将自动用作键。 |
| instances | array[object] | 是 |
| | LLM 实例配置。 |
| instances.name | string | 是 |
| | LLM 服务实例的名称。 |
-| instances.provider | string | 是 |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, openai-compatible] | LLM 服务提供商。设置为 `openai`
时，插件将代理请求到 `api.openai.com`。设置为 `deepseek` 时，插件将代理请求到 `api.deepseek.com`。设置为
`aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.aimlapi.com`。设置为 `anthropic`
时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到 `api.anthropic.com`。设置为 `openrouter` 时，插件使用
OpenAI 兼容驱动程序，默认将请求代理到 `openrouter.ai`。设置为 `gemini` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到 `generativelanguage.googleapis.com`。设置为 `vertex-ai`
时，插件默认将请求代理到 `aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。设置为
`openai-
compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
-| instances.provider_conf | object | 否 |
| | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置
`override` 时必填。 |
+| instances.provider | string | 是 |
| [openai, deepseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, bedrock, openai-compatible] | LLM 服务提供商。设置为
`openai` 时，插件将代理请求到 `api.openai.com`。设置为 `deepseek` 时，插件将代理请求到
`api.deepseek.com`。设置为 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`api.aimlapi.com`。设置为 `anthropic` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`api.anthropic.com`。设置为 `openrouter` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`openrouter.ai`。设置为 `gemini` 时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`generativelanguage.googleapis.com`。设置为 `vertex-ai` 时，插件默认将请求代理到
`aiplatform.googleapis.com`，且需要配置 `provider_conf` 或 `override`。设置为
`bedrock` 时，插件将代理请求到 AWS Bedrock Converse
API（`bedrock-runtime.<region>.amazonaws.com`），并使用 AWS SigV4 对请求进行签名。设置为
`openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
+| instances.provider_conf | object | 否 |
| | 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置
`override` 时必填。当 `provider` 设置为 `bedrock` 时必填。 |
| instances.provider_conf.project_id | string | 是 |
| | Google Cloud 项目 ID。 |
| instances.provider_conf.region | string | 是 |
| | Google Cloud 区域。 |
+| instances.provider_conf.region | string | 是 |
| minLength = 1 | 用于构造 Bedrock 端点并使用 SigV4 对请求进行签名的 AWS 区域。当
`provider` 设置为 `bedrock` 时必填。 |
Review Comment:
属性表中 `instances.provider_conf.region` 现在出现了两次（一次表示 Google Cloud Region，一次表示
AWS Region）。同一表格内重复键会导致含义不清晰。建议合并为同一行并在描述中注明不同 provider 的语义/必选条件，或将
provider-specific 配置拆分到单独小节。
```suggestion
| instances.provider_conf.region | string | 是 |
| minLength = 1 | 区域配置：当 `provider` 设置为 `vertex-ai` 时表示
Google Cloud 区域；当 `provider` 设置为 `bedrock` 时表示 AWS 区域，用于构造 Bedrock 端点并使用 SigV4
对请求进行签名，且该场景下必填。 |
```
##########
docs/zh/latest/plugins/ai-proxy.md:
##########
@@ -47,25 +47,42 @@ description: ai-proxy 插件通过将插件配置转换为所需的请求格式
| `messages.role` | String | 是 | 消息的角色（`system`、`user`、`assistant`）。|
| `messages.content` | String | 是 | 消息的内容。 |
+### Bedrock Converse 请求格式
+
+当 `provider` 设置为 `bedrock` 时，插件期望请求采用 [Bedrock Converse
API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html)
格式。请求 URI 必须以 `/converse` 结尾，且请求体必须包含 `messages` 数组。
+
+| 名称 | 类型 | 必选项 | 描述
|
+| ------------------ | ------ | -------- |
----------------------------------------------------------------------------------------------------
|
+| `messages` | Array | 是 | 消息对象数组。
|
+| `messages.role` | String | 是 | 消息的角色（`user`、`assistant`）。
|
+| `messages.content` | Array | 是 | 内容块数组。每个块包含一个 `text` 字段（例如 `[{"text":
"What is 1+1?"}]`）。 |
+| `system` | Array | 否 | 可选的系统提示块（例如 `[{"text": "You are a
helpful assistant."}]`）。 |
+| `inferenceConfig` | Object | 否 | 可选的推理参数，如
`maxTokens`、`temperature`、`topP` 等。 |
+
## 属性
| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述
|
|--------------------|--------|----------|---------|------------------------------------------|-------------|
-| provider | string | 是 | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai,
openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件将代理请求到
`https://api.openai.com/chat/completions`。当设置为 `deepseek` 时，插件将代理请求到
`https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为 `anthropic`
时，插件将代理请求到 `https://api.anthropic.com/v1/chat/completions`。当设置为 `openrouter`
时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`https://openrouter.ai/api/v1/chat/completions`。当设置为 `gemini` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到
`https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`。当设置为
`vertex-ai` 时，插件默认将请求
代理到 `https://aiplatform.googleapis.com`，且需要配置 `provider_conf` 或
`override`。当设置为 `openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
-| provider_conf | object | 否 | |
| 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。 |
+| provider | string | 是 | | [openai, deepseek,
azure-openai, aimlapi, anthropic, openrouter, gemini, vertex-ai, bedrock,
openai-compatible] | LLM 服务提供商。当设置为 `openai` 时，插件将代理请求到
`https://api.openai.com/chat/completions`。当设置为 `deepseek` 时，插件将代理请求到
`https://api.deepseek.com/chat/completions`。当设置为 `aimlapi` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到 `https://api.aimlapi.com/v1/chat/completions`。当设置为 `anthropic`
时，插件将代理请求到 `https://api.anthropic.com/v1/chat/completions`。当设置为 `openrouter`
时，插件使用 OpenAI 兼容驱动程序，默认将请求代理到
`https://openrouter.ai/api/v1/chat/completions`。当设置为 `gemini` 时，插件使用 OpenAI
兼容驱动程序，默认将请求代理到
`https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`。当设置为
`vertex-ai` 时，插件默认
将请求代理到 `https://aiplatform.googleapis.com`，且需要配置 `provider_conf` 或
`override`。当设置为 `bedrock` 时，插件将代理请求到 AWS Bedrock Converse
API（`https://bedrock-runtime.<region>.amazonaws.com`），并使用 AWS SigV4
对请求进行签名。当设置为 `openai-compatible` 时，插件将代理请求到在 `override` 中配置的自定义端点。 |
+| provider_conf | object | 否 | |
| 特定提供商的配置。当 `provider` 设置为 `vertex-ai` 且未配置 `override` 时必填。当
`provider` 设置为 `bedrock` 时必填。 |
| provider_conf.project_id | string | 是 | |
| Google Cloud 项目 ID。 |
| provider_conf.region | string | 是 | |
| Google Cloud 区域。 |
+| provider_conf.region | string | 是 | | minLength = 1
| 用于构造 Bedrock 端点并使用 SigV4 对请求进行签名的 AWS 区域。当 `provider` 设置为
`bedrock` 时必填。 |
Review Comment:
属性表中 `provider_conf.region` 现在出现了两次（一次表示 Google Cloud Region，一次表示 AWS
Region）。这会让读者难以区分该字段在不同 provider 下的含义/必选条件。建议合并为同一行并在描述中注明不同 provider 的语义与要求，或将
provider-specific 配置拆分到单独小节，避免重复键。
```suggestion
| provider_conf.region | string | 是 | | minLength = 1
| 区域配置：当 `provider` 设置为 `vertex-ai` 时，表示 Google Cloud 区域；当
`provider` 设置为 `bedrock` 时，表示 AWS 区域，用于构造 Bedrock 端点并使用 SigV4 对请求进行签名。在
`bedrock` 场景下该字段必填且不能为空。 |
```
##########
docs/en/latest/plugins/ai-proxy-multi.md:
##########
@@ -7,7 +7,7 @@ keywords:
- ai-proxy-multi
- AI
- LLM
-description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy
with load balancing, retries, fallbacks, and health chekcs, simplifying the
integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter,
Gemini, Vertex AI, and other OpenAI-compatible APIs.
+description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy
with load balancing, retries, fallbacks, and health chekcs, simplifying the
integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter,
Gemini, Vertex AI, Amazon Bedrock, and other OpenAI-compatible APIs.
Review Comment:
Typo in the front-matter description: “health chekcs” should be “health
checks”.
```suggestion
description: The ai-proxy-multi Plugin extends the capabilities of ai-proxy
with load balancing, retries, fallbacks, and health checks, simplifying the
integration with OpenAI, DeepSeek, Azure, AIMLAPI, Anthropic, OpenRouter,
Gemini, Vertex AI, Amazon Bedrock, and other OpenAI-compatible APIs.
```
##########
apisix/plugins/ai-proxy/schema.lua:
##########
@@ -200,10 +264,14 @@ _M.ai_proxy_schema = {
description = "To be specified to override the endpoint of
the AI Instance",
},
},
+ required = { "endpoint" },
},
},
required = {"provider", "auth"},
- encrypt_fields = {"auth.header", "auth.query",
"auth.gcp.service_account_json"},
+ encrypt_fields = {
+ "auth.header", "auth.query", "auth.gcp.service_account_json",
+ "auth.aws.secret_access_key", "auth.aws.session_token",
+ },
Review Comment:
For the single-instance `ai-proxy` schema, Bedrock-specific requirements
described in this PR (e.g., `provider_conf.region` and `auth.aws` being
required for `provider=bedrock`) are not enforced here—only the encrypt_fields
list was updated. This allows misconfigured Bedrock routes to pass schema
validation and fail later at runtime during SigV4 signing/path construction.
Please add Bedrock-specific conditional requirements to `ai_proxy_schema`
(similar to the `instances` schema in this file).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
