Copilot commented on code in PR #13174: URL: https://github.com/apache/apisix/pull/13174#discussion_r3056230299
########## docs/en/latest/plugins/ai-proxy.md: ########## @@ -314,6 +314,184 @@ You should receive a response similar to the following: } ``` +### Proxy to Gemini + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Google's Gemini API for chat completion. + +[Obtain a Gemini API key](https://ai.google.dev/gemini-api/docs/api-key) and save it to an environment variable: + +```shell +export GEMINI_API_KEY=<your-api-key> # replace with your API key +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-gemini-route", + "uri": "/anything", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "gemini", + "auth": { + "header": { + "Authorization": "Bearer '"$GEMINI_API_KEY"'" + } + }, + "options": { + "model": "gemini-2.5-flash" + } + } + } + }' Review Comment: The header value will literally include single quotes around the key (`Bearer '...') due to the shell-escaping pattern used inside the JSON string. This will likely fail authentication for providers expecting `Bearer <token>` without quotes (and the same quoting pattern appears in other examples in this doc). Recommend adjusting the doc examples so the resulting header value does not contain extra quote characters (e.g., build the JSON with a heredoc / `jq` to safely interpolate variables). ```suggestion --data @- <<EOF { "id": "ai-proxy-gemini-route", "uri": "/anything", "methods": ["POST"], "plugins": { "ai-proxy": { "provider": "gemini", "auth": { "header": { "Authorization": "Bearer ${GEMINI_API_KEY}" } }, "options": { "model": "gemini-2.5-flash" } } } } EOF ``` ########## docs/en/latest/plugins/ai-proxy.md: ########## @@ -391,6 +569,236 @@ You should receive a response similar to the following: } ``` +### Proxy to Vertex AI Embedding Models + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Vertex AI embedding models using GCP service account authentication. + +Before proceeding, follow the same steps as the [Proxy to Vertex AI Chat Completion](#proxy-to-vertex-ai-chat-completion) example to enable Vertex AI and obtain GCP service account credentials. + +Save the JSON to an environment variable: + +```shell +export GCP_SA_JSON="$(cat credentials.json)" +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-vertex-ai-embeddings-route", + "uri": "/embeddings", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "vertex-ai", + "auth": { + "gcp": { + "service_account_json": "'"$GCP_SA_JSON"'" + } + }, + "provider_conf": { + "project_id": "your-project-id", + "region": "us-central1" + }, + "options": { + "model": "gemini-embedding-001" + } + } + } + }' +``` + +The configuration above specifies `vertex-ai` as the provider and uses a Vertex AI Gemini embedding model. Replace the `service_account_json`, `project_id`, and `region` with your GCP service account credentials and project details. + +Send a POST request to the Route with an input string: + +```shell +curl "http://127.0.0.1:9080/embeddings"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "input": "hello world" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "model": "gemini-embedding-001", + "usage": { + "total_tokens": 2, + "prompt_tokens": 2 + }, + "object": "list", + "data": [ + { + "index": 0, + "object": "embedding", + "embedding": [ + -0.0241838414222, + 0.0098769934847951, + 0.0074856607243419, + -0.067302219569683, + ... + ] + } + ] +} +``` + +### Proxy to Anthropic + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Anthropic's Claude API for chat completion. + +Obtain an Anthropic [API key](https://console.anthropic.com/settings/keys) and save it to an environment variable: + +```shell +export ANTHROPIC_API_KEY=<your-api-key> # replace with your API key +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-anthropic-route", + "uri": "/anything", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "anthropic", + "auth": { + "header": { + "x-api-key": "'"$ANTHROPIC_API_KEY"'" + } + }, + "options": { + "model": "claude-sonnet-4-20250514" + } + } + } + }' +``` + +The configuration above specifies `anthropic` as the provider and attaches the Anthropic API key in the `x-api-key` header. + +Send a POST request to the Route with a system prompt and a sample user question in the request body: + +```shell +curl "http://127.0.0.1:9080/anything"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + { "role": "system", "content": "You are a mathematician" }, + { "role": "user", "content": "What is 1+1?" } + ] + }' +``` + +You should receive a response similar to the following: + +```json +{ + "id": "msg_01XFDUDYJgAACzvnptvVoYEL", + "type": "message", + "role": "assistant", + "content": [ + { + "type": "text", + "text": "1+1 equals 2." + } + ], + "model": "claude-sonnet-4-20250514", + "stop_reason": "end_turn", + "usage": { + "input_tokens": 19, + "output_tokens": 11 + } +} +``` + +### Convert Anthropic Requests to OpenAI-Compatible Backend + +The following example demonstrates how the `ai-proxy` Plugin can accept requests in the Anthropic Messages API format and automatically convert them to the OpenAI-compatible format before forwarding to any OpenAI-compatible backend (such as OpenAI, DeepSeek, or other compatible services). This is useful when client applications send Anthropic-formatted requests but you want to use a different LLM backend. + +The protocol conversion is triggered automatically when the Route URI is set to `/v1/messages` (the Anthropic Messages API endpoint). The Plugin will convert Anthropic-formatted requests to OpenAI-compatible format and transform the responses back to Anthropic format. + +Obtain an API key for your chosen OpenAI-compatible backend service and save it to an environment variable. This example uses OpenAI: + +```shell +export BACKEND_API_KEY=<your-api-key> # replace with your API key +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-anthropic-convert-route", + "uri": "/v1/messages", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "openai", + "auth": { + "header": { + "Authorization": "Bearer '"$BACKEND_API_KEY"'" + } + }, + "options": { + "model": "gpt-4" + } + } + } + }' +``` + +The configuration above sets the URI to `/v1/messages` to trigger automatic Anthropic protocol conversion. The backend provider can be any OpenAI-compatible provider, such as `openai`, `deepseek`, or others. + +Send a POST request to the Route in Anthropic Messages API format: + +```shell +curl "http://127.0.0.1:9080/v1/messages"; -X POST \ + -H "Content-Type: application/json" \ + -H "x-api-key: ${BACKEND_API_KEY}" \ Review Comment: This example reuses `BACKEND_API_KEY` (the upstream OpenAI-compatible backend key) as the client-sent Anthropic `x-api-key`. That’s easy to misread as “clients must send the backend key”, which risks accidental key exposure. Recommend either using a clearly separate placeholder variable for the *client* header (or explicitly stating it can be any dummy value / is ignored), and clarifying that the backend key is configured on the route (not provided by the client). ```suggestion The configuration above sets the URI to `/v1/messages` to trigger automatic Anthropic protocol conversion. The backend provider can be any OpenAI-compatible provider, such as `openai`, `deepseek`, or others. The backend API key is configured on the Route in the `Authorization` header above; clients should not send the backend provider key to this endpoint. Send a POST request to the Route in Anthropic Messages API format. The Anthropic-style `x-api-key` header below is only a client-side placeholder for the converted request format and should not reuse the backend provider key: ```shell curl "http://127.0.0.1:9080/v1/messages"; -X POST \ -H "Content-Type: application/json" \ -H "x-api-key: DUMMY_ANTHROPIC_API_KEY" \ ``` ########## docs/zh/latest/plugins/ai-proxy.md: ########## @@ -81,7 +81,7 @@ description: ai-proxy 插件通过将插件配置转换为所需的请求格式 :::note -您可以使用以下命令从 `config.yaml` 获取 `admin_key` 并保存到环境变量中： +你可以使用以下命令从 `config.yaml` 获取 `admin_key` 并保存到环境变量中： Review Comment: This (and a few other changed lines) switches from the formal “您” to the informal “你”, which can make the tone inconsistent within the same Chinese doc. Recommend standardizing on one form across the page (most docs use “您” for consistency/formality). ```suggestion 您可以使用以下命令从 `config.yaml` 获取 `admin_key` 并保存到环境变量中： ``` ########## docs/zh/latest/plugins/ai-proxy.md: ########## @@ -314,6 +314,184 @@ curl "http://127.0.0.1:9080/anything"; -X POST \ } ``` +### 代理到 Gemini + +以下示例演示了如何配置 `ai-proxy` 插件以将请求代理到 Google 的 Gemini API 进行聊天补全。 + +[获取 Gemini API 密钥](https://ai.google.dev/gemini-api/docs/api-key)并保存到环境变量： + +```shell +export GEMINI_API_KEY=<your-api-key> # 替换为你的 API 密钥 +``` + +创建路由并配置 `ai-proxy` 插件： + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-gemini-route", + "uri": "/anything", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "gemini", + "auth": { + "header": { + "Authorization": "Bearer '"$GEMINI_API_KEY"'" + } + }, + "options": { + "model": "gemini-2.5-flash" + } + } + } + }' +``` + +上述配置将 `gemini` 指定为提供商，并在 `Authorization` 标头中附加 Gemini API 密钥。插件将请求代理到聊天补全端点 `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`。要将请求代理到嵌入模型，请在 `override` 字段中显式配置嵌入模型端点。 + +向路由发送 POST 请求，在请求体中包含系统提示和示例用户问题： + +```shell +curl "http://127.0.0.1:9080/anything"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + { "role": "system", "content": "You are a helpful AI assistant" }, + { "role": "user", "content": "What is the capital of France?" } + ] + }' +``` + +你应该收到类似以下的响应： + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "The capital of France is **Paris**.", + "role": "assistant" + } + } + ], + "model": "gemini-2.5-flash", + "object": "chat.completion", + "usage": { + "completion_tokens": 8, + "prompt_tokens": 15, + "total_tokens": 41 + }, + ... +} +``` + +### 代理到 Vertex AI 聊天补全 + +以下示例演示了如何配置 `ai-proxy` 插件，使用 GCP 服务帐户身份验证将请求代理到 Google Cloud 的 Vertex AI 平台。 + +在开始之前： + +* 为你的 GCP 项目[启用 Vertex AI](https://docs.cloud.google.com/vertex-ai/docs/featurestore/setup) 和计费功能。 +* 按照[服务帐户凭证](https://developers.google.com/workspace/guides/create-credentials#service-account)部分在 GCP 中创建服务帐户，为该帐户分配 "Vertex AI User" 角色，并获取 JSON 格式的帐户凭证。 + +你的凭证文件应类似于以下内容： + +```json +{ + "type": "service_account", + "project_id": "your-project-id", + "private_key_id": "...", + "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n", + "client_email": "[email protected]", + "client_id": "....", + "auth_uri": "https://accounts.google.com/o/oauth2/auth";, + "token_uri": "https://oauth2.googleapis.com/token";, + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs";, + "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/your-sa%40your-project-id.iam.gserviceaccount.com";, + "universe_domain": "googleapis.com" +} +``` + +将 JSON 保存到环境变量： + +```shell +export GCP_SA_JSON="$(cat credentials.json)" Review Comment: 与英文文档相同：`cat credentials.json` 得到的是原始 JSON（包含换行和引号），后续再把它拼进 Admin API 的 JSON 字符串字段时，通常会导致提交的 JSON 无效。建议在文档中改为先把凭证文件 JSON-escape 成单行字符串，再安全地注入到 `service_account_json` 字段中（例如用 `jq` 之类的方式生成转义后的字符串）。 ########## docs/en/latest/plugins/ai-proxy.md: ########## @@ -314,6 +314,184 @@ You should receive a response similar to the following: } ``` +### Proxy to Gemini + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Google's Gemini API for chat completion. + +[Obtain a Gemini API key](https://ai.google.dev/gemini-api/docs/api-key) and save it to an environment variable: + +```shell +export GEMINI_API_KEY=<your-api-key> # replace with your API key +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-gemini-route", + "uri": "/anything", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "gemini", + "auth": { + "header": { + "Authorization": "Bearer '"$GEMINI_API_KEY"'" + } + }, + "options": { + "model": "gemini-2.5-flash" + } + } + } + }' +``` + +The configuration above specifies `gemini` as the provider and attaches the Gemini API key in the `Authorization` header. The Plugin proxies requests to the chat completion endpoint at `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`. To proxy requests to an embeddings model, explicitly configure the embeddings model endpoint in the `override` field. + +Send a POST request to the Route with a system prompt and a sample user question in the request body: + +```shell +curl "http://127.0.0.1:9080/anything"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + { "role": "system", "content": "You are a helpful AI assistant" }, + { "role": "user", "content": "What is the capital of France?" } + ] + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "The capital of France is **Paris**.", + "role": "assistant" + } + } + ], + "model": "gemini-2.5-flash", + "object": "chat.completion", + "usage": { + "completion_tokens": 8, + "prompt_tokens": 15, + "total_tokens": 41 + }, + ... +} +``` + +### Proxy to Vertex AI Chat Completion + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Google Cloud's Vertex AI platform using GCP service account authentication. + +Before proceeding: + +* [Enable Vertex AI](https://docs.cloud.google.com/vertex-ai/docs/featurestore/setup) and billing for your GCP project. Review Comment: The link target is for Featurestore setup, which doesn’t match the bullet text “Enable Vertex AI and billing”. Please update to a Vertex AI enablement/getting-started page that actually covers enabling the Vertex AI API and billing. ```suggestion * [Enable Vertex AI](https://cloud.google.com/vertex-ai/docs/start/quickstarts/quickstart-multimodal) and billing for your GCP project. ``` ########## docs/en/latest/plugins/ai-proxy.md: ########## @@ -314,6 +314,184 @@ You should receive a response similar to the following: } ``` +### Proxy to Gemini + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Google's Gemini API for chat completion. + +[Obtain a Gemini API key](https://ai.google.dev/gemini-api/docs/api-key) and save it to an environment variable: + +```shell +export GEMINI_API_KEY=<your-api-key> # replace with your API key +``` + +Create a Route and configure the `ai-proxy` Plugin as such: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes"; -X PUT \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "id": "ai-proxy-gemini-route", + "uri": "/anything", + "methods": ["POST"], + "plugins": { + "ai-proxy": { + "provider": "gemini", + "auth": { + "header": { + "Authorization": "Bearer '"$GEMINI_API_KEY"'" + } + }, + "options": { + "model": "gemini-2.5-flash" + } + } + } + }' +``` + +The configuration above specifies `gemini` as the provider and attaches the Gemini API key in the `Authorization` header. The Plugin proxies requests to the chat completion endpoint at `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions`. To proxy requests to an embeddings model, explicitly configure the embeddings model endpoint in the `override` field. + +Send a POST request to the Route with a system prompt and a sample user question in the request body: + +```shell +curl "http://127.0.0.1:9080/anything"; -X POST \ + -H "Content-Type: application/json" \ + -d '{ + "messages": [ + { "role": "system", "content": "You are a helpful AI assistant" }, + { "role": "user", "content": "What is the capital of France?" } + ] + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "The capital of France is **Paris**.", + "role": "assistant" + } + } + ], + "model": "gemini-2.5-flash", + "object": "chat.completion", + "usage": { + "completion_tokens": 8, + "prompt_tokens": 15, + "total_tokens": 41 + }, + ... +} +``` + +### Proxy to Vertex AI Chat Completion + +The following example demonstrates how you can configure the `ai-proxy` Plugin to proxy requests to Google Cloud's Vertex AI platform using GCP service account authentication. + +Before proceeding: + +* [Enable Vertex AI](https://docs.cloud.google.com/vertex-ai/docs/featurestore/setup) and billing for your GCP project. +* Follow the [service account credentials](https://developers.google.com/workspace/guides/create-credentials#service-account) section to create a service account in GCP, assign the account with the "Vertex AI User" role, and obtain the account credentials in JSON. + +Your credentials file should look similar to the following: + +```json +{ + "type": "service_account", + "project_id": "your-project-id", + "private_key_id": "...", + "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n", + "client_email": "[email protected]", + "client_id": "....", + "auth_uri": "https://accounts.google.com/o/oauth2/auth";, + "token_uri": "https://oauth2.googleapis.com/token";, + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs";, + "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/your-sa%40your-project-id.iam.gserviceaccount.com";, + "universe_domain": "googleapis.com" +} +``` + +Save the JSON to an environment variable: + +```shell +export GCP_SA_JSON="$(cat credentials.json)" Review Comment: `GCP_SA_JSON` is later injected into an Admin API JSON payload as a JSON string value, but `cat credentials.json` preserves raw quotes/newlines and is not JSON-escaped; this commonly produces invalid JSON in the `curl -d '...'` payload. Recommend updating the doc to store an escaped JSON string (for example, using a command that JSON-escapes the file into a single-line string) and then inject it in a way that keeps the overall Admin payload valid JSON. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
