This is an automated email from the ASF dual-hosted git repository.
Yilialinn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apisix.git
The following commit(s) were added to refs/heads/master by this push:
new 449460168 docs(ai-request-rewrite): re-port with Admin API, ADC, and
Ingress Controller tabs (#13210)
449460168 is described below
commit 44946016806b12273b39aa122f6c3ab649406169
Author: Yilia Lin <[email protected]>
AuthorDate: Wed Apr 22 14:45:03 2026 +0800
docs(ai-request-rewrite): re-port with Admin API, ADC, and Ingress
Controller tabs (#13210)
---
docs/en/latest/plugins/ai-request-rewrite.md | 869 +++++++++++++++++++++++++--
docs/zh/latest/plugins/ai-request-rewrite.md | 863 ++++++++++++++++++++++++--
2 files changed, 1611 insertions(+), 121 deletions(-)
diff --git a/docs/en/latest/plugins/ai-request-rewrite.md
b/docs/en/latest/plugins/ai-request-rewrite.md
index 6304f59fe..df518598d 100644
--- a/docs/en/latest/plugins/ai-request-rewrite.md
+++ b/docs/en/latest/plugins/ai-request-rewrite.md
@@ -5,7 +5,7 @@ keywords:
- AI Gateway
- Plugin
- ai-request-rewrite
-description: The ai-request-rewrite plugin intercepts client requests before
they are forwarded to the upstream service. It sends a predefined prompt, along
with the original request body, to a specified LLM service. The LLM processes
the input and returns a modified request body, which is then used for the
upstream request. This allows dynamic transformation of API requests based on
AI-generated content.
+description: The ai-request-rewrite plugin forwards client requests to LLM
services for processing before sending them upstream, enabling AI-driven
redaction, enrichment, and reformatting.
---
<!--
@@ -27,70 +27,87 @@ description: The ai-request-rewrite plugin intercepts
client requests before the
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/ai-request-rewrite"; />
+</head>
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
## Description
-The `ai-request-rewrite` plugin intercepts client requests before they are
forwarded to the upstream service. It sends a predefined prompt, along with the
original request body, to a specified LLM service. The LLM processes the input
and returns a modified request body, which is then used for the upstream
request. This allows dynamic transformation of API requests based on
AI-generated content.
+The `ai-request-rewrite` Plugin processes client requests by forwarding them
to LLM services for transformation before relaying them to Upstream services.
This enables LLM-powered modifications such as data redaction, content
enrichment, or reformatting. The Plugin supports integration with OpenAI,
DeepSeek, Gemini, Vertex AI, Anthropic, OpenRouter, and other OpenAI-compatible
APIs.
## Plugin Attributes
-| **Field** | **Required** | **Type** | **Description**
|
-| ------------------------- | ------------ | -------- |
------------------------------------------------------------------------------------
|
-| prompt | Yes | String | The prompt send to LLM
service. |
-| provider | Yes | String | Name of the LLM
service. Available options: openai, deekseek, azure-openai, aimlapi, anthropic,
openrouter, gemini, vertex-ai, and openai-compatible. When `aimlapi` is
selected, the plugin uses the OpenAI-compatible driver with a default endpoint
of `https://api.aimlapi.com/v1/chat/completions`. |
-| provider_conf | No | Object | Configuration for the
specific provider. Required when `provider` is set to `vertex-ai` and
`override` is not configured. |
-| provider_conf.project_id | Yes | String | Google Cloud Project
ID. |
-| provider_conf.region | Yes | String | Google Cloud Region. |
-| auth | Yes | Object | Authentication
configuration |
-| auth.header | No | Object | Authentication
headers. Key must match pattern `^[a-zA-Z0-9._-]+$`. |
-| auth.query | No | Object | Authentication query
parameters. Key must match pattern `^[a-zA-Z0-9._-]+$`. |
-| auth.gcp | No | Object | Configuration for
Google Cloud Platform (GCP) authentication. |
-| auth.gcp.service_account_json | No | String | Content of the GCP
service account JSON file. This can also be configured by setting the
`GCP_SERVICE_ACCOUNT` environment variable. |
-| auth.gcp.max_ttl | No | Integer | Maximum TTL (in
seconds) for caching the GCP access token. Minimum: 1. |
-| auth.gcp.expire_early_secs| No | Integer | Seconds to expire the
access token before its actual expiration time to avoid edge cases. Minimum: 0.
Default: 60. |
-| options | No | Object | Key/value settings for
the model |
-| options.model | No | String | Model to execute.
Examples: "gpt-3.5-turbo" for openai, "deepseek-chat" for deekseek, or
"qwen-turbo" for openai-compatible or aimlapi services |
-| override.endpoint | No | String | Override the default
endpoint when using OpenAI-compatible services (e.g., self-hosted models or
third-party LLM services). When the provider is 'openai-compatible', the
endpoint field is required. |
-| timeout | No | Integer | Total timeout in
milliseconds for requests to LLM service, including connect, send, and read
timeouts. Range: 1 - 60000. Default: 30000|
-| keepalive | No | Boolean | Enable keepalive for
requests to LLM service. Default: true |
-| keepalive_timeout | No | Integer | Keepalive timeout in
milliseconds for requests to LLM service. Minimum: 1000. Default: 60000 |
-| keepalive_pool | No | Integer | Keepalive pool size
for requests to LLM service. Minimum: 1. Default: 30 |
-| ssl_verify | No | Boolean | SSL verification for
requests to LLM service. Default: true |
-
-## How it works
-
-
+| Name | Type | Required | Default | Valid values | Description |
+| --- | --- | --- | --- | --- | --- |
+| `prompt` | string | True | | | The prompt to send to the LLM service for
rewriting the client request. |
+| `provider` | string | True | | [openai, deepseek, azure-openai, aimlapi,
gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM service
provider. When set to `aimlapi`, the Plugin uses the OpenAI-compatible driver
and proxies the request to `https://api.aimlapi.com/v1/chat/completions`. When
set to `openai-compatible`, the Plugin proxies requests to the custom endpoint
configured in `override`. When set to `azure-openai`, the Plugin also proxies
requests to the custom endpo [...]
+| `auth` | object | True | | | Authentication configurations. |
+| `auth.header` | object | False | | | Authentication headers. Key must match
pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query` should be
configured. |
+| `auth.query` | object | False | | | Authentication query parameters. Key
must match pattern `^[a-zA-Z0-9._-]+$`. At least one of `header` and `query`
should be configured. |
+| `options` | object | False | | | Model configurations. In addition to
`model`, you can configure additional parameters and they will be forwarded to
the upstream LLM service in the request body. For instance, if you are working
with OpenAI, you can configure additional parameters such as `temperature`,
`top_p`, and `stream`. See your LLM provider's API documentation for more
available options. |
+| `options.model` | string | False | | | Name of the LLM model, such as
`gpt-4` or `gpt-3.5`. See your LLM provider's API documentation for more
available models. |
+| `override` | object | False | | | Override setting. |
+| `override.endpoint` | string | False | | | LLM provider endpoint. Required
when `provider` is `openai-compatible`. |
+| `timeout` | integer | False | 30000 | 1 - 60000 | Request timeout in
milliseconds when requesting the LLM service. |
+| `keepalive` | boolean | False | true | | If true, keep the connection alive
when requesting the LLM service. |
+| `keepalive_timeout` | integer | False | 60000 | ≥ 1000 | Keepalive timeout
in milliseconds for requests to the LLM service. |
+| `keepalive_pool` | integer | False | 30 | ≥ 1 | Keepalive pool size for
connections to the LLM service. |
+| `ssl_verify` | boolean | False | true | | If true, verify the LLM service's
SSL certificate. |
+
+## How It Works
+
+
## Examples
The examples below demonstrate how you can configure `ai-request-rewrite` for
different scenarios.
+The examples use OpenAI as the LLM service. To follow along, obtain an OpenAI
[API key](https://openai.com/blog/openai-api) and save it to an environment
variable:
+
+```shell
+export OPENAI_API_KEY=<your-api-key>
+```
+
:::note
-You can fetch the admin_key from config.yaml and save to an environment
variable with the following command:
+You can fetch the `admin_key` from `config.yaml` and save to an environment
variable with the following command:
+```shell
admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed
's/"//g')
+```
:::
-### Redact sensitive information
+### Redact Sensitive Information
+
+The following example demonstrates how to use the `ai-request-rewrite` Plugin
to redact sensitive information before the request reaches the Upstream service.
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+Create a Route and configure the `ai-request-rewrite` Plugin. The `provider`
is set to `openai`, the OpenAI API key is passed in the `Authorization` header,
and the `prompt` instructs the LLM to identify and mask sensitive information:
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
-H "X-API-KEY: ${admin_key}" \
-d '{
"uri": "/anything",
+ "methods": ["POST"],
"plugins": {
"ai-request-rewrite": {
- "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged.",
"provider": "openai",
"auth": {
"header": {
- "Authorization": "Bearer <some-token>"
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
}
},
- "options": {
+ "options":{
"model": "gpt-4"
- }
+ },
+ "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
}
},
"upstream": {
@@ -102,53 +119,623 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X
PUT \
}'
```
-Now send a request:
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+Create a Route with the `ai-request-rewrite` Plugin:
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+Synchronize the configuration to the gateway:
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+```
+
+</TabItem>
+</Tabs>
+
+Apply the configuration to your cluster:
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+Send a POST request to the Route with some personally identifiable information:
+
+```shell
+curl "http://127.0.0.1:9080/anything"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "content": "John said his debit card number is 4111 1111 1111 1111 and SIN
is 123-45-6789."
+ }'
+```
+
+You should receive a response similar to the following:
+
+```json
+{
+ "args": {},
+ "data": "{\"content\": \"John said his debit card number is **** **** ****
1111 and SIN is ***-**-***.\"}",
+ ...,
+ "json": {
+ "content": "John said his debit card number is **** **** **** 1111 and SIN
is ***-**-***."
+ },
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.170",
+ "url": "http://127.0.0.1/anything";
+}
+```
+
+### Reformat Data
+
+The following example demonstrates how to use the `ai-request-rewrite` Plugin
to reformat data before the request reaches the Upstream service.
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+Create a Route and configure the `ai-request-rewrite` Plugin. The `prompt`
instructs the LLM to convert natural language queries into structured JSON:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/anything",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-request-rewrite": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options":{
+ "model": "gpt-4"
+ },
+ "prompt": "Convert natural language queries into structured JSON
format with intent and extracted parameters."
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "httpbin.org:80": 1
+ }
+ }
+ }'
+```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+Create a Route with the `ai-request-rewrite` Plugin:
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON
format with intent and extracted parameters."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+Synchronize the configuration to the gateway:
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON format
with intent and extracted parameters."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON
format with intent and extracted parameters."
+```
+
+</TabItem>
+</Tabs>
+
+Apply the configuration to your cluster:
```shell
-curl "http://127.0.0.1:9080/anything"; \
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+Send a POST request to the Route:
+
+```shell
+curl "http://127.0.0.1:9080/anything"; -X POST \
-H "Content-Type: application/json" \
-d '{
- "name": "John Doe",
- "email": "[email protected]",
- "credit_card": "4111 1111 1111 1111",
- "ssn": "123-45-6789",
- "address": "123 Main St"
+ "content": "Book a flight from NYC to LA on April 10, 2022."
}'
```
-The request body send to the LLM Service is as follows:
+You should receive a response similar to the following:
```json
{
- "messages": [
- {
- "role": "system",
- "content": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., '*** **** **** 1234') for
credit card numbers). Ensure the JSON structure remains unchanged."
- },
- {
- "role": "user",
- "content": "{\n\"name\":\"John
Doe\",\n\"email\":\"[email protected]\",\n\"credit_card\":\"4111 1111 1111
1111\",\n\"ssn\":\"123-45-6789\",\n\"address\":\"123 Main St\"\n}"
- }
- ]
+ "args": {},
+ "data": "{\n \"intent\": \"BookFlight\",\n \"parameters\": {\n
\"origin\": \"NYC\",\n \"destination\": \"LA\",\n \"date\":
\"2022-04-10\"\n }\n}",
+ ...,
+ "json": {
+ "intent": "BookFlight",
+ "parameters": {
+ "date": "2022-04-10",
+ "destination": "LA",
+ "origin": "NYC"
+ }
+ },
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.167",
+ "url": "http://127.0.0.1/anything";
}
+```
+
+### Summarize Information
+The following example demonstrates how to use the `ai-request-rewrite` Plugin
to summarize information before the request reaches the Upstream service.
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+Create a Route and configure the `ai-request-rewrite` Plugin. The `prompt`
instructs the LLM to summarize lengthy input while preserving key details:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/anything",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-request-rewrite": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options":{
+ "model": "gpt-4"
+ },
+ "prompt": "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "httpbin.org:80": 1
+ }
+ }
+ }'
+```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+Create a Route with the `ai-request-rewrite` Plugin:
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
```
-The LLM processes the input and returns a modified request body, which replace
detected sensitive values with a masked format then used for the upstream
request:
+Synchronize the configuration to the gateway:
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details. Ensure
the summary remains concise and informative."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+```
+
+</TabItem>
+</Tabs>
+
+Apply the configuration to your cluster:
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+Send a POST request to the Route with lengthy content:
+
+```shell
+curl "http://127.0.0.1:9080/anything"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "content": "Hey! So, I'\''m planning a trip to Japan next spring for about
three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\''m not sure
how to split my time between them. I really love history and cultural sites, so
temples and shrines are a must. I'\''m also a big foodie, especially into ramen
and sushi, so I'\''d love recommendations on the best spots. I prefer quieter
areas for accommodation, but I don'\''t mind traveling into busy areas for
sightseeing. Oh, and I [...]
+ }'
+```
+
+You should receive a response similar to the following:
```json
{
- "name": "John Doe",
- "email": "[email protected]",
- "credit_card": "**** **** **** 1111",
- "ssn": "***-**-6789",
- "address": "123 Main St"
+ "args": {},
+ "data": "The individual is planning a three-week trip to Japan in the
spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in
history, culture, temples, and shrines. They love ramen and sushi, so are
seeking food recommendations. Accommodation should be in quieter areas, but
they are open to busy sites for sightseeing. Along with these cities, they plan
to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms
in early April. The best transport met [...]
+ ...,
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.171",
+ "url": "http://127.0.0.1/anything";
}
```
-### Send request to an OpenAI compatible LLM
+### Send Request to an OpenAI-Compatible LLM
-Create a route with the `ai-request-rewrite` plugin with `provider` set to
`openai-compatible` and the endpoint of the model set to `override.endpoint`
like so:
+The following example demonstrates how to use the `ai-request-rewrite` Plugin
with an OpenAI-compatible LLM provider by setting `provider` to
`openai-compatible` and configuring the custom endpoint in `override.endpoint`.
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+Create a Route and configure the `ai-request-rewrite` Plugin:
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
@@ -157,11 +744,11 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT
\
"uri": "/anything",
"plugins": {
"ai-request-rewrite": {
- "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., '*** **** **** 1234') for
credit card numbers). Ensure the JSON structure remains unchanged.",
+ "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged.",
"provider": "openai-compatible",
"auth": {
"header": {
- "Authorization": "Bearer <some-token>"
+ "Authorization": "Bearer <your-api-key>"
}
},
"options": {
@@ -182,3 +769,161 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT
\
}
}'
```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+Create a Route with the `ai-request-rewrite` Plugin:
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ plugins:
+ ai-request-rewrite:
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer <your-api-key>"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+Synchronize the configuration to the gateway:
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ prompt: "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+```
+
+</TabItem>
+</Tabs>
+
+Apply the configuration to your cluster:
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
diff --git a/docs/zh/latest/plugins/ai-request-rewrite.md
b/docs/zh/latest/plugins/ai-request-rewrite.md
index 3768c0391..bd0c1c058 100644
--- a/docs/zh/latest/plugins/ai-request-rewrite.md
+++ b/docs/zh/latest/plugins/ai-request-rewrite.md
@@ -5,7 +5,7 @@ keywords:
- AI 网关
- Plugin
- ai-request-rewrite
-description: ai-request-rewrite 插件在客户端请求转发到上游服务之前拦截请求。它将预定义的提示与原始请求体一起发送到指定的
LLM 服务。LLM 处理输入并返回修改后的请求体，然后用于上游请求。这允许基于 AI 生成的内容动态转换 API 请求。
+description: ai-request-rewrite 插件在将客户端请求发送到上游服务之前，将其转发到 LLM 服务进行处理，实现 AI
驱动的脱敏、内容增强和格式转换。
---
<!--
@@ -27,70 +27,302 @@ description: ai-request-rewrite 插件在客户端请求转发到上游服务之
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/ai-request-rewrite"; />
+</head>
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
## 描述
-`ai-request-rewrite` 插件在客户端请求转发到上游服务之前拦截请求。它将预定义的提示与原始请求体一起发送到指定的 LLM 服务。LLM
处理输入并返回修改后的请求体，然后用于上游请求。这允许基于 AI 生成的内容动态转换 API 请求。
+`ai-request-rewrite` 插件在将客户端请求转发到上游服务之前，先将请求发送到 LLM 服务进行转换处理。这使得 LLM
能够对请求进行数据脱敏、内容增强或格式转换等修改。该插件支持集成 OpenAI、DeepSeek、Gemini、Vertex
AI、Anthropic、OpenRouter 以及其他 OpenAI 兼容的 API。
## 插件属性
-| **字段** | **必选项** | **类型** | **描述**
|
-| ------------------------- | ------------ | -------- |
------------------------------------------------------------------------------------
|
-| prompt | 是 | String | 发送到 LLM 服务的提示。
|
-| provider | 是 | String | LLM
服务的名称。可用选项：openai、deekseek、azure-openai、aimlapi、anthropic、openrouter、gemini、vertex-ai
和 openai-compatible。当选择 `aimlapi` 时，插件使用 OpenAI 兼容驱动程序，默认端点为
`https://api.aimlapi.com/v1/chat/completions`。 |
-| provider_conf | 否 | Object | 特定提供商的配置。当 `provider`
设置为 `vertex-ai` 且未配置 `override` 时必填。 |
-| provider_conf.project_id | 是 | String | Google Cloud 项目 ID。 |
-| provider_conf.region | 是 | String | Google Cloud 区域。 |
-| auth | 是 | Object | 身份验证配置
|
-| auth.header | 否 | Object | 身份验证头部。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。 |
-| auth.query | 否 | Object | 身份验证查询参数。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。 |
-| auth.gcp | 否 | Object | Google Cloud Platform
(GCP) 身份验证配置。 |
-| auth.gcp.service_account_json | 否 | String | GCP 服务账号 JSON
文件的内容。也可以通过设置“GCP_SERVICE_ACCOUNT”环境变量来配置。 |
-| auth.gcp.max_ttl | 否 | Integer | 缓存 GCP 访问令牌的最大
TTL（秒）。最小值：1。 |
-| auth.gcp.expire_early_secs| 否 | Integer |
在访问令牌实际过期时间之前使其过期的秒数，以避免边缘情况。最小值：0。默认值：60。 |
-| options | 否 | Object | 模型的键/值设置
|
-| options.model | 否 | String | 要执行的模型。示例：openai 的
"gpt-3.5-turbo"，deekseek 的 "deepseek-chat"，或 openai-compatible 或 aimlapi 服务的
"qwen-turbo" |
-| override.endpoint | 否 | String | 使用 OpenAI
兼容服务时覆盖默认端点（例如，自托管模型或第三方 LLM 服务）。当提供商为 'openai-compatible' 时，endpoint 字段是必需的。 |
-| timeout | 否 | Integer | 对 LLM
服务请求的总超时时间（毫秒），包括连接、发送和读取超时。范围：1 - 60000。默认值：30000|
-| keepalive | 否 | Boolean | 为对 LLM 服务的请求启用
keepalive。默认值：true |
-| keepalive_timeout | 否 | Integer | 对 LLM 服务请求的 keepalive
超时时间（毫秒）。最小值：1000。默认值：60000 |
-| keepalive_pool | 否 | Integer | 对 LLM 服务请求的 keepalive
池大小。最小值：1。默认值：30 |
-| ssl_verify | 否 | Boolean | 对 LLM 服务请求的 SSL
验证。默认值：true |
+| 名称 | 类型 | 必选项 | 默认值 | 有效值 | 描述 |
+| --- | --- | --- | --- | --- | --- |
+| `prompt` | string | 是 | | | 发送到 LLM 服务用于重写客户端请求的提示词。 |
+| `provider` | string | 是 | | [openai, deepseek, azure-openai, aimlapi,
gemini, vertex-ai, anthropic, openrouter, openai-compatible] | LLM 服务提供商。设置为
`aimlapi` 时，插件使用 OpenAI 兼容驱动并将请求代理到
`https://api.aimlapi.com/v1/chat/completions`。设置为 `openai-compatible`
时，插件将请求代理到 `override` 中配置的自定义端点。设置为 `azure-openai` 时，插件同样将请求代理到 `override`
中配置的自定义端点，并会额外移除用户请求中的 `model` 参数。 |
+| `auth` | object | 是 | | | 身份验证配置。 |
+| `auth.header` | object | 否 | | | 身份验证请求头。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。`header` 和 `query` 至少需要配置其中一个。 |
+| `auth.query` | object | 否 | | | 身份验证查询参数。键必须匹配模式
`^[a-zA-Z0-9._-]+$`。`header` 和 `query` 至少需要配置其中一个。 |
+| `options` | object | 否 | | | 模型配置。除了 `model` 之外，还可以配置其他参数，这些参数会在请求体中转发给上游
LLM 服务。例如，使用 OpenAI 时，可以配置 `temperature`、`top_p` 和 `stream` 等参数。更多可用选项请参阅 LLM
提供商的 API 文档。 |
+| `options.model` | string | 否 | | | LLM 模型名称，例如 `gpt-4` 或 `gpt-3.5`。更多可用模型请参阅
LLM 提供商的 API 文档。 |
+| `override` | object | 否 | | | 覆盖设置。 |
+| `override.endpoint` | string | 否 | | | LLM 提供商端点。当 `provider` 为
`openai-compatible` 时必填。 |
+| `timeout` | integer | 否 | 30000 | 1 - 60000 | 请求 LLM 服务的超时时间（毫秒）。 |
+| `keepalive` | boolean | 否 | true | | 是否在请求 LLM 服务时保持连接。 |
+| `keepalive_timeout` | integer | 否 | 60000 | ≥ 1000 | 请求 LLM 服务的 keepalive
超时时间（毫秒）。 |
+| `keepalive_pool` | integer | 否 | 30 | ≥ 1 | 连接 LLM 服务的 keepalive 连接池大小。 |
+| `ssl_verify` | boolean | 否 | true | | 是否验证 LLM 服务的 SSL 证书。 |
## 工作原理
-
+
## 示例
-以下示例演示了如何为不同场景配置 `ai-request-rewrite`。
+以下示例演示如何为不同场景配置 `ai-request-rewrite`。
+
+示例使用 OpenAI 作为 LLM 服务。请先获取 OpenAI [API
密钥](https://openai.com/blog/openai-api)并将其保存到环境变量：
+
+```shell
+export OPENAI_API_KEY=<your-api-key>
+```
:::note
-您可以使用以下命令从 config.yaml 获取 admin_key 并保存到环境变量中：
+你可以使用以下命令从 `config.yaml` 中获取 `admin_key` 并保存到环境变量：
+```shell
admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed
's/"//g')
+```
:::
-### 编辑敏感信息
+### 脱敏敏感信息
+
+以下示例演示如何使用 `ai-request-rewrite` 插件在请求到达上游服务之前脱敏敏感信息。
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+创建路由并配置 `ai-request-rewrite` 插件。`provider` 设置为 `openai`，OpenAI API 密钥通过
`Authorization` 请求头传递，`prompt` 指示 LLM 识别和屏蔽敏感信息：
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
-H "X-API-KEY: ${admin_key}" \
-d '{
"uri": "/anything",
+ "methods": ["POST"],
"plugins": {
"ai-request-rewrite": {
- "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged.",
"provider": "openai",
"auth": {
"header": {
- "Authorization": "Bearer <some-token>"
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
}
},
- "options": {
+ "options":{
"model": "gpt-4"
- }
+ },
+ "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "httpbin.org:80": 1
+ }
+ }
+ }'
+```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+创建带有 `ai-request-rewrite` 插件的路由：
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+将配置同步到网关：
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+```
+
+</TabItem>
+</Tabs>
+
+将配置应用到集群：
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+发送一个包含个人敏感信息的 POST 请求到路由：
+
+```shell
+curl "http://127.0.0.1:9080/anything"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "content": "John said his debit card number is 4111 1111 1111 1111 and SIN
is 123-45-6789."
+ }'
+```
+
+你应该收到类似以下的响应：
+
+```json
+{
+ "args": {},
+ "data": "{\"content\": \"John said his debit card number is **** **** ****
1111 and SIN is ***-**-***.\"}",
+ ...,
+ "json": {
+ "content": "John said his debit card number is **** **** **** 1111 and SIN
is ***-**-***."
+ },
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.170",
+ "url": "http://127.0.0.1/anything";
+}
+```
+
+### 格式转换
+
+以下示例演示如何使用 `ai-request-rewrite` 插件在请求到达上游服务之前对数据进行格式转换。
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+创建路由并配置 `ai-request-rewrite` 插件。`prompt` 指示 LLM 将自然语言查询转换为结构化 JSON：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/anything",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-request-rewrite": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options":{
+ "model": "gpt-4"
+ },
+ "prompt": "Convert natural language queries into structured JSON
format with intent and extracted parameters."
}
},
"upstream": {
@@ -102,53 +334,408 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X
PUT \
}'
```
-现在发送一个请求：
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+创建带有 `ai-request-rewrite` 插件的路由：
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON
format with intent and extracted parameters."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+将配置同步到网关：
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON format
with intent and extracted parameters."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Convert natural language queries into structured JSON
format with intent and extracted parameters."
+```
+
+</TabItem>
+</Tabs>
+
+将配置应用到集群：
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+发送一个 POST 请求到路由：
```shell
-curl "http://127.0.0.1:9080/anything"; \
+curl "http://127.0.0.1:9080/anything"; -X POST \
-H "Content-Type: application/json" \
-d '{
- "name": "John Doe",
- "email": "[email protected]",
- "credit_card": "4111 1111 1111 1111",
- "ssn": "123-45-6789",
- "address": "123 Main St"
+ "content": "Book a flight from NYC to LA on April 10, 2022."
}'
```
-发送到 LLM 服务的请求体如下：
+你应该收到类似以下的响应：
```json
{
- "messages": [
- {
- "role": "system",
- "content": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., '*** **** **** 1234') for
credit card numbers). Ensure the JSON structure remains unchanged."
- },
- {
- "role": "user",
- "content": "{\n\"name\":\"John
Doe\",\n\"email\":\"[email protected]\",\n\"credit_card\":\"4111 1111 1111
1111\",\n\"ssn\":\"123-45-6789\",\n\"address\":\"123 Main St\"\n}"
- }
- ]
+ "args": {},
+ "data": "{\n \"intent\": \"BookFlight\",\n \"parameters\": {\n
\"origin\": \"NYC\",\n \"destination\": \"LA\",\n \"date\":
\"2022-04-10\"\n }\n}",
+ ...,
+ "json": {
+ "intent": "BookFlight",
+ "parameters": {
+ "date": "2022-04-10",
+ "destination": "LA",
+ "origin": "NYC"
+ }
+ },
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.167",
+ "url": "http://127.0.0.1/anything";
}
+```
+
+### 信息摘要
+
+以下示例演示如何使用 `ai-request-rewrite` 插件在请求到达上游服务之前对信息进行摘要。
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+创建路由并配置 `ai-request-rewrite` 插件。`prompt` 指示 LLM 在保留关键细节的同时对冗长输入进行摘要：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/anything",
+ "methods": ["POST"],
+ "plugins": {
+ "ai-request-rewrite": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options":{
+ "model": "gpt-4"
+ },
+ "prompt": "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+ }
+ },
+ "upstream": {
+ "type": "roundrobin",
+ "nodes": {
+ "httpbin.org:80": 1
+ }
+ }
+ }'
+```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+创建带有 `ai-request-rewrite` 插件的路由：
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ methods:
+ - POST
+ plugins:
+ ai-request-rewrite:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer ${OPENAI_API_KEY}"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+将配置同步到网关：
+
+```shell
+adc sync -f adc.yaml
+```
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details. Ensure
the summary remains concise and informative."
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ method: POST
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ methods:
+ - POST
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ provider: openai
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: gpt-4
+ prompt: "Summarize lengthy input while preserving key details.
Ensure the summary remains concise and informative."
+```
+
+</TabItem>
+</Tabs>
+
+将配置应用到集群：
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
+
+发送一个包含冗长内容的 POST 请求到路由：
+
+```shell
+curl "http://127.0.0.1:9080/anything"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "content": "Hey! So, I'\''m planning a trip to Japan next spring for about
three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I'\''m not sure
how to split my time between them. I really love history and cultural sites, so
temples and shrines are a must. I'\''m also a big foodie, especially into ramen
and sushi, so I'\''d love recommendations on the best spots. I prefer quieter
areas for accommodation, but I don'\''t mind traveling into busy areas for
sightseeing. Oh, and I [...]
+ }'
```
-LLM 处理输入并返回修改后的请求体，将检测到的敏感值替换为掩码格式，然后用于上游请求：
+你应该收到类似以下的响应：
```json
{
- "name": "John Doe",
- "email": "[email protected]",
- "credit_card": "**** **** **** 1111",
- "ssn": "***-**-6789",
- "address": "123 Main St"
+ "args": {},
+ "data": "The individual is planning a three-week trip to Japan in the
spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in
history, culture, temples, and shrines. They love ramen and sushi, so are
seeking food recommendations. Accommodation should be in quieter areas, but
they are open to busy sites for sightseeing. Along with these cities, they plan
to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms
in early April. The best transport met [...]
+ ...,
+ "method": "POST",
+ "origin": "192.168.97.1, 103.97.2.171",
+ "url": "http://127.0.0.1/anything";
}
```
### 向 OpenAI 兼容的 LLM 发送请求
-创建一个带有 `ai-request-rewrite` 插件的路由，将 `provider` 设置为
`openai-compatible`，并将模型的端点设置为 `override.endpoint`，如下所示：
+以下示例演示如何通过将 `provider` 设置为 `openai-compatible` 并在 `override.endpoint`
中配置自定义端点，来使用 `ai-request-rewrite` 插件与 OpenAI 兼容的 LLM 提供商。
+
+<Tabs groupId="api">
+<TabItem value="admin-api" label="Admin API">
+
+创建路由并配置 `ai-request-rewrite` 插件：
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
@@ -157,11 +744,11 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT
\
"uri": "/anything",
"plugins": {
"ai-request-rewrite": {
- "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., '*** **** **** 1234') for
credit card numbers). Ensure the JSON structure remains unchanged.",
+ "prompt": "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver'\''s license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged.",
"provider": "openai-compatible",
"auth": {
"header": {
- "Authorization": "Bearer <some-token>"
+ "Authorization": "Bearer <your-api-key>"
}
},
"options": {
@@ -182,3 +769,161 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT
\
}
}'
```
+
+</TabItem>
+<TabItem value="adc" label="ADC">
+
+创建带有 `ai-request-rewrite` 插件的路由：
+
+```yaml title="adc.yaml"
+services:
+ - name: ai-request-rewrite-service
+ routes:
+ - name: ai-request-rewrite-route
+ uris:
+ - /anything
+ plugins:
+ ai-request-rewrite:
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer <your-api-key>"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+ upstream:
+ type: roundrobin
+ nodes:
+ - host: httpbin.org
+ port: 80
+ weight: 1
+```
+
+将配置同步到网关：
+
+```shell
+adc sync -f adc.yaml
+```
+
+</TabItem>
+<TabItem value="ingress" label="Ingress Controller">
+
+<Tabs groupId="k8s-api">
+<TabItem value="gateway-api" label="Gateway API">
+
+```yaml title="ai-request-rewrite-gw.yaml"
+apiVersion: v1
+kind: Service
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ type: ExternalName
+ externalName: httpbin.org
+---
+apiVersion: apisix.apache.org/v1alpha1
+kind: PluginConfig
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-plugin-config
+spec:
+ plugins:
+ - name: ai-request-rewrite
+ config:
+ prompt: "Given a JSON request body, identify and mask any sensitive
information such as credit card numbers, social security numbers, and personal
identification numbers (e.g., passport or driver's license numbers). Replace
detected sensitive values with a masked format (e.g., \"*** **** **** 1234\")
for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ parentRefs:
+ - name: apisix
+ rules:
+ - matches:
+ - path:
+ type: Exact
+ value: /anything
+ filters:
+ - type: ExtensionRef
+ extensionRef:
+ group: apisix.apache.org
+ kind: PluginConfig
+ name: ai-request-rewrite-plugin-config
+ backendRefs:
+ - name: httpbin-external-domain
+ port: 80
+```
+
+</TabItem>
+<TabItem value="ingress" label="APISIX Ingress Controller">
+
+```yaml title="ai-request-rewrite-ic.yaml"
+apiVersion: apisix.apache.org/v2
+kind: ApisixUpstream
+metadata:
+ namespace: aic
+ name: httpbin-external-domain
+spec:
+ ingressClassName: apisix
+ externalNodes:
+ - type: Domain
+ name: httpbin.org
+---
+apiVersion: apisix.apache.org/v2
+kind: ApisixRoute
+metadata:
+ namespace: aic
+ name: ai-request-rewrite-route
+spec:
+ ingressClassName: apisix
+ http:
+ - name: ai-request-rewrite-route
+ match:
+ paths:
+ - /anything
+ upstreams:
+ - name: httpbin-external-domain
+ plugins:
+ - name: ai-request-rewrite
+ enable: true
+ config:
+ prompt: "Given a JSON request body, identify and mask any
sensitive information such as credit card numbers, social security numbers, and
personal identification numbers (e.g., passport or driver's license numbers).
Replace detected sensitive values with a masked format (e.g., \"*** **** ****
1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
+ provider: openai-compatible
+ auth:
+ header:
+ Authorization: "Bearer your-api-key"
+ options:
+ model: qwen-plus
+ max_tokens: 1024
+ temperature: 1
+ override:
+ endpoint:
"https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions";
+```
+
+</TabItem>
+</Tabs>
+
+将配置应用到集群：
+
+```shell
+kubectl apply -f ai-request-rewrite-ic.yaml
+```
+
+</TabItem>
+</Tabs>
