This is an automated email from the ASF dual-hosted git repository.
wenming pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apisix.git
The following commit(s) were added to refs/heads/master by this push:
new a52231ff7 docs: improve `ai-prompt-template` plugin docs (#12897)
a52231ff7 is described below
commit a52231ff7ab6a355b9d68851aceb21dd14d49302
Author: Yilia Lin <[email protected]>
AuthorDate: Tue Jan 20 10:54:03 2026 +0800
docs: improve `ai-prompt-template` plugin docs (#12897)
---
docs/en/latest/plugins/ai-prompt-template.md | 223 +++++++++++++++++++++++----
docs/zh/latest/plugins/ai-prompt-template.md | 220 ++++++++++++++++++++++----
2 files changed, 379 insertions(+), 64 deletions(-)
diff --git a/docs/en/latest/plugins/ai-prompt-template.md
b/docs/en/latest/plugins/ai-prompt-template.md
index 9ca4e1f70..12b4fb7dd 100644
--- a/docs/en/latest/plugins/ai-prompt-template.md
+++ b/docs/en/latest/plugins/ai-prompt-template.md
@@ -5,7 +5,7 @@ keywords:
- API Gateway
- Plugin
- ai-prompt-template
-description: This document contains information about the Apache APISIX
ai-prompt-template Plugin.
+description: The ai-prompt-template plugin supports pre-configuring prompt
templates that only accept user inputs in designated template variables, in a
fill-in-the-blank fashion.
---
<!--
@@ -27,49 +27,73 @@ description: This document contains information about the
Apache APISIX ai-promp
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-template"; />
+</head>
+
## Description
-The `ai-prompt-template` plugin simplifies access to LLM providers, such as
OpenAI and Anthropic, and their models by predefining the request format
-using a template, which only allows users to pass customized values into
template variables.
+The `ai-prompt-template` Plugin simplifies access to LLM providers, such as
OpenAI and Anthropic, and their models. It pre-configures prompt templates that
only accept user inputs in designated template variables, in a "fill in the
blank" fashion.
## Plugin Attributes
-| **Field** | **Required** | **Type** |
**Description**
|
-| ------------------------------------- | ------------ | -------- |
---------------------------------------------------------------------------------------------------------------------------
|
-| `templates` | Yes | Array | An array
of template objects
|
-| `templates.name` | Yes | String | Name of
the template.
|
-| `templates.template.model` | Yes | String | Model of
the AI Model, for example `gpt-4` or `gpt-3.5`. See your LLM provider API
documentation for more available models. |
-| `templates.template.messages.role` | Yes | String | Role of
the message (`system`, `user`, `assistant`)
|
-| `templates.template.messages.content` | Yes | String | Content of
the message.
|
+| **Field** | **Required** | **Type** | **Description** |
+| :--- | :--- | :--- | :--- |
+| `templates` | Yes | Array | An array of template objects. |
+| `templates.name` | Yes | String | Name of the template. When requesting the
route, the request should include the template name that corresponds to the
configured template. |
+| `templates.template` | Yes | Object | Template specification. |
+| `templates.template.model` | Yes | String | Name of the AI Model, such as
`gpt-4` or `gpt-3.5`. See your LLM provider API documentation for more
available models. |
+| `templates.template.messages` | Yes | Array | Template message
specification. |
+| `templates.template.messages.role` | Yes | String | Role of the message,
such as `system`, `user`, or `assistant`. |
+| `templates.template.messages.content` | Yes | String | Content of the
message (prompt). |
-## Example usage
+## Examples
-Create a route with the `ai-prompt-template` plugin like so:
+The following examples will be using OpenAI as the Upstream service provider.
Before proceeding, create an [OpenAI account](https://openai.com) and an [API
key](https://openai.com/blog/openai-api). You can optionally save the key to an
environment variable as such:
+
+```shell
+export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
+```
+
+If you are working with other LLM providers, please refer to the provider's
documentation to obtain an API key.
+
+### Configure a Template for Open Questions in Custom Complexity
+
+The following example demonstrates how to use the `ai-prompt-template` Plugin
to configure a template which can be used to answer open questions and accepts
user-specified response complexity.
+
+Create a Route to the chat completion endpoint with pre-configured prompt
templates as such:
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
- -H "X-API-KEY: ${ADMIN_API_KEY}" \
+ -H "X-API-KEY: ${admin_key}" \
-d '{
"uri": "/v1/chat/completions",
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "api.openai.com:443": 1
- },
- "scheme": "https",
- "pass_host": "node"
- },
"plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4"
+ }
+ },
"ai-prompt-template": {
"templates": [
{
- "name": "level of detail",
+ "name": "QnA with complexity",
"template": {
"model": "gpt-4",
"messages": [
+ {
+ "role": "system",
+ "content": "Answer in {{complexity}}."
+ },
{
"role": "user",
- "content": "Explain about {{ topic }} in {{ level }}."
+ "content": "Explain {{prompt}}."
}
]
}
@@ -80,23 +104,156 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT
\
}'
```
+Send a POST request to the Route with a sample question and desired answer
complexity in the request body.
+
Now send a request:
```shell
-curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type:
application/json' -d '{
- "template_name": "level of detail",
- "topic": "psychology",
- "level": "brief"
-}' -H "Authorization: Bearer <your token here>"
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "QnA with complexity",
+ "complexity": "brief",
+ "prompt": "quick sort"
+ }'
+```
+
+You should receive a response similar to the following:
+
+```json
+{
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "Quick sort is a highly efficient sorting algorithm that
uses a divide-and-conquer approach to arrange elements in a list or array in
order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element
from the list as a 'pivot'. Common methods include choosing the first element,
the last element, the middle element, or a random element.\n\n2.
**Partitioning**: Rearrange the elements in the list such that all elements
less than the pivot are moved before it, [...]
+ "role": "assistant"
+ }
+ }
+ ],
+ "created": 1723194057,
+ "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x",
+ "model": "gpt-4o-2024-05-13",
+ "object": "chat.completion",
+ "system_fingerprint": "fp_abc28019ad",
+ "usage": {
+ "completion_tokens": 234,
+ "prompt_tokens": 18,
+ "total_tokens": 252
+ }
+}
+```
+
+### Configure Multiple Templates
+
+The following example demonstrates how you can configure multiple templates on
the same Route. When requesting the Route, users will be able to pass custom
inputs to different templates by specifying the template name.
+
+The example continues with the [last
example](#configure-a-template-for-open-questions-in-custom-complexity). Update
the Plugin with another template:
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/v1/chat/completions",
+ "plugins": {
+ "ai-prompt-template": {
+ "templates": [
+ {
+ "name": "QnA with complexity",
+ "template": {
+ "model": "gpt-4",
+ "messages": [
+ {
+ "role": "system",
+ "content": "Answer in {{complexity}}."
+ },
+ {
+ "role": "user",
+ "content": "Explain {{prompt}}."
+ }
+ ]
+ }
+ },
+ {
+ "name": "echo",
+ "template": {
+ "model": "gpt-4",
+ "messages": [
+ {
+ "role": "system",
+ "content": "You are an echo bot. You must repeat exactly
what the user says without any changes or additional text."
+ },
+ {
+ "role": "user",
+ "content": "Echo {{prompt}}."
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+You should now be able to use both templates through the same Route.
+
+Send a POST request to the Route and use the first template:
+
+```shell
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "QnA with complexity",
+ "complexity": "brief",
+ "prompt": "quick sort"
+ }'
+```
+
+You should receive a response similar to the following:
+
+```json
+{
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "Quick sort is a highly efficient sorting algorithm that
uses a divide-and-conquer approach to arrange elements in a list or array in
order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element
from the list as a 'pivot'. Common methods include choosing the first element,
the last element, the middle element, or a random element.\n\n2.
**Partitioning**: Rearrange the elements in the list such that all elements
less than the pivot are moved before it, [...]
+ "role": "assistant"
+ }
+ }
+ ],
+ ...
+}
+```
+
+Send a POST request to the Route and use the second template:
+
+```shell
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "echo",
+ "prompt": "hello APISIX"
+ }'
```
-Then the request body will be modified to something like this:
+You should receive a response similar to the following:
```json
{
- "model": "some model",
- "messages": [
- { "role": "user", "content": "Explain about psychology in brief." }
- ]
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "hello APISIX",
+ "role": "assistant"
+ }
+ }
+ ],
+ ...
}
```
diff --git a/docs/zh/latest/plugins/ai-prompt-template.md
b/docs/zh/latest/plugins/ai-prompt-template.md
index fb329280c..6bd046a11 100644
--- a/docs/zh/latest/plugins/ai-prompt-template.md
+++ b/docs/zh/latest/plugins/ai-prompt-template.md
@@ -5,7 +5,7 @@ keywords:
- API 网关
- Plugin
- ai-prompt-template
-description: 本文档包含有关 Apache APISIX ai-prompt-template 插件的信息。
+description: ai-prompt-template 插件支持预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用填空的方式。
---
<!--
@@ -27,48 +27,166 @@ description: 本文档包含有关 Apache APISIX ai-prompt-template 插件的信
#
-->
+<head>
+ <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-template"; />
+</head>
+
## 描述
-`ai-prompt-template` 插件通过使用模板预定义请求格式，简化了对 LLM 提供商（如 OpenAI 和
Anthropic）及其模型的访问，只允许用户将自定义值传递到模板变量中。
+`ai-prompt-template` 插件简化了对 OpenAI、Anthropic
等大语言模型提供商及其模型的访问。它预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用“填空”的方式。
## 插件属性
-| **字段** | **必选项** | **类型** | **描述**
|
-| ------------------------------------- | ---------- | -------- |
--------------------------------------------------------------------------------------------------------------------
|
-| `templates` | 是 | Array | 模板对象数组
|
-| `templates.name` | 是 | String | 模板的名称。
|
-| `templates.template.model` | 是 | String | AI 模型的名称，例如
`gpt-4` 或 `gpt-3.5`。有关更多可用模型，请参阅您的 LLM 提供商 API 文档。 |
-| `templates.template.messages.role` | 是 | String |
消息的角色（`system`、`user`、`assistant`）
|
-| `templates.template.messages.content` | 是 | String | 消息的内容。
|
+| **字段** | **是否必填** | **类型** | **描述** |
+| :--- | :--- | :--- | :--- |
+| `templates` | 是 | Array | 模板对象数组。 |
+| `templates.name` | 是 | String | 模板的名称。在请求路由时，请求中应包含与所配置模板相对应的模板名称。 |
+| `templates.template` | 是 | Object | 模板规范。 |
+| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或
`gpt-3.5`。更多可用模型请参阅 LLM 提供商的 API 文档。 |
+| `templates.template.messages` | 是 | Array | 模板消息规范。 |
+| `templates.template.messages.role` | 是 | String | 消息的角色，例如 `system`、`user` 或
`assistant`。 |
+| `templates.template.messages.content` | 是 | String | 消息（提示词）的内容。 |
## 使用示例
-创建一个带有 `ai-prompt-template` 插件的路由，如下所示：
+以下示例将使用 OpenAI 作为上游服务提供商。开始之前，请先创建一个 OpenAI 账户和一个 API 密钥。你可以选择将密钥保存到环境变量中，如下所示：
+
+```shell
+export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> # 替换为你的 API 密钥
+```
+
+如果你正在使用其他 LLM 提供商，请参阅该提供商的文档以获取 API 密钥。
+
+### 为自定义复杂度的开放式问题配置模板
+
+以下示例演示了如何使用 `ai-prompt-template` 插件配置一个模板，该模板可用于回答开放式问题并接受用户指定的回答复杂度。
+
+创建一个指向聊天补全端点的路由，并配置预定义的提示词模板：
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
- -H "X-API-KEY: ${ADMIN_API_KEY}" \
+ -H "X-API-KEY: ${admin_key}" \
-d '{
"uri": "/v1/chat/completions",
- "upstream": {
- "type": "roundrobin",
- "nodes": {
- "api.openai.com:443": 1
+ "plugins": {
+ "ai-proxy": {
+ "provider": "openai",
+ "auth": {
+ "header": {
+ "Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+ },
+ "options": {
+ "model": "gpt-4"
+ }
},
- "scheme": "https",
- "pass_host": "node"
- },
+ "ai-prompt-template": {
+ "templates": [
+ {
+ "name": "QnA with complexity",
+ "template": {
+ "model": "gpt-4",
+ "messages": [
+ {
+ "role": "system",
+ "content": "Answer in {{complexity}}."
+ },
+ {
+ "role": "user",
+ "content": "Explain {{prompt}}."
+ }
+ ]
+ }
+ }
+ ]
+ }
+ }
+ }'
+```
+
+向该路由发送一个 POST 请求，在请求体中包含示例问题和期望的回答复杂度。
+
+发送请求：
+
+```shell
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "QnA with complexity",
+ "complexity": "brief",
+ "prompt": "quick sort"
+ }'
+```
+
+你应该会收到类似于以下的响应：
+
+```json
+{
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "Quick sort is a highly efficient sorting algorithm that
uses a divide-and-conquer approach to arrange elements in a list or array in
order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element
from the list as a 'pivot'. Common methods include choosing the first element,
the last element, the middle element, or a random element.\n\n2.
**Partitioning**: Rearrange the elements in the list such that all elements
less than the pivot are moved before it, [...]
+ "role": "assistant"
+ }
+ }
+ ],
+ "created": 1723194057,
+ "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x",
+ "model": "gpt-4o-2024-05-13",
+ "object": "chat.completion",
+ "system_fingerprint": "fp_abc28019ad",
+ "usage": {
+ "completion_tokens": 234,
+ "prompt_tokens": 18,
+ "total_tokens": 252
+ }
+}
+```
+
+### 配置多个模板
+
+以下示例演示了如何在同一条路由上配置多个模板。请求该路由时，用户将能够通过指定模板名称向不同模板传递自定义输入。
+
+该示例延续自[上一个示例](#为自定义复杂度的开放式问题配置模板)。使用另一个模板更新插件配置：
+
+```shell
+curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \
+ -H "X-API-KEY: ${admin_key}" \
+ -d '{
+ "uri": "/v1/chat/completions",
"plugins": {
"ai-prompt-template": {
"templates": [
{
- "name": "详细程度",
+ "name": "QnA with complexity",
+ "template": {
+ "model": "gpt-4",
+ "messages": [
+ {
+ "role": "system",
+ "content": "Answer in {{complexity}}."
+ },
+ {
+ "role": "user",
+ "content": "Explain {{prompt}}."
+ }
+ ]
+ }
+ },
+ {
+ "name": "echo",
"template": {
"model": "gpt-4",
"messages": [
+ {
+ "role": "system",
+ "content": "You are an echo bot. You must repeat exactly
what the user says without any changes or additional text."
+ },
{
"role": "user",
- "content": "用{{ level }}的方式解释{{ topic }}。"
+ "content": "Echo {{prompt}}."
}
]
}
@@ -79,23 +197,63 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
}'
```
-现在发送一个请求：
+现在，你应该能够通过同一条路由使用这两个模板。
+
+向路由发送一个 POST 请求，使用第一个模板：
+
+```shell
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "QnA with complexity",
+ "complexity": "brief",
+ "prompt": "quick sort"
+ }'
+```
+
+你应该会收到类似于以下的响应：
+
+```json
+{
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "Quick sort is a highly efficient sorting algorithm that
uses a divide-and-conquer approach to arrange elements in a list or array in
order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element
from the list as a 'pivot'. Common methods include choosing the first element,
the last element, the middle element, or a random element.\n\n2.
**Partitioning**: Rearrange the elements in the list such that all elements
less than the pivot are moved before it, [...]
+ "role": "assistant"
+ }
+ }
+ ],
+ ...
+}
+```
+
+向路由发送一个 POST 请求，使用第二个模板：
```shell
-curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type:
application/json' -d '{
- "template_name": "详细程度",
- "topic": "心理学",
- "level": "简要"
-}' -H "Authorization: Bearer <your token here>"
+curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \
+ -H "Content-Type: application/json" \
+ -d '{
+ "template_name": "echo",
+ "prompt": "hello APISIX"
+ }'
```
-然后请求体将被修改为类似这样：
+你应该会收到类似于以下的响应：
```json
{
- "model": "some model",
- "messages": [
- { "role": "user", "content": "用简要的方式解释心理学。" }
- ]
+ "choices": [
+ {
+ "finish_reason": "stop",
+ "index": 0,
+ "message": {
+ "content": "hello APISIX",
+ "role": "assistant"
+ }
+ }
+ ],
+ ...
}
```
