Copilot commented on code in PR #12897: URL: https://github.com/apache/apisix/pull/12897#discussion_r2686867051
########## docs/zh/latest/plugins/ai-prompt-template.md: ########## @@ -27,27 +27,138 @@ description: 本文档包含有关 Apache APISIX ai-prompt-template 插件的信 # --> +<head> + <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-template"; /> +</head> + ## 描述 -`ai-prompt-template` 插件通过使用模板预定义请求格式，简化了对 LLM 提供商（如 OpenAI 和 Anthropic）及其模型的访问，只允许用户将自定义值传递到模板变量中。 +`ai-prompt-template` 插件简化了对 OpenAI、Anthropic 等大语言模型提供商及其模型的访问。它预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用“填空”的方式。 ## 插件属性 -| **字段** | **必选项** | **类型** | **描述** | -| ------------------------------------- | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------- | -| `templates` | 是 | Array | 模板对象数组 | -| `templates.name` | 是 | String | 模板的名称。 | -| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。有关更多可用模型，请参阅您的 LLM 提供商 API 文档。 | -| `templates.template.messages.role` | 是 | String | 消息的角色（`system`、`user`、`assistant`） | -| `templates.template.messages.content` | 是 | String | 消息的内容。 | +| **字段** | **是否必填** | **类型** | **描述** | +| :--- | :--- | :--- | :--- | +| `templates` | 是 | Array | 模板对象数组。 | +| `templates.name` | 是 | String | 模板的名称。在请求路由时，请求中应包含与所配置模板相对应的模板名称。 | +| `templates.template` | 是 | Object | 模板规范。 | +| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。更多可用模型请参阅 LLM 提供商的 API 文档。 | +| `templates.template.messages` | 是 | Array | 模板消息规范。 | +| `templates.template.messages.role` | 是 | String | 消息的角色，例如 `system`、`user` 或 `assistant`。 | +| `templates.template.messages.content` | 是 | String | 消息（提示词）的内容。 | ## 使用示例 -创建一个带有 `ai-prompt-template` 插件的路由，如下所示： +以下示例将使用 OpenAI 作为上游服务提供商。开始之前，请先创建一个 OpenAI 账户 和一个 API 密钥。你可以选择将密钥保存到环境变量中，如下所示： + +```shell +export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> # 替换为你的 API 密钥 +``` + +如果你正在使用其他 LLM 提供商，请参阅该提供商的文档以获取 API 密钥。 + +### 为自定义复杂度的开放式问题配置模板 + +以下示例演示了如何使用 `ai-prompt-template` 插件配置一个模板，该模板可用于回答开放式问题并接受用户指定的回答复杂度。 + +创建一个指向聊天补全端点的路由，并配置预定义的提示词模板： ```shell curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ - -H "X-API-KEY: ${ADMIN_API_KEY}" \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "uri": "/v1/chat/completions", + "upstream": { + "type": "roundrobin", + "nodes": { + "api.openai.com:443": 1 + }, + "scheme": "https", + "pass_host": "node" + }, + "plugins": { + "proxy-rewrite": { + "headers": { + "set": { + "Authorization": "Bearer '"$OPENAI_API_KEY"'" + } + } + }, + "ai-prompt-template": { + "templates": [ + { + "name": "QnA with complexity", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "Answer in {{complexity}}." + }, + { + "role": "user", + "content": "Explain {{prompt}}." + } + ] + } + } + ] + } + } + }' +``` + +向该路由发送一个 POST 请求，在请求体中包含示例问题和期望的回答复杂。 Review Comment: Typo in the sentence. The word "复杂" should be "复杂度" (complexity level). The sentence currently says "期望的回答复杂" which is grammatically incomplete. It should read "期望的回答复杂度" to properly mean "desired answer complexity". ```suggestion 向该路由发送一个 POST 请求，在请求体中包含示例问题和期望的回答复杂度。 ``` ########## docs/zh/latest/plugins/ai-prompt-template.md: ########## @@ -79,23 +206,66 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` -现在发送一个请求： +现在，你应该能够通过同一条路由使用这两个模板。 + +向路由发送一个 POST 请求，使用第一个模板： + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +你应该会收到类似于以下的响应： + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + ... +} +``` + +向路由发送一个 POST 请求，使用第二个模板： ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "详细程度", - "topic": "心理学", - "level": "简要" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + # highlight-next-line Review Comment: The comment syntax used here is invalid for JSON. JSON does not support comments with the `#` syntax. This line will cause a JSON parsing error when users try to use this example. Either remove the comment or place it outside the JSON block as a separate note. ```suggestion ``` ########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +108,164 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. + Now send a request: ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "level of detail", - "topic": "psychology", - "level": "brief" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + "created": 1723194057, + "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x", + "model": "gpt-4o-2024-05-13", + "object": "chat.completion", + "system_fingerprint": "fp_abc28019ad", + "usage": { + "completion_tokens": 234, + "prompt_tokens": 18, + "total_tokens": 252 + } +} +``` + +### Configure Multiple Templates + +The following example demonstrates how you can configure multiple templates on the same Route. When requesting the Route, users will be able to pass custom inputs to different templates by specifying the template name. + +The example continues with the [last example](#configure-a-template-for-open-questions-in-custom-complexity). Update the Plugin with another template: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "uri": "/v1/chat/completions", + "upstream": { + "type": "roundrobin", + "nodes": { + "api.openai.com:443": 1 + }, + "scheme": "https", + "pass_host": "node" + }, + "plugins": { + "ai-prompt-template": { + "templates": [ + { + "name": "QnA with complexity", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "Answer in {{complexity}}." + }, + { + "role": "user", + "content": "Explain {{prompt}}." + } + ] + } + }, + { + "name": "echo", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "user", + "content": "Echo {{prompt}}." + } + ] + } + } + ] + } + } + }' +``` + +You should now be able to use both templates through the same Route. + +Send a POST request to the Route and use the first template: + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + ... +} +``` + +Send a POST request to the Route and use the second template: + +```shell +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + # highlight-next-line Review Comment: The comment syntax used here is invalid for JSON. JSON does not support comments with the `#` syntax. This line will cause a JSON parsing error when users try to use this example. Either remove the comment or place it outside the JSON block as a separate note. ```suggestion # highlight-next-line curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ -H "Content-Type: application/json" \ -H "Host: api.openai.com:443" \ -d '{ ``` ########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +108,164 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. + Now send a request: ```shell -curl http://127.0.0.1:9080/v1/chat/completions -i -XPOST -H 'Content-Type: application/json' -d '{ - "template_name": "level of detail", - "topic": "psychology", - "level": "brief" -}' -H "Authorization: Bearer <your token here>" +curl "http://127.0.0.1:9080/v1/chat/completions"; -X POST \ + -H "Content-Type: application/json" \ + -H "Host: api.openai.com:443" \ + -d '{ + "template_name": "QnA with complexity", + "complexity": "brief", + "prompt": "quick sort" + }' +``` + +You should receive a response similar to the following: + +```json +{ + "choices": [ + { + "finish_reason": "stop", + "index": 0, + "message": { + "content": "Quick sort is a highly efficient sorting algorithm that uses a divide-and-conquer approach to arrange elements in a list or array in order. Here’s a brief explanation:\n\n1. **Choose a Pivot**: Select an element from the list as a 'pivot'. Common methods include choosing the first element, the last element, the middle element, or a random element.\n\n2. **Partitioning**: Rearrange the elements in the list such that all elements less than the pivot are moved before it, and all elements greater than the pivot are moved after it. The pivot is now in its final position.\n\n3. **Recursively Apply**: Recursively apply the same process to the sub-lists of elements to the left and right of the pivot.\n\nThe base case of the recursion is lists of size zero or one, which are already sorted.\n\nQuick sort has an average-case time complexity of O(n log n), making it suitable for large datasets. However, its worst-case time complexity is O(n^2), which occurs when the smalle st or largest element is always chosen as the pivot. This can be mitigated by using good pivot selection strategies or randomization.", + "role": "assistant" + } + } + ], + "created": 1723194057, + "id": "chatcmpl-9uFmTYN4tfwaXZjyOQwcp0t5law4x", + "model": "gpt-4o-2024-05-13", + "object": "chat.completion", + "system_fingerprint": "fp_abc28019ad", + "usage": { + "completion_tokens": 234, + "prompt_tokens": 18, + "total_tokens": 252 + } +} +``` + +### Configure Multiple Templates + +The following example demonstrates how you can configure multiple templates on the same Route. When requesting the Route, users will be able to pass custom inputs to different templates by specifying the template name. + +The example continues with the [last example](#configure-a-template-for-open-questions-in-custom-complexity). Update the Plugin with another template: + +```shell +curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PATCH \ + -H "X-API-KEY: ${admin_key}" \ + -d '{ + "uri": "/v1/chat/completions", + "upstream": { + "type": "roundrobin", + "nodes": { + "api.openai.com:443": 1 + }, + "scheme": "https", + "pass_host": "node" + }, + "plugins": { + "ai-prompt-template": { + "templates": [ + { + "name": "QnA with complexity", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "system", + "content": "Answer in {{complexity}}." + }, + { + "role": "user", + "content": "Explain {{prompt}}." + } + ] + } + }, + { + "name": "echo", + "template": { + "model": "gpt-4", + "messages": [ + { + "role": "user", + "content": "Echo {{prompt}}." + } + ] + } + } + ] + } + } + }' +``` Review Comment: The `proxy-rewrite` plugin configuration is missing from this PATCH request, but it's required to set the Authorization header with the OpenAI API key. Since this is a PATCH request that only updates the `ai-prompt-template` plugin, the Authorization header from the previous configuration may not be preserved. Users following this example after the first example would lose authentication and get API errors. Consider either using PUT to include the full configuration or adding a note that the proxy-rewrite configuration from the previous example is still active. ########## docs/zh/latest/plugins/ai-prompt-template.md: ########## @@ -27,27 +27,138 @@ description: 本文档包含有关 Apache APISIX ai-prompt-template 插件的信 # --> +<head> + <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-template"; /> +</head> + ## 描述 -`ai-prompt-template` 插件通过使用模板预定义请求格式，简化了对 LLM 提供商（如 OpenAI 和 Anthropic）及其模型的访问，只允许用户将自定义值传递到模板变量中。 +`ai-prompt-template` 插件简化了对 OpenAI、Anthropic 等大语言模型提供商及其模型的访问。它预先配置提示词模板，这些模板仅接受用户在指定的模板变量中输入，采用“填空”的方式。 ## 插件属性 -| **字段** | **必选项** | **类型** | **描述** | -| ------------------------------------- | ---------- | -------- | -------------------------------------------------------------------------------------------------------------------- | -| `templates` | 是 | Array | 模板对象数组 | -| `templates.name` | 是 | String | 模板的名称。 | -| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。有关更多可用模型，请参阅您的 LLM 提供商 API 文档。 | -| `templates.template.messages.role` | 是 | String | 消息的角色（`system`、`user`、`assistant`） | -| `templates.template.messages.content` | 是 | String | 消息的内容。 | +| **字段** | **是否必填** | **类型** | **描述** | +| :--- | :--- | :--- | :--- | +| `templates` | 是 | Array | 模板对象数组。 | +| `templates.name` | 是 | String | 模板的名称。在请求路由时，请求中应包含与所配置模板相对应的模板名称。 | +| `templates.template` | 是 | Object | 模板规范。 | +| `templates.template.model` | 是 | String | AI 模型的名称，例如 `gpt-4` 或 `gpt-3.5`。更多可用模型请参阅 LLM 提供商的 API 文档。 | +| `templates.template.messages` | 是 | Array | 模板消息规范。 | +| `templates.template.messages.role` | 是 | String | 消息的角色，例如 `system`、`user` 或 `assistant`。 | +| `templates.template.messages.content` | 是 | String | 消息（提示词）的内容。 | ## 使用示例 -创建一个带有 `ai-prompt-template` 插件的路由，如下所示： +以下示例将使用 OpenAI 作为上游服务提供商。开始之前，请先创建一个 OpenAI 账户 和一个 API 密钥。你可以选择将密钥保存到环境变量中，如下所示： + +```shell +export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> # 替换为你的 API 密钥 Review Comment: The curl command example should specify the variable placeholder format more explicitly. The text instructs users to "替换为你的 API 密钥" (replace with your API key) but uses angle brackets in the command. For consistency with how environment variables are used elsewhere in the document (like `${admin_key}`), consider using a more explicit format or removing the comment since the export command already handles this. ```suggestion export OPENAI_API_KEY="your-openai-api-key" ``` ########## docs/en/latest/plugins/ai-prompt-template.md: ########## @@ -80,23 +108,164 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` +Send a POST request to the Route with a sample question and desired answer complexity in the request body. Review Comment: Missing period at the end of the sentence. The sentence should end with proper punctuation for consistency with other sentences in the documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
