kayx23 commented on code in PR #12917:
URL: https://github.com/apache/apisix/pull/12917#discussion_r3032170738
##
docs/en/latest/plugins/ai-prompt-decorator.md:
##
@@ -27,83 +27,277 @@ description: This document contains information about the
Apache APISIX ai-promp
#
-->
+
+ https://docs.api7.ai/hub/ai-prompt-decorator"; />
+
+
## Description
-The `ai-prompt-decorator` plugin simplifies access to LLM providers, such as
OpenAI and Anthropic, and their models by appending or prepending prompts into
the request.
+The `ai-prompt-decorator` Plugin simplifies access to LLM providers, such as
OpenAI and Anthropic, and their models. It modifies user input prompts by
prefixing and appending pre-engineered prompts to set contexts in content
generation. This practice helps the model operate within desired guidelines
during interactions.
## Plugin Attributes
| **Field** | **Required**| **Type** | **Description**
|
| - | --- | |
--- |
-| `prepend` | Conditionally\* | Array| An array of prompt objects
to be prepended |
-| `prepend.role`| Yes | String | Role of the message
(`system`, `user`, `assistant`) |
-| `prepend.content` | Yes | String | Content of the message.
Minimum length: 1 |
-| `append` | Conditionally\* | Array| An array of prompt objects
to be appended |
-| `append.role` | Yes | String | Role of the message
(`system`, `user`, `assistant`) |
-| `append.content` | Yes | String | Content of the message.
Minimum length: 1 |
+| `prepend` | Conditionally\* | Array| An array of prompt objects
to be prepended. |
+| `prepend.role`| Yes | String | Role of the message, such
as `system`, `user`, or `assistant`. |
+| `prepend.content` | Yes | String | Content of the message
(prompt). |
+| `append` | Conditionally\* | Array| An array of prompt objects
to be appended. |
+| `append.role` | Yes | String | Role of the message, such
as `system`, `user`, or `assistant`. |
+| `append.content` | Yes | String | Content of the message
(prompt). |
\* **Conditionally Required**: At least one of `prepend` or `append` must be
provided.
-## Example usage
+## Example
-Create a route with the `ai-prompt-decorator` plugin like so:
+The following example will be using OpenAI as the upstream service provider.
Before proceeding, create an [OpenAI account](https://openai.com) and an [API
key](https://openai.com/blog/openai-api). You can optionally save the key to an
environment variable as such:
+
+```shell
+export OPENAI_API_KEY=
+```
+
+If you are working with other LLM providers, please refer to the provider's
documentation to obtain an API key.
+
+### Prepend and Append Messages
+
+The following example demonstrates how to configure the `ai-prompt-decorator`
Plugin to prepend a system message and append a user message to the user input
message.
+
+
+
+
+
+Create a Route to the chat completion endpoint with pre-configured prompt
templates as such:
```shell
curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \
- -H "X-API-KEY: ${ADMIN_API_KEY}" \
+ -H "X-API-KEY: ${admin_key}" \
-d '{
"uri": "/v1/chat/completions",
"plugins": {
+ "ai-proxy": {
+"provider": "openai",
+"auth": {
+ "header": {
+"Authorization": "Bearer '"$OPENAI_API_KEY"'"
+ }
+}
+ },
"ai-prompt-decorator": {
"prepend":[
{
"role": "system",
-"content": "I have exams tomorrow so explain conceptually and
briefly"
+"content": "Answer briefly and conceptually."
}
],
"append":[
{
-"role": "system",
-"content": "End the response with an analogy."
+"role": "user",
+"content": "End the answer with a simple analogy."
}
]
}
-},
-"upstream": {
- "type": "roundrobin",
- "nodes": {
-"api.openai.com:443": 1
- },
- "pass_host": "node",
- "scheme": "https"
}
}'
```
-Now send a request:
+
+
+
+
+Create a route with the `ai-proxy` and `ai-prompt-decorator` plugins
configured as such:
+
+```yaml title="adc.yaml"
+services:
+ - name: prompt-decorator-service
+routes:
+ - name: prompt-decorator-route
+uris:
+ - /openai-chat
+methods:
+ - POST
+plugins:
+ ai-proxy:
+provider: openai
+auth:
+ header:
+Authorization: "Bearer ${admin_key}"
Review Comment:
It should not be `${admin_key}`. This must have been mistakenly
