kayx23 commented on code in PR #12917: URL: https://github.com/apache/apisix/pull/12917#discussion_r3032170738
########## docs/en/latest/plugins/ai-prompt-decorator.md: ########## @@ -27,83 +27,277 @@ description: This document contains information about the Apache APISIX ai-promp # --> +<head> + <link rel="canonical" href="https://docs.api7.ai/hub/ai-prompt-decorator"; /> +</head> + ## Description -The `ai-prompt-decorator` plugin simplifies access to LLM providers, such as OpenAI and Anthropic, and their models by appending or prepending prompts into the request. +The `ai-prompt-decorator` Plugin simplifies access to LLM providers, such as OpenAI and Anthropic, and their models. It modifies user input prompts by prefixing and appending pre-engineered prompts to set contexts in content generation. This practice helps the model operate within desired guidelines during interactions. ## Plugin Attributes | **Field** | **Required** | **Type** | **Description** | | ----------------- | --------------- | -------- | --------------------------------------------------- | -| `prepend` | Conditionally\* | Array | An array of prompt objects to be prepended | -| `prepend.role` | Yes | String | Role of the message (`system`, `user`, `assistant`) | -| `prepend.content` | Yes | String | Content of the message. Minimum length: 1 | -| `append` | Conditionally\* | Array | An array of prompt objects to be appended | -| `append.role` | Yes | String | Role of the message (`system`, `user`, `assistant`) | -| `append.content` | Yes | String | Content of the message. Minimum length: 1 | +| `prepend` | Conditionally\* | Array | An array of prompt objects to be prepended. | +| `prepend.role` | Yes | String | Role of the message, such as `system`, `user`, or `assistant`. | +| `prepend.content` | Yes | String | Content of the message (prompt). | +| `append` | Conditionally\* | Array | An array of prompt objects to be appended. | +| `append.role` | Yes | String | Role of the message, such as `system`, `user`, or `assistant`. | +| `append.content` | Yes | String | Content of the message (prompt). | \* **Conditionally Required**: At least one of `prepend` or `append` must be provided. -## Example usage +## Example -Create a route with the `ai-prompt-decorator` plugin like so: +The following example will be using OpenAI as the upstream service provider. Before proceeding, create an [OpenAI account](https://openai.com) and an [API key](https://openai.com/blog/openai-api). You can optionally save the key to an environment variable as such: + +```shell +export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> +``` + +If you are working with other LLM providers, please refer to the provider's documentation to obtain an API key. + +### Prepend and Append Messages + +The following example demonstrates how to configure the `ai-prompt-decorator` Plugin to prepend a system message and append a user message to the user input message. + +<Tabs +groupId="api" +defaultValue="admin-api" +values={[ +{label: 'Admin API', value: 'admin-api'}, +{label: 'ADC', value: 'adc'}, +{label: 'Ingress Controller', value: 'aic'} +]}> + +<TabItem value="admin-api"> + +Create a Route to the chat completion endpoint with pre-configured prompt templates as such: ```shell curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ - -H "X-API-KEY: ${ADMIN_API_KEY}" \ + -H "X-API-KEY: ${admin_key}" \ -d '{ "uri": "/v1/chat/completions", "plugins": { + "ai-proxy": { + "provider": "openai", + "auth": { + "header": { + "Authorization": "Bearer '"$OPENAI_API_KEY"'" + } + } + }, "ai-prompt-decorator": { "prepend":[ { "role": "system", - "content": "I have exams tomorrow so explain conceptually and briefly" + "content": "Answer briefly and conceptually." } ], "append":[ { - "role": "system", - "content": "End the response with an analogy." + "role": "user", + "content": "End the answer with a simple analogy." } ] } - }, - "upstream": { - "type": "roundrobin", - "nodes": { - "api.openai.com:443": 1 - }, - "pass_host": "node", - "scheme": "https" } }' ``` -Now send a request: +</TabItem> + +<TabItem value="adc"> + +Create a route with the `ai-proxy` and `ai-prompt-decorator` plugins configured as such: + +```yaml title="adc.yaml" +services: + - name: prompt-decorator-service + routes: + - name: prompt-decorator-route + uris: + - /openai-chat + methods: + - POST + plugins: + ai-proxy: + provider: openai + auth: + header: + Authorization: "Bearer ${admin_key}" Review Comment: It should not be `${admin_key}`. This must have been mistakenly replaced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
