kayx23 commented on code in PR #13210: URL: https://github.com/apache/apisix/pull/13210#discussion_r3109776826
########## docs/en/latest/plugins/ai-request-rewrite.md: ########## @@ -102,66 +119,646 @@ curl "http://127.0.0.1:9180/apisix/admin/routes/1"; -X PUT \ }' ``` -Now send a request: +</TabItem> +<TabItem value="adc" label="ADC"> + +Create a Route with the `ai-request-rewrite` Plugin: + +```yaml title="adc.yaml" +services: + - name: ai-request-rewrite-service + routes: + - name: ai-request-rewrite-route + uris: + - /anything + methods: + - POST + plugins: + ai-request-rewrite: + provider: openai + auth: + header: + Authorization: "Bearer ${OPENAI_API_KEY}" + options: + model: gpt-4 + prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged." + upstream: + type: roundrobin + nodes: + - host: httpbin.org + port: 80 + weight: 1 +``` + +Synchronize the configuration to the gateway: ```shell -curl "http://127.0.0.1:9080/anything"; \ +adc sync -f adc.yaml +``` + +</TabItem> +<TabItem value="ingress" label="Ingress Controller"> + +<Tabs groupId="k8s-api"> +<TabItem value="gateway-api" label="Gateway API"> + +```yaml title="ai-request-rewrite-gw.yaml" +apiVersion: v1 +kind: Service +metadata: + namespace: aic + name: httpbin-external-domain +spec: + type: ExternalName + externalName: httpbin.org +--- +apiVersion: apisix.apache.org/v1alpha1 +kind: PluginConfig +metadata: + namespace: aic + name: ai-request-rewrite-plugin-config +spec: + plugins: + - name: ai-request-rewrite + config: + provider: openai + auth: + header: + Authorization: "Bearer your-api-key" + options: + model: gpt-4 + prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged." +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + namespace: aic + name: ai-request-rewrite-route +spec: + parentRefs: + - name: apisix + rules: + - matches: + - path: + type: Exact + value: /anything + method: POST + filters: + - type: ExtensionRef + extensionRef: + group: apisix.apache.org + kind: PluginConfig + name: ai-request-rewrite-plugin-config + backendRefs: + - name: httpbin-external-domain + port: 80 +``` + +</TabItem> +<TabItem value="ingress" label="APISIX Ingress Controller"> + +```yaml title="ai-request-rewrite-ic.yaml" +apiVersion: apisix.apache.org/v2 +kind: ApisixUpstream +metadata: + namespace: aic + name: httpbin-external-domain +spec: + ingressClassName: apisix + externalNodes: + - type: Domain + name: httpbin.org +--- +apiVersion: apisix.apache.org/v2 +kind: ApisixRoute +metadata: + namespace: aic + name: ai-request-rewrite-route +spec: + ingressClassName: apisix + http: + - name: ai-request-rewrite-route + match: + paths: + - /anything + methods: + - POST + upstreams: + - name: httpbin-external-domain + plugins: + - name: ai-request-rewrite + enable: true + config: + provider: openai + auth: + header: + Authorization: "Bearer your-api-key" + options: + model: gpt-4 + prompt: "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged." +``` + +</TabItem> +</Tabs> + +Apply the configuration to your cluster: + +```shell +kubectl apply -f ai-request-rewrite-ic.yaml +``` + +</TabItem> +</Tabs> + +Send a POST request to the Route with some personally identifiable information: + +```shell +curl "http://127.0.0.1:9080/anything"; -X POST \ -H "Content-Type: application/json" \ -d '{ - "name": "John Doe", - "email": "[email protected]", - "credit_card": "4111 1111 1111 1111", - "ssn": "123-45-6789", - "address": "123 Main St" + "content": "John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789." }' ``` -The request body send to the LLM Service is as follows: +You should receive a response similar to the following: ```json { - "messages": [ - { - "role": "system", - "content": "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver's license numbers). Replace detected sensitive values with a masked format (e.g., '*** **** **** 1234') for credit card numbers). Ensure the JSON structure remains unchanged." - }, - { - "role": "user", - "content": "{\n\"name\":\"John Doe\",\n\"email\":\"[email protected]\",\n\"credit_card\":\"4111 1111 1111 1111\",\n\"ssn\":\"123-45-6789\",\n\"address\":\"123 Main St\"\n}" - } - ] + "args": {}, + "data": "{\n \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-****.\"\n }", Review Comment: This sample response does not seem to match the plugin behavior. `ai-request-rewrite` replaces the original request body with the extracted LLM text before proxying upstream, so httpbin should reflect the rewritten body, not a `messages` array with `model: "openai"` or unrelated system content like `Client information from customer service calls`. As written, this example is likely to confuse users about what actually gets forwarded upstream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
