shreemaan-abhishek opened a new pull request, #13528:
URL: https://github.com/apache/apisix/pull/13528
### Description
The `ai-aws-content-moderation` plugin sent the **raw** HTTP request body to
AWS Comprehend instead of parsing the LLM request and extracting the actual
prompt content. As a result, moderation scored the undecoded JSON envelope,
while the upstream LLM acts on the decoded content, so the two see different
text:
- A body containing `"content":"toxic"` was scored by Comprehend as the
literal escaped string, whereas the upstream model decodes it to `toxic`.
- The full `{"model":...,"messages":[...]}` envelope was scored as-is,
adding noise to the toxicity result.
This is inconsistent with the sibling `ai-aliyun-content-moderation` plugin,
which is protocol-aware.
#### What changed
Make the plugin protocol-aware, in the same style as
`ai-aliyun-content-moderation`:
- Require/validate the `application/json` content type.
- Parse the JSON body, detect the client protocol via `ai-protocols`, and
extract the LLM-visible content (`messages[].content`, Responses
input/instructions, Anthropic message content, etc.).
- Send only the normalized, decoded content to Comprehend.
Because the plugin runs in the `rewrite` phase (before `ai-proxy`), it
detects the protocol directly rather than relying on `ctx.ai_client_protocol`.
Requests that are not recognized AI requests (non-JSON content type,
unparseable bodies, or JSON that carries no LLM content) are handled by the
existing `fail_mode` (`skip` by default), so non-AI traffic on Consumer-bound
plugins keeps its current pass-through behavior.
#### Behavior change
Making the plugin JSON/LLM-only is a behavior change for anyone running it
on non-JSON routes. With the default `fail_mode: skip`, such requests now pass
through unchecked instead of being forwarded verbatim to Comprehend; set
`fail_mode: error` to reject them.
#### Which issue(s) this PR fixes:
Fixes #
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not,
please discuss on the [APISIX mailing
list](https://github.com/apache/apisix/tree/master#community) first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
