Croway commented on code in PR #25410:
URL: https://github.com/apache/camel/pull/25410#discussion_r3749888956
##########
catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/openai-operations.adoc:
##########
@@ -168,6 +168,112 @@ The following headers are set after an embeddings request:
| `CamelOpenAISimilarityScore` | Double | Cosine similarity (if reference
embedding provided)
|===
+== Moderation Operation
+
+The `moderation` operation checks text against the OpenAI usage policies. It
is the canonical pre-filter for untrusted
+input on a public-facing route: rejecting policy-violating content before
spending chat tokens or triggering tool calls.
+
+The message body is passed through unchanged and the verdict is exposed as
headers, so the result can be used for
+content-based routing while the original content stays available to the rest
of the route.
+
+=== Guarding a Route
+
+[tabs]
+====
+Java::
++
+[source,java]
+----
+from("platform-http:/chat")
+ .to("openai:moderation?moderationModel=omni-moderation-latest")
+ .choice()
+ .when(header(OpenAIConstants.MODERATION_FLAGGED).isEqualTo(true))
+ .setBody(constant("Your message violates our usage policy."))
+ .otherwise()
+ .to("openai:chat-completion?model=gpt-5")
+ .end();
+----
+
+YAML::
++
+[source,yaml]
+----
+- from:
+ uri: platform-http:/chat
+ steps:
+ - to: openai:moderation?moderationModel=omni-moderation-latest
+ - choice:
+ when:
+ - simple: "${header.CamelOpenAIModerationFlagged} == true"
+ steps:
+ - setBody:
+ constant: "Your message violates our usage policy."
+ otherwise:
+ steps:
+ - to: openai:chat-completion?model=gpt-5
+----
+====
+
+=== Batch Moderation
+
+A `List` body moderates every element in a single API call.
`CamelOpenAIModerationFlagged` is then `true` when at least
+one element was flagged, while the category headers hold one entry per input,
in the order of the input list.
+
+The header shape follows the body shape: a `String` body yields `Map` headers
and a `List` body yields `List` headers,
+including a list with a single element. That way a route processing batches
does not have to special-case a batch that
+happens to hold one item:
+
+[source,java]
+----
+from("direct:moderate-batch")
+ .to("openai:moderation")
+ .process(exchange -> {
+ List<Map<String, Boolean>> categories
+ =
exchange.getMessage().getHeader(OpenAIConstants.MODERATION_CATEGORIES,
List.class);
+ // categories.get(0) belongs to the first input, and so on
+ });
+----
+
+=== Moderation Parameters
+
+[cols="1,1,1,3"]
+|===
+| Parameter | Type | Default | Description
+
+| `moderationModel` | String | `omni-moderation-latest` | The moderation model
to use
+| `storeFullResponse` | boolean | `false` | Store the full SDK response in the
`CamelOpenAIModerationResponse` exchange property
+|===
+
+=== Failure Modes
+
+The operation is meant to gate untrusted content, so it fails the exchange
rather than letting a message through
+without a verdict:
+
+* the API returning a number of results that does not match the number of
inputs raises a `CamelExchangeException`,
+ instead of leaving `CamelOpenAIModerationFlagged` as `false`;
+* a missing body, an empty list, or a list containing `null` elements raises
an `IllegalArgumentException`.
+
+=== Moderation Output Headers
+
+The following headers are set after a moderation request:
+
+[cols="1,1,3"]
+|===
+| Header | Type | Description
+
+| `CamelOpenAIModerationFlagged` | Boolean | Whether the input violates the
usage policies. For a batch, `true` when at least one input was flagged
+| `CamelOpenAIModerationCategories` | Map/List | Category name to violation
flag. A `Map` for a single input, a `List` of maps for a batch
+| `CamelOpenAIModerationCategoryScores` | Map/List | Category name to
confidence score. A `Map` for a single input, a `List` of maps for a batch
+| `CamelOpenAIModerationResponseModel` | String | The model used for moderation
+|===
+
+The category names are the ones returned by the API, for example `hate`,
`hate/threatening`, `self-harm/intent`,
+`sexual/minors` and `violence/graphic`.
+
+NOTE: The `illicit` and `illicit/violent` categories are optional in the API
model. OpenAI returns them, but an
+xref:others:openai-providers.adoc[OpenAI-compatible provider] may not, in
which case they are absent from the
+category map. The score map always contains every category.
+
Review Comment:
would it be possible to provide an example (and a test) that show how the
header _CamelOpenAIModerationCategoryScores_ can be used? for example:
```
simple("${header.CamelOpenAIModerationCategoryScores[hate]} > 0.85")
```
I do think this use case is interesting for the users
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
