[
https://issues.apache.org/jira/browse/CAMEL-23369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18076567#comment-18076567
]
Federico Mariani commented on CAMEL-23369:
------------------------------------------
[~pcongiusti] I'd like to work on the implementation, but I'd like to have your
feedback. For the spans I plan to add _gen_ai.*_ constants in _TagConstants_ as
per opentelemetry gen-ai convention, then create a base class for the GenAISpan
Decorator, and finally, concrete implementations for OpenAI, Langchain4j,
SpringAI and so on.
For the metrics, I'd create an EventNotifier that filter the AI components and
gather information about the model, tokens and so on. At the moment we do not
have any specific implementation for metrics in Camel, since we rely on
runtimes (I guess), but I think that for AI workloads it would be nice to have
these information (I should be able to know how many tokens I am using).
> Add OpenTelemetry GenAI observability (traces and metrics) for AI components
> ----------------------------------------------------------------------------
>
> Key: CAMEL-23369
> URL: https://issues.apache.org/jira/browse/CAMEL-23369
> Project: Camel
> Issue Type: Improvement
> Affects Versions: 4.20.0
> Reporter: Federico Mariani
> Priority: Major
>
> Currently none of the Camel AI components (camel-openai,
> camel-langchain4j-chat, camel-langchain4j-embeddings, camel-spring-ai-chat,
> camel-google-vertexai, camel-aws-bedrock, camel-ibm-watsonx-ai) are
> instrumented by the _camel-telemetry_ tracing framework or the metrics
> frameworks (camel-opentelemetry-metrics, camel-micrometer).
> Similarly, the existing EventNotifier-based metrics (exchange duration,
> in-flight gauges) don't capture AI-specific data like token counts.
>
>
>
> The https://opentelemetry.io/docs/specs/semconv/gen-ai/ define standard span
> attributes (gen_ai.operation.name, gen_ai.provider.name,
> gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens,
> gen_ai.response.finish_reasons, etc.) and metrics (gen_ai.client.token.usage
> histogram, gen_ai.client.operation.duration histogram) for LLM operations.
> Camel AIs components already expose token counts and response metadata as
> Camel message headers, but this data does not flow into distributed traces or
> metrics.
> We might improve the current camel AI implementations, and expose all the
> important metrics if not already exposed (for example, cache read and cache
> write tokens)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
