Claus Ibsen created CAMEL-27313:
-----------------------------------

             Summary: camel-ai-observability - a genai dev console with the 
agent runs, model calls and tool calls
                 Key: CAMEL-27313
                 URL: https://issues.apache.org/jira/browse/CAMEL-27313
             Project: Camel
          Issue Type: Sub-task
          Components: camel-ai
            Reporter: Claus Ibsen
             Fix For: 4.25.0


The GenAI observability (CAMEL-27131) records model calls, agent runs, tool 
calls and, at level {{full}}, the messages as OpenTelemetry spans. Today every 
client works them out from the raw span dump:
* the Camel TUI: CAMEL-27135
* the Camel MCP server: CAMEL-27312, through a span model shared in 
camel-jbang-core

A {{genai}} dev console in camel-ai-observability would give every client the 
same JSON straight from the runtime, Kaoto and other tools included:
* the model calls per model (tokens in, out and cached, latency, finish reasons)
* the agent runs with their steps in order
* the level in use
* on request, the conversation of one run

It would also work without a tracer, if the console keeps its own bounded 
buffer.

The console would carry the same rules as the views it serves. Only model 
operations count as model calls. A run is matched by its spans (or 
observations). Message content is shown only at level {{full}}, only when 
asked, and capped, because it can hold customers' data.

The CLI, the TUI and the MCP server (CAMEL-27312) could then read the console 
instead of the span dump.

Suggested by the local model benchmark work, planned by Claus for 4.25.0.

_Created by Claude Code on behalf of Claus Ibsen_



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to