joeyutong opened a new pull request, #924:
URL: https://github.com/apache/flink-agents/pull/924
Linked discussion: #900
Related: #710, #841, #923
### Purpose of change
This PR implements the recording side of Agent Trace proposed in #900.
The existing Event Log records business Events but does not provide enough
runtime context to identify one input run, distinguish concrete
Action/LLM/Parser/Tool executions, reconstruct nested execution relationships,
or attribute execution failures.
This PR adds execution identity and lifecycle recording to the existing
Event Log path. It complements the business Event lineage introduced by #923
rather than defining another Event-to-Action lineage field:
- Business Event lineage remains responsible for cross-Action causality.
- This PR records concrete execution identity and parent-child containment.
- Business Events emitted by an Action retain that Action execution's
`execution_id`, connecting the two models.
#### API and trace model
- `ExecutionTraceContext` carries run identity, execution hierarchy, entity
identity, and entity metadata.
- `ExecutionLifecycleEvents` defines `started`, `finished`, `failed`, and
`reused` lifecycle Events.
- `ExecutionReporter` and `ExecutionReporters` provide an optional,
best-effort capability for reporting nested executions.
- `ToolExecutionMetadataProvider` allows Tools to contribute small
structured execution metadata.
- `EventLogger` accepts an optional `ExecutionTraceContext`; its default
overload preserves compatibility with existing implementations.
- `AgentPlan` carries the Agent name used by trace records.
- `event-log.trace.enabled` controls Trace persistence and defaults to
`false`.
#### Runtime integration
- `ActionExecutionOperator` creates one run context per processed input and
one execution context per Action invocation.
- `ActionTask` carries execution context and active child-execution state
across continuations and Flink state restoration.
- Action executions emit lifecycle Events around invocation and emit
`reused` when completed Action state is reused.
- Business Events continue through `EventRouter`.
- Built-in Execution Events use `ExecutionEventSink` and do not enter the
business Event routing path.
- `RunnerContextImpl` implements `ExecutionReporter`, creates child
execution contexts, and pairs start and terminal reports.
- `ExecutionEventLogger` sends Execution Events to the shared
`EventLogWriter`.
- `EventLogWriter` owns logger open, append, flush, and close operations;
write failures remain best-effort.
#### Action and resource instrumentation
- `ChatModelAction` records one LLM execution per framework model invocation.
- Structured-output parsing is recorded as a separate Parser execution.
- `ToolCallAction` records one Tool execution per concrete tool call,
including error responses and thrown exceptions.
- The instrumentation does not change existing retry, ignore, or Tool-call
execution semantics.
- MCP Tools remain Tool executions; MCP Server identity is stored as
metadata.
- Explicit `load_skill` calls record the loaded Skill as Tool execution
metadata.
- Failure Events preserve the deepest available cause and use stable problem
categories.
#### Python alignment
- Python exposes the same `ExecutionReporter` capability and entity/problem
vocabulary as Java.
- Python LLM, Parser, and Tool boundaries report the same lifecycle
semantics.
- Python reporting delegates to the runtime context without introducing a
second trace model.
- Python Event IDs represent unique Event occurrences rather than
content-derived identities.
#### Event Log format and compatibility
`EventLogRecord` combines `EventContext`, optional `ExecutionTraceContext`,
and `Event`.
The JSONL representation is flattened for querying and aggregation. It
includes Event occurrence fields, run identity, execution hierarchy, entity
information, lifecycle status, failure category, and Event attributes.
The framework deserializer continues to read the previous nested Event Log
format. Existing external consumers that parse the raw JSON shape must migrate
to the normalized field names.
Trace persistence is disabled by default. When disabled, business Events
continue to be logged without trace context and Execution Events are not
persisted.
A restored `ActionTask` retains its run and execution identities. A
source-replayed input not represented by restored Action state starts a new run.
### Tests
- Added and updated Java tests for trace contexts, lifecycle Events, Event
Log serde, Event Log writing, Action execution, retries, Tool reporting, MCP
metadata, and Skill metadata.
- Added and updated Python tests for reporter behavior, LLM/Parser retries,
Tool reporting, MCP metadata, and Skill metadata.
- `mvn -pl runtime -am -DskipITs -Dsurefire.failIfNoSpecifiedTests=false
-Dtest=EventLogRecordJsonSerdeTest,ExecutionTraceContextTest,ActionExecutionOperatorTest
test`
- 50 tests passed.
- `uv run pytest flink_agents/api/tests/test_execution_reporter.py -q`
- 2 tests passed.
### API
This PR introduces the public Trace APIs described above and extends
`EventLogger` with a backward-compatible default overload accepting optional
trace context.
The normalized raw Event Log JSON shape is a compatibility change for
external consumers. Legacy records remain readable through the framework
deserializer.
### Documentation
- [ ] `doc-needed`
- [ ] `doc-not-needed`
- [x] `doc-included`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
