GitHub user Zhuoxi2000 added a comment to the discussion: [Discuss][Observability] OpenTelemetry GenAI export for Agent Traces (follow-up to #900)
> Thanks for working through these. I think this narrows the remaining > questions considerably. > > Assuming each converter invocation receives a fixed, complete set of > already-collected files, the batch shape seems to resolve the ordering and > timeout questions: the converter can group records by `inputRunId`, sort > them, and use end-of-input rather than a timeout to determine that no more > records are coming. > > Incomplete executions may still be present, though. `EventLogWriter` calls > append and flush for each record, but treats failures as best-effort and does > not retry them. A crash can also leave a start record in the collected files > without a terminal record. A restored Action execution keeps its > `executionId` and `inputRunId`, since both are checkpointed with the > `ActionTask`. By contrast, the state pairing nested LLM, Parser, and Tool > starts with their terminal reports is transient, so recovery can produce new > child executions under the same run. > > How would we like those incomplete executions to appear in the exported > trace? Would omitting them hide useful evidence of failed work, while > synthesizing an end timestamp risk implying a duration we did not observe? > Could emitting them as errors with an explicit "incomplete/synthetic close" > marker preserve that distinction, or would a diagnostic-only record be > clearer? > > Could > [`trace_tree.py`](https://github.com/apache/flink-agents/blob/1c55683deb5c7e8d44997851403e75b5c0fbdb57/python/flink_agents/cli/trace_tree.py) > provide a useful precedent for input validation and diagnostics? It reports > conditions such as `MISSING_PARENT` for business-event lineage and > `MALFORMED_RECORD` for invalid input. Although it intentionally ignores > execution lifecycle records and therefore does not answer the incomplete-span > policy itself, might its warning model be reusable? > > It also discovers only `events-*.log`, which is the `FileEventLogger` naming > contract rather than the default SLF4J output. Would it help for Phase 1 to > state whether the converter consumes specifically `FileEventLogger` JSONL > files, including how a complete multi-subtask file set is identified, or > normalized records collected from any Event Log sink? > > I also wonder whether the Parser mapping could be explicit. The > run/root/parent fields establish its topology, but the converter still needs > to decide its span name, `gen_ai.operation.name`, and OTel span kind. Since > #924 records PARSER as a first-class execution type, would defining one > stable mapping here make traces more consistent across converter versions? > > The OTel GenAI convention permits a custom `gen_ai.operation.name` when none > of its well-known values applies, and OpenLLMetry uses that mechanism for > names such as `execute_task` and `llm_request`. OpenInference's `CHAIN`, > `GUARDRAIL`, and `EVALUATOR` values belong to the separate > `openinference.span.kind` classification. Since Phoenix and Langfuse > recognize that convention, would it be worth deciding whether Phase 1 emits > only OTel GenAI attributes or also an OpenInference classification for > AI-aware rendering? > > Separately, for the OTel span kind, would `SpanKind.INTERNAL` fit Parser, and > perhaps Action, as in-process work, while preserving `CLIENT` for spans that > directly represent remote calls? Thanks! totally agreed that batch input gives us a clear ordering/completion model. The converter can group by `inputRunId`, sort by timestamp, and treat end-of-input as completion. I’ll make that explicit in #970. **Incomplete executions.** I’d combine the two approaches: still emit the span, close it at the only timestamp we observed (so zero duration rather than inventing an end time), keep the status `UNSET`, add something like `flink_agents.execution.incomplete = true`, and surface a diagnostic as well. I’d avoid `ERROR` because a missing terminal record doesn’t necessarily mean the execution failed — it could also be a dropped best-effort append or a recovery/replay artifact. Marking it incomplete keeps that evidence queryable without claiming an outcome we didn’t observe. A terminal record with no matching start would get the same treatment, with a `MISSING_START` diagnostic. **Diagnostics.** I agree we should follow `trace_tree.py` here and reuse the same machine-readable shape (`code`, id, `message`, `filePath`, `lineNumber`). We can keep `MALFORMED_RECORD` and add `INCOMPLETE_EXECUTION` / `MISSING_START` rather than inventing a separate model. **Input contract.** Phase 1 should operate on the JSON records themselves, not depend on a file naming convention. It can take explicit files, ignore unknown fields, and handle concatenated JSON objects. Directory discovery for `events-*.log` can just be a convenience matching `trace_tree.py`. For multi-subtask jobs, completeness remains the caller’s responsibility, but per-file counts and diagnostics should make missing input easier to spot. **Parser mapping.** Agreed: `parse {entityName}`, `gen_ai.operation.name = "parse"`, `SpanKind.INTERNAL`. **OpenInference.** I’d keep Phase 1 OTel GenAI-only. `openinference.span.kind` can be an additive opt-in follow-up for backends like Phoenix/Langfuse without changing topology or IDs. **Span kind.** Agreed on `INTERNAL` for Parser and Action. I’d use `INTERNAL` for Tool as well for now, since the Event Log doesn’t tell us whether a tool is local or remote. LLM is the only clearly remote operation, so that can be `CLIENT`. If tool transport is recorded later in `entityMetadata`, remote/MCP tools can be classified as `CLIENT` then. I’ll fold these into #970’s scope/design notes. GitHub link: https://github.com/apache/flink-agents/discussions/929#discussioncomment-17956216 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
