This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 3eecf146044f CAMEL-23706: TUI - Add OTel Spans tab with Jaeger-style
waterfall view (#23819)
3eecf146044f is described below
commit 3eecf146044f12d35a1832164e5f1fb1fa270a3e
Author: Claus Ibsen <[email protected]>
AuthorDate: Mon Jun 8 07:20:10 2026 +0200
CAMEL-23706: TUI - Add OTel Spans tab with Jaeger-style waterfall view
(#23819)
* CAMEL-23706: Add camel cmd span CLI command for OTel span display
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Add OTel Spans tab with Jaeger-style waterfall view
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Enrich OTel spans with routeId/processorId, add MCP
tool, improve waterfall
- DevConsole: enrich span JSON with routeId (from route endpoint URI
matching)
and processorId (from span name parsing), propagate routeId to child spans
- TUI: add tui_get_spans MCP tool for raw OTel span data access with traceId
filtering and limit support
- SpansTab: improve waterfall rendering with route-based coloring, duration
bars,
span kind labels, and detail panel showing attributes
- Filed CAMEL-23708 for missing RECEIVED spans in nested async stub/seda
paths
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23708: Add reproducer test for multicast+seda/stub OTel spans
Add MulticastSedaTest with 4 scenarios testing span completeness for
multicast to seda/stub topologies including the production-like
route-topology example. All tests pass - confirms the missing spans
are caused by DevSpanExporter capacity eviction (500 bounded queue),
not a framework instrumentation bug.
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23708: DevSpanExporter trace-aware eviction and capacity bump
Switch from per-span FIFO eviction (LinkedBlockingQueue) to
trace-aware eviction (LinkedHashMap grouped by traceId). When
capacity is exceeded, entire traces are evicted oldest-first
so we never have half-complete traces with orphaned spans.
Bump default capacity from 500 to 2000 (~60 traces at 33
spans/trace, covering ~5 minutes of data at 5s intervals).
Co-Authored-By: Claude <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Improve OTel trace list with sorting, filtering, and
remote detection
- Add ROUTE and FROM columns, detect remote components (kafka, http, etc.)
- Add /-activated filter matching traceId, exchangeId, routes, and
components
- Add column sorting (s/S) cycling newest/duration/spans/routes/status with
visual sort indicator arrows on sortable column headers
- Add F1 help menu with full OTel Spans documentation
- Compute SPANS and DEPTH from effective waterfall (excluding EVENT_PROCESS)
using lightweight counting instead of full tree builds for performance
- Add getTableDataAsJson() for MCP tui_get_table support
- Guard filter text input against tab-switching number keys in CamelMonitor
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - MCP navigate/getTable support for More submenu tabs
- Extend navigateToTab() to handle More submenu tabs (Beans, Browse,
Circuit Breaker, Classpath, Configuration, Consumers, Inflight,
Memory, Metrics, Spans, Startup, Threads) so MCP tui_navigate and
tui_get_table can reach them by name
- Include More submenu tab names in getTabNames() for MCP discovery
- Enrich F1 help text with waterfall view documentation and sample
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Fix sort header styles to all use yellow
All column headers in the Spans trace list now use yellow consistently.
The sort arrow indicator on the active column is sufficient to show
which column is being sorted.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Sort cycle follows column order left-to-right
Sort cycle now matches table column order: newest, spans, routes,
status, duration (was newest, duration, spans, routes, status).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Add route and from as sortable columns
- Add route and from to sort cycle (trace-id, route, from, spans,
routes, status, duration) matching left-to-right column order
- Rename sort key from 'newest' to 'trace-id' to match column name
- Sort route/from alphabetically (case-insensitive), nulls last
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Pre-select first trace when opening Spans tab
Auto-select the first row when traces are available and nothing is
selected yet, avoiding the visual shift when pressing cursor down.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23706: TUI - Stable trace selection across tab switches
Track selected trace by traceId instead of row index. When returning
to the Spans tab after data refresh or trace eviction, the selection
stays on the same trace. Falls back to the first row if the previously
selected trace was evicted.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-23672: TUI - Fix throughput chart not updating
The throughput sample window (1000ms) was too tight for the diagram-
throttled refresh interval (also 1000ms). With thread scheduling
jitter, samples expired before the next refresh arrived, leaving
the sliding window with only 1 sample — too few to compute a rate.
Widen the sample retention window to 2000ms so at least 2 samples
are always available for rate calculation regardless of refresh
interval. Fixes both throughput and endpoint rate charts.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
---------
Signed-off-by: Claus Ibsen <[email protected]>
Co-authored-by: Claude <[email protected]>
---
.../camel/opentelemetry2/DevSpanExporter.java | 51 +-
.../opentelemetry2/OpenTelemetryDevConsole.java | 86 +-
.../camel/opentelemetry2/MulticastSedaTest.java | 301 ++++++
.../pages/jbang-commands/camel-jbang-cmd-span.adoc | 29 +
.../ROOT/pages/jbang-commands/camel-jbang-cmd.adoc | 1 +
.../camel/cli/connector/LocalCliConnector.java | 22 +
.../META-INF/camel-jbang-commands-metadata.json | 2 +-
.../dsl/jbang/core/commands/CamelJBangMain.java | 1 +
.../core/commands/action/CamelSpanAction.java | 270 +++++
.../dsl/jbang/core/commands/tui/CamelMonitor.java | 160 ++-
.../jbang/core/commands/tui/MetricsCollector.java | 4 +-
.../dsl/jbang/core/commands/tui/OverviewTab.java | 3 +-
.../dsl/jbang/core/commands/tui/SpanEntry.java | 66 ++
.../dsl/jbang/core/commands/tui/SpansTab.java | 1078 ++++++++++++++++++++
.../dsl/jbang/core/commands/tui/TuiHelper.java | 4 +-
.../dsl/jbang/core/commands/tui/TuiMcpServer.java | 26 +-
16 files changed, 2076 insertions(+), 28 deletions(-)
diff --git
a/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/DevSpanExporter.java
b/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/DevSpanExporter.java
index 3dfca59782a5..5767e9f76f3b 100644
---
a/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/DevSpanExporter.java
+++
b/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/DevSpanExporter.java
@@ -18,35 +18,40 @@ package org.apache.camel.opentelemetry2;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.LinkedHashMap;
import java.util.List;
-import java.util.Queue;
-import java.util.concurrent.LinkedBlockingQueue;
+import java.util.Map;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.trace.data.SpanData;
import io.opentelemetry.sdk.trace.export.SpanExporter;
/**
- * In-memory {@link SpanExporter} for development use. Stores finished spans
in a bounded queue so they can be queried
+ * In-memory {@link SpanExporter} for development use. Stores finished spans
in a bounded store so they can be queried
* by the dev console and other local tooling.
*
+ * Uses trace-aware eviction: when the total span count exceeds capacity,
entire traces are evicted (oldest first) to
+ * avoid half-complete traces with orphaned spans.
+ *
* Auto-configured by {@link OpenTelemetryTracer} when the Camel profile is
"dev".
*/
final class DevSpanExporter implements SpanExporter {
- static final int DEFAULT_CAPACITY = 500;
+ static final int DEFAULT_CAPACITY = 2000;
- private final Queue<SpanData> spans;
private final int capacity;
private volatile boolean stopped;
+ // LinkedHashMap preserves insertion order of traces (oldest first for
eviction)
+ private final Map<String, List<SpanData>> traces = new LinkedHashMap<>();
+ private int totalSpanCount;
+
DevSpanExporter() {
this(DEFAULT_CAPACITY);
}
DevSpanExporter(int capacity) {
this.capacity = capacity;
- this.spans = new LinkedBlockingQueue<>(capacity);
}
@Override
@@ -54,9 +59,20 @@ final class DevSpanExporter implements SpanExporter {
if (stopped) {
return CompletableResultCode.ofSuccess();
}
- for (SpanData span : spanDataList) {
- while (!spans.offer(span)) {
- spans.poll();
+ synchronized (traces) {
+ for (SpanData span : spanDataList) {
+ String traceId = span.getTraceId();
+ traces.computeIfAbsent(traceId, k -> new
ArrayList<>()).add(span);
+ totalSpanCount++;
+ }
+ // Evict oldest complete traces until we're under capacity
+ while (totalSpanCount > capacity && traces.size() > 1) {
+ var it = traces.entrySet().iterator();
+ if (it.hasNext()) {
+ var oldest = it.next();
+ totalSpanCount -= oldest.getValue().size();
+ it.remove();
+ }
}
}
return CompletableResultCode.ofSuccess();
@@ -74,11 +90,19 @@ final class DevSpanExporter implements SpanExporter {
}
List<SpanData> getFinishedSpans() {
- return new ArrayList<>(spans);
+ synchronized (traces) {
+ List<SpanData> result = new ArrayList<>(totalSpanCount);
+ for (List<SpanData> traceSpans : traces.values()) {
+ result.addAll(traceSpans);
+ }
+ return result;
+ }
}
int getSpanCount() {
- return spans.size();
+ synchronized (traces) {
+ return totalSpanCount;
+ }
}
int getCapacity() {
@@ -86,6 +110,9 @@ final class DevSpanExporter implements SpanExporter {
}
void reset() {
- spans.clear();
+ synchronized (traces) {
+ traces.clear();
+ totalSpanCount = 0;
+ }
}
}
diff --git
a/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/OpenTelemetryDevConsole.java
b/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/OpenTelemetryDevConsole.java
index 42059cbe85c1..d84fd020c850 100644
---
a/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/OpenTelemetryDevConsole.java
+++
b/components/camel-opentelemetry2/src/main/java/org/apache/camel/opentelemetry2/OpenTelemetryDevConsole.java
@@ -16,10 +16,13 @@
*/
package org.apache.camel.opentelemetry2;
+import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.sdk.trace.data.SpanData;
+import org.apache.camel.Route;
import org.apache.camel.spi.annotations.DevConsole;
import org.apache.camel.support.CamelContextHelper;
import org.apache.camel.support.console.AbstractDevConsole;
@@ -86,9 +89,38 @@ public class OpenTelemetryDevConsole extends
AbstractDevConsole {
List<SpanData> spans = exporter.getFinishedSpans();
int start = Math.max(0, spans.size() - limit);
+ // Build lookup map for enriching spans with route context
+ Map<String, String> endpointToRouteId = new HashMap<>();
+ buildEnrichmentMaps(endpointToRouteId);
+
+ // First pass: convert spans to JSON and resolve routeIds for
endpoint spans
JsonArray arr = new JsonArray();
+ Map<String, String> spanIdToRouteId = new HashMap<>();
+ Map<String, String> spanIdToParent = new HashMap<>();
for (int i = start; i < spans.size(); i++) {
- arr.add(spanToJson(spans.get(i)));
+ SpanData sd = spans.get(i);
+ JsonObject jo = spanToJson(sd, endpointToRouteId);
+ arr.add(jo);
+ // Track routeId and parent relationships for propagation
+ if (jo.containsKey("routeId")) {
+ spanIdToRouteId.put(sd.getSpanId(),
jo.getString("routeId"));
+ }
+ String pid = sd.getParentSpanId();
+ if (pid != null && !pid.isEmpty() &&
!"0000000000000000".equals(pid)) {
+ spanIdToParent.put(sd.getSpanId(), pid);
+ }
+ }
+
+ // Second pass: propagate routeId to processor spans by walking
parent chain
+ for (int i = 0; i < arr.size(); i++) {
+ JsonObject jo = (JsonObject) arr.get(i);
+ if (!jo.containsKey("routeId")) {
+ String spanId = jo.getString("spanId");
+ String inheritedRouteId = findAncestorRouteId(spanId,
spanIdToRouteId, spanIdToParent);
+ if (inheritedRouteId != null) {
+ jo.put("routeId", inheritedRouteId);
+ }
+ }
}
root.put("spans", arr);
} else {
@@ -103,8 +135,21 @@ public class OpenTelemetryDevConsole extends
AbstractDevConsole {
return CamelContextHelper.findSingleByType(getCamelContext(),
DevSpanExporter.class);
}
+ private void buildEnrichmentMaps(Map<String, String> endpointToRouteId) {
+ try {
+ // Map route endpoint URIs (sanitized) to route IDs
+ for (Route route : getCamelContext().getRoutes()) {
+ if (route.getEndpoint() != null && route.getId() != null) {
+
endpointToRouteId.put(route.getEndpoint().getEndpointUri(), route.getId());
+ }
+ }
+ } catch (Exception e) {
+ // ignore
+ }
+ }
+
@SuppressWarnings("unchecked")
- private static JsonObject spanToJson(SpanData span) {
+ private static JsonObject spanToJson(SpanData span, Map<String, String>
endpointToRouteId) {
JsonObject jo = new JsonObject();
jo.put("traceId", span.getTraceId());
jo.put("spanId", span.getSpanId());
@@ -126,6 +171,43 @@ public class OpenTelemetryDevConsole extends
AbstractDevConsole {
jo.put("attributes", attrs);
}
+ // Enrich with route context from endpoint URI
+ String uri =
span.getAttributes().get(AttributeKey.stringKey("camel.uri"));
+ if (uri != null) {
+ String routeId = endpointToRouteId.get(uri);
+ if (routeId != null) {
+ jo.put("routeId", routeId);
+ }
+ }
+
+ // Enrich processor spans with processorId extracted from span name
(format: id-shortName)
+ String op = span.getAttributes().get(AttributeKey.stringKey("op"));
+ if ("EVENT_PROCESS".equals(op)) {
+ String name = span.getName();
+ int dash = name.lastIndexOf('-');
+ if (dash > 0) {
+ jo.put("processorId", name.substring(0, dash));
+ }
+ }
+
return jo;
}
+
+ private static String findAncestorRouteId(
+ String spanId, Map<String, String> spanIdToRouteId, Map<String,
String> spanIdToParent) {
+ String current = spanId;
+ int maxDepth = 50;
+ while (current != null && maxDepth-- > 0) {
+ String parent = spanIdToParent.get(current);
+ if (parent == null) {
+ return null;
+ }
+ String routeId = spanIdToRouteId.get(parent);
+ if (routeId != null) {
+ return routeId;
+ }
+ current = parent;
+ }
+ return null;
+ }
}
diff --git
a/components/camel-opentelemetry2/src/test/java/org/apache/camel/opentelemetry2/MulticastSedaTest.java
b/components/camel-opentelemetry2/src/test/java/org/apache/camel/opentelemetry2/MulticastSedaTest.java
new file mode 100644
index 000000000000..fda0f6f91b2f
--- /dev/null
+++
b/components/camel-opentelemetry2/src/test/java/org/apache/camel/opentelemetry2/MulticastSedaTest.java
@@ -0,0 +1,301 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.opentelemetry2;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+import io.opentelemetry.api.common.AttributeKey;
+import io.opentelemetry.sdk.trace.data.SpanData;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.RoutesBuilder;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.opentelemetry2.CamelOpenTelemetryExtension.OtelTrace;
+import org.junit.jupiter.api.Test;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Tests OTel span completeness for multicast to seda/stub topologies with
varying levels of async nesting.
+ *
+ * Verifies that EVENT_RECEIVED spans are properly created and ended for all
consumer routes, including those consuming
+ * from seda/stub endpoints produced by a multicast inside another seda
consumer route (nested async paths).
+ *
+ * See CAMEL-23708 for background on the production observation that triggered
this investigation.
+ */
+public class MulticastSedaTest extends OpenTelemetryTracerTestSupport {
+
+ @Override
+ protected CamelContext createCamelContext() throws Exception {
+ OpenTelemetryTracer tst = new OpenTelemetryTracer();
+ tst.setTracer(otelExtension.getOpenTelemetry().getTracer("traceTest"));
+
tst.setContextPropagators(otelExtension.getOpenTelemetry().getPropagators());
+ tst.setDisableCoreProcessors(true);
+ CamelContext context = super.createCamelContext();
+ CamelContextAware.trySetCamelContext(tst, context);
+ tst.init(context);
+ return context;
+ }
+
+ /**
+ * Baseline: direct → seda → multicast → seda consumer routes.
+ */
+ @Test
+ void testMulticastToSeda() throws Exception {
+ MockEndpoint mockAlpha = getMockEndpoint("mock:alpha");
+ MockEndpoint mockBeta = getMockEndpoint("mock:beta");
+ mockAlpha.expectedMessageCount(1);
+ mockBeta.expectedMessageCount(1);
+
+ template.sendBody("direct:start", "Hello");
+
+ mockAlpha.assertIsSatisfied(5000);
+ mockBeta.assertIsSatisfied(5000);
+
+ await().atMost(10, TimeUnit.SECONDS)
+ .pollInterval(200, TimeUnit.MILLISECONDS)
+ .untilAsserted(() -> {
+ Map<String, OtelTrace> traces = otelExtension.getTraces();
+ assertEquals(1, traces.size());
+ OtelTrace trace = traces.values().iterator().next();
+ assertTrue(trace.getSpans().size() >= 10,
+ "Expected at least 10 spans, got " +
trace.getSpans().size());
+ });
+
+ OtelTrace trace = otelExtension.getTraces().values().iterator().next();
+ List<SpanData> spans = trace.getSpans();
+
+ assertReceivedSpanExists(spans, "seda://orders");
+ assertReceivedSpanExists(spans, "seda://alpha");
+ assertReceivedSpanExists(spans, "seda://beta");
+ }
+
+ /**
+ * Nested async: seda → multicast → seda → leaf seda consumers.
+ */
+ @Test
+ void testNestedSedaMulticast() throws Exception {
+ MockEndpoint mockAlphaOut = getMockEndpoint("mock:alpha-out");
+ MockEndpoint mockBetaOut = getMockEndpoint("mock:beta-out");
+ mockAlphaOut.expectedMessageCount(1);
+ mockBetaOut.expectedMessageCount(1);
+
+ template.sendBody("seda:orders", "Hello");
+
+ mockAlphaOut.assertIsSatisfied(5000);
+ mockBetaOut.assertIsSatisfied(5000);
+
+ await().atMost(10, TimeUnit.SECONDS)
+ .pollInterval(200, TimeUnit.MILLISECONDS)
+ .untilAsserted(() -> {
+ List<SpanData> allSpans = otelExtension.getSpans();
+ assertTrue(allSpans.size() >= 10, "Expected at least 10
spans, got " + allSpans.size());
+ });
+
+ List<SpanData> allSpans = otelExtension.getSpans();
+
+ assertReceivedSpanExists(allSpans, "seda://orders");
+ assertReceivedSpanExists(allSpans, "seda://alpha");
+ assertReceivedSpanExists(allSpans, "seda://beta");
+ }
+
+ /**
+ * Production-like topology matching the route-topology example with stub
replacing kafka.
+ *
+ * <pre>
+ * direct:order-entry → direct:process → direct:validate → stub:orders
+ * → dispatcher route (multicast)
+ * → stub:fulfillment → fulfillment route → stub:warehouse →
mock:warehouse
+ * → stub:notifications → notification route → stub:email →
mock:email
+ * </pre>
+ */
+ @Test
+ void testProductionTopology() throws Exception {
+ MockEndpoint mockWarehouse = getMockEndpoint("mock:warehouse");
+ MockEndpoint mockEmail = getMockEndpoint("mock:email");
+ mockWarehouse.expectedMessageCount(1);
+ mockEmail.expectedMessageCount(1);
+
+ template.sendBody("direct:order-entry", "order-001");
+
+ mockWarehouse.assertIsSatisfied(5000);
+ mockEmail.assertIsSatisfied(5000);
+
+ await().atMost(10, TimeUnit.SECONDS)
+ .pollInterval(200, TimeUnit.MILLISECONDS)
+ .untilAsserted(() -> {
+ List<SpanData> allSpans = otelExtension.getSpans();
+ assertTrue(allSpans.size() >= 12, "Expected at least 12
spans, got " + allSpans.size());
+ });
+
+ List<SpanData> allSpans = otelExtension.getSpans();
+
+ // Direct routes RECEIVED spans
+ assertReceivedSpanExists(allSpans, "direct://order-entry");
+ assertReceivedSpanExists(allSpans, "direct://process");
+ assertReceivedSpanExists(allSpans, "direct://validate");
+
+ // First async hop: dispatcher consumes from stub:orders
+ assertReceivedSpanExists(allSpans, "stub://orders");
+
+ // Second async hop: leaf routes consume from stub endpoints produced
by multicast
+ assertReceivedSpanExists(allSpans, "stub://fulfillment");
+ assertReceivedSpanExists(allSpans, "stub://notifications");
+
+ // Third async hop: terminal consumers
+ assertReceivedSpanExists(allSpans, "stub://warehouse");
+ assertReceivedSpanExists(allSpans, "stub://email");
+ }
+
+ /**
+ * Multiple messages through the production topology to verify span
consistency.
+ */
+ @Test
+ void testProductionTopologyMultipleMessages() throws Exception {
+ int count = 5;
+ MockEndpoint mockWarehouse = getMockEndpoint("mock:warehouse");
+ MockEndpoint mockEmail = getMockEndpoint("mock:email");
+ mockWarehouse.expectedMessageCount(count);
+ mockEmail.expectedMessageCount(count);
+
+ for (int i = 0; i < count; i++) {
+ template.sendBody("direct:order-entry", "order-" + i);
+ }
+
+ mockWarehouse.assertIsSatisfied(10000);
+ mockEmail.assertIsSatisfied(10000);
+
+ await().atMost(15, TimeUnit.SECONDS)
+ .pollInterval(500, TimeUnit.MILLISECONDS)
+ .untilAsserted(() -> {
+ Map<String, OtelTrace> traces = otelExtension.getTraces();
+ assertEquals(count, traces.size(), "Should have " + count
+ " traces");
+ });
+
+ Map<String, OtelTrace> traces = otelExtension.getTraces();
+ int tracesWithMissingSpans = 0;
+
+ for (OtelTrace trace : traces.values()) {
+ List<SpanData> spans = trace.getSpans();
+
+ boolean hasFulfillmentReceived = hasReceivedSpan(spans,
"stub://fulfillment");
+ boolean hasNotificationReceived = hasReceivedSpan(spans,
"stub://notifications");
+
+ if (!hasFulfillmentReceived || !hasNotificationReceived) {
+ tracesWithMissingSpans++;
+ }
+ }
+
+ assertEquals(0, tracesWithMissingSpans,
+ tracesWithMissingSpans + " of " + count
+ + " traces have missing
RECEIVED spans for stub:fulfillment or stub:notifications");
+ }
+
+ private void assertReceivedSpanExists(List<SpanData> spans, String uri) {
+ List<SpanData> found = spans.stream()
+ .filter(s ->
uri.equals(s.getAttributes().get(AttributeKey.stringKey("camel.uri"))))
+ .filter(s ->
"EVENT_RECEIVED".equals(s.getAttributes().get(AttributeKey.stringKey("op"))))
+ .collect(Collectors.toList());
+ assertEquals(1, found.size(), uri + " RECEIVED span should exist");
+ assertTrue(found.get(0).hasEnded(), uri + " RECEIVED span should have
ended");
+ }
+
+ private boolean hasReceivedSpan(List<SpanData> spans, String uri) {
+ return spans.stream()
+ .anyMatch(s ->
uri.equals(s.getAttributes().get(AttributeKey.stringKey("camel.uri")))
+ &&
"EVENT_RECEIVED".equals(s.getAttributes().get(AttributeKey.stringKey("op"))));
+ }
+
+ @Override
+ protected RoutesBuilder createRouteBuilder() {
+ return new RouteBuilder() {
+ @Override
+ public void configure() {
+ // === Simple seda topology ===
+ from("direct:start")
+ .routeId("entry")
+ .to("seda:orders");
+
+ from("seda:orders")
+ .routeId("dispatcher")
+ .multicast()
+ .to("seda:alpha", "seda:beta")
+ .end();
+
+ from("seda:alpha")
+ .routeId("alpha")
+ .to("seda:alpha-out")
+ .to("mock:alpha");
+
+ from("seda:beta")
+ .routeId("beta")
+ .to("seda:beta-out")
+ .to("mock:beta");
+
+ from("seda:alpha-out")
+ .routeId("alpha-out")
+ .to("mock:alpha-out");
+
+ from("seda:beta-out")
+ .routeId("beta-out")
+ .to("mock:beta-out");
+
+ // === Production-like topology (stub simulating kafka) ===
+ from("direct:order-entry")
+ .routeId("order-entry")
+ .to("direct:process");
+
+ from("direct:process")
+ .routeId("process-order")
+ .to("direct:validate")
+ .to("stub:orders");
+
+ from("direct:validate")
+ .routeId("validate-order")
+ .log("Validating: ${body}");
+
+ from("stub:orders")
+ .routeId("order-dispatcher")
+ .multicast()
+ .to("stub:fulfillment", "stub:notifications")
+ .end();
+
+ from("stub:fulfillment")
+ .routeId("fulfillment")
+ .to("stub:warehouse");
+
+ from("stub:notifications")
+ .routeId("notification")
+ .to("stub:email");
+
+ from("stub:warehouse")
+ .routeId("warehouse")
+ .to("mock:warehouse");
+
+ from("stub:email")
+ .routeId("email")
+ .to("mock:email");
+ }
+ };
+ }
+}
diff --git
a/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd-span.adoc
b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd-span.adoc
new file mode 100644
index 000000000000..0bd6f3271913
--- /dev/null
+++
b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd-span.adoc
@@ -0,0 +1,29 @@
+
+// AUTO-GENERATED by camel-package-maven-plugin - DO NOT EDIT THIS FILE
+= camel cmd span
+
+Display OpenTelemetry spans from running Camel integrations
+
+
+== Usage
+
+[source,bash]
+----
+camel cmd span [options]
+----
+
+
+
+== Options
+
+[cols="2,5,1,2",options="header"]
+|===
+| Option | Description | Default | Type
+| `--filter` | Filter spans by name (substring match) | | String
+| `--limit` | Maximum number of spans to display | 100 | int
+| `--logging-color` | Use colored logging | true | boolean
+| `--sort` | Sort by name, duration, or status | | String
+| `-h,--help` | Display the help and sub-commands | | boolean
+|===
+
+
diff --git
a/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd.adoc
b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd.adoc
index 4a9792305122..8687fdd9751c 100644
--- a/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd.adoc
+++ b/docs/user-manual/modules/ROOT/pages/jbang-commands/camel-jbang-cmd.adoc
@@ -32,6 +32,7 @@ camel cmd [options]
| xref:jbang-commands/camel-jbang-cmd-route-structure.adoc[route-structure] |
Dump Camel route structure
| xref:jbang-commands/camel-jbang-cmd-route-topology.adoc[route-topology] |
Display inter-route topology connections
| xref:jbang-commands/camel-jbang-cmd-send.adoc[send] | Send messages to
endpoints
+| xref:jbang-commands/camel-jbang-cmd-span.adoc[span] | Display OpenTelemetry
spans from running Camel integrations
| xref:jbang-commands/camel-jbang-cmd-start-group.adoc[start-group] | Start
Camel route groups
| xref:jbang-commands/camel-jbang-cmd-start-route.adoc[start-route] | Start
Camel routes
| xref:jbang-commands/camel-jbang-cmd-stop-group.adoc[stop-group] | Stop Camel
route groups
diff --git
a/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/LocalCliConnector.java
b/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/LocalCliConnector.java
index ae024871ebd4..5fbb9eeb16ea 100644
---
a/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/LocalCliConnector.java
+++
b/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/LocalCliConnector.java
@@ -365,6 +365,8 @@ public class LocalCliConnector extends ServiceSupport
implements CliConnector, C
doActionTraceTask(root);
} else if ("browse".equals(action)) {
doActionBrowseTask(root);
+ } else if ("span".equals(action)) {
+ doActionSpanTask(root);
} else if ("receive".equals(action)) {
doActionReceiveTask(root);
} else if ("readme".equals(action)) {
@@ -887,6 +889,26 @@ public class LocalCliConnector extends ServiceSupport
implements CliConnector, C
}
}
+ private void doActionSpanTask(JsonObject root) throws IOException {
+ DevConsole dc =
camelContext.getCamelContextExtension().getContextPlugin(DevConsoleRegistry.class)
+ .resolveById("opentelemetry");
+ if (dc != null) {
+ Map<String, Object> params = new HashMap<>();
+ params.put("dump", "true");
+ String limit = root.getString("limit");
+ if (limit != null) {
+ params.put("limit", limit);
+ }
+ JsonObject json = (JsonObject) dc.call(DevConsole.MediaType.JSON,
params);
+ LOG.trace("Updating output file: {}", outputFile);
+ IOHelper.writeText(json.toJson(), outputFile);
+ } else {
+ JsonObject json = new JsonObject();
+ json.put("enabled", false);
+ IOHelper.writeText(json.toJson(), outputFile);
+ }
+ }
+
private void doActionReceiveTask(JsonObject root) throws Exception {
DevConsole dc =
camelContext.getCamelContextExtension().getContextPlugin(DevConsoleRegistry.class)
.resolveById("receive");
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/generated/resources/META-INF/camel-jbang-commands-metadata.json
b/dsl/camel-jbang/camel-jbang-core/src/generated/resources/META-INF/camel-jbang-commands-metadata.json
index 019f53a12b41..cd8c5f4cd094 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/generated/resources/META-INF/camel-jbang-commands-metadata.json
+++
b/dsl/camel-jbang/camel-jbang-core/src/generated/resources/META-INF/camel-jbang-commands-metadata.json
@@ -3,7 +3,7 @@
{ "name": "ask", "fullName": "ask", "description": "Ask a question about a
running Camel application using AI", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.Ask", "options": [ { "names":
"--api-key", "description": "API key. Also reads ANTHROPIC_API_KEY,
OPENAI_API_KEY, or LLM_API_KEY env vars", "javaType": "java.lang.String",
"type": "string" }, { "names": "--api-type", "description": "API type:
'ollama', 'openai', or 'anthropic'", "javaType": "LlmClient.ApiType", "type"
[...]
{ "name": "bind", "fullName": "bind", "description": "DEPRECATED: Bind
source and sink Kamelets as a new Camel integration", "deprecated": true,
"sourceClass": "org.apache.camel.dsl.jbang.core.commands.bind.Bind", "options":
[ { "names": "--error-handler", "description": "Add error handler
(none|log|sink:<endpoint>). Sink endpoints are expected in the format
[[apigroup\/]version:]kind:[namespace\/]name, plain Camel URIs or Kamelet
name.", "javaType": "java.lang.String", "type": "stri [...]
{ "name": "catalog", "fullName": "catalog", "description": "List artifacts
from Camel Catalog", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.catalog.CatalogCommand", "options": [
{ "names": "-h,--help", "description": "Display the help and sub-commands",
"javaType": "boolean", "type": "boolean" } ], "subcommands": [ { "name":
"component", "fullName": "catalog component", "description": "List components
from the Camel Catalog", "sourceClass": "org.apache.camel.dsl.jbang.co [...]
- { "name": "cmd", "fullName": "cmd", "description": "Performs commands in
the running Camel integrations, such as start\/stop route, or change logging
levels.", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.action.CamelAction", "options": [ {
"names": "-h,--help", "description": "Display the help and sub-commands",
"javaType": "boolean", "type": "boolean" } ], "subcommands": [ { "name":
"browse", "fullName": "cmd browse", "description": "Browse pending messages on
endpoints [...]
+ { "name": "cmd", "fullName": "cmd", "description": "Performs commands in
the running Camel integrations, such as start\/stop route, or change logging
levels.", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.action.CamelAction", "options": [ {
"names": "-h,--help", "description": "Display the help and sub-commands",
"javaType": "boolean", "type": "boolean" } ], "subcommands": [ { "name":
"browse", "fullName": "cmd browse", "description": "Browse pending messages on
endpoints [...]
{ "name": "completion", "fullName": "completion", "description": "Generate
completion script for bash\/zsh", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.Complete", "options": [ { "names":
"-h,--help", "description": "Display the help and sub-commands", "javaType":
"boolean", "type": "boolean" } ] },
{ "name": "config", "fullName": "config", "description": "Get and set user
configuration values", "sourceClass":
"org.apache.camel.dsl.jbang.core.commands.config.ConfigCommand", "options": [ {
"names": "-h,--help", "description": "Display the help and sub-commands",
"javaType": "boolean", "type": "boolean" } ], "subcommands": [ { "name": "get",
"fullName": "config get", "description": "Display user configuration value",
"sourceClass": "org.apache.camel.dsl.jbang.core.commands.config. [...]
{ "name": "debug", "fullName": "debug", "description": "Debug local Camel
integration", "sourceClass": "org.apache.camel.dsl.jbang.core.commands.Debug",
"options": [ { "names": "--ago", "description": "Use ago instead of yyyy-MM-dd
HH:mm:ss in timestamp.", "javaType": "boolean", "type": "boolean" }, { "names":
"--background", "description": "Run in the background", "defaultValue":
"false", "javaType": "boolean", "type": "boolean" }, { "names":
"--background-wait", "description": "To [...]
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/CamelJBangMain.java
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/CamelJBangMain.java
index 989fdd70e0cd..4f5a5ede5b51 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/CamelJBangMain.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/CamelJBangMain.java
@@ -119,6 +119,7 @@ public class CamelJBangMain implements Callable<Integer> {
.addSubcommand("route-structure", new CommandLine(new
CamelRouteStructureAction(this)))
.addSubcommand("route-topology", new CommandLine(new
CamelRouteTopologyAction(this)))
.addSubcommand("send", new CommandLine(new
CamelSendAction(this)))
+ .addSubcommand("span", new CommandLine(new
CamelSpanAction(this)))
.addSubcommand("start-group", new CommandLine(new
CamelRouteGroupStartAction(this)))
.addSubcommand("start-route", new CommandLine(new
CamelRouteStartAction(this)))
.addSubcommand("stop-group", new CommandLine(new
CamelRouteGroupStopAction(this)))
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelSpanAction.java
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelSpanAction.java
new file mode 100644
index 000000000000..89c1bd958d30
--- /dev/null
+++
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/action/CamelSpanAction.java
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.action;
+
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+
+import com.github.freva.asciitable.AsciiTable;
+import com.github.freva.asciitable.Column;
+import com.github.freva.asciitable.HorizontalAlign;
+import com.github.freva.asciitable.OverflowBehaviour;
+import org.apache.camel.dsl.jbang.core.commands.CamelJBangMain;
+import org.apache.camel.dsl.jbang.core.common.PathUtils;
+import org.apache.camel.dsl.jbang.core.common.ProcessHelper;
+import org.apache.camel.dsl.jbang.core.common.TerminalWidthHelper;
+import org.apache.camel.util.TimeUtils;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+import picocli.CommandLine;
+
[email protected](name = "span",
+ description = "Display OpenTelemetry spans from running
Camel integrations",
+ sortOptions = false, showDefaultValues = true,
+ footer = {
+ "%nExamples:",
+ " camel cmd span",
+ " camel cmd span --limit=50",
+ " camel cmd span --filter=direct" })
+public class CamelSpanAction extends ActionBaseCommand {
+
+ public static class SortCompletionCandidates implements Iterable<String> {
+
+ public SortCompletionCandidates() {
+ }
+
+ @Override
+ public Iterator<String> iterator() {
+ return List.of("name", "duration", "status").iterator();
+ }
+ }
+
+ @CommandLine.Parameters(description = "Name or pid of running Camel
integration", arity = "0..1")
+ String name = "*";
+
+ @CommandLine.Option(names = { "--limit" }, defaultValue = "100",
+ description = "Maximum number of spans to display")
+ int limit = 100;
+
+ @CommandLine.Option(names = { "--filter" },
+ description = "Filter spans by name (substring match)")
+ String filter;
+
+ @CommandLine.Option(names = { "--sort" }, completionCandidates =
SortCompletionCandidates.class,
+ description = "Sort by name, duration, or status")
+ String sort;
+
+ @CommandLine.Option(names = { "--logging-color" }, defaultValue = "true",
+ description = "Use colored logging")
+ boolean loggingColor = true;
+
+ private volatile long pid;
+
+ public CamelSpanAction(CamelJBangMain main) {
+ super(main);
+ }
+
+ @Override
+ public Integer doCall() throws Exception {
+ List<Long> pids = findPids(name);
+ if (pids.isEmpty()) {
+ return 1;
+ } else if (pids.size() > 1) {
+ printer().println("Name or pid " + name + " matches " + pids.size()
+ + " running Camel integrations. Specify a name
or PID that matches exactly one.");
+ return 1;
+ }
+
+ this.pid = pids.get(0);
+
+ Path outputFile = getOutputFile(Long.toString(pid));
+ PathUtils.deleteFile(outputFile);
+
+ JsonObject root = new JsonObject();
+ root.put("action", "span");
+ root.put("dump", "true");
+ root.put("limit", Integer.toString(limit));
+
+ Path f = getActionFile(Long.toString(pid));
+ try {
+ PathUtils.writeTextSafely(root.toJson(), f);
+ } catch (Exception e) {
+ // ignore
+ }
+
+ JsonObject jo = getJsonObject(outputFile);
+ if (jo != null) {
+ Boolean enabled = jo.getBoolean("enabled");
+ if (enabled == null || !enabled) {
+ printer().println(
+ "OpenTelemetry in-memory exporter is not enabled. Use
--observe flag when running the integration.");
+ PathUtils.deleteFile(outputFile);
+ return 0;
+ }
+
+ JsonArray arr = jo.getCollection("spans");
+ if (arr == null || arr.isEmpty()) {
+ printer().println("No spans captured yet.");
+ PathUtils.deleteFile(outputFile);
+ return 0;
+ }
+
+ List<Row> rows = new ArrayList<>();
+ root = loadStatus(this.pid);
+ String integrationName = null;
+ String ago = null;
+ if (root != null) {
+ JsonObject context = (JsonObject) root.get("context");
+ if (context != null) {
+ integrationName = context.getString("name");
+ if ("CamelJBang".equals(integrationName)) {
+ ProcessHandle ph =
ProcessHandle.of(this.pid).orElse(null);
+ integrationName = ProcessHelper.extractName(root, ph);
+ }
+ }
+ ProcessHandle ph = ProcessHandle.of(this.pid).orElse(null);
+ long uptime = extractSince(ph);
+ ago = TimeUtils.printSince(uptime);
+ }
+
+ for (int i = 0; i < arr.size(); i++) {
+ JsonObject span = (JsonObject) arr.get(i);
+ Row row = new Row();
+ row.pid = Long.toString(this.pid);
+ row.name = integrationName;
+ row.ago = ago;
+ row.traceId = span.getString("traceId");
+ row.spanId = span.getString("spanId");
+ row.parentSpanId = span.getString("parentSpanId");
+ row.spanName = span.getString("name");
+ row.kind = span.getString("kind");
+ row.status = span.getString("status");
+ Long durationMs = span.getLong("durationMs");
+ row.durationMs = durationMs != null ? durationMs : 0;
+
+ if (filter != null && !matchesFilter(row.spanName, filter)) {
+ continue;
+ }
+
+ rows.add(row);
+ }
+
+ if (sort != null) {
+ rows.sort(this::sortRow);
+ }
+
+ tableSpans(rows);
+ } else {
+ printer().printErr("Response from running Camel with PID " + pid +
" not received within 5 seconds");
+ return 1;
+ }
+
+ PathUtils.deleteFile(outputFile);
+ return 0;
+ }
+
+ private boolean matchesFilter(String spanName, String pattern) {
+ if (spanName == null) {
+ return false;
+ }
+ return spanName.toLowerCase().contains(pattern.toLowerCase());
+ }
+
+ protected void tableSpans(List<Row> rows) {
+ int tw = terminalWidth();
+ int fixedWidth = 10 + 10 + 10 + 12 + 8 + 10;
+ int borderOverhead = TerminalWidthHelper.noBorderOverhead(7);
+ int nameWidth = TerminalWidthHelper.flexWidth(tw, fixedWidth,
borderOverhead, 15, 60);
+
+ printer().println(AsciiTable.getTable(AsciiTable.NO_BORDERS, rows,
Arrays.asList(
+ new
Column().header("TRACE-ID").headerAlign(HorizontalAlign.CENTER)
+ .with(r -> shortId(r.traceId)),
+ new
Column().header("SPAN-ID").headerAlign(HorizontalAlign.CENTER)
+ .with(r -> shortId(r.spanId)),
+ new
Column().header("PARENT").headerAlign(HorizontalAlign.CENTER)
+ .with(r -> shortId(r.parentSpanId)),
+ new Column().header("NAME").dataAlign(HorizontalAlign.LEFT)
+ .maxWidth(nameWidth, OverflowBehaviour.ELLIPSIS_RIGHT)
+ .with(r -> r.spanName),
+ new Column().header("KIND").headerAlign(HorizontalAlign.CENTER)
+ .with(r -> r.kind),
+ new
Column().header("STATUS").headerAlign(HorizontalAlign.CENTER)
+ .with(r -> r.status),
+ new
Column().header("DURATION").headerAlign(HorizontalAlign.RIGHT)
+ .dataAlign(HorizontalAlign.RIGHT)
+ .with(r -> r.durationMs + "ms"))));
+ }
+
+ protected int sortRow(Row o1, Row o2) {
+ String s = sort;
+ int negate = 1;
+ if (s.startsWith("-")) {
+ s = s.substring(1);
+ negate = -1;
+ }
+ switch (s) {
+ case "name":
+ return compareNullSafe(o1.spanName, o2.spanName) * negate;
+ case "duration":
+ return Long.compare(o1.durationMs, o2.durationMs) * negate;
+ case "status":
+ return compareNullSafe(o1.status, o2.status) * negate;
+ default:
+ return 0;
+ }
+ }
+
+ private static int compareNullSafe(String a, String b) {
+ if (a == null && b == null) {
+ return 0;
+ }
+ if (a == null) {
+ return -1;
+ }
+ if (b == null) {
+ return 1;
+ }
+ return a.compareToIgnoreCase(b);
+ }
+
+ private static String shortId(String id) {
+ if (id == null || id.isEmpty()) {
+ return "";
+ }
+ if (id.length() > 8) {
+ return id.substring(0, 8);
+ }
+ return id;
+ }
+
+ private static class Row {
+ String pid;
+ String name;
+ String ago;
+ String traceId;
+ String spanId;
+ String parentSpanId;
+ String spanName;
+ String kind;
+ String status;
+ long durationMs;
+ }
+
+}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
index a53e2f560922..9c21d6d3a6cf 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/CamelMonitor.java
@@ -151,6 +151,8 @@ public class CamelMonitor extends CamelCommand {
// Trace/history data — shared between CamelMonitor and tabs
private final AtomicReference<List<TraceEntry>> traces = new
AtomicReference<>(Collections.emptyList());
private final Map<String, Long> traceFilePositions = new
ConcurrentHashMap<>();
+ // OTel span data — shared between CamelMonitor and SpansTab
+ private final AtomicReference<List<SpanEntry>> otelSpans = new
AtomicReference<>(List.of());
// selectedPid is stored on ctx (MonitorContext) so tabs can access it
@@ -217,6 +219,7 @@ public class CamelMonitor extends CamelCommand {
private InflightTab inflightTab;
private MemoryTab memoryTab;
private ThreadsTab threadsTab;
+ private SpansTab spansTab;
private OverviewTab overviewTab;
// "Switch integration" popup state
@@ -285,6 +288,7 @@ public class CamelMonitor extends CamelCommand {
inflightTab = new InflightTab(ctx);
memoryTab = new MemoryTab(ctx, metrics);
threadsTab = new ThreadsTab(ctx);
+ spansTab = new SpansTab(ctx, otelSpans);
overviewTab = new OverviewTab(
ctx, metrics, stoppingPids,
this::resetIntegrationTabState);
@@ -413,7 +417,7 @@ public class CamelMonitor extends CamelCommand {
return true;
}
if (ke.isDown()) {
- morePopupState.selectNext(11);
+ morePopupState.selectNext(12);
return true;
}
int shortcutSel = morePopupShortcut(ke);
@@ -435,8 +439,9 @@ public class CamelMonitor extends CamelCommand {
case 6 -> inflightTab;
case 7 -> memoryTab;
case 8 -> metricsTab;
- case 9 -> startupTab;
- case 10 -> threadsTab;
+ case 9 -> spansTab;
+ case 10 -> startupTab;
+ case 11 -> threadsTab;
default -> null;
};
if (activeMoreTab != null) {
@@ -510,7 +515,9 @@ public class CamelMonitor extends CamelCommand {
}
boolean probeEditing = tabsState.selected() == TAB_HTTP &&
httpTab.isProbeMode();
boolean logSearchActive = tabsState.selected() == TAB_LOG &&
logTab.isSearchInputActive();
- boolean textEditing = probeEditing || logSearchActive;
+ boolean spanFilterActive = tabsState.selected() == TAB_MORE &&
activeMoreTab == spansTab
+ && spansTab.isFilterInputActive();
+ boolean textEditing = probeEditing || logSearchActive ||
spanFilterActive;
if (!textEditing && (ke.isCharIgnoreCase('q') || ke.isCtrlC())) {
runner.quit();
return true;
@@ -1215,6 +1222,7 @@ public class CamelMonitor extends CamelCommand {
ListItem.from(Line.from(Span.raw(" "), Span.styled("I",
keyStyle), Span.raw("nflight"))),
ListItem.from(Line.from(Span.raw(" "), Span.styled("M",
keyStyle), Span.raw("emory"))),
ListItem.from(Line.from(Span.raw(" M"), Span.styled("e",
keyStyle), Span.raw("trics"))),
+ ListItem.from(Line.from(Span.raw(" "), Span.styled("O",
keyStyle), Span.raw("Tel Spans"))),
ListItem.from(Line.from(Span.raw(" "), Span.styled("S",
keyStyle), Span.raw("tartup"))),
ListItem.from(Line.from(Span.raw(" "), Span.styled("T",
keyStyle), Span.raw("hreads"))),
};
@@ -1321,12 +1329,15 @@ public class CamelMonitor extends CamelCommand {
if (ke.isChar('e')) {
return 8;
}
- if (ke.isChar('s')) {
+ if (ke.isChar('o')) {
return 9;
}
- if (ke.isChar('t')) {
+ if (ke.isChar('s')) {
return 10;
}
+ if (ke.isChar('t')) {
+ return 11;
+ }
return -1;
}
@@ -1917,6 +1928,51 @@ public class CamelMonitor extends CamelCommand {
}
refreshTraceData(selectedPids);
}
+ if (tabsState.selected() == TAB_MORE && activeMoreTab == spansTab
+ && ctx.selectedPid != null && spansTab.spanRefreshRequested) {
+ spansTab.spanRefreshRequested = false;
+ refreshSpanData();
+ }
+ }
+
+ @SuppressWarnings("unchecked")
+ private void refreshSpanData() {
+ String pid = ctx.selectedPid;
+ if (pid == null) {
+ return;
+ }
+ try {
+ // Send action to request span data
+ Path outputFile = ctx.getOutputFile(pid);
+ PathUtils.deleteFile(outputFile);
+
+ JsonObject action = new JsonObject();
+ action.put("action", "span");
+ action.put("dump", "true");
+ action.put("limit", "500");
+ Path actionFile = ctx.getActionFile(pid);
+ PathUtils.writeTextSafely(action.toJson(), actionFile);
+
+ // Poll for response
+ JsonObject response = MonitorContext.pollJsonResponse(outputFile,
3000);
+ if (response != null) {
+ Boolean enabled = response.getBoolean("enabled");
+ if (enabled != null && enabled) {
+ JsonArray arr = response.getCollection("spans");
+ if (arr != null) {
+ List<SpanEntry> entries = new ArrayList<>();
+ for (int i = 0; i < arr.size(); i++) {
+ JsonObject spanObj = (JsonObject) arr.get(i);
+ entries.add(SpanEntry.fromJson(spanObj));
+ }
+ otelSpans.set(entries);
+ }
+ }
+ PathUtils.deleteFile(outputFile);
+ }
+ } catch (Exception e) {
+ // ignore
+ }
}
@SuppressWarnings("unchecked")
@@ -2352,6 +2408,11 @@ public class CamelMonitor extends CamelCommand {
drawOverlay.clear();
}
+ private static final String[] MORE_TAB_NAMES = {
+ "Beans", "Browse", "Circuit Breaker", "Classpath", "Configuration",
+ "Consumers", "Inflight", "Memory", "Metrics", "Spans", "Startup",
"Threads"
+ };
+
String navigateToTab(String tabName) {
for (int i = 0; i < TAB_NAMES.length; i++) {
if (TAB_NAMES[i].equalsIgnoreCase(tabName)) {
@@ -2359,6 +2420,34 @@ public class CamelMonitor extends CamelCommand {
return TAB_NAMES[i];
}
}
+ // Check More submenu tabs
+ for (int i = 0; i < MORE_TAB_NAMES.length; i++) {
+ if (MORE_TAB_NAMES[i].equalsIgnoreCase(tabName)) {
+ morePopupState.select(i);
+ lastMoreSelection = i;
+ activeMoreTab = switch (i) {
+ case 0 -> beansTab;
+ case 1 -> browseTab;
+ case 2 -> circuitBreakerTab;
+ case 3 -> classpathTab;
+ case 4 -> configurationTab;
+ case 5 -> consumersTab;
+ case 6 -> inflightTab;
+ case 7 -> memoryTab;
+ case 8 -> metricsTab;
+ case 9 -> spansTab;
+ case 10 -> startupTab;
+ case 11 -> threadsTab;
+ default -> null;
+ };
+ if (activeMoreTab != null) {
+ overviewTab.selectCurrentIntegration();
+ tabsState.select(TAB_MORE);
+ activeMoreTab.onTabSelected();
+ }
+ return MORE_TAB_NAMES[i];
+ }
+ }
return null;
}
@@ -2378,7 +2467,10 @@ public class CamelMonitor extends CamelCommand {
}
List<String> getTabNames() {
- return List.of(TAB_NAMES);
+ List<String> names = new ArrayList<>();
+ names.addAll(List.of(TAB_NAMES));
+ names.addAll(List.of(MORE_TAB_NAMES));
+ return names;
}
List<String> getActionLabels() {
@@ -2612,6 +2704,60 @@ public class CamelMonitor extends CamelCommand {
return diagramTab.getTopologyDataAsJson();
}
+ @SuppressWarnings("unchecked")
+ JsonObject getSpanData(String traceId, int limit) {
+ String pid = ctx.selectedPid;
+ if (pid == null) {
+ JsonObject err = new JsonObject();
+ err.put("error", "No integration selected");
+ return err;
+ }
+ try {
+ Path outputFile = ctx.getOutputFile(pid);
+ PathUtils.deleteFile(outputFile);
+
+ JsonObject action = new JsonObject();
+ action.put("action", "span");
+ action.put("dump", "true");
+ action.put("limit", String.valueOf(limit));
+ Path actionFile = ctx.getActionFile(pid);
+ PathUtils.writeTextSafely(action.toJson(), actionFile);
+
+ JsonObject response = MonitorContext.pollJsonResponse(outputFile,
3000);
+ if (response != null) {
+ PathUtils.deleteFile(outputFile);
+ Boolean enabled = response.getBoolean("enabled");
+ if (enabled == null || !enabled) {
+ JsonObject err = new JsonObject();
+ err.put("error", "OpenTelemetry not enabled (requires
--observe flag)");
+ return err;
+ }
+ if (traceId != null && !traceId.isBlank()) {
+ JsonArray all = response.getCollection("spans");
+ if (all != null) {
+ JsonArray filtered = new JsonArray();
+ for (int i = 0; i < all.size(); i++) {
+ JsonObject span = (JsonObject) all.get(i);
+ String tid = span.getString("traceId");
+ if (tid != null && tid.contains(traceId)) {
+ filtered.add(span);
+ }
+ }
+ response.put("spans", filtered);
+ }
+ }
+ return response;
+ }
+ JsonObject err = new JsonObject();
+ err.put("error", "Timeout waiting for span data");
+ return err;
+ } catch (Exception e) {
+ JsonObject err = new JsonObject();
+ err.put("error", e.getMessage());
+ return err;
+ }
+ }
+
String navigateDiagramToRoute(String routeId) {
navigateToTab("Diagram");
if (diagramTab.selectRoute(routeId)) {
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MetricsCollector.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MetricsCollector.java
index c342c77fe1ee..a2db00029ec8 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MetricsCollector.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/MetricsCollector.java
@@ -160,7 +160,7 @@ class MetricsCollector {
LinkedList<long[]> samples = throughputSamples.computeIfAbsent(pid, k
-> new LinkedList<>());
samples.add(new long[] { now, currentTotal, currentFailed });
- while (!samples.isEmpty() && now - samples.get(0)[0] > 1000) {
+ while (!samples.isEmpty() && now - samples.get(0)[0] > 2000) {
samples.remove(0);
}
@@ -271,7 +271,7 @@ class MetricsCollector {
Map<String, LinkedList<Long>> inHistMap, Map<String,
LinkedList<Long>> outHistMap) {
LinkedList<long[]> samples = samplesMap.computeIfAbsent(pid, k -> new
LinkedList<>());
samples.add(new long[] { now, inTotal, outTotal });
- while (!samples.isEmpty() && now - samples.get(0)[0] > 1000) {
+ while (!samples.isEmpty() && now - samples.get(0)[0] > 2000) {
samples.remove(0);
}
if (samples.size() >= 2) {
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OverviewTab.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OverviewTab.java
index f7507415cfb6..3e731a45699f 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OverviewTab.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/OverviewTab.java
@@ -383,7 +383,8 @@ class OverviewTab implements MonitorTab {
Line titleLine;
if (chartMode == CHART_SINGLE && ctx.selectedPid != null) {
IntegrationInfo chartSel = ctx.findSelectedIntegration();
- String chartName = chartSel != null ?
TuiHelper.truncate(chartSel.name, 12) : ctx.selectedPid;
+ String chartName = chartSel != null ?
TuiHelper.truncate(chartSel.name, 12)
+ : ctx.selectedPid != null ? ctx.selectedPid : "?";
titleLine = Line.from(
Span.raw(" ["),
Span.styled(chartName, Style.EMPTY.fg(Color.YELLOW)),
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpanEntry.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpanEntry.java
new file mode 100644
index 000000000000..8825d21f89e6
--- /dev/null
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpanEntry.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.util.Map;
+
+import org.apache.camel.util.json.JsonObject;
+
+record SpanEntry(
+ String traceId,
+ String spanId,
+ String parentSpanId,
+ String name,
+ String kind,
+ String status,
+ long startEpochNanos,
+ long endEpochNanos,
+ long durationMs,
+ String routeId,
+ String processorId,
+ Map<String, Object> attributes) {
+
+ @SuppressWarnings("unchecked")
+ static SpanEntry fromJson(JsonObject jo) {
+ Map<String, Object> attrs = null;
+ JsonObject attrsObj = jo.getMap("attributes");
+ if (attrsObj != null && !attrsObj.isEmpty()) {
+ attrs = attrsObj;
+ }
+ return new SpanEntry(
+ jo.getString("traceId"),
+ jo.getString("spanId"),
+ jo.getString("parentSpanId"),
+ jo.getString("name"),
+ jo.getString("kind"),
+ jo.getString("status"),
+ jo.getLongOrDefault("startEpochNanos", 0),
+ jo.getLongOrDefault("endEpochNanos", 0),
+ jo.getLongOrDefault("durationMs", 0),
+ jo.getString("routeId"),
+ jo.getString("processorId"),
+ attrs);
+ }
+
+ boolean isRoot() {
+ return parentSpanId == null || parentSpanId.isEmpty();
+ }
+
+ boolean isError() {
+ return "ERROR".equals(status);
+ }
+}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpansTab.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpansTab.java
new file mode 100644
index 000000000000..be861745cd23
--- /dev/null
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/SpansTab.java
@@ -0,0 +1,1078 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.jbang.core.commands.tui;
+
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import dev.tamboui.layout.Constraint;
+import dev.tamboui.layout.Layout;
+import dev.tamboui.layout.Rect;
+import dev.tamboui.style.Color;
+import dev.tamboui.style.Style;
+import dev.tamboui.terminal.Frame;
+import dev.tamboui.text.Line;
+import dev.tamboui.text.Span;
+import dev.tamboui.text.Text;
+import dev.tamboui.tui.event.KeyCode;
+import dev.tamboui.tui.event.KeyEvent;
+import dev.tamboui.widgets.block.Block;
+import dev.tamboui.widgets.block.BorderType;
+import dev.tamboui.widgets.input.TextInputState;
+import dev.tamboui.widgets.paragraph.Paragraph;
+import dev.tamboui.widgets.scrollbar.Scrollbar;
+import dev.tamboui.widgets.scrollbar.ScrollbarState;
+import dev.tamboui.widgets.table.Cell;
+import dev.tamboui.widgets.table.Row;
+import dev.tamboui.widgets.table.Table;
+import dev.tamboui.widgets.table.TableState;
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+
+class SpansTab implements MonitorTab {
+
+ private static final String[] SORT_COLUMNS = { "trace-id", "route",
"from", "spans", "routes", "status", "duration" };
+
+ private final MonitorContext ctx;
+ private final AtomicReference<List<SpanEntry>> spans;
+
+ private final TableState traceListState = new TableState();
+ private final ScrollbarState waterfallScrollState = new ScrollbarState();
+
+ private boolean waterfallView;
+ private String selectedTraceId;
+ private String selectedListTraceId;
+ private int waterfallScroll;
+ private int waterfallSelected;
+ private boolean showProcessors = true;
+ private String sortColumn = "trace-id";
+ private int sortIndex;
+ private boolean sortReversed;
+ private boolean filterInputActive;
+ private TextInputState filterInputState = new TextInputState("");
+ private String filterTerm;
+
+ boolean spanRefreshRequested;
+
+ SpansTab(MonitorContext ctx, AtomicReference<List<SpanEntry>> spans) {
+ this.ctx = ctx;
+ this.spans = spans;
+ }
+
+ @Override
+ public void onTabSelected() {
+ spanRefreshRequested = true;
+ }
+
+ @Override
+ public void onIntegrationChanged() {
+ spanRefreshRequested = true;
+ waterfallView = false;
+ selectedTraceId = null;
+ }
+
+ @Override
+ public boolean handleKeyEvent(KeyEvent ke) {
+ if (waterfallView) {
+ return handleWaterfallKeys(ke);
+ }
+ if (filterInputActive) {
+ return handleFilterInput(ke);
+ }
+ if (ke.isConfirm()) {
+ Integer sel = traceListState.selected();
+ List<TraceSummary> summaries = buildFilteredTraceSummaries();
+ if (sel != null && sel >= 0 && sel < summaries.size()) {
+ selectedTraceId = summaries.get(sel).traceId;
+ waterfallView = true;
+ waterfallScroll = 0;
+ waterfallSelected = 0;
+ }
+ return true;
+ }
+ if (ke.isKey(KeyCode.F5)) {
+ spanRefreshRequested = true;
+ return true;
+ }
+ if (ke.isChar('/')) {
+ filterInputActive = true;
+ filterInputState = new TextInputState(filterTerm != null ?
filterTerm : "");
+ return true;
+ }
+ if (ke.isChar('s')) {
+ sortIndex = (sortIndex + 1) % SORT_COLUMNS.length;
+ sortColumn = SORT_COLUMNS[sortIndex];
+ sortReversed = false;
+ return true;
+ }
+ if (ke.isChar('S')) {
+ sortReversed = !sortReversed;
+ return true;
+ }
+ return false;
+ }
+
+ private boolean handleFilterInput(KeyEvent ke) {
+ if (ke.isKey(KeyCode.ESCAPE)) {
+ filterInputActive = false;
+ return true;
+ }
+ if (ke.isConfirm()) {
+ String text = filterInputState.text().trim();
+ filterTerm = text.isEmpty() ? null : text;
+ filterInputActive = false;
+ selectedListTraceId = null;
+ traceListState.select(0);
+ return true;
+ }
+ FormHelper.handleTextInput(ke, filterInputState);
+ return true;
+ }
+
+ boolean isFilterInputActive() {
+ return filterInputActive;
+ }
+
+ private boolean handleWaterfallKeys(KeyEvent ke) {
+ if (ke.isUp()) {
+ if (waterfallSelected > 0) {
+ waterfallSelected--;
+ }
+ return true;
+ }
+ if (ke.isDown()) {
+ List<WaterfallNode> nodes = buildWaterfallNodes(selectedTraceId);
+ if (waterfallSelected < nodes.size() - 1) {
+ waterfallSelected++;
+ }
+ return true;
+ }
+ if (ke.isPageUp() || ke.isKey(KeyCode.PAGE_UP)) {
+ waterfallSelected = Math.max(0, waterfallSelected - 20);
+ return true;
+ }
+ if (ke.isPageDown() || ke.isKey(KeyCode.PAGE_DOWN)) {
+ List<WaterfallNode> nodes = buildWaterfallNodes(selectedTraceId);
+ waterfallSelected = Math.min(nodes.size() - 1, waterfallSelected +
20);
+ return true;
+ }
+ if (ke.isChar('p')) {
+ showProcessors = !showProcessors;
+ waterfallSelected = 0;
+ waterfallScroll = 0;
+ return true;
+ }
+ if (ke.isKey(KeyCode.F5)) {
+ spanRefreshRequested = true;
+ return true;
+ }
+ return false;
+ }
+
+ @Override
+ public boolean handleEscape() {
+ if (filterInputActive) {
+ filterInputActive = false;
+ return true;
+ }
+ if (waterfallView) {
+ waterfallView = false;
+ selectedTraceId = null;
+ return true;
+ }
+ if (filterTerm != null) {
+ filterTerm = null;
+ selectedListTraceId = null;
+ traceListState.select(0);
+ return true;
+ }
+ return false;
+ }
+
+ @Override
+ public boolean setFilter(String filter) {
+ filterTerm = (filter != null && !filter.isEmpty()) ? filter : null;
+ filterInputActive = false;
+ selectedListTraceId = null;
+ traceListState.select(0);
+ return true;
+ }
+
+ @Override
+ public void navigateUp() {
+ if (!waterfallView) {
+ traceListState.selectPrevious();
+ syncSelectedListTraceId();
+ }
+ }
+
+ @Override
+ public void navigateDown() {
+ if (!waterfallView) {
+ traceListState.selectNext(buildFilteredTraceSummaries().size());
+ syncSelectedListTraceId();
+ }
+ }
+
+ private void syncSelectedListTraceId() {
+ Integer sel = traceListState.selected();
+ if (sel != null) {
+ List<TraceSummary> summaries = buildFilteredTraceSummaries();
+ if (sel >= 0 && sel < summaries.size()) {
+ selectedListTraceId = summaries.get(sel).traceId;
+ }
+ }
+ }
+
+ @Override
+ public void render(Frame frame, Rect area) {
+ IntegrationInfo info = ctx.findSelectedIntegration();
+ if (info == null) {
+ MonitorContext.renderNoSelection(frame, area);
+ return;
+ }
+
+ List<SpanEntry> currentSpans = spans.get();
+ if (currentSpans.isEmpty()) {
+ frame.renderWidget(
+ Paragraph.builder()
+ .text(Text.from(Line.from(
+ Span.styled(" No OTel spans captured. Use
--observe flag when running.",
+ Style.EMPTY.dim()))))
+
.block(Block.builder().borderType(BorderType.ROUNDED)
+ .title(" OTel Spans ").build())
+ .build(),
+ area);
+ return;
+ }
+
+ if (waterfallView && selectedTraceId != null) {
+ renderWaterfallView(frame, area);
+ } else {
+ renderTraceList(frame, area);
+ }
+ }
+
+ private List<TraceSummary> buildFilteredTraceSummaries() {
+ List<TraceSummary> all = buildTraceSummaries();
+ if (filterTerm == null) {
+ return all;
+ }
+ String filter = filterTerm.toLowerCase();
+ List<TraceSummary> filtered = new ArrayList<>();
+ for (TraceSummary ts : all) {
+ if (ts.searchText.contains(filter)) {
+ filtered.add(ts);
+ }
+ }
+ return filtered;
+ }
+
+ private void renderTraceList(Frame frame, Rect area) {
+ List<TraceSummary> allSummaries = buildTraceSummaries();
+ List<TraceSummary> summaries = filterTerm != null ?
buildFilteredTraceSummaries() : allSummaries;
+
+ // Restore selection by traceId (survives data refresh/eviction)
+ if (!summaries.isEmpty()) {
+ if (selectedListTraceId != null) {
+ int idx = -1;
+ for (int i = 0; i < summaries.size(); i++) {
+ if (summaries.get(i).traceId.equals(selectedListTraceId)) {
+ idx = i;
+ break;
+ }
+ }
+ if (idx >= 0) {
+ traceListState.select(idx);
+ } else {
+ traceListState.select(0);
+ selectedListTraceId = summaries.get(0).traceId;
+ }
+ } else {
+ traceListState.select(0);
+ selectedListTraceId = summaries.get(0).traceId;
+ }
+ }
+
+ List<Row> rows = new ArrayList<>();
+ for (TraceSummary ts : summaries) {
+ Style statusStyle;
+ if (ts.hasError) {
+ statusStyle = Style.EMPTY.fg(Color.LIGHT_RED);
+ } else {
+ statusStyle = Style.EMPTY.fg(Color.GREEN);
+ }
+
+ rows.add(Row.from(
+ Cell.from(shortId(ts.traceId)),
+ Cell.from(ts.rootRouteId != null ? ts.rootRouteId : ""),
+ Cell.from(ts.rootName != null ? ts.rootName : "?"),
+ MonitorContext.rightCell(String.valueOf(ts.spanCount), 5),
+ MonitorContext.rightCell(String.valueOf(ts.routeCount), 5),
+ Cell.from(ts.remoteComponents.isEmpty() ? "-" :
ts.remoteComponents),
+ Cell.from(Span.styled(ts.hasError ? "ERROR" : "OK",
statusStyle)),
+ Cell.from(ts.totalDurationMs + "ms"),
+ MonitorContext.rightCell(String.valueOf(ts.maxDepth), 5)));
+ }
+
+ String title;
+ if (filterTerm != null) {
+ title = String.format(" OTel Traces — %d/%d traces [%s] ",
+ summaries.size(), allSummaries.size(), filterTerm);
+ } else {
+ title = String.format(" OTel Traces — %d traces, %d spans ",
allSummaries.size(), spans.get().size());
+ }
+ Table table = Table.builder()
+ .rows(rows)
+ .header(Row.from(
+ Cell.from(Span.styled(sortLabel("TRACE-ID",
"trace-id"), sortStyle("trace-id"))),
+ Cell.from(Span.styled(sortLabel("ROUTE", "route"),
sortStyle("route"))),
+ Cell.from(Span.styled(sortLabel("FROM", "from"),
sortStyle("from"))),
+ Cell.from(Span.styled(sortLabel("SPANS", "spans"),
sortStyle("spans"))),
+ Cell.from(Span.styled(sortLabel("ROUTES", "routes"),
sortStyle("routes"))),
+ Cell.from(Span.styled("REMOTE",
Style.EMPTY.fg(Color.YELLOW).bold())),
+ Cell.from(Span.styled(sortLabel("STATUS", "status"),
sortStyle("status"))),
+ Cell.from(Span.styled(sortLabel("DURATION",
"duration"), sortStyle("duration"))),
+ Cell.from(Span.styled("DEPTH",
Style.EMPTY.fg(Color.YELLOW).bold()))))
+ .widths(
+ Constraint.length(10),
+ Constraint.length(20),
+ Constraint.length(20),
+ Constraint.length(7),
+ Constraint.length(8),
+ Constraint.fill(),
+ Constraint.length(8),
+ Constraint.length(12),
+ Constraint.length(7))
+ .highlightStyle(Style.EMPTY.fg(Color.WHITE).bold().onBlue())
+
.block(Block.builder().borderType(BorderType.ROUNDED).title(title).build())
+ .build();
+ frame.renderStatefulWidget(table, area, traceListState);
+ }
+
+ private void renderWaterfallView(Frame frame, Rect area) {
+ List<WaterfallNode> nodes = buildWaterfallNodes(selectedTraceId);
+ if (nodes.isEmpty()) {
+ waterfallView = false;
+ return;
+ }
+
+ long traceStart = Long.MAX_VALUE;
+ long traceEnd = 0;
+ long minDuration = Long.MAX_VALUE;
+ long maxDuration = 0;
+ for (WaterfallNode n : nodes) {
+ traceStart = Math.min(traceStart, n.span.startEpochNanos());
+ traceEnd = Math.max(traceEnd, n.span.endEpochNanos());
+ if (n.span.durationMs() > 0) {
+ minDuration = Math.min(minDuration, n.span.durationMs());
+ maxDuration = Math.max(maxDuration, n.span.durationMs());
+ }
+ }
+ if (minDuration == Long.MAX_VALUE) {
+ minDuration = 0;
+ }
+ long traceDuration = (traceEnd - traceStart) / 1_000_000;
+
+ // Split: waterfall top, detail bottom
+ List<Rect> chunks = Layout.vertical()
+ .constraints(Constraint.fill(), Constraint.length(10))
+ .split(area);
+
+ renderWaterfall(frame, chunks.get(0), nodes, traceStart,
traceDuration, minDuration, maxDuration);
+ renderSpanDetail(frame, chunks.get(1), nodes);
+ }
+
+ private void renderWaterfall(
+ Frame frame, Rect area, List<WaterfallNode> nodes,
+ long traceStart, long traceDuration, long minDuration, long
maxDuration) {
+
+ String title = String.format(" Trace %s — %d spans, %dms ",
+ shortId(selectedTraceId), nodes.size(), traceDuration);
+ Block block = Block.builder()
+ .borderType(BorderType.ROUNDED)
+ .title(title)
+ .build();
+ Rect inner = block.inner(area);
+ frame.renderWidget(block, area);
+
+ if (inner.height() < 1 || inner.width() < 20) {
+ return;
+ }
+
+ int visibleLines = inner.height();
+ int maxScroll = Math.max(0, nodes.size() - visibleLines);
+ waterfallScroll = Math.min(waterfallScroll, maxScroll);
+
+ // Auto-scroll to keep selection visible
+ if (waterfallSelected < waterfallScroll) {
+ waterfallScroll = waterfallSelected;
+ } else if (waterfallSelected >= waterfallScroll + visibleLines) {
+ waterfallScroll = waterfallSelected - visibleLines + 1;
+ }
+
+ int labelWidth = 0;
+ for (WaterfallNode n : nodes) {
+ int indent = n.depth * 2;
+ labelWidth = Math.max(labelWidth, indent +
spanLabel(n.span).length());
+ }
+ labelWidth = Math.min(labelWidth + 2, inner.width() / 3);
+
+ int barMaxWidth = Math.max(10, inner.width() - labelWidth - 12);
+
+ int end = Math.min(waterfallScroll + visibleLines, nodes.size());
+ List<Line> lines = new ArrayList<>();
+ for (int i = waterfallScroll; i < end; i++) {
+ WaterfallNode n = nodes.get(i);
+ boolean selected = i == waterfallSelected;
+ lines.add(renderWaterfallLine(n, labelWidth, barMaxWidth,
+ traceStart, traceDuration, minDuration, maxDuration,
selected));
+ }
+
+ List<Rect> hChunks = Layout.horizontal()
+ .constraints(Constraint.fill(), Constraint.length(1))
+ .split(inner);
+
+ frame.renderWidget(Paragraph.builder().text(Text.from(lines)).build(),
hChunks.get(0));
+
+ if (nodes.size() > visibleLines) {
+ waterfallScrollState
+ .contentLength(nodes.size())
+ .viewportContentLength(visibleLines)
+ .position(waterfallScroll);
+ frame.renderStatefulWidget(Scrollbar.builder().build(),
hChunks.get(1), waterfallScrollState);
+ }
+ }
+
+ private static Line renderWaterfallLine(
+ WaterfallNode node, int labelWidth, int maxBarWidth,
+ long traceStart, long traceDuration, long minDuration, long
maxDuration,
+ boolean selected) {
+ String indicator = selected ? "▸ " : " ";
+ String indent = " ".repeat(node.depth);
+ String label = indent + spanLabel(node.span);
+ if (label.length() > labelWidth) {
+ label = label.substring(0, labelWidth - 1) + "…";
+ } else {
+ label = String.format("%-" + labelWidth + "s", label);
+ }
+
+ // Calculate bar offset and width relative to trace timeline
+ long spanStart = node.span.startEpochNanos() - traceStart;
+ long spanDuration = node.span.endEpochNanos() -
node.span.startEpochNanos();
+
+ double offsetRatio = traceDuration > 0 ? (double) (spanStart /
1_000_000) / traceDuration : 0;
+ double widthRatio = traceDuration > 0 ? (double) (spanDuration /
1_000_000) / traceDuration : 0;
+
+ int barOffset = (int) Math.round(offsetRatio * maxBarWidth);
+ int barWidth = Math.max(1, (int) Math.round(widthRatio * maxBarWidth));
+ barOffset = Math.min(barOffset, maxBarWidth - 1);
+ barWidth = Math.min(barWidth, maxBarWidth - barOffset);
+
+ String gap = " ".repeat(barOffset);
+ String bar = "█".repeat(barWidth);
+
+ String durationStr = node.span.durationMs() + "ms";
+ int pad = Math.max(1, 8 - durationStr.length());
+
+ boolean error = node.span.isError();
+ Style labelStyle;
+ Style bandStyle;
+ if (error) {
+ labelStyle = selected ? Style.EMPTY.fg(Color.LIGHT_RED).bold() :
Style.EMPTY.fg(Color.LIGHT_RED);
+ bandStyle = Style.EMPTY.fg(Color.LIGHT_RED);
+ } else {
+ labelStyle = selected ? Style.EMPTY.fg(Color.CYAN).bold() :
Style.EMPTY.fg(Color.CYAN);
+ bandStyle = TuiHelper.colorForDuration(node.span.durationMs(),
minDuration, maxDuration);
+ }
+
+ String errorTag = error ? " ERR" : "";
+
+ return Line.from(
+ Span.styled(indicator, Style.EMPTY.fg(Color.YELLOW).bold()),
+ Span.styled(label, labelStyle),
+ Span.raw(gap),
+ Span.styled(bar, bandStyle),
+ Span.styled(errorTag, Style.EMPTY.fg(Color.LIGHT_RED).bold()),
+ Span.raw(" ".repeat(pad)),
+ Span.styled(durationStr, error
+ ? Style.EMPTY.fg(Color.LIGHT_RED).bold()
+ : Style.EMPTY.fg(Color.WHITE).bold()));
+ }
+
+ private void renderSpanDetail(Frame frame, Rect area, List<WaterfallNode>
nodes) {
+ if (waterfallSelected < 0 || waterfallSelected >= nodes.size()) {
+ return;
+ }
+ SpanEntry span = nodes.get(waterfallSelected).span;
+
+ List<Line> lines = new ArrayList<>();
+
+ // Row 1: span identity
+ Style statusStyle = span.isError() ?
Style.EMPTY.fg(Color.LIGHT_RED).bold() : Style.EMPTY.fg(Color.GREEN);
+ lines.add(Line.from(
+ Span.styled(" Span: ", Style.EMPTY.dim()),
+ Span.styled(span.spanId(), Style.EMPTY.fg(Color.WHITE).bold()),
+ Span.styled(" Parent: ", Style.EMPTY.dim()),
+ Span.raw(span.parentSpanId() != null ? span.parentSpanId() :
"-"),
+ Span.styled(" Kind: ", Style.EMPTY.dim()),
+ Span.raw(span.kind() != null ? span.kind() : "")));
+
+ // Row 2: status and duration
+ lines.add(Line.from(
+ Span.styled(" Status: ", Style.EMPTY.dim()),
+ Span.styled(span.isError() ? "ERROR" : "OK", statusStyle),
+ Span.styled(" Duration: ", Style.EMPTY.dim()),
+ Span.raw(span.durationMs() + "ms")));
+
+ // Row 3: route and processor context
+ if (span.routeId() != null || span.processorId() != null) {
+ List<Span> ctx = new ArrayList<>();
+ if (span.routeId() != null) {
+ ctx.add(Span.styled(" Route: ", Style.EMPTY.dim()));
+ ctx.add(Span.styled(span.routeId(),
Style.EMPTY.fg(Color.YELLOW)));
+ }
+ if (span.processorId() != null) {
+ ctx.add(Span.styled(" Processor: ", Style.EMPTY.dim()));
+ ctx.add(Span.styled(span.processorId(),
Style.EMPTY.fg(Color.YELLOW)));
+ }
+ lines.add(Line.from(ctx));
+ }
+
+ // Attributes as individual key: value lines
+ if (span.attributes() != null && !span.attributes().isEmpty()) {
+ lines.add(Line.from(Span.raw("")));
+ for (Map.Entry<String, Object> entry :
span.attributes().entrySet()) {
+ lines.add(Line.from(
+ Span.styled(" " + entry.getKey() + ": ",
Style.EMPTY.dim()),
+ Span.raw(String.valueOf(entry.getValue()))));
+ }
+ }
+
+ Style titleStyle = span.isError() ? Style.EMPTY.fg(Color.LIGHT_RED) :
Style.EMPTY.fg(Color.CYAN);
+ frame.renderWidget(
+ Paragraph.builder()
+ .text(Text.from(lines))
+ .block(Block.builder().borderType(BorderType.ROUNDED)
+ .title(dev.tamboui.widgets.block.Title.from(
+ Line.from(Span.styled(" " +
spanLabel(span) + " ", titleStyle))))
+ .build())
+ .build(),
+ area);
+ }
+
+ @Override
+ public void renderFooter(List<Span> spans) {
+ if (waterfallView) {
+ MonitorContext.hint(spans, "Esc", "back");
+ MonitorContext.hint(spans, "F5", "refresh");
+ MonitorContext.hint(spans, "p", showProcessors ? "processors [on]"
: "processors [off]");
+ MonitorContext.hint(spans, "↑↓", "navigate");
+ MonitorContext.hintLast(spans, "PgUp/Dn", "page");
+ } else if (filterInputActive) {
+ spans.add(Span.styled(" /", Style.EMPTY.fg(Color.YELLOW).bold()));
+ spans.add(Span.raw(filterInputState.text() + "█ "));
+ MonitorContext.hint(spans, "Enter", "filter");
+ MonitorContext.hintLast(spans, "Esc", "cancel");
+ } else {
+ MonitorContext.hint(spans, "Esc", filterTerm != null ? "clear" :
"back");
+ MonitorContext.hint(spans, "F5", "refresh");
+ MonitorContext.hint(spans, "Enter", "waterfall");
+ if (filterTerm != null) {
+ spans.add(Span.styled(" /",
Style.EMPTY.fg(Color.YELLOW).bold()));
+ spans.add(Span.raw("\"" + filterTerm + "\" "));
+ } else {
+ MonitorContext.hint(spans, "/", "filter");
+ }
+ MonitorContext.hintLast(spans, "↑↓", "navigate");
+ }
+ }
+
+ private List<TraceSummary> buildTraceSummaries() {
+ List<SpanEntry> currentSpans = spans.get();
+ Map<String, TraceSummary> byTrace = new LinkedHashMap<>();
+
+ for (SpanEntry span : currentSpans) {
+ TraceSummary ts = byTrace.computeIfAbsent(span.traceId(), k -> new
TraceSummary(k));
+ if (span.isRoot()) {
+ ts.rootRouteId = span.routeId();
+ ts.rootName = compactUri(span);
+ }
+ if (span.isError()) {
+ ts.hasError = true;
+ }
+ }
+
+ List<TraceSummary> result = new ArrayList<>(byTrace.values());
+ for (TraceSummary ts : result) {
+ List<SpanEntry> traceSpans = currentSpans.stream()
+ .filter(s -> s.traceId().equals(ts.traceId))
+ .toList();
+ // Fallback root: use the earliest span
+ if (ts.rootName == null && !traceSpans.isEmpty()) {
+ SpanEntry earliest = traceSpans.stream()
+
.min(Comparator.comparingLong(SpanEntry::startEpochNanos))
+ .orElse(null);
+ if (earliest != null) {
+ ts.rootName = compactUri(earliest);
+ if (ts.rootRouteId == null) {
+ ts.rootRouteId = earliest.routeId();
+ }
+ }
+ }
+ // Compute trace envelope duration (same as waterfall view)
+ long traceStart = Long.MAX_VALUE;
+ long traceEnd = 0;
+ Set<String> routes = new HashSet<>();
+ Set<String> exchangeIds = new HashSet<>();
+ Set<String> remoteSchemes = new LinkedHashSet<>();
+ for (SpanEntry s : traceSpans) {
+ traceStart = Math.min(traceStart, s.startEpochNanos());
+ traceEnd = Math.max(traceEnd, s.endEpochNanos());
+ if (s.routeId() != null) {
+ routes.add(s.routeId());
+ }
+ if (s.attributes() != null) {
+ Object eid = s.attributes().get("exchangeId");
+ if (eid != null) {
+ exchangeIds.add(eid.toString());
+ }
+ Object scheme = s.attributes().get("url.scheme");
+ if (scheme != null && isRemoteScheme(scheme.toString())) {
+ remoteSchemes.add(scheme.toString());
+ }
+ }
+ }
+ ts.totalDurationMs = traceStart < Long.MAX_VALUE ? (traceEnd -
traceStart) / 1_000_000 : 0;
+ ts.routeCount = routes.size();
+ ts.remoteComponents = remoteSchemes.isEmpty() ? "" :
String.join(",", remoteSchemes);
+ // Build search text for filtering (traceId, exchangeIds, routes,
remote components)
+ StringBuilder sb = new StringBuilder();
+ sb.append(ts.traceId).append(' ');
+ exchangeIds.forEach(e -> sb.append(e).append(' '));
+ routes.forEach(r -> sb.append(r).append(' '));
+ if (!ts.remoteComponents.isEmpty()) {
+ sb.append(ts.remoteComponents);
+ }
+ ts.searchText = sb.toString().toLowerCase();
+ // Count effective spans (exclude EVENT_PROCESS which are
collapsed in waterfall)
+ for (SpanEntry s : traceSpans) {
+ if (!isEventProcess(s)) {
+ ts.spanCount++;
+ }
+ }
+ ts.maxDepth = computeEffectiveDepth(traceSpans);
+ }
+ result.sort((a, b) -> {
+ int cmp = sortTrace(a, b, currentSpans);
+ return sortReversed ? -cmp : cmp;
+ });
+
+ return result;
+ }
+
+ private List<WaterfallNode> buildWaterfallNodes(String traceId) {
+ List<SpanEntry> traceSpans = spans.get().stream()
+ .filter(s -> traceId.equals(s.traceId()))
+ .sorted(Comparator.comparingLong(SpanEntry::startEpochNanos))
+ .toList();
+
+ if (traceSpans.isEmpty()) {
+ return List.of();
+ }
+
+ // Build parent-child tree
+ Map<String, List<SpanEntry>> childrenMap = new LinkedHashMap<>();
+ SpanEntry root = null;
+ for (SpanEntry span : traceSpans) {
+ if (span.isRoot()) {
+ root = span;
+ }
+ String parentId = span.parentSpanId();
+ if (parentId != null && !parentId.isEmpty()) {
+ childrenMap.computeIfAbsent(parentId, k -> new
ArrayList<>()).add(span);
+ }
+ }
+
+ if (root == null && !traceSpans.isEmpty()) {
+ root = traceSpans.get(0);
+ }
+
+ Set<String> included = new HashSet<>();
+ Map<String, Integer> spanIdToDepth = new LinkedHashMap<>();
+ List<WaterfallNode> result = new ArrayList<>();
+ addToWaterfall(result, root, childrenMap, 0, included, spanIdToDepth);
+
+ // Add orphan spans whose parent isn't reachable from the root.
+ // Try to nest them under their parent if the parent is in the result
+ // or was collapsed (spanIdToDepth tracks both shown and collapsed
spans).
+ boolean changed = true;
+ while (changed) {
+ changed = false;
+ for (SpanEntry span : traceSpans) {
+ if (included.contains(span.spanId())) {
+ continue;
+ }
+ int depth = 0;
+ if (span.parentSpanId() != null &&
spanIdToDepth.containsKey(span.parentSpanId())) {
+ depth = spanIdToDepth.get(span.parentSpanId()) + 1;
+ }
+ result.add(new WaterfallNode(span, depth));
+ included.add(span.spanId());
+ spanIdToDepth.put(span.spanId(), depth);
+ changed = true;
+ }
+ }
+ return result;
+ }
+
+ private void addToWaterfall(
+ List<WaterfallNode> result, SpanEntry span,
+ Map<String, List<SpanEntry>> childrenMap, int depth,
+ Set<String> included, Map<String, Integer> spanIdToDepth) {
+ if (!included.add(span.spanId())) {
+ return;
+ }
+ // Record depth for every visited span (even collapsed ones)
+ // so orphans can find their parent's depth
+ spanIdToDepth.put(span.spanId(), depth);
+
+ List<SpanEntry> children = childrenMap.get(span.spanId());
+ // Collapse internal producer+consumer pairs:
+ // When an EVENT_SENT span has a single EVENT_RECEIVED child with the
same name,
+ // skip the producer and show only the consumer (the route execution).
+ // Never collapse if the skipped span has an error — the error would
be hidden.
+ if (isEventSent(span) && !span.isError() && children != null &&
children.size() == 1
+ && isEventReceived(children.get(0))
+ && span.name().equals(children.get(0).name())) {
+ addToWaterfall(result, children.get(0), childrenMap, depth,
included, spanIdToDepth);
+ return;
+ }
+ // Collapse processor+send pairs:
+ // When an EVENT_PROCESS span (e.g. to4-to) has a single EVENT_SENT
child,
+ // skip the processor wrapper and show only the send span.
+ if (isEventProcess(span) && !span.isError() && children != null &&
children.size() == 1
+ && isEventSent(children.get(0))) {
+ addToWaterfall(result, children.get(0), childrenMap, depth,
included, spanIdToDepth);
+ return;
+ }
+ // Hide processor spans when toggle is off — promote children to same
depth
+ // Keep error processors visible so errors aren't hidden
+ if (!showProcessors && !span.isError() && isEventProcess(span)) {
+ if (children != null) {
+ for (SpanEntry child : children) {
+ addToWaterfall(result, child, childrenMap, depth,
included, spanIdToDepth);
+ }
+ }
+ return;
+ }
+ result.add(new WaterfallNode(span, depth));
+ if (children != null) {
+ for (SpanEntry child : children) {
+ addToWaterfall(result, child, childrenMap, depth + 1,
included, spanIdToDepth);
+ }
+ }
+ }
+
+ private static boolean isEventSent(SpanEntry span) {
+ return span.attributes() != null &&
"EVENT_SENT".equals(span.attributes().get("op"));
+ }
+
+ private static boolean isEventReceived(SpanEntry span) {
+ return span.attributes() != null &&
"EVENT_RECEIVED".equals(span.attributes().get("op"));
+ }
+
+ private static boolean isEventProcess(SpanEntry span) {
+ return span.attributes() != null &&
"EVENT_PROCESS".equals(span.attributes().get("op"));
+ }
+
+ private static String spanLabel(SpanEntry span) {
+ String name = span.name();
+ Map<String, Object> attrs = span.attributes();
+ if (attrs != null) {
+ // Use camel.uri for endpoint spans (direct, stub, timer, etc.)
+ Object uri = attrs.get("camel.uri");
+ if (uri != null) {
+ String label = uri.toString();
+ if (span.routeId() != null) {
+ label += " (" + span.routeId() + ")";
+ }
+ return label;
+ }
+ }
+ // Processor spans: use processorId with routeId context
+ if (span.processorId() != null) {
+ String label = span.processorId();
+ if (span.routeId() != null) {
+ label += " (" + span.routeId() + ")";
+ }
+ return label;
+ }
+ return name;
+ }
+
+ private int sortTrace(TraceSummary a, TraceSummary b, List<SpanEntry>
currentSpans) {
+ return switch (sortColumn) {
+ case "route" -> compareNullable(a.rootRouteId, b.rootRouteId);
+ case "from" -> compareNullable(a.rootName, b.rootName);
+ case "duration" -> Long.compare(b.totalDurationMs,
a.totalDurationMs);
+ case "spans" -> Integer.compare(b.spanCount, a.spanCount);
+ case "routes" -> Integer.compare(b.routeCount, a.routeCount);
+ case "status" -> {
+ int as = a.hasError ? 1 : 0;
+ int bs = b.hasError ? 1 : 0;
+ yield Integer.compare(bs, as);
+ }
+ default -> {
+ // newest first
+ long at = currentSpans.stream()
+ .filter(s -> s.traceId().equals(a.traceId))
+ .mapToLong(SpanEntry::startEpochNanos).max().orElse(0);
+ long bt = currentSpans.stream()
+ .filter(s -> s.traceId().equals(b.traceId))
+ .mapToLong(SpanEntry::startEpochNanos).max().orElse(0);
+ yield Long.compare(bt, at);
+ }
+ };
+ }
+
+ private String compactUri(SpanEntry span) {
+ if (span.attributes() != null) {
+ Object uri = span.attributes().get("camel.uri");
+ if (uri != null) {
+ String s = uri.toString();
+ // strip scheme:// prefix to just scheme:path
+ s = s.replace("://", ":");
+ // strip query parameters
+ int q = s.indexOf('?');
+ if (q > 0) {
+ s = s.substring(0, q);
+ }
+ return s;
+ }
+ }
+ return span.name();
+ }
+
+ private static boolean isRemoteScheme(String scheme) {
+ return scheme != null
+ && !"direct".equals(scheme) && !"seda".equals(scheme)
+ && !"mock".equals(scheme) && !"log".equals(scheme)
+ && !"bean".equals(scheme) && !"class".equals(scheme);
+ }
+
+ private static int compareNullable(String a, String b) {
+ if (a == null && b == null) {
+ return 0;
+ }
+ if (a == null) {
+ return 1;
+ }
+ if (b == null) {
+ return -1;
+ }
+ return a.compareToIgnoreCase(b);
+ }
+
+ private String sortLabel(String label, String column) {
+ return MonitorContext.sortLabel(label, column, sortColumn,
sortReversed);
+ }
+
+ private Style sortStyle(String column) {
+ return Style.EMPTY.fg(Color.YELLOW).bold();
+ }
+
+ private static int computeEffectiveDepth(List<SpanEntry> traceSpans) {
+ // Build parent-child tree excluding EVENT_PROCESS spans
+ Map<String, String> parentMap = new HashMap<>();
+ Set<String> nonProcessIds = new HashSet<>();
+ for (SpanEntry s : traceSpans) {
+ if (!isEventProcess(s)) {
+ nonProcessIds.add(s.spanId());
+ // Walk up to find nearest non-process parent
+ String parentId = s.parentSpanId();
+ while (parentId != null && !nonProcessIds.contains(parentId)) {
+ String nextParent = null;
+ for (SpanEntry p : traceSpans) {
+ if (p.spanId().equals(parentId)) {
+ nextParent = p.parentSpanId();
+ break;
+ }
+ }
+ parentId = nextParent;
+ }
+ if (parentId != null) {
+ parentMap.put(s.spanId(), parentId);
+ }
+ }
+ }
+ // Compute max depth from the effective tree
+ int maxDepth = 0;
+ for (String id : nonProcessIds) {
+ int depth = 0;
+ String cur = id;
+ while (parentMap.containsKey(cur)) {
+ depth++;
+ cur = parentMap.get(cur);
+ }
+ maxDepth = Math.max(maxDepth, depth);
+ }
+ return maxDepth + 1;
+ }
+
+ private static String shortId(String id) {
+ if (id == null || id.isEmpty()) {
+ return "";
+ }
+ return id.length() > 8 ? id.substring(0, 8) : id;
+ }
+
+ @Override
+ public String getHelpText() {
+ return """
+ # OTel Spans
+
+ The Spans tab shows OpenTelemetry traces captured from the
running
+ integration. Requires the `--observe` flag when starting the
integration.
+
+ ## Trace List
+
+ The main view shows a table of traces with:
+
+ - **TRACE-ID** — Short 8-character trace identifier
+ - **ROUTE** — The root route that started the trace
+ - **FROM** — The entry endpoint URI (e.g., timer:orders)
+ - **SPANS** — Total number of spans in the trace
+ - **ROUTES** — Number of distinct routes touched
+ - **REMOTE** — External components used (kafka, http, sql,
etc.)
+ - **STATUS** — OK or ERROR
+ - **DURATION** — Total trace duration (wall-clock envelope)
+ - **DEPTH** — Maximum nesting depth of the span tree
+
+ ## Waterfall View
+
+ Press **Enter** on a trace to see the Jaeger-style waterfall
showing
+ the span tree with proportional duration bars. Each span shows
its
+ endpoint URI, processor ID, route context in parentheses, and
duration.
+
+ Indentation shows the parent-child relationship between spans.
Duration
+ bars are proportional to the trace envelope so you can
visually spot
+ where time is spent. Colors indicate relative duration: green
(fast),
+ yellow (medium), red (slow).
+
+ Processor spans (setBody, log, etc.) are shown by default.
Press **p**
+ to toggle them off for a cleaner view focused on
endpoint-to-endpoint
+ flow. Error spans are always shown regardless of the toggle.
+
+ Press **Esc** to return to the trace list.
+
+ Example waterfall for an order-processing integration:
+
+ ```
+ Trace 4bb73039 — 15 spans, 4ms
+ ▸ timer://orders (order-generator)
█████████████████████████████████ 2ms
+ setBody1 (order-generator) █ 0ms
+ direct://process-order (process) ████████████████ 1ms
+ direct://validate-order (validate) █ 0ms
+ log2 (validate-order) █ 0ms
+ log1 (process-order) █ 0ms
+ kafka://orders (order-dispatcher)
████████████████ 1ms
+ log3 (order-dispatcher) █ 0ms
+ multicast1 (order-dispatcher) █ 0ms
+ kafka://fulfillment (fulfill)
█ 0ms
+ log4 (fulfillment)
█ 0ms
+ kafka://warehouse (fulfill)
█ 0ms
+ kafka://notifications (notif)
█ 0ms
+ log5 (notification)
█ 0ms
+ kafka://email-outbox (notif)
█ 0ms
+ ```
+
+ ## Keyboard Shortcuts
+
+ | Key | Action |
+ |-----|--------|
+ | Enter | Drill into trace waterfall |
+ | Esc | Back to list / clear filter |
+ | / | Open filter input (matches trace ID, exchange ID, route,
component) |
+ | s | Cycle sort column (trace-id, route, from, spans, routes,
status, duration) |
+ | S | Reverse sort direction |
+ | p | Toggle processor spans in waterfall |
+ | F5 | Refresh span data |
+
+ ## Filtering
+
+ Press `/` to open the filter input. Type a search term and
press Enter.
+ Matches against trace ID, exchange ID, route names, and remote
component
+ names. For example, type `kafka` to find traces that use
Kafka, or paste
+ an exchange ID from a log line to find its trace.
+ """;
+ }
+
+ @Override
+ public JsonObject getTableDataAsJson() {
+ List<TraceSummary> summaries = buildTraceSummaries();
+ JsonObject result = new JsonObject();
+ result.put("tab", "Spans");
+ JsonArray rows = new JsonArray();
+ for (TraceSummary ts : summaries) {
+ JsonObject row = new JsonObject();
+ row.put("traceId", ts.traceId);
+ row.put("route", ts.rootRouteId);
+ row.put("from", ts.rootName);
+ row.put("spans", ts.spanCount);
+ row.put("routes", ts.routeCount);
+ row.put("remote", ts.remoteComponents);
+ row.put("status", ts.hasError ? "ERROR" : "OK");
+ row.put("duration", ts.totalDurationMs);
+ row.put("depth", ts.maxDepth);
+ rows.add(row);
+ }
+ result.put("rows", rows);
+ result.put("totalRows", summaries.size());
+ Integer sel = traceListState.selected();
+ result.put("selectedIndex", sel != null ? sel : -1);
+ return result;
+ }
+
+ static class TraceSummary {
+ final String traceId;
+ String rootRouteId;
+ String rootName;
+ int spanCount;
+ long totalDurationMs;
+ boolean hasError;
+ int maxDepth;
+ int routeCount;
+ String remoteComponents;
+ String searchText;
+
+ TraceSummary(String traceId) {
+ this.traceId = traceId;
+ }
+ }
+
+ record WaterfallNode(SpanEntry span, int depth) {
+ }
+}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiHelper.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiHelper.java
index c8ab629ca13c..3fe39d4bb64c 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiHelper.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiHelper.java
@@ -388,7 +388,7 @@ final class TuiHelper {
Style.EMPTY.fg(Color.LIGHT_GREEN),
Style.EMPTY.fg(Color.YELLOW),
Style.EMPTY.fg(Color.rgb(0xFF, 0xA5, 0x00)),
- Style.EMPTY.fg(Color.RED),
+ Style.EMPTY.fg(Color.MAGENTA),
};
static Style colorForDuration(long duration, long minDuration, long
maxDuration) {
@@ -397,7 +397,7 @@ final class TuiHelper {
}
double ratio = (Math.log1p(duration) - Math.log1p(minDuration))
/ (Math.log1p(maxDuration) - Math.log1p(minDuration));
- int bandIndex = Math.min((int) (ratio * 5), 4);
+ int bandIndex = Math.max(0, Math.min((int) (ratio * 5), 4));
return DURATION_BAND_STYLES[bandIndex];
}
diff --git
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiMcpServer.java
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiMcpServer.java
index 612be553db7e..66d85ad5f26d 100644
---
a/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiMcpServer.java
+++
b/dsl/camel-jbang/camel-jbang-plugin-tui/src/main/java/org/apache/camel/dsl/jbang/core/commands/tui/TuiMcpServer.java
@@ -262,7 +262,8 @@ class TuiMcpServer {
"tui_navigate",
"Navigates the TUI: switch tabs and/or select an integration. "
+ "All parameters are optional — set whichever
you want to change. "
- + "Tab names: Overview, Log, Diagram, Routes,
Consumers, Endpoints, HTTP, Health, Inspect, Circuit Breaker. "
+ + "Tab names: Overview, Log, Diagram, Routes,
Endpoints, HTTP, Health, Inspect, "
+ + "Circuit Breaker. "
+ "Use 'route' to select a route in the
Diagram topology, "
+ "and 'node' to drill down into a route and
select a specific processor/EIP node. "
+ "Returns screen content and selection
metadata after navigating.",
@@ -498,6 +499,18 @@ class TuiMcpServer {
"Integration name. If omitted, uses the currently
selected integration."),
"file", propDef("string",
"Filename to read. If omitted, returns the
file list instead."))));
+ toolList.add(toolDef(
+ "tui_get_spans",
+ "Returns raw OpenTelemetry span data as structured JSON from
the selected integration. "
+ + "Each span includes: traceId, spanId,
parentSpanId, name, kind, status, "
+ + "startEpochNanos, endEpochNanos,
durationMs, routeId, processorId, and attributes. "
+ + "Use traceId to filter spans for a specific
trace. "
+ + "The parentSpanId chain shows the span
hierarchy for building waterfall views.",
+ Map.of("traceId", propDef("string",
+ "Filter to spans matching this trace ID (substring
match). "
+ + "If omitted, returns all
recent spans."),
+ "limit", propDef("integer",
+ "Maximum number of spans to return (default
500)"))));
toolList.add(toolDef(
"tui_locate",
"Locates elements on the TUI screen and returns their exact
screen coordinates (x, y, width, height). "
@@ -558,6 +571,7 @@ class TuiMcpServer {
case "tui_get_readme" -> callGetReadme(args);
case "tui_control" -> callControl(args);
case "tui_get_files" -> callGetFiles(args);
+ case "tui_get_spans" -> callGetSpans(args);
case "tui_locate" -> callLocate(args);
case "tui_draw_shape" -> callDrawShape(args);
default -> {
@@ -1148,6 +1162,16 @@ class TuiMcpServer {
return Jsoner.serialize(data);
}
+ private String callGetSpans(Map<String, Object> args) {
+ String traceId = args.get("traceId") instanceof String s ? s : null;
+ int limit = 500;
+ if (args.get("limit") instanceof Number n) {
+ limit = n.intValue();
+ }
+ JsonObject data = monitor.getSpanData(traceId, limit);
+ return Jsoner.serialize(data);
+ }
+
private String callSendMessage(Map<String, Object> args) {
String endpoint = (String) args.get("endpoint");
if (endpoint == null || endpoint.isBlank()) {
