This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-website.git
The following commit(s) were added to refs/heads/main by this push:
new 35688fd8 Blog: GenAI observability two-part series (JBang + Spring
Boot) (#1734)
35688fd8 is described below
commit 35688fd84c48c1078f89e820e5e804f6396fdc11
Author: Omar Atie <[email protected]>
AuthorDate: Mon Aug 31 13:06:14 2026 -0700
Blog: GenAI observability two-part series (JBang + Spring Boot) (#1734)
* Blog: GenAI observability two-part series (JBang + Spring Boot)
Publish camel.apache.org blog posts for Camel 4.23 GenAI observability:
JBang/TUI quick start and Spring Boot with Prometheus/VictoriaTraces.
Co-authored-by: Cursor <[email protected]>
* Address review feedback and fix broken observability-services links
- Use Camel CLI terminology instead of JBang in prose
- Point examples to camel-jbang-examples and camel-spring-boot-examples PRs
- Fix Observability Services link (/components/next/others/...)
- Use camel.aiObservability.enabled (correct property name)
- Mark posts as draft until publication date is set
- Remove inline Claus thanks; add camel infra, export, and Java route
content
- Document CLI dependency auto-discovery as future improvement
Co-authored-by: Cursor Agent <[email protected]>
* Publish GenAI observability blog posts with TUI Spans screenshot
- Set draft: false and publication date 2026-08-30
- Add TUI Spans screenshot showing gen_ai OpenTelemetry attributes
- Fix route example to resolve property placeholders in Java builder
- Note pending JIRA for CLI dependency auto-discovery
Co-authored-by: Cursor Agent <[email protected]>
* Set GenAI observability blog publication date to 2026-09-01
Move both posts from 2026/08/ to 2026/09/ and update cross-links.
Co-authored-by: Cursor <[email protected]>
* Move TUI spans screenshot to 2026/09 blog folder and fix broken CLI link
- Move camel-genai-observability-spans.png alongside the September blog post
- Fix broken /manual/camel-jbang-jdk-installation.html link (404) to
/manual/camel-jbang.html
Co-authored-by: Cursor <[email protected]>
* Update GenAI observability blogs for reworked Spring Boot example
Align Part 2 with camel-spring-boot-examples#192: camel infra run
observability, management port 9876, Micrometer Tracing OTLP export,
two-model YAML routes, Perses GenAI dashboard screenshots, and
camel.aiobservability.enabled property name.
Co-authored-by: Cursor <[email protected]>
---------
Co-authored-by: Omar Atie <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
---
.../camel-genai-observability-spans.png | Bin 0 -> 172299 bytes
.../09/camel-genai-observability-jbang/index.md | 258 ++++++++++++++++
.../genai-dashboard-calls.png | Bin 0 -> 237089 bytes
.../genai-dashboard-tokens.png | Bin 0 -> 94387 bytes
.../camel-genai-observability-spring-boot/index.md | 332 +++++++++++++++++++++
5 files changed, 590 insertions(+)
diff --git
a/content/blog/2026/09/camel-genai-observability-jbang/camel-genai-observability-spans.png
b/content/blog/2026/09/camel-genai-observability-jbang/camel-genai-observability-spans.png
new file mode 100644
index 00000000..352b6e00
Binary files /dev/null and
b/content/blog/2026/09/camel-genai-observability-jbang/camel-genai-observability-spans.png
differ
diff --git a/content/blog/2026/09/camel-genai-observability-jbang/index.md
b/content/blog/2026/09/camel-genai-observability-jbang/index.md
new file mode 100644
index 00000000..ed737bf2
--- /dev/null
+++ b/content/blog/2026/09/camel-genai-observability-jbang/index.md
@@ -0,0 +1,258 @@
+---
+title: "Observe Your Camel AI Routes with GenAI OpenTelemetry"
+date: 2026-09-01
+draft: false
+authors: [atiaomar1978-hub]
+categories: ["AI", "Howtos"]
+keywords: ["apache camel", "genai", "observability", "opentelemetry",
"ollama", "langchain4j", "camel cli", "camel tui", "camel 4.23", "llm"]
+preview: "Apache Camel 4.23 adds GenAI observability — OpenTelemetry spans and
Micrometer metrics for LLM routes. Prototype it in minutes with the Camel CLI,
Ollama, and the Camel TUI."
+---
+
+Once LLMs live inside Camel routes, the next question is always the same: *how
much are we spending, and
+where is latency coming from?* Apache Camel **4.23** introduces **GenAI
observability** — OpenTelemetry spans
+and Micrometer metrics for LLM producers, aligned with the
+[OpenTelemetry GenAI semantic
conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/).
+
+This is **Blog 1** in a two-part series. We start with the fastest path to
visible AI telemetry: the
+[Camel CLI](/manual/camel-jbang.html), a LangChain4j chat route,
[Ollama](https://ollama.com/),
+and the Camel TUI — no Spring Boot required.
+
+## The three phases of Camel Gen AI
+
+| Phase | Name | What you do |
+|-------|------|-------------|
+| 1 | **Connect** | Wire LLMs into routes (`langchain4j-chat`, `openai`,
Spring AI) |
+| 2 | **Act** | Give AI tools via `ai-tool:`, agents, MCP server, A2A |
+| 3 | **Operate** | Run in production with guardrails, RAG, and **GenAI
observability** |
+
+This post covers Phase 1 plus the first observability prototype.
+[Part 2](/blog/2026/09/camel-genai-observability-spring-boot/) moves to Spring
Boot with
+`camel infra run observability`, per-model Perses dashboards, and
VictoriaTraces.
+
+## What you'll build
+
+A timer-driven route that calls Ollama every 15 seconds. For each LLM
invocation you get:
+
+- Micrometer timer: `gen_ai.client.operation`
+- Micrometer counter: `gen_ai.client.token.usage` (tags `input` / `output`)
+- OpenTelemetry child spans with `gen_ai.request.model`, token counts, finish
reason
+- Exchange headers: `CamelLangChain4jChatRequestModel`,
`CamelLangChain4jChatResponseModel`
+- TUI **Spans** tab visualization + AI Usage view (**Ctrl+U**)
+
+```
+timer:genai (every 15s)
+ └─ langchain4j-chat (Ollama ChatModel)
+ ├─ camel-ai-observability-api → start GenAI span + record metrics
+ ├─ camel-opentelemetry2 → export span attributes
+ └─ camel-micrometer → gen_ai.client.* metrics
+
+camel run --observe
+ └─ camel-observability-services
+ ├─ /observe/health
+ ├─ /observe/metrics (Prometheus format)
+ └─ TUI Spans collector (embedded OTLP for dev)
+
+camel tui
+ ├─ Spans tab (shortcut: o)
+ └─ AI panel + Ctrl+U (CLI ask + route GenAI usage combined)
+```
+
+## Prerequisites
+
+- Camel CLI **4.23+** (`camel version`)
+- Ollama installed and running
+
+```shell
+ollama pull llama3.2
+ollama serve
+```
+
+## Run the example
+
+The example lives in the
[camel-jbang-examples](https://github.com/apache/camel-jbang-examples)
repository at
+[`ai/genai-observability`](https://github.com/apache/camel-jbang-examples/tree/main/ai/genai-observability).
+It is registered in the CLI example catalog so you can also browse and launch
it from the
+[Camel TUI](/manual/camel-jbang-tui.html) example browser.
+
+Clone the example:
+
+```shell
+git clone https://github.com/apache/camel-jbang-examples.git
+cd camel-jbang-examples/ai/genai-observability
+```
+
+Run with observability enabled:
+
+```shell
+camel run GenAiObservabilityRoute.java application.properties \
+ --observe \
+ --dependency=camel-langchain4j-chat \
+ --dependency=camel-ai-observability \
+ --dependency=langchain4j-ollama
+```
+
+> **Note:** Today you must pass `--dependency` for LangChain4j and GenAI
observability components.
+> A JIRA ticket will track auto-discovery of these dependencies in a future
Camel CLI release.
+
+The `--observe` flag adds `camel-observability-services`, enables health
checks, Micrometer metrics,
+OpenTelemetry tracing, and powers the TUI **Spans** tab on the management port
(default `9876`).
+
+Alternatively, start the bundled observability stack with
+[`camel infra run
observability`](/blog/2026/08/camel422-whatsnew/#observability-stack) and run
the route
+with `--observe` — metrics and traces flow to Prometheus and VictoriaTraces
without extra setup.
+
+Wait for log lines like:
+
+```text
+LLM reply: Apache Camel is an open source integration framework...
+Request model: llama3.2
+Response model: llama3.2
+```
+
+## Inspect Prometheus metrics
+
+```shell
+curl -s http://127.0.0.1:9876/observe/metrics | grep gen_ai
+```
+
+Example output (abbreviated):
+
+```text
+# HELP gen_ai_client_operation GenAI client operation duration
+gen_ai_client_operation_count{gen_ai_operation_name="chat",gen_ai_system="langchain4j",...}
3.0
+
+# HELP gen_ai_client_token_usage GenAI token usage
+gen_ai_client_token_usage_total{gen_ai_token_type="input",...} 42.0
+gen_ai_client_token_usage_total{gen_ai_token_type="output",...} 18.0
+```
+
+## Explore spans in the TUI
+
+Open a second terminal:
+
+```shell
+camel tui
+```
+
+1. Select the running integration (`genai-observability` or similar)
+2. Press **o** or navigate to **More → Spans**
+3. Wait for the next timer tick (~15s) and watch a new span appear
+
+
+
+Each LLM call creates a child span with `gen_ai.operation.name=chat`, model
attributes, and token usage.
+See the [OpenTelemetry Spans
section](/manual/camel-jbang-tui.html#_opentelemetry_spans) in the TUI manual
+for a walkthrough of the Spans tab layout.
+
+Key span attributes:
+
+| Attribute | Meaning |
+|-----------|---------|
+| `gen_ai.operation.name` | e.g. `chat`, `embeddings` |
+| `gen_ai.system` | Provider abstraction (e.g. `langchain4j`) |
+| `gen_ai.request.model` | Model requested |
+| `gen_ai.response.model` | Model that served the response |
+| `gen_ai.usage.input_tokens` | Prompt tokens |
+| `gen_ai.usage.output_tokens` | Completion tokens |
+| `camel.component` | e.g. `langchain4j-chat` |
+
+Press **Ctrl+U** in the AI panel to toggle the **AI Usage** view — token
consumption from both
+TUI `camel ask` prompts and route LLM calls on one screen.
+
+## Example route
+
+The example uses Java today. Camel is also on a mission to make LLM routes
approachable in YAML DSL and
+Kamelets for non-Java developers — expect GenAI observability to work the same
way once those DSLs support
+the same components.
+
+```java
+import dev.langchain4j.model.chat.ChatModel;
+import dev.langchain4j.model.ollama.OllamaChatModel;
+import org.apache.camel.builder.RouteBuilder;
+
+import static java.time.Duration.ofSeconds;
+
+public class GenAiObservabilityRoute extends RouteBuilder {
+
+ @Override
+ public void configure() throws Exception {
+ String baseUrl =
getContext().resolvePropertyPlaceholders("{{ollama.baseUrl:http://localhost:11434}}";);
+ String modelName =
getContext().resolvePropertyPlaceholders("{{ollama.model:llama3.2}}");
+ ChatModel chatModel = OllamaChatModel.builder()
+ .baseUrl(baseUrl)
+ .modelName(modelName)
+ .temperature(0.2)
+ .timeout(ofSeconds(120))
+ .build();
+ getContext().getRegistry().bind("chatModel", chatModel);
+
+ from("timer:genai?period={{genai.period:15000}}")
+ .routeId("genai-chat")
+ .setBody(constant("In one sentence, what is Apache Camel
integration?"))
+ .to("langchain4j-chat:demo?chatModel=#chatModel")
+ .log("LLM reply: ${body}")
+ .log("Request model:
${header.CamelLangChain4jChatRequestModel}")
+ .log("Response model:
${header.CamelLangChain4jChatResponseModel}");
+ }
+}
+```
+
+### application.properties
+
+```properties
+ollama.baseUrl=http://localhost:11434
+ollama.model=llama3.2
+genai.period=15000
+
+# GenAI observability — enabled by default when backends present
+camel.aiobservability.enabled=true
+```
+
+GenAI observability also covers `openai:` — run with
`--dependency=camel-openai` and the same
+`--observe` flag; spans use the same `gen_ai.*` attributes.
+
+## What's instrumented today
+
+| Component | Operations observed |
+|-----------|---------------------|
+| `langchain4j-chat` | Chat completions |
+| `langchain4j-tools` | Tool-augmented LLM calls |
+| `langchain4j-agent` | AI Service agent loops |
+| `langchain4j-embeddings` | Embedding generation |
+| `openai` | Chat, embeddings, Responses API, etc. |
+
+## Troubleshooting
+
+| Symptom | Fix |
+|---------|-----|
+| No `gen_ai` metrics | Confirm `--observe` or manual observability config;
check `camel-ai-observability` on classpath |
+| Empty Spans tab | Wait for at least one LLM call; verify
`camel.opentelemetry2.enabled=true` |
+| Ollama connection refused | Run `ollama serve`; check `ollama.baseUrl` |
+| Metrics port unreachable | Default management port is `9876`; look for
`Management service available` in logs |
+
+## Next up — Spring Boot and the observability stack
+
+[Part 2](/blog/2026/09/camel-genai-observability-spring-boot/) wires the same
`gen_ai.*` signals into
+Spring Boot with the shared observability stack, per-model Perses dashboards,
and VictoriaTraces —
+the pattern most teams use in production.
+
+When you are ready to move from CLI prototyping to Spring Boot, export the
route with:
+
+```shell
+camel export GenAiObservabilityRoute.java --runtime spring-boot --dir
./genai-sb
+```
+
+Then continue in the generated Maven project (see [Part
2](/blog/2026/09/camel-genai-observability-spring-boot/)).
+
+## Learn more
+
+- [AI Observability component
source](https://github.com/apache/camel/blob/main/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc)
(4.23+)
+- [Camel TUI manual](/manual/camel-jbang-tui.html)
+- [`camel ask` command](/manual/jbang-commands/camel-jbang-ask.html)
+- [Camel AI components](/components/next/ai-summary.html)
+- [Observability Services](/components/next/others/observability-services.html)
+- [JBang example](https://github.com/apache/camel-jbang-examples/pull/73)
+
+---
+
+*This post was written by Omar Atie
([@atiaomar1978-hub](https://github.com/atiaomar1978-hub)) with assistance from
Cursor Cloud Agent.*
diff --git
a/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-calls.png
b/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-calls.png
new file mode 100644
index 00000000..98569acb
Binary files /dev/null and
b/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-calls.png
differ
diff --git
a/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-tokens.png
b/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-tokens.png
new file mode 100644
index 00000000..89bc1cf6
Binary files /dev/null and
b/content/blog/2026/09/camel-genai-observability-spring-boot/genai-dashboard-tokens.png
differ
diff --git
a/content/blog/2026/09/camel-genai-observability-spring-boot/index.md
b/content/blog/2026/09/camel-genai-observability-spring-boot/index.md
new file mode 100644
index 00000000..aa264167
--- /dev/null
+++ b/content/blog/2026/09/camel-genai-observability-spring-boot/index.md
@@ -0,0 +1,332 @@
+---
+title: "GenAI Observability with Spring Boot and the Camel Observability Stack"
+date: 2026-09-01
+draft: false
+authors: [atiaomar1978-hub]
+categories: ["AI", "Howtos"]
+keywords: ["apache camel", "genai", "observability", "spring boot",
"prometheus", "opentelemetry", "victoriatraces", "perses", "ollama",
"langchain4j", "camel 4.23"]
+preview: "Take Camel GenAI observability from CLI/TUI prototypes to production
— Spring Boot, camel infra run observability, per-model Perses dashboards, and
VictoriaTraces."
+---
+
+In [Part 1](/blog/2026/09/camel-genai-observability-jbang/) we prototyped
GenAI observability with the
+Camel CLI and TUI. This follow-up — **Phase 3 (Operate)** — shows the same
`gen_ai.*` telemetry in a
+**Spring Boot** application wired to the observability stack Camel ships for
local development:
+**Prometheus**, **VictoriaTraces**, and **Perses**.
+
+The runnable sample lives in the
[camel-spring-boot-examples](https://github.com/apache/camel-spring-boot-examples)
+repository at
[`genai-observability`](https://github.com/apache/camel-spring-boot-examples/tree/main/genai-observability)
+(reworked in [PR
#192](https://github.com/apache/camel-spring-boot-examples/pull/192)).
+
+## Architecture
+
+```
+Terminal 1: ollama serve
+Terminal 2: camel infra run observability
+Terminal 3: mvn spring-boot:run
+
+┌─────────────────┐ scrape :9876/observe/metrics ┌──────────────┐
+│ Spring Boot │ ───────────────────────────────► │ Prometheus │
+│ Camel + 2 LLMs │ │ :9090 │
+│ app :8080 │ └──────┬───────┘
+│ mgmt :9876 │ │
+└────────┬────────┘ ▼
+ │ OTLP (Micrometer Tracing) ┌──────────────┐
+ ▼ │ Perses │
+┌─────────────────┐ dashboards │ :3000 │
+│ VictoriaTraces │ ◄────────────────────────────────└──────────────┘
+│ :10428 │
+└─────────────────┘
+```
+
+Two timer routes call two small Ollama models (`llama3.2:1b` and
`qwen3:0.6b`), so every GenAI metric
+and span carries a distinct `gen_ai.request.model` tag — ideal for Perses
dashboards that compare
+latency and token cost per model.
+
+The `camel-observability-services-starter` moves Actuator endpoints to
management port **9876** under
+`/observe`, with Prometheus at `/observe/metrics`. Both standard Camel metrics
and `gen_ai.*` metrics
+share the same Micrometer registry.
+
+Traces use the Spring Boot idiomatic setup: **Micrometer Tracing with the
OpenTelemetry bridge**
+(`spring-boot-micrometer-tracing-opentelemetry`,
`micrometer-tracing-bridge-otel`,
+`opentelemetry-exporter-otlp`). Spring Boot auto-configures the OTLP exporter
and a tracing handler
+on the `ObservationRegistry`. Camel route spans (via `camel-opentelemetry2`)
and `gen_ai.*` client
+spans land in the same VictoriaTraces trace.
+
+## Quick start
+
+### Prerequisites
+
+```shell
+java -version # 17+
+mvn -version # 3.9+
+camel version # Camel CLI 4.22+
+docker --version # used by camel infra run observability
+ollama pull llama3.2:1b
+ollama pull qwen3:0.6b
+```
+
+### Terminal 1 — Ollama
+
+```shell
+ollama serve
+```
+
+### Terminal 2 — Observability stack
+
+```shell
+camel infra run observability
+```
+
+This bundles Prometheus, VictoriaTraces, VictoriaLogs, and Perses — the same
stack described in the
+[Camel 4.22 what's new
post](/blog/2026/08/camel422-whatsnew/#observability-stack). You can also start
+it from the [Camel TUI](/manual/camel-jbang-tui.html) infrastructure panel.
+
+The bundled Prometheus is pre-configured to scrape
`host.docker.internal:9876/observe/metrics`.
+
+| Service | Port | Role |
+|---------|------|------|
+| Prometheus | 9090 | Scrapes `/observe/metrics` on management port 9876 |
+| VictoriaTraces | 10428 | Stores OTLP traces; UI at `/select/vmui` |
+| Perses | 3000 | Metrics dashboards |
+
+### Terminal 3 — Spring Boot
+
+```shell
+git clone https://github.com/apache/camel-spring-boot-examples.git
+cd camel-spring-boot-examples/genai-observability
+mvn spring-boot:run
+```
+
+Wait for log lines like:
+
+```text
+Started GenAiObservabilityApplication
+[llama3.2:1b] LLM reply: Apache Camel is an integration framework...
+[qwen3:0.6b] LLM reply: Enterprise Integration Patterns are...
+```
+
+## Verify GenAI metrics
+
+Metrics are on the **management port**, not the application port:
+
+```shell
+curl -s http://localhost:9876/observe/metrics | grep gen_ai
+```
+
+Expected Micrometer names:
+
+- `gen_ai_client_operation` — timer of LLM call duration
+- `gen_ai_client_token_usage` — counter with tag
`gen_ai_token_type=input|output`
+
+Both series are tagged with `gen_ai_request_model` so you can split by model
in Prometheus and Perses.
+
+### Prometheus queries
+
+Open [http://localhost:9090](http://localhost:9090) and try:
+
+```promql
+# Output tokens by model
+sum by (gen_ai_request_model) (
+ rate(gen_ai_client_token_usage_total{gen_ai_token_type="output"}[5m])
+)
+
+# LLM call rate per model
+sum by (gen_ai_request_model) (rate(gen_ai_client_operation_count[5m]))
+
+# Mean latency per model
+avg by (gen_ai_request_model) (gen_ai_client_operation_seconds_sum)
+ / avg by (gen_ai_request_model) (gen_ai_client_operation_seconds_count)
+```
+
+## Perses GenAI dashboard
+
+The observability stack includes a general Camel overview dashboard at
+[http://localhost:3000/projects/camel/dashboards/overview](http://localhost:3000/projects/camel/dashboards/overview).
+
+Create the GenAI dashboard from the example definition:
+
+```shell
+curl -X POST http://localhost:3000/api/v1/projects \
+ -H 'Content-Type: application/json' \
+ -d '{"kind":"Project","metadata":{"name":"camel_genai"},"spec":{}}'
+
+curl -X POST http://localhost:3000/api/v1/projects/camel_genai/dashboards \
+ -H 'Content-Type: application/json' \
+ --data @perses-genai-dashboard.json
+```
+
+Open
[http://localhost:3000/projects/camel_genai/dashboards/overview](http://localhost:3000/projects/camel_genai/dashboards/overview).
+
+
+
+The **GenAI Summary** row shows running totals: LLM calls, errors, in-flight
calls, and input/output
+token counters. **Call rate** and **Mean / Max LLM latency** split by
`gen_ai.request.model` — a fast
+model settles at the timer frequency; a slow *thinking* model's rate is capped
by its own latency.
+
+
+
+**Token throughput** and **Avg tokens per call** come from
`gen_ai.client.token.usage`, split by model
+and token type. With `llama3.2:1b` and a reasoning model like `qwen3`, the
same one-sentence prompt can
+produce an order-of-magnitude difference in output tokens — exactly the
cost/latency trade-off these
+panels surface.
+
+> **Note:** Perses state lives in the container. After restarting `camel infra
run observability`,
+> re-run the two `curl` commands above to recreate the GenAI dashboard.
+
+## Explore traces in VictoriaTraces
+
+Open [http://localhost:10428/select/vmui](http://localhost:10428/select/vmui)
+
+Useful trace search filters:
+
+- `gen_ai.operation.name="chat"`
+- `gen_ai.request.model="llama3.2:1b"`
+- `camel.component="langchain4j-chat"`
+
+Each span carries `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`,
+`gen_ai.response.finish_reasons`, and `gen_ai.system` (e.g. `langchain4j`).
+
+## Key configuration
+
+### Maven dependencies (excerpt)
+
+```xml
+<dependency>
+ <groupId>org.apache.camel.springboot</groupId>
+ <artifactId>camel-observability-services-starter</artifactId>
+</dependency>
+<dependency>
+ <groupId>org.apache.camel.springboot</groupId>
+ <artifactId>camel-ai-observability-starter</artifactId>
+</dependency>
+<dependency>
+ <groupId>org.apache.camel.springboot</groupId>
+ <artifactId>camel-langchain4j-chat-starter</artifactId>
+</dependency>
+<dependency>
+ <groupId>org.apache.camel.springboot</groupId>
+ <artifactId>camel-yaml-dsl-starter</artifactId>
+</dependency>
+<dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-ai-observability</artifactId>
+ <version>${camel-version}</version>
+</dependency>
+<dependency>
+ <groupId>dev.langchain4j</groupId>
+ <artifactId>langchain4j-ollama</artifactId>
+ <version>${langchain4j-version}</version>
+</dependency>
+<!-- Micrometer Tracing + OpenTelemetry bridge for OTLP export -->
+<dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-micrometer-tracing-opentelemetry</artifactId>
+</dependency>
+<dependency>
+ <groupId>io.micrometer</groupId>
+ <artifactId>micrometer-tracing-bridge-otel</artifactId>
+</dependency>
+<dependency>
+ <groupId>io.opentelemetry</groupId>
+ <artifactId>opentelemetry-exporter-otlp</artifactId>
+</dependency>
+```
+
+The example builds `ChatModel` beans explicitly with plain
`langchain4j-ollama` rather than
+`langchain4j-ollama-spring-boot-starter`, which is not yet compatible with
Spring Boot 4
+([langchain4j#6236](https://github.com/langchain4j/langchain4j/issues/6236)).
+
+### application.properties
+
+```properties
+# Two Ollama models (ChatModelConfiguration builds chatModel1/chatModel2 beans)
+langchain4j.ollama.chat-model.base-url=http://localhost:11434
+langchain4j.ollama.chat-model.temperature=0.2
+langchain4j.ollama.chat-model.timeout=PT120S
+langchain4j.ollama.chat-model-1.model-name=llama3.2:1b
+langchain4j.ollama.chat-model-2.model-name=qwen3:0.6b
+
+# YAML routes under src/main/resources/camel/
+camel.main.routes-include-pattern=camel/*
+
+# GenAI observability
+camel.aiobservability.enabled=true
+camel.opentelemetry2.enabled=true
+
+# Actuator / Prometheus on management port 9876 (/observe/*)
+management.endpoints.web.exposure.include=health,prometheus,info
+management.prometheus.metrics.export.enabled=true
+
+# OTLP trace export to VictoriaTraces (camel infra run observability)
+management.opentelemetry.tracing.export.otlp.endpoint=http://localhost:10428/insert/opentelemetry/v1/traces
+management.tracing.sampling.probability=1.0
+```
+
+### YAML routes (excerpt)
+
+Routes live in `src/main/resources/camel/genai-route.camel.yaml`:
+
+```yaml
+- route:
+ id: genai-chat-1
+ from:
+ uri: timer:genai1
+ parameters:
+ period: "15000"
+ steps:
+ - setBody:
+ constant: "In one sentence, what is Apache Camel integration?"
+ - to:
+ uri: langchain4j-chat:model1
+ parameters:
+ chatModel: "#chatModel1"
+ - log:
+ message: "[${header.CamelLangChain4jChatResponseModel}] LLM reply:
${body}"
+```
+
+A second route calls `#chatModel2` on a 20-second timer with a different
prompt.
+
+## CLI/TUI vs Spring Boot
+
+| Concern | Part 1 (CLI/TUI) | Part 2 (Spring Boot) |
+|---------|------------------|----------------------|
+| Time to first span | Minutes (`camel run --observe`) | Minutes + `camel
infra run observability` |
+| Metrics endpoint | `/observe/metrics` on port 9876 | `/observe/metrics` on
port 9876 |
+| Trace UI (dev) | TUI Spans tab (built-in) | VictoriaTraces VMUI |
+| Per-model dashboards | Prometheus queries | Perses GenAI dashboard |
+| Production fit | Prototyping, CI demos | Standard Spring ops (Actuator, K8s
probes) |
+
+Both parts use the same `camel infra run observability` stack, so you can
prototype in the CLI/TUI
+and switch to Spring Boot without reconfiguring collectors.
+
+## Production checklist
+
+1. Set `camel.aiobservability.enabled=true` explicitly in all environments
+2. Scrape `/observe/metrics` on the management port (9876 by default with
observability-services)
+3. Export OTLP via Micrometer Tracing to your org's collector (Jaeger, Tempo,
VictoriaTraces, etc.)
+4. Alert on `gen_ai_client_token_usage` rate and `gen_ai_client_operation`
p99, split by model
+5. Use route IDs (`genai-chat-1`, `genai-chat-2`) in dashboards to attribute
cost per integration
+
+Disable globally with `camel.aiobservability.enabled=false` when running load
tests without LLM overhead.
+
+## Troubleshooting
+
+| Issue | Resolution |
+|-------|------------|
+| Prometheus empty targets | Spring Boot must be running; stack scrapes
`host.docker.internal:9876` |
+| No traces in VictoriaTraces | Verify
`management.opentelemetry.tracing.export.otlp.endpoint` points to port
**10428** |
+| `chatModel1` not found | Confirm `ChatModelConfiguration` and Ollama
property prefixes |
+| No `gen_ai` metrics | Add `camel-ai-observability-starter` +
`camel-observability-services-starter` |
+| Perses dashboard missing | Re-run the two `curl` commands after restarting
the observability stack |
+
+## Learn more
+
+- [Part 1: CLI and TUI](/blog/2026/09/camel-genai-observability-jbang/)
+- [AI Observability component
source](https://github.com/apache/camel/blob/main/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc)
(4.23+)
+- [Observability Services](/components/next/others/observability-services.html)
+- [Camel AI components](/components/next/ai-summary.html)
+- [Spring Boot
example](https://github.com/apache/camel-spring-boot-examples/tree/main/genai-observability)
+
+---
+
+*This post was written by Omar Atie
([@atiaomar1978-hub](https://github.com/atiaomar1978-hub)) with assistance from
Cursor Cloud Agent.*
