Ma77Ball commented on code in PR #5377:
URL: https://github.com/apache/texera/pull/5377#discussion_r4233112669
##########
bin/single-node/docker-compose.yml:
##########
@@ -530,6 +530,189 @@ services:
command: >
sh -c 'apk add --no-cache curl jq bash > /dev/null 2>&1 && bash
/examples/load-examples.sh'
+ # ========================================================================
+ # Part 5: Observability stack (PR 6).
Review Comment:
Removed the "PR 6" references and tightened the observability comments to
state only what each service does.
##########
bin/single-node/docker-compose.yml:
##########
@@ -530,6 +530,189 @@ services:
command: >
sh -c 'apk add --no-cache curl jq bash > /dev/null 2>&1 && bash
/examples/load-examples.sh'
+ # ========================================================================
+ # Part 5: Observability stack (PR 6).
+ #
+ # All six services live in this single compose file (no separate
+ # observability compose file, no monolithic observability profile).
+ # Per-signal profiles let an operator drop one without disturbing
+ # the others — see bin/single-node/up.sh and bin/single-node/.env
+ # (COMPOSE_PROFILES default includes every observability profile so
+ # `docker compose up` runs the whole stack).
+ #
+ # Network posture: every receiver/HTTP port binds to loopback
+ # (127.0.0.1) on the host or stays inside the texera-single-node
+ # bridge network entirely. No 0.0.0.0 host bindings, no Ingress.
+ # ========================================================================
+
+ # OpenTelemetry Collector — the single OTLP ingress for all three
+ # signals. Reads bin/observability/otel-collector/config.yaml.
+ # Apache-2.0; pinned to the contrib distribution.
+ otel-collector:
+ image: otel/opentelemetry-collector-contrib:0.153.0
+ container_name: texera-otel-collector
+ profiles: [observability-collector]
+ restart: always
+ user: "10001:10001"
+ read_only: true
+ security_opt:
+ - no-new-privileges:true
+ volumes:
+ -
../observability/otel-collector/config.yaml:/etc/otelcol-contrib/config.yaml:ro,z
Review Comment:
The release workflow now archives bin/observability and rewrites the
../observability mount paths to ./observability, so the compose resolves from
the released single-node bundle.
##########
bin/single-node/.env:
##########
@@ -96,3 +96,48 @@ LLM_ENDPOINT=http://nginx:8080
TEXERA_DASHBOARD_SERVICE_ENDPOINT=http://dashboard-service:8080
WORKFLOW_COMPILING_SERVICE_ENDPOINT=http://workflow-compiling-service:9090
WORKFLOW_EXECUTION_SERVICE_ENDPOINT=http://workflow-runtime-coordinator-service:8085
+
+# ============================================================================
+# Observability (PR 6)
+# ============================================================================
+# Each backend lives behind its own docker-compose profile. By default
+# COMPOSE_PROFILES enables every observability profile so `docker compose up`
+# brings the whole stack online — there is no separate observability compose
+# file or monolithic observability profile.
+#
+# To disable a signal:
+# * Prefer using bin/single-node/up.sh, which translates the
+# TEXERA_OBSERVABILITY_* env vars below into the right COMPOSE_PROFILES.
+# * Or edit COMPOSE_PROFILES directly here.
+#
+# Disable env-var conventions (consumed by up.sh):
Review Comment:
Observability is now opt-in: COMPOSE_PROFILES defaults to empty, so no
observability service starts unless the operator enables the profiles.
##########
bin/single-node/.env:
##########
@@ -96,3 +96,48 @@ LLM_ENDPOINT=http://nginx:8080
TEXERA_DASHBOARD_SERVICE_ENDPOINT=http://dashboard-service:8080
WORKFLOW_COMPILING_SERVICE_ENDPOINT=http://workflow-compiling-service:9090
WORKFLOW_EXECUTION_SERVICE_ENDPOINT=http://workflow-runtime-coordinator-service:8085
+
+# ============================================================================
+# Observability (PR 6)
+# ============================================================================
+# Each backend lives behind its own docker-compose profile. By default
+# COMPOSE_PROFILES enables every observability profile so `docker compose up`
+# brings the whole stack online — there is no separate observability compose
+# file or monolithic observability profile.
+#
+# To disable a signal:
+# * Prefer using bin/single-node/up.sh, which translates the
+# TEXERA_OBSERVABILITY_* env vars below into the right COMPOSE_PROFILES.
+# * Or edit COMPOSE_PROFILES directly here.
+#
+# Disable env-var conventions (consumed by up.sh):
+# TEXERA_OBSERVABILITY_LOGS=disabled drops victorialogs
+# TEXERA_OBSERVABILITY_METRICS=disabled drops victoriametrics
+# TEXERA_OBSERVABILITY_TRACES=disabled drops jaeger
+# TEXERA_OBSERVABILITY_PROFILES=disabled drops parca + parca-agent
+# TEXERA_OBSERVABILITY_COLLECTOR=disabled drops the otel-collector (rare)
Review Comment:
Dropped the TEXERA_OBSERVABILITY_* / up.sh convention and removed up.sh;
COMPOSE_PROFILES, which docker compose reads natively through the canonical
launcher, is now the single switch.
##########
bin/single-node/docker-compose.yml:
##########
@@ -530,6 +530,189 @@ services:
command: >
sh -c 'apk add --no-cache curl jq bash > /dev/null 2>&1 && bash
/examples/load-examples.sh'
+ # ========================================================================
+ # Part 5: Observability stack (PR 6).
+ #
+ # All six services live in this single compose file (no separate
+ # observability compose file, no monolithic observability profile).
+ # Per-signal profiles let an operator drop one without disturbing
+ # the others — see bin/single-node/up.sh and bin/single-node/.env
+ # (COMPOSE_PROFILES default includes every observability profile so
+ # `docker compose up` runs the whole stack).
+ #
+ # Network posture: every receiver/HTTP port binds to loopback
+ # (127.0.0.1) on the host or stays inside the texera-single-node
+ # bridge network entirely. No 0.0.0.0 host bindings, no Ingress.
+ # ========================================================================
+
+ # OpenTelemetry Collector — the single OTLP ingress for all three
+ # signals. Reads bin/observability/otel-collector/config.yaml.
+ # Apache-2.0; pinned to the contrib distribution.
+ otel-collector:
+ image: otel/opentelemetry-collector-contrib:0.153.0
+ container_name: texera-otel-collector
+ profiles: [observability-collector]
+ restart: always
+ user: "10001:10001"
+ read_only: true
+ security_opt:
+ - no-new-privileges:true
+ volumes:
+ -
../observability/otel-collector/config.yaml:/etc/otelcol-contrib/config.yaml:ro,z
+ command: ["--config=/etc/otelcol-contrib/config.yaml"]
+ # Local dev override: publish OTLP receiver ports on the host
+ # loopback so a Scala backend running outside docker (sbt /
+ # IntelliJ) can emit telemetry. In compose-only deploys, services
+ # talk to otel-collector:4317/:4318 via the bridge network.
+ ports:
+ - "127.0.0.1:4317:4317"
Review Comment:
Moved the OTLP host-port publishing into
bin/single-node/docker-compose.override.yml, auto-loaded for local dev; the
base compose no longer publishes them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
