GitHub user Ma77Ball created a discussion: Approach: landing Modern Observability (#4070) as a stack of small, self-contained PRs
## Proposal: land "Modern Observability" (#4070) as a stack of small, self-contained PRs Opening this to get feedback on the **approach and procedure** before raising the PRs. ### Problem The observability work for #4070 grew into one large branch (~16k insertions across ~230 files): OTel bootstrap + log/metric/trace/profile pipelines, a tenant-scoped query gateway, an Angular dashboard, docker-compose backends — plus a codebase-wide logging/readability pass that rode along. That is far too big to review or merge safely as one PR. ### Approach Decompose it into **26 small branches in two independent tracks**, each: - **≤ ~1000 lines of logic**, compiles green on its own, and is **non-breaking** (the feature is default-off via `OTEL_SDK_DISABLED`, and the dashboard is health-gated), and - adds one coherent slice toward the larger feature. **Track A — the feature (18 stacked PRs):** foundations (log sanitizer → log bridge → OTel bootstrap → metrics → tracing) → deploy (compose/collector, Parca/eBPF) → query gateway (dtos → core → **logs / metrics / traces / profiles each its own PR**) → UI (shell → logs / metrics / traces / profiles panels). **Track B — logging & a security fix (8 PRs, independent, off `main`):** human-readable lifecycle/error logging across engine, web layer, dashboard resources, workflow-core storage, the microservices, common/auth, common/dao; plus a fix to stop logging JWTs/headers/bodies in access-control. ### Procedure - Each PR is tracked as a sub-issue of #4070 (#5327–#5352) and maps 1:1 to a branch. - Track A PRs are **stacked** (each opens against the previous); Track B PRs target `main` directly and can merge in any order. - Suggested land order: security fix + logging PRs (parallel) → the feature stack in numeric order. Shortest path to a first shippable signal (logs end-to-end): bootstrap → gateway-core → gateway-logs → ui-shell → ui-logs. - Per-signal verification: backend via `sbt <module>/compile` (+ specs), frontend via `ng build`. ### Questions for reviewers 1. Is the two-track split + per-signal gateway/UI granularity the right reviewing unit, or would you prefer coarser PRs? 2. Should the codebase-wide logging track live under #4070, or as its own umbrella issue? 3. Any concerns with the stacked-PR procedure for the feature track? GitHub link: https://github.com/apache/texera/discussions/5355 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
