hanahmily opened a new pull request, #1189:
URL: https://github.com/apache/skywalking-banyandb/pull/1189
## What
Storage-node **post-trace pipeline** for BanyanDB — a design + contract
(proto + Go SDK) with **no engine code yet** — letting operators retain/sample
traces during storage lifecycle events on data nodes.
This supersedes #1144 (closed). The design is unchanged except that the
sampler is generalized into a **generic, chainable `Plugin`**.
## Contents
- **Design** — `docs/design/post-trace-pipeline.md`: decoupled **gating**
(Hot-phase `PIPELINE_EVENT_MERGE` / `PIPELINE_EVENT_FINALIZE`) and **per-stage
retention** (at each stage's migration-out boundary), the
trace-completeness/timing model, integration with the time-aging system, and
two operational scenarios grounded in the skywalking-showcase cluster
(`sw_trace`, `sw_zipkinTrace`). All diagrams are mermaid.
- **Proto** — `api/proto/banyandb/pipeline/v1/trace_pipeline.proto`:
`TracePipelineConfig`, `StageRule`, the generic `Plugin` envelope,
`SamplerPlugin`, `PipelineEvent`, and the `TracePipelineRegistryService` CRUD
surface (HTTP under `/v1/trace-pipeline/schema`).
- **Go SDK** — `pkg/pipeline/sdk`: the pinned surface plugin authors build
against, with a reference plugin at `_example/segment-tail-sampler`.
## Generic Plugin + chain
- `Plugin` is a kind-tagged **oneof envelope** (`{ name; oneof kind {
SamplerPlugin sampler } }`); the set arm is the discriminator. Today the only
kind is the **sampler**; adding a kind (transformer / exporter / router) is
purely additive — a new oneof arm + an SDK sub-interface + engine dispatch
(design §2.6).
- `TracePipelineConfig.plugins` (gating) and `StageRule.plugins` (per-stage
retention) each wire an **ordered chain** (`repeated Plugin`). The chain is a
**sequential pipe**: links run in declared order, each evaluating the traces
the previous link kept, so an all-sampler chain is the conjunction of the
links' keep-masks. **Fail-open is per link.** A single sampler is just a
one-element chain.
- SDK: the generic `Plugin` interface (`Kind` / `Project` / `Close`);
`Sampler` embeds it and adds `Decide`. The `.so` boundary crosses only stdlib
slices plus the stable `pbv1.ValueType` enum, mirroring the native trace block;
the batch is strictly read-only.
## Scope / non-goals
Design + contract only. No engine, loader, chain executor, or projection
materializer. No Transformer/Exporter/Router kind is implemented (sketched in
§2.6 only) — the read-only batch invariant holds precisely because the only
shipped kind is a filter.
## Verification
- `make generate` clean and idempotent; the generated `Plugin` oneof Go
compiles.
- `pkg/pipeline/sdk`: `go build`, `go test`, and `golangci-lint` are green;
the reference plugin builds via `-buildmode=plugin`.
- All six mermaid diagrams validate (mmdc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
