dulvac commented on code in PR #3058: URL: https://github.com/apache/jackrabbit-oak/pull/3058#discussion_r3712063611
########## oak-doc/src/site/markdown/security/audit-design.md: ########## @@ -0,0 +1,435 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Audit Pipeline Design +-------------------------------------------------------------------------------- + +This document describes the design of Oak's audit pipeline: the SPI surface +in `oak-core-spi`, the security-domain constants in `oak-security-spi`, the +pipeline implementation in `oak-core`, OSGi and embedded wiring, and the +threading and ordering rules the implementation relies on. + +For the consumer-facing guide (event model, listener contract, trust model), +see [Audit SPI](audit.html). + +<a name="overview"></a> +### Overview + +The audit pipeline transports structured `AuditEvent`s from producers to +bundle-registered `AuditEventListener` consumers, gated by a feature toggle +and a per-domain listener registry. + +There are two delivery paths: + +- **Commit-attached.** Oak-internal capture sites (e.g. `UserManagerImpl`) + call `AuditEvents.record(root, event)`. Events land in a per-session + `ThreadLocal` buffer (`AuditBuffer`), and a `NodeStore` `Observer` + (`AuditDrainObserver`) drains and dispatches them after the surrounding + `Root.commit()` durably persists. At drain time each event is decorated + with `commit.sessionId`, `commit.userId`, and `commit.timestamp` payload + entries; a failed commit drops the buffer. +- **Fire-and-forget.** Any OSGi bundle resolves `AuditEventEmitter` via + `@Reference` and calls `emit(event)`. The event is dispatched synchronously + on the calling thread: no buffering, no commit boundary, no payload + decoration. Caller-supplied values for the three reserved `commit.*` + attestation keys are stripped before delivery (the trust contract on + `AuditEvent#getPayload()` is the normative statement). + +Both paths converge on `AuditEventListener.onEvents(List<AuditEvent>)` and +share one listener registry. Failure isolation is layered: an outer +`Throwable` barrier in `AuditDrainObserver.contentChanged` keeps audit from +masquerading as a commit failure, and an inner per-listener `Throwable` +barrier on both paths keeps one misbehaving listener from stopping the +others. + +Pipeline state is owned by `AuditConfigurationImpl` in `oak-core`, which +holds the feature toggle, the buffer, the listener registry, the sink +installed into the `AuditEvents` facade, and the singleton drain observer. +It is registered as an OSGi service of type `AuditConfiguration`. Audit is a +top-level Oak concern, not a `SecurityConfiguration`. + +<a name="pipeline_diagram"></a> +### Pipeline diagram + + + +The upper row is the commit-attached path, the lower row fire-and-forget. +They converge on the listener registry, which filters by domain and orders by +rank before invoking each listener. + +On the commit-attached path, `BufferSink.record` gates on the feature toggle +and on whether any listener is registered for the event's domain, so a +capture site allocates nothing when audit is off. The observer runs on the +commit thread once the merge has persisted, drains the buffer for that +session, and stamps the three `commit.*` keys. On the fire-and-forget path, +`BufferSink.dispatch` applies the same toggle and listener gates, strips +caller-supplied `commit.*` values, and dispatches inline. + +The short-circuit order in the observer, and the exception barriers on both +paths, are described under Implementation below. + +<a name="commit_flow"></a> +### Commit flow + + + +The observer fires synchronously on the commit thread, after durable +persistence and before `store.merge` returns. That is the property the +per-session buffer depends on: the drain has to happen on the thread that +filled it. Every production `NodeStore` notifies observers that way, though +not all by the same route. The document and composite stores dispatch +through `ChangeDispatcher`; `MemoryNodeStore` iterates its registered +observers directly from `setRoot`. Either way the notification completes +before `merge` returns. + +Two segment-store configurations are worth knowing about, because in both the Review Comment: fair. the key constants are package-private in oak-core, so a listener has no choice but to hardcode the string. I'll add a static predicate plus public key constants to the SPI, and cut the doc down to naming it instead of describing the check. But the predicate can only ever be "these keys are present", since Oak doesn't sign anything so I'll name it for what it checks rather than implying a stronger guarantee. And it belongs in `oak-core-spi` next to `AuditEvent,` so listeners keep their single-module dependency. The API change lands in the implementation PR (#3059) and I'll update this page to match. ########## oak-doc/src/site/markdown/security/audit-design.md: ########## @@ -0,0 +1,435 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +Audit Pipeline Design +-------------------------------------------------------------------------------- + +This document describes the design of Oak's audit pipeline: the SPI surface +in `oak-core-spi`, the security-domain constants in `oak-security-spi`, the +pipeline implementation in `oak-core`, OSGi and embedded wiring, and the +threading and ordering rules the implementation relies on. + +For the consumer-facing guide (event model, listener contract, trust model), +see [Audit SPI](audit.html). + +<a name="overview"></a> +### Overview + +The audit pipeline transports structured `AuditEvent`s from producers to +bundle-registered `AuditEventListener` consumers, gated by a feature toggle +and a per-domain listener registry. + +There are two delivery paths: + +- **Commit-attached.** Oak-internal capture sites (e.g. `UserManagerImpl`) + call `AuditEvents.record(root, event)`. Events land in a per-session + `ThreadLocal` buffer (`AuditBuffer`), and a `NodeStore` `Observer` + (`AuditDrainObserver`) drains and dispatches them after the surrounding + `Root.commit()` durably persists. At drain time each event is decorated + with `commit.sessionId`, `commit.userId`, and `commit.timestamp` payload + entries; a failed commit drops the buffer. +- **Fire-and-forget.** Any OSGi bundle resolves `AuditEventEmitter` via + `@Reference` and calls `emit(event)`. The event is dispatched synchronously + on the calling thread: no buffering, no commit boundary, no payload + decoration. Caller-supplied values for the three reserved `commit.*` + attestation keys are stripped before delivery (the trust contract on + `AuditEvent#getPayload()` is the normative statement). + +Both paths converge on `AuditEventListener.onEvents(List<AuditEvent>)` and +share one listener registry. Failure isolation is layered: an outer +`Throwable` barrier in `AuditDrainObserver.contentChanged` keeps audit from +masquerading as a commit failure, and an inner per-listener `Throwable` +barrier on both paths keeps one misbehaving listener from stopping the +others. + +Pipeline state is owned by `AuditConfigurationImpl` in `oak-core`, which +holds the feature toggle, the buffer, the listener registry, the sink +installed into the `AuditEvents` facade, and the singleton drain observer. +It is registered as an OSGi service of type `AuditConfiguration`. Audit is a +top-level Oak concern, not a `SecurityConfiguration`. + +<a name="pipeline_diagram"></a> +### Pipeline diagram + + + +The upper row is the commit-attached path, the lower row fire-and-forget. +They converge on the listener registry, which filters by domain and orders by +rank before invoking each listener. + +On the commit-attached path, `BufferSink.record` gates on the feature toggle +and on whether any listener is registered for the event's domain, so a +capture site allocates nothing when audit is off. The observer runs on the +commit thread once the merge has persisted, drains the buffer for that +session, and stamps the three `commit.*` keys. On the fire-and-forget path, +`BufferSink.dispatch` applies the same toggle and listener gates, strips +caller-supplied `commit.*` values, and dispatches inline. + +The short-circuit order in the observer, and the exception barriers on both +paths, are described under Implementation below. + +<a name="commit_flow"></a> +### Commit flow + + + +The observer fires synchronously on the commit thread, after durable +persistence and before `store.merge` returns. That is the property the +per-session buffer depends on: the drain has to happen on the thread that +filled it. Every production `NodeStore` notifies observers that way, though +not all by the same route. The document and composite stores dispatch +through `ChangeDispatcher`; `MemoryNodeStore` iterates its registered +observers directly from `setRoot`. Either way the notification completes +before `merge` returns. + +Two segment-store configurations are worth knowing about, because in both the +commit-attached path silently produces nothing. `SegmentNodeStore.addObserver` +returns a no-op handle unless change dispatch is enabled, and an observer +attached through that handle is never notified. That applies to a +cold-standby instance, where the primary store turns dispatch off, and to a +store configured through `SegmentNodeStoreFactory`, where dispatch is off +unless `dispatchChanges` is set explicitly. + +Draining from an `Observer` rather than from a commit hook has two +consequences worth spelling out. Events never transit the `CommitContext`, +which is a shared string-keyed channel readable by any `CommitHook` in the +same commit; keeping audit events out of it avoids a class of cross-bundle +information disclosure. And dispatch happens only after the commit is +durable, so there is no window in which a listener sees an event for a +write that subsequently fails. + +<a name="spi_layout"></a> +### SPI layout + +#### oak-core-spi + +Package `org.apache.jackrabbit.oak.spi.audit` holds the domain-neutral SPI: + +| Type | Role | +|---|---| +| `AuditEvent` | Event interface: domain, type, timestamp, payload. Static factory `AuditEvent.of(...)`. | +| `AuditEventListener` | Consumer SPI: `onEvents(List<AuditEvent>)`, scoped to one domain via `getDomain()`, ordered by `getRank()`. | +| `AuditEventEmitter` | OSGi service surface for fire-and-forget emission from any bundle. | +| `AuditEvents` | Static facade: `record(root, event)` and `dispatch(event)`, routing to the installed `Sink`; `isEnabled()` / `isEnabledFor(domain)` gates. | +| `AuditEvents.Sink` | SPI implemented by the pipeline. `AuditConfigurationImpl` installs a `BufferSink`. | +| `AuditBufferLifecycle` | Session lifecycle callouts: drain on refresh and on commit failure. | +| `AuditConfiguration` | Typed handle on pipeline state (`isActive()`, `NOOP`). | + +`AuditConfiguration.isActive()` returns `true` when the feature toggle is +enabled and at least one listener is registered. The two predicates AND +together so a deployed-but-unused pipeline reports `false`, matching the +no-allocation semantics of `AuditEvents.isEnabled()`. Both read the same +volatile sink state, so they cannot drift apart. The interface ships a +`NOOP` constant for callers that want a guaranteed-non-null handle. + +Cardinality is unary optional: multiple `AuditConfiguration` implementations +are not supported. The buffer lifecycle is a singleton install, and two +observers on the same root `NodeStore` would each produce a dispatch. +Multiplexing belongs at the listener layer. + +#### oak-security-spi + +Security-domain constants live next to the SPI they describe: + +- `spi/security/audit/SecurityAuditDomain` holds the single domain string + `"oak.security"` shared by all events Oak's security stack emits. The + `oak.` prefix namespaces the domain so listeners in mixed deployments + (Sling, application bundles) can tell Oak's security events apart from + same-named domains defined by other layers. +- `spi/security/user/UserAuditTypes` holds the user-membership vocabulary: + type strings (`MEMBER_ADDED`, `MEMBER_REMOVED`) and payload keys + (`PAYLOAD_GROUP_PATH`, `PAYLOAD_MEMBER_IDS`, `PAYLOAD_MEMBER_PATHS`, + `PAYLOAD_MEMBERSHIP_SOURCE`, `PAYLOAD_IS_CONTENT_ID`, + `PAYLOAD_FAILED_IDS`). A single and a bulk membership change share the + same type; a bulk change is one whose `PAYLOAD_MEMBER_IDS` list holds more + than one entry. + +Future ACL, principal, or token events declare their own `*AuditTypes` +classes in the respective SPI sub-packages. + +Producer-side factories are deliberately not part of the SPI. They live as +package-private classes next to their only callers, e.g. +`UserAuditEvents` next to `UserManagerImpl` in `oak-core`. The asymmetry +(read-side vocabulary public, write-side factories impl-private) raises the +bar for casually forging Oak-attested events, but it is not a hard boundary: +any bundle can call `AuditEvent.of(domain, type, payload)` directly. +Listeners that need to distinguish Oak-attested commit-attached events from +fire-and-forget emissions check for the three reserved `commit.*` payload +keys, per the trust contract on `AuditEvent#getPayload()`. Review Comment: fair. the key constants are package-private in [oak-core](https://issues.apache.org/jira/browse/OAK-core), so a listener has no choice but to hardcode the string. I'll add a static predicate plus public key constants to the SPI, and cut the doc down to naming it instead of describing the check. But the predicate can only ever be "these keys are present", since Oak doesn't sign anything so I'll name it for what it checks rather than implying a stronger guarantee. And it belongs in oak-core-spi next to AuditEvent, so listeners keep their single-module dependency. The API change lands in the implementation PR (https://github.com/apache/jackrabbit-oak/pull/3059) and I'll update this page to match. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
