[
https://issues.apache.org/jira/browse/NIFI-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-15979:
-------------------------------
Description:
NIFI-14991 wired Process Group identity into the SLF4J MDC for every component
log, but it has no concept of a `Connector` (it pre-dates that feature). As a
result:
- Logs from inside a connector-managed flow carry `processGroupId/Name/Path`
but
nothing identifying the owning connector or its NAR bundle. The connector's
own
log lines emit no MDC at all (`ExtensionBuilder.buildConnector` constructs the
`ComponentLog` with the no-arg `StandardLoggingContext`).
- Reporting tasks that walk `EventAccess.getControllerStatus()` never see
connector-managed Process Groups — they have no parent and live in
`ConnectorRepository`, separate from the root group.
- A connector implementation has no way to surface custom logging context
(e.g. source schema, table namespace).
+*What changes*+
*nifi-api* (Covered by NIP-11)
- `ConnectorInitializationContext.setLoggingAttributes(Map<String, String>)` —
new `default` method; `AbstractConnector` provides a `protected` helper.
- `ProcessGroupStatus` — new `loggingAttributes` field (`Map.of()` default),
included in `clone()` / `merge()`.
- `EventAccess.getConnectorStatuses()` — new `default` returning empty;
implementations override.
*NiFi Framework*
- `StandardConnectorNode` holds a `loggingAttributes` map, implements
`GroupedComponent`, and is initialized with reserved framework keys:
`connectorId`, `connectorName`, `connectorComponent` (FQN, mirroring the
existing processor `component` convention), and
`connectorBundleGroup`/`Artifact`/`Version` from the bundle coordinate.
- Reserved keys cannot be overridden by `setLoggingAttributes`; collisions are
dropped with a single WARN. A second WARN fires when a connector sets more
than ~10 custom keys, to surface cardinality risk.
- `StandardProcessGroup.setLoggingAttributes()` merges the owning connector's
keys and recurses into descendants, so every component inside the managed
flow inherits them without needing to know about Connectors.
- `ExtensionBuilder.buildConnector` (and the ghost path) now constructs
`SimpleProcessLogger` with a `StandardLoggingContext` bound to the
`StandardConnectorNode` via the existing `setComponent` late-bind path.
- `AbstractEventAccess.createProcessGroupStatus` snapshots
`group.getLoggingAttributes()` onto the status DTO.
`AbstractEventAccess.getConnectorStatuses()` walks the Connector registry and
returns one status per managed root.
+*Tests + docs*+
Unit tests for framework keys, propagation into inner components,
reserved-key filtering, `setName` updates, and the new `EventAccess` methods.
Short `administration-guide.adoc` section listing the new keys alongside the
NIFI-14991 documentation.
+*Delivery*+
Two PRs:
# `nifi-api`: BW-compatible additions only (default impls / optional DTO
field; no behavior change).
# `nifi`: Framework wiring + tests + docs.
was:
NIFI-14991 wired Process Group
identity into the SLF4J MDC for every component log, but it has no concept of a
`Connector`. As a result:
- Logs from inside a connector-managed flow carry `processGroupId/Name/Path`
but
nothing identifying the owning connector or its NAR bundle. The connector's
own
log lines emit no MDC at all (`ExtensionBuilder.buildConnector` constructs the
`ComponentLog` with the no-arg `StandardLoggingContext`).
- Reporting tasks that walk `EventAccess.getControllerStatus()` never see
connector-managed Process Groups — they have no parent and live in
`ConnectorRepository`, separate from the root group.
- A connector implementation has no way to surface custom logging context
(e.g. source schema, table namespace).
#
##
### What changes
*{*}nifi-api{*}*
- `ConnectorInitializationContext.setLoggingAttributes(Map<String, String>)` —
new `default` method; `AbstractConnector` provides a `protected` helper.
- `ProcessGroupStatus` — new `loggingAttributes` field (`Map.of()` default),
included in `clone()` / `merge()`.
- `EventAccess.getConnectorStatuses()` — new `default` returning empty;
implementations override.
*{*}Framework{*}*
- `StandardConnectorNode` holds a `loggingAttributes` map, implements
`GroupedComponent`, and is initialized with reserved framework keys:
`connectorId`, `connectorName`, `connectorComponent` (FQN, mirroring the
existing processor `component` convention), and
`connectorBundleGroup`/`Artifact`/`Version` from the bundle coordinate.
- Reserved keys cannot be overridden by `setLoggingAttributes`; collisions are
dropped with a single WARN. A second WARN fires when a connector sets more
than ~10 custom keys, to surface cardinality risk.
- `StandardProcessGroup.setLoggingAttributes()` merges the owning connector's
keys and recurses into descendants, so every component inside the managed
flow inherits them without needing to know about Connectors.
- `ExtensionBuilder.buildConnector` (and the ghost path) now constructs
`SimpleProcessLogger` with a `StandardLoggingContext` bound to the
`StandardConnectorNode` via the existing `setComponent` late-bind path.
- `AbstractEventAccess.createProcessGroupStatus` snapshots
`group.getLoggingAttributes()` onto the status DTO.
`AbstractEventAccess.getConnectorStatuses()` walks the Connector registry and
returns one status per managed root.
*{*}Tests + docs{*}*
Unit tests for framework keys, propagation into inner components,
reserved-key filtering, `setName` updates, and the new `EventAccess` methods.
Short `administration-guide.adoc` section listing the new keys alongside the
NIFI-14991 documentation.
#
##
### Delivery
Two PRs:
1. `nifi-api`: BW-compatible additions only (default impls / optional DTO
field; no behavior change).
2. `nifi`: Framework wiring + tests + docs.
> Extend existing MDC logging and reporting-task visibility to Connectors
> -----------------------------------------------------------------------
>
> Key: NIFI-15979
> URL: https://issues.apache.org/jira/browse/NIFI-15979
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, NiFi API
> Reporter: Kevin Doran
> Assignee: Kevin Doran
> Priority: Major
>
> NIFI-14991 wired Process Group identity into the SLF4J MDC for every
> component log, but it has no concept of a `Connector` (it pre-dates that
> feature). As a result:
> - Logs from inside a connector-managed flow carry `processGroupId/Name/Path`
> but
> nothing identifying the owning connector or its NAR bundle. The connector's
> own
> log lines emit no MDC at all (`ExtensionBuilder.buildConnector` constructs
> the
> `ComponentLog` with the no-arg `StandardLoggingContext`).
> - Reporting tasks that walk `EventAccess.getControllerStatus()` never see
> connector-managed Process Groups — they have no parent and live in
> `ConnectorRepository`, separate from the root group.
> - A connector implementation has no way to surface custom logging context
> (e.g. source schema, table namespace).
> +*What changes*+
> *nifi-api* (Covered by NIP-11)
> - `ConnectorInitializationContext.setLoggingAttributes(Map<String, String>)`
> —
> new `default` method; `AbstractConnector` provides a `protected` helper.
> - `ProcessGroupStatus` — new `loggingAttributes` field (`Map.of()` default),
> included in `clone()` / `merge()`.
> - `EventAccess.getConnectorStatuses()` — new `default` returning empty;
> implementations override.
> *NiFi Framework*
> - `StandardConnectorNode` holds a `loggingAttributes` map, implements
> `GroupedComponent`, and is initialized with reserved framework keys:
> `connectorId`, `connectorName`, `connectorComponent` (FQN, mirroring the
> existing processor `component` convention), and
> `connectorBundleGroup`/`Artifact`/`Version` from the bundle coordinate.
> - Reserved keys cannot be overridden by `setLoggingAttributes`; collisions
> are
> dropped with a single WARN. A second WARN fires when a connector sets more
> than ~10 custom keys, to surface cardinality risk.
> - `StandardProcessGroup.setLoggingAttributes()` merges the owning connector's
> keys and recurses into descendants, so every component inside the managed
> flow inherits them without needing to know about Connectors.
> - `ExtensionBuilder.buildConnector` (and the ghost path) now constructs
> `SimpleProcessLogger` with a `StandardLoggingContext` bound to the
> `StandardConnectorNode` via the existing `setComponent` late-bind path.
> - `AbstractEventAccess.createProcessGroupStatus` snapshots
> `group.getLoggingAttributes()` onto the status DTO.
> `AbstractEventAccess.getConnectorStatuses()` walks the Connector registry
> and
> returns one status per managed root.
> +*Tests + docs*+
> Unit tests for framework keys, propagation into inner components,
> reserved-key filtering, `setName` updates, and the new `EventAccess` methods.
> Short `administration-guide.adoc` section listing the new keys alongside the
> NIFI-14991 documentation.
> +*Delivery*+
> Two PRs:
> # `nifi-api`: BW-compatible additions only (default impls / optional DTO
> field; no behavior change).
> # `nifi`: Framework wiring + tests + docs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
