[
https://issues.apache.org/jira/browse/NIP-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bob Paulin updated NIP-36:
--------------------------
Summary: Lockable Connector Properties (was: Add immutable option to
ConnectorPropertyDescriptor)
> Lockable Connector Properties
> -----------------------------
>
> Key: NIP-36
> URL: https://issues.apache.org/jira/browse/NIP-36
> Project: NiFi Improvement Proposal
> Issue Type: Improvement
> Reporter: Bob Paulin
> Priority: Major
>
> h2. Motivation
> Some Connector configuration properties describe decisions that cannot be
> safely changed after the Connector has started producing data. Typical
> examples:
> * The output format written to a destination. Once objects have been written
> to an S3 bucket as Parquet, silently switching the property to Avro produces
> a bucket whose contents can no longer be read by a single reader.
> * The physical destination identifier. The S3 bucket, Kafka topic, or
> object-store prefix that the Connector writes into. Changing this after data
> has been written silently forks the destination and leaves the previously
> written data orphaned.
> * The identity used to track source progress. The Kafka consumer group id or
> source offset-tracking key. Changing this after commit either re-reads
> already-processed data or leaves committed offsets behind, in both cases
> producing duplicates or gaps.
> Today there is no first-class way for a Connector author to declare that such
> a property is frozen after the Connector has started. The intent lives only
> in ad hoc validation code inside each Connector, is not exposed to the REST
> API, is not visible in the extension manifest, and is not visible in the UI.
> Users can freely edit these fields in the configuration dialog and see a
> generic error only when they attempt to apply the change, if the Connector
> happens to defend it at all.
> This proposal makes _"once the Connector has been started at least once, this
> property is frozen at its current value"_ a first-class descriptor attribute,
> enforced by the framework, surfaced through the REST API and extension
> manifest, and rendered as read-only in the UI so the user never has the
> opportunity to make an unappliable change in the first place.
> h2. Scope
> Out of Scope:
> Immutable Properties for Processors or Controller Services are not part of
> the scope of this NIP. Processors or Controller Services have done without
> this feature and have implemented ways to handle property value changes
> within the component code. Connectors integrate an entire flow worth of
> components together so trying to force connectors to handle any possible
> connector property change may not always be possible.
> {{nifi-api}} additions:
> * {{ConnectorPropertyDescriptor.isImmutable()}} and
> {{{}ConnectorPropertyDescriptor.Builder.immutable(boolean){}}}, defaulting to
> {{{}false{}}}. {{Builder.from(other)}} copies the flag.
> * {{VersionedConnector.hasBeenStarted}} ({{{}boolean{}}}) with
> getter/setter, serialized alongside the Connector's other flow-level metadata.
> * {{XmlConnectorDocumentationWriter.writeConnectorProperty(...)}} emits a
> new {{<immutable>}} element so the extension manifest exposes the flag.
> Framework additions:
> * New methods on {{{}org.apache.nifi.components.connector.ConnectorNode{}}}:
> {{boolean hasBeenStarted()}} and {{{}void setHasBeenStarted(boolean){}}}.
> * {{StandardConnectorNode}} sets {{hasBeenStarted = true}} after every
> successful transition to {{{}RUNNING{}}}, and rejects changes to any
> immutable property in {{setConfiguration(...)}} and
> {{replaceWorkingConfiguration(...)}} once the flag is set. Comparison is done
> on the raw {{ConnectorValueReference}} instances so asset and secret
> references are compared by identity, not by their resolved values.
> * {{VersionedComponentFlowMapper.mapConnector(...)}} persists the flag;
> {{StandardConnectorRepository.createConnectorNode(VersionedConnector)}}
> restores it via {{{}ConnectorNode.setHasBeenStarted(...){}}}. The flag
> survives restarts, flow imports, and Connector-version upgrades.
> Model / DTO / manifest plumbing:
> * {{ConnectorPropertyDescriptorDTO}} (nifi-client-dto) gains an
> {{immutable}} field, populated by
> {{{}DtoFactory.createConnectorPropertyDescriptorDto(...){}}}.
> * {{ConnectorDTO}} (nifi-client-dto) gains a {{hasBeenStarted}} field,
> populated by {{DtoFactory}} from {{{}ConnectorNode.hasBeenStarted(){}}}, so
> that any client can determine whether a Connector's immutable properties are
> currently locked.
> * {{org.apache.nifi.extension.manifest.ConnectorProperty}} and
> {{org.apache.nifi.c2.protocol.component.api.ConnectorPropertyDescriptor}}
> gain the same {{immutable}} field;
> {{StandardRuntimeManifestBuilder.getConnectorPropertyDescriptor(...)}} copies
> it through.
> UI additions ({{{}nifi-frontend{}}}):
> * The Connector property-editor components consume {{descriptor.immutable}}
> together with {{{}connector.hasBeenStarted{}}}. When both are true, the
> corresponding form control is rendered as disabled and no editable value
> control is offered.
> * A locked visual indicator (padlock icon plus tooltip text) is displayed
> next to the disabled control, explaining that the property was fixed at the
> value from the Connector's first successful start and cannot be changed..
> h2. Description
> Descriptor contract.
> A property built with {{.immutable(true)}} behaves identically to a
> non-immutable property until the Connector transitions to {{RUNNING}} for the
> first time. Attempts to change it before that point succeed, which preserves
> normal initial configuration, verification, and staged flow deployment.
> Immutability begins the moment {{StandardConnectorNode}} sets
> {{{}hasBeenStarted = true{}}}, and from then on the value locked in is
> whatever was in the active configuration at the point of transition.
> Enforcement point.
> The check lives in a single private helper —
> {{StandardConnectorNode.verifyImmutablePropertiesUnchanged(stepName,
> incoming, replace)}} — called from the two mutation entry points a user
> request can reach:
> * {{setConfiguration(String, StepConfiguration)}} — the merge path used by
> REST configuration updates and by {{{}ConnectorTestRunner.configure(...){}}}.
> * {{replaceWorkingConfiguration(String, StepConfiguration)}} — the
> whole-step replacement path used by configuration providers.
> Restore paths (flow deserialization, snapshot inheritance,
> {{{}recreateWorkingFlowContext{}}}) deliberately bypass the check because
> they must be able to reconstruct the Connector's existing state without the
> framework second-guessing itself. A violating change throws
> {{FlowUpdateException}} with a message that names the offending property, the
> step, and the Connector.
> Persistence.
> {{hasBeenStarted}} is stored on {{{}VersionedConnector{}}}, not on the
> descriptor. On save, {{VersionedComponentFlowMapper}} reads
> {{{}connectorNode.hasBeenStarted(){}}}. On load,
> {{StandardConnectorRepository}} reads the field off the
> {{VersionedConnector}} and calls {{setHasBeenStarted(true)}} before the
> Connector is exposed to any user request, so enforcement is active
> immediately after restart. Because the flag lives on the Connector and not on
> the descriptor, values locked under Connector v1 remain locked under
> Connector v2 even if v2 changes the descriptor set: whether a property is
> immutable is decided by the current v2 descriptors, and _"the Connector has
> been started"_ is decided by the persisted flag.
> Backwards compatibility.
> {{isImmutable()}} defaults to {{{}false{}}}, so existing Connectors are
> unaffected. {{VersionedConnector.hasBeenStarted}} defaults to {{false}} in
> the JSON representation, so flows saved by older frameworks come back
> unlocked, which is the same behaviour as today. The new DTO / manifest / c2
> fields are additive, and building an older Connector against the new
> {{nifi-api}} continues to compile because both new descriptor methods have
> default implementations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
