[
https://issues.apache.org/jira/browse/NIFI-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088356#comment-18088356
]
ASF subversion and git services commented on NIFI-15880:
--------------------------------------------------------
Commit 6668f104397a5983acc9d53415a08259d21282b6 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6668f104397 ]
NIFI-15880: Allow Connectors to enter a Troubleshooting mode (#11175)
* NIFI-15880: Allow Connectors to enter a Troubleshooting mode where the flow
becomes modifiable and user-managed (temporarily) rather than Connector managed.
NIFI-15880: Surface why END_TROUBLESHOOTING is unavailable in the allowable
action; fix findProcessGroup for connector child groups
- Refactored verifyCanEndTroubleshooting to delegate to a new
getReasonCannotEndTroubleshooting() method that returns Optional<String>.
The verify method throws if the Optional is present;
createEndTroubleshootingAction
uses the same method to populate the reason field on the allowable action so
callers see why the action is unavailable without having to attempt it.
- Replaced the throwing verifyAllComponentsStoppedAndDisabled helper with
findReasonComponentsNotStopped, which returns the first reason found as an
Optional<String>. The expensive flow-revert preflight only runs after the
cheap component-state scan passes.
- Fixed StandardProcessGroup.findProcessGroup to use the unconditional
flowManager.getGroup(id) instead of the connector-ID-filtered overload.
Only the root managed process group carries a connectorId; child groups do
not, so the filtered lookup always returned null for non-root groups,
causing a 404 when navigating into a child group in Troubleshooting mode.
The existing isOwner check already enforces hierarchy ownership.
NIFI-15880: Add system test for fetching a non-root child process group via the
Connector flow endpoint
Adds testGetFlowForChildProcessGroupInTroubleshooting to
ConnectorTroubleshootingIT to verify that the Connector's
/flow/process-groups/{groupId} endpoint returns the correct flow for a
child process group within the managed flow while in Troubleshooting mode.
NIFI-15880: Fix compile errors in nifi-framework-core after nifi-api rebuild
- Use fully-qualified org.apache.nifi.controller.ScheduledState for local
variables in findReasonComponentsNotStopped to resolve the naming conflict
with the imported org.apache.nifi.flow.ScheduledState.
- Pass ConnectorSyncMode.LOCAL_ONLY to ConnectorRepository.getConnectors()
in FlowController.findInputPortIncludingConnectorManaged and
findOutputPortIncludingConnectorManaged; the no-arg overload was removed
from the interface and these internal lookups do not need a provider sync.
NIFI-15880: Add getActiveFlow override to test connector stubs
DependentSecretVerifyConnector, RequiredSecretConnector, and
TrackingConnector did not implement the new abstract
Connector.getActiveFlow(FlowContext) method added to nifi-api.
Add a stub returning null to each, since none of the tests
exercised that method.
NIFI-15880: Pass ConnectorSyncMode to ConnectorRepository.getConnectors in
component DAOs
After rebasing onto main (NIFI-15919), ConnectorRepository.getConnector
and ConnectorRepository.getConnectors now require a ConnectorSyncMode.
Update the connector-managed component lookups added to StandardFunnelDAO,
StandardInputPortDAO, StandardLabelDAO, StandardOutputPortDAO, and
StandardRemoteProcessGroupDAO to pass ConnectorSyncMode.LOCAL_ONLY.
These are internal walks across in-memory connector state to locate a
component by id and have no need to consult the configuration provider.
NIFI-15880: Fix broken assertion in testEndTroubleshootingBlockedByQueuedData
assertEquals(-1, getQueuedCount(...)) compared an int against an
OptionalInt, so the comparison was never meaningful. Replace it with
assertFalse(getQueuedCount(...).isPresent(), ...) which expresses the
intended check: after ending Troubleshooting the connection added during
Troubleshooting must no longer exist in the managed flow.
NIFI-15880: Address Connector troubleshooting review feedback
- Replace boolean includeConnectorManaged overloads on DAOs and
AuthorizableLookup with new ConnectorManagedComponentLookup and
ConnectorManagedAuthorizableLookup facades to keep
connector-managed access off the primary lookup APIs.
- Add ProcessGroup#findOwningConnector default implementation.
- Deprecate one-arg VersionedComponentFlowMapper#mapConnector for
removal to flag the latent footgun for callers.
- Add ConnectorRepository#verifyEndTroubleshooting and a matching
ConnectorConfigurationProvider hook so the repository runs
verification symmetrically with enterTroubleshooting and notifies
third-party providers of the transition.
- Fix Javadoc typo and ScheduledState reference on ConnectorRepository.
- Add ClusteredConnectorTroubleshootingIT to run the troubleshooting
suite against a two-node cluster, and add restart helper that waits
for all nodes to reconnect to avoid races against post-restart
cluster rejoin.
NIFI-15880: Remove unused one-arg VersionedComponentFlowMapper#mapConnector
The single-arg overload had no remaining callers after the
two-arg form replaced it; it is framework-internal and not part
of any public API, so delete it rather than carry it as a
deprecated method.
NiFi API 2.9.0
NIFI-15148 Add service support for Branch creation in Registry Clients (#11072)
NIFI-15880: Allow scheduling components and controller services in
Connector-managed child groups
scheduleComponents, enableComponents, and activateControllerServices on
StandardProcessGroupDAO used the connector-blind ProcessGroup lookup. As a
result, schedule/enable/activate requests targeting a child Process Group
inside a Connector's managed flow failed with 404 (or 400 for the activate
path) even when the Connector was in TROUBLESHOOTING. Route all three through
locateProcessGroup(..., includeConnectorManaged=true) so the lookup succeeds
for managed child groups and the existing connector-state guard at the REST
layer continues to authorize access. Drop the now-unreachable null check in
activateControllerServices since locateProcessGroup throws ResourceNotFound
when the group does not exist.
Add system tests that drive the activate and schedule/enable REST endpoints
against a child Process Group inside a TROUBLESHOOTING Connector's managed
flow. The tests run in both standalone and clustered mode through the
existing ClusteredConnectorTroubleshootingIT subclass.
* NIFI-15880: Remove duplicate imports in StandardNiFiServiceFacadeTest
PMD's UnnecessaryImport rule flagged two duplicate imports
(FlowRegistryDAO and TestingAuthenticationToken) introduced when
ConnectorManagedComponentLookup was added to the test. Collapse each
pair to a single import to make the Scan workflow pass.
* NIFI-15880: Use FlowManager.getGroup() in StandardControllerServiceProvider
to resolve connector-managed groups
* NIFI-15880: Skip expensive end-troubleshooting preflight on entity GET; add
cluster-reconnect sync test
createEndTroubleshootingAction is invoked on every REST GET of the Connector
entity. The previous implementation ran the full flow-revert preflight on every
call, which resolves the authoritative flow from the Connector plugin and runs
a flow-comparison that becomes expensive for large managed flows. Split the
check into a quick variant (state + components stopped) and the full variant.
The quick variant feeds the Allowable Action; the full variant continues to
run from verifyCanEndTroubleshooting (and therefore from the verify REST
endpoint and endTroubleshooting itself).
Add a clustered system test that disconnects a node while a Connector is in
TROUBLESHOOTING and reconnects it. The path that runs syncConnector against an
existing in-memory TROUBLESHOOTING Connector (including the setName call) is
not covered by the existing restart-based tests, which create a fresh
STOPPED Connector before sync restores TROUBLESHOOTING.
* NIFI-15880: Use FlowManager directly in ControllerFacade status lookups
The status/status-history/status-analytics methods on ControllerFacade looked
up Connectors, Processors, Connections, and Process Groups by walking the root
Process Group tree (findX) and, for getConnectionStatus, falling back to
FlowManager only when the recursive walk returned null. The FlowManager
already indexes all of these components in O(1) maps and includes the entries
registered by Connector-managed Process Groups (StandardProcessGroup invokes
the onXAdded hooks unconditionally). Switch the affected lookups to call
FlowManager directly and drop the redundant fallback in getConnectionStatus.
Remote Process Groups are left as-is because FlowManager has no direct
getRemoteProcessGroup(id) accessor today.
* NIFI-15880: Apply review feedback to ConnectorTroubleshootingIT
Assert exact queued FlowFile count survives restart, use getFirst(),
exercise GET/POST while not in Troubleshooting and GET/PUT/POST/DELETE
while in Troubleshooting, filter Stateless Process Groups instead of
swallowing scheduling exceptions, fold Processor coverage into the
managed-flow component access test, extract a managedGroupId local,
trim verbose Javadocs, drop "the fix" / historical references, add
blank lines between back-to-back blocks, and replace the ConflictingCall
and FlowClientCall inner functional interfaces with
java.util.concurrent.Callable.
* NIFI-15880: Address review feedback from kevdoran
- Changed troubleshooting methods in StandardConnectorDAO to use
LOCAL_ONLY sync mode instead of SYNC_WITH_PROVIDER, consistent
with other mutation methods in the class.
- Fixed endTroubleshooting error message to use e.getMessage()
instead of e.toString() to avoid duplicating the exception class
name in the user-facing message.
- Routed verifyDeleteConnector through ConnectorDAO and
ConnectorRepository instead of calling ConnectorNode.verifyCanDelete()
directly, mirroring the pattern used by verifyCreate and providing
a hook point for ConnectorConfigurationProvider consultation.
* NIFI-15880: Fix action history failing for connector-managed components
The Flow Configuration History page (GET /nifi-api/flow/history) was
returning a 409 Conflict when any action history entry referenced a
component inside a connector-managed flow whose connector was not in
Troubleshooting mode. The authorizeAction method tried to look up the
component through the standard DAO path, which calls
verifyAccessibleForComponentOperation and throws IllegalStateException.
Introduced ConnectorManagedAccessException (extends IllegalStateException)
that carries the owning Connector's identifier. When authorizeAction
catches this exception, it resolves the owning Connector through
authorizableLookup.getConnector() and authorizes the action against
the Connector rather than the inaccessible component.
> Implement a Troubleshooting mode for Connectors - Framework
> -----------------------------------------------------------
>
> Key: NIFI-15880
> URL: https://issues.apache.org/jira/browse/NIFI-15880
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Allow a user to switch the ownership of a Connector's flow between
> Connector-owned and user-owned temporarily for troubleshooting purposes.
> See NIP-28 for details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
