[ https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729343#comment-17729343 ]
ASF subversion and git services commented on NIFI-11464: -------------------------------------------------------- Commit 64b110c2929b9d9de4d354d0ecc05c8b9c0fb8dc in nifi's branch refs/heads/support/nifi-1.x from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=64b110c292 ] NIFI-11464 Improvements for importing nested versioned flows (#7218) * NIFI-11464 Improvements for importing nested versioned flows - Introduce FlowSnapshotContainer to return root snapshot + children - Introduce ControllerServiceResolver to extract logic from service facade - Update resolution logic to correctly consider all services in the hierarchy - Merge additional parameter contexts and parameter providers from child to parent - Add unit test for controller service resolver - Replace use of emptSet/emptyMap with new set/map instance > importing a versioned flow with a nested versioned flow shows nested > versioned flow with invalid controller services. > --------------------------------------------------------------------------------------------------------------------- > > Key: NIFI-11464 > URL: https://issues.apache.org/jira/browse/NIFI-11464 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning > Affects Versions: 1.21.0 > Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04) > Reporter: Patrick A. Mol > Assignee: Bryan Bende > Priority: Major > Attachments: exported_flow_versions_pretty.zip, > image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png, > nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip > > Time Spent: 1h 10m > Remaining Estimate: 0h > > When a flow (reusable_flow_Q) has controller services inherited from the > hierarchy (process group reusables) and a version of the flow is stored, the > flow version contains the references to these external controller services > (as seen in an exported flow version [see below]). > When this versioned flow is imported in another flow (tenant_flows) the > controller services need to be reset to the controller services in the new > hierarchy. > When we have a working flow with the nested versioned flow ready in > development we can check this flow into version control. > When we then deploy the flow in production, the nested versioned flow shows > up with invalid components. It shows the external controller service > identifiers as stored in the flow version. > When we then go back to development version of tenant_flows and make a minor > change to the nested versioned flow reusable_flow_Q and commit this change to > version control. > Due to this version change, we need to also commit the changes for the > tenant_flows process group. > When we now go back to production, and import this new version of > tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid > controller services. > If you have several flows under development using the same reusable > components, > you will likely end up with invalid components after import. > Depending on the amount of versioned flows used, it could be a lot of work. > It could also lead to issues when using the ExecuteStateless processor. > Please see attached template nested_version_flow_issue.xml for a starting > point to reproduce the issue. It contains the steps. > Screenprints are attached in a zip file show the process and diagnosis. > Controller services identifiers in version 2. > {code:java} > $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty > "controllerServices": [ > { > "identifier": "dc884171-4d75-3854-8604-afab91bd0e60", > "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904", > "name": "reusables_avro_reader", > "comments": "", > "type": "org.apache.nifi.avro.AvroReader", > "bundle": { > "group": "org.apache.nifi", > -- > }, > { > "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133", > "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd", > "name": "reusables_avro_writer", > "comments": "", > "type": "org.apache.nifi.avro.AvroRecordSetWriter", > "bundle": { > "group": "org.apache.nifi", > $ head -15 reusable_flow_Q-version-2.json.pretty > { > "externalControllerServices": { > "dc884171-4d75-3854-8604-afab91bd0e60": { > "identifier": "dc884171-4d75-3854-8604-afab91bd0e60", > "name": "reusables_avro_reader" > }, > "b512b238-cdee-3642-b5cb-0c98d30dd133": { > "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133", > "name": "reusables_avro_writer" > } > }, > "flowContents": { > "comments": "used to perform Q ...", > "componentType": "PROCESS_GROUP", > "connections": [ > {code} > Controller services identifiers with version 3 committed in process group > "tenant_flows". > {code:java} > pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C > 4 tenant_flow_avro tenant_flows-version-1.json.pretty > ], > "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d", > "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22", > "instanceIdentifier": "8f6cb319-0187-1000-b7fa-83340f7055f7", > "name": "tenant_flow_avro_writer", > "properties": { > "compression-format": "NONE", > "Schema Write Strategy": "avro-embedded", > "schema-name": "${schema.name}", > -- > ], > "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d", > "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807", > "instanceIdentifier": "8f6c8a94-0187-1000-af54-2fee12838618", > "name": "tenant_flow_avro_reader", > "properties": { > "schema-name": "${schema.name}", > "cache-size": "1000", > "schema-access-strategy": "embedded-avro-schema", > pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ head -15 > reusable_flow_Q-version-3.json.pretty > { > "externalControllerServices": { > "8ff96d88-3dc8-30ed-aeb8-757c26a7b807": { > "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807", > "name": "tenant_flow_avro_reader" > }, > "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22": { > "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22", > "name": "tenant_flow_avro_writer" > } > }, > "flowContents": { > "comments": "used to perform Q ...", > "componentType": "PROCESS_GROUP", > "connections": [ > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)