Thanks guys, that makes sense in the context of the review model. Obviously some kind of general pre-event hook is the most flexible, but I would say we should also consider whether we really want to be calling out to arbitrary scripts during the main API requests, as well as consider what someone would have to do to implement a scripted rule. The current events are all metadata related, they don't have the content of the snapshots, so in this case we'd have to pass the entire json string of a snapshot as an arg to the script and then the script has to parse through it looking for fields to validate/check. I wonder if we should consider defining the types of rules ourselves and allowing some kind of rules config file to be provided. This way the rules are loaded during start up and applied in Java code by our framework code. Downside is it is less flexible in that we probably won't be able to know every rule someone wants ahead of time. I guess the other options are to consider whether tagging or nifi side rules are better suited to solve this problem.
On Wed, Jan 17, 2024 at 10:48 AM Simon Bence <[email protected]> wrote: > > HI, > > I think, both of you are correct, my initial example of the possible usage > was not the best. Having a control over committing versions for flows is > maybe the most important benefit we can gain. Contrary to naming conventions > this can be a more complex endeavour. So I suggest moving on with this > example instead of the original one. > > Regards, > Bence > > > On 2024. Jan 17., at 16:02, Pierre Villard <[email protected]> > > wrote: > > > > I guess the issue relates to some recurrent discussions around the lack of > > mechanism for a Pull Request kind of model for "approving" a versioned > > flow. An alternative we've been discussing for a long time is the ability > > to set tags to versions so that a versioned flow could be reviewed (diff > > with previous version(s)) and if approved a tag would be applied and this > > tag would be used to potentially trigger an automatic deployment. > > > > The proposal is not helping with the "PR model" but would allow someone to > > control what is versioned. A user may have the permissions to commit a new > > version but one may not want to accept a flow where a processor is > > configured with 50 concurrent tasks (for example). That would get deployed > > in prod and potentially impact existing deployed flows. I guess one could > > see this approach as a "checkstyle" plugin to allow or not the commit of a > > new version. > > > > Le mer. 17 janv. 2024 à 18:56, Bryan Bende <[email protected]> a écrit : > > > >> I think before we embed something into synchronous API requests we > >> should try to define a few more real uses besides just a bucket naming > >> pattern to see if we really need to add something like this. For > >> bucket naming pattern, it could just be a simple property in > >> nifi-registry.properties with a regex to check names against. If we > >> did add some kind of pluggable validator, I don't think these > >> validators should be making authorization decisions, it should be > >> strictly for rules about the content being version controlled. If we > >> are worried about a user version controlling something that interferes > >> with a CI/CD pipeline, that seems to imply our current authorization > >> policy model needs improvement, or it is not being used correctly > >> (i.e. why does a user have write to a bucket they shouldn't be writing > >> to?). > >> > >> On Wed, Jan 17, 2024 at 8:53 AM Pierre Villard > >> <[email protected]> wrote: > >>> > >>> I do think this would be a good idea. That would provide users of the > >> NiFi > >>> Registry a way to implement custom conditions on what can be versioned or > >>> not based on their requirements. Right now if one has write permissions > >> for > >>> a bucket/flow, you could commit anything as a new version which could > >>> potentially be an issue when using CI/CD pipelines to automate > >> deployments. > >>> > >>> As a side-note, another improvement could be to prevent a flow from being > >>> versioned in the registry if a required rule is violated (I'm talking > >> about > >>> the new feature coming with NiFi 2.0). Just a thought for an additional > >>> improvement. > >>> > >>> Overall I do think that giving options to users for better controlling > >> what > >>> is being versioned is good. > >>> > >>> Le mer. 17 janv. 2024 à 17:13, Simon Bence <[email protected]> a > >>> écrit : > >>> > >>>> Hi, > >>>> > >>>> I recently found the need for custom validation to maintain NiFi > >> Registry > >>>> content. This includes checks such as enforcing naming conventions when > >>>> creating a Bucket and similar usage specific cases. While exploring the > >>>> Registry's codebase, I came across the EventHookProvider, which aligns > >> with > >>>> a similar concept. However, it does not cover the case here due to its > >>>> asynchronous nature and being a "post-event" activity. > >>>> > >>>> Although the EventHookProvider is not suitable for this specific need, > >> I > >>>> find the Event construct and the "whitelist" concept pretty overlapping > >>>> with my objectives. Consequently, I propose the addition of a new type > >> of > >>>> Provider covering for "pre-event" validation, operating in a manner > >> similar > >>>> to the EventHookProvider: a call from the request methods to the set of > >>>> providers but filtered using a whitelist. Similarly to the mentioned > >>>> provider, I believe an implementation capable of executing scripts > >> (akin to > >>>> ScriptEventHookProvider) would be a good starting point, to cover a > >> most > >>>> use cases. > >>>> > >>>> I am keen to hear your opinion on this proposal and welcome any further > >>>> ideas. Thank you for your consideration! > >>>> > >>>> PS.: using the "event" term comes from the already existing > >>>> EventHookProvider. In practice these are the methods of the Registry > >> web > >>>> API. > >>>> > >>>> Regards, > >>>> Bence Simon > >> >
