[ 
https://issues.apache.org/jira/browse/NIFI-13724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885403#comment-17885403
 ] 

Dye357 commented on NIFI-13724:
-------------------------------

Sure thing Bryan! I didn't specifically create a ticket because the following 
may be unique to our use case but here is what I found:

What I attempted to do was use the nifi-registry-client within an event-hook 
implementation I was providing nifi registry via an extension. So in other 
words I was running my custom extension in nifi-registry, I was using the 
nifi-registry-client to gather additional metadata about the generated event 
(see my PR) to enrich the event and then dispatch the event onto an event bus. 
(Yeah, it's kinda awkward to use the rest client within the extension to get 
additional metadata about flows through the registry rest api) The problems I 
referred to were as follows:

On Nifi 1.24.0 I discovered a very annoying issue with Javax & the other Java 
EE libraries. Because those libraries are available on the registry's classpath 
via the nifi-registry-web-api in a shaded jar. {*}A java Linking exception is 
thrown when instantiating the nifi-registry-client within the extensio{*}n. My 
research concluded this is due to some little-known issues with shading Java EE 
libraries. At first the javax rs-api was the primary culprit. I was able to get 
around the issue by switching out javax for a different implementation 
(jakarata) - the reason this works is because Java doesn't have to compare if 
the provided impl is the same class as the existing class on the registry's 
classpath.

With JavaX settled I thought I was in the clear and then discovered a secondary 
issue with Glassfish. Glassfish uses an DI framework called H2 which when the 
registry-client is instantiated resolves various dependencies for doing the 
JSON serialization/deserialization. The same problem exists here because those 
Glassfish classes already exist on the classpath of the nifi-registry-web-api 
and the linking exception is thrown. 

I attempted this with both shadow and non-shadow extension jars.

I tried this on Java 11 and Java 17.

I attempted shading the Java EE dependencies and doing package re-location 
which breaks GlassFish because there are hard-coded classpath coordinates in 
the H2 code so relocation is not possible.

I would not expect to see any problems with the CLI - If I invoked the CLI via 
a JNI call or something I expect this would have worked. 

I did not test this with 2.x but the solution I put in the PR should be just 
fine on 2.x and 1.x.

I'm not an expert on the Java EE and Jakarta stuff but I believe a *potential* 
resolution to this issue would be to not shade those libraries into 
registry-web-api war but I haven't done anything to test that. 

Fundamentally the issue is that the events Nifi Registry creates don't include 
enough metadata about the entities being created/modified/deleted so the PR 
should make it easier for folks to do things like put these events on a service 
bus (Like SNS) to generate a notification that something changed. Also the 
implementation of us using the nifi-registry-client was rather poor because if 
a delete event occurs we can't query the registry web api to get the 
bucket/flow name because it doesn't exist anymore so the PR also eliminates 
timing issues for accountability.

 

> Nifi Registry Events Include Additional Metadata
> ------------------------------------------------
>
>                 Key: NIFI-13724
>                 URL: https://issues.apache.org/jira/browse/NIFI-13724
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: NiFi Registry
>    Affects Versions: 1.27.0, 2.0.0-M4
>            Reporter: Dye357
>            Assignee: Dye357
>            Priority: Minor
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently the Nifi Registry EventFactory adds a few pieces of metadata to the 
> EventType - when writing a custom extension for Nifi Registry a developer is 
> limited in the amount of information they have about the event – this ticket 
> would expand the scope of metadata included in each event to include the 
> Name, Description and Timestamps. 
> As a work-around we were using the nifi-registry-client to use the provided 
> IDs to enrich our dataset however we've run into problems:
> 1. The nifi registry client is broken when running inside an Nifi Registry 
> extension due to JavaX/Jakarta classpath issues. (Separate ticket)
> 2. This the most important of the two points – if we have a Bucket Deleted or 
> Flow Deleted event only having the ID is useless because we can no longer 
> query the API to get the name and description for the Entity in question 
> which has led to some auditing/accountability concerns. 
> I'm putting together a PR to include the additional specified metadata in the 
> EventFactory. The PR should be merge-able on 1.x and 2.x



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to