That is accurate. Generally the intention isn't really to extend processors across NARs. It is better to create a shared module with a base class that both processors can extend from, or utility code they can both call, but in the case that shared module doesn't exist then the main option is to copy over the code you need.
On Wed, Oct 17, 2018 at 2:52 PM John McGinn <figgie...@sbcglobal.net> wrote: > > Thanks Bryan. > > That was a mistake on my part. I thought that I could use part of the NiFi > standard processors in my custom processor. I suppose this also means it's > not possible to extend a standard processor class, say Wait, and override a > method, for instance, onTrigger(), to do different work. Instead, as you > state, I should copy the portions I would need to my custom processor. Is > that accurate? > > In the end, as I was looking through how to modify Wait, I think it is bigger > than the need that I have, so I restarted with a new generated processor > project, and just added the cache service, and appropriate relationships, and > simple logic to control flow through my processor. I'll work on getting a > github account and uploading my simple processor. > > Thanks for the assistance > John McGinn > > -------------------------------------------- > On Tue, 10/16/18, Bryan Bende <bbe...@gmail.com> wrote: > > Subject: Re: Compiling custom processor > To: figgie...@sbcglobal.net > Date: Tuesday, October 16, 2018, 4:45 PM > > You shouldn't really try to > depend on code inside other NARs like > standard processors, you'll want to copy > WaitNotifyProtocol into your > own NAR. > On > Tue, Oct 16, 2018 at 4:40 PM John McGinn <figgie...@sbcglobal.net> > wrote: > > > > The > commons-lang3-3.7.jar file is located in the lib/bootstrap > directory, therefore, I figured it could be considered > provided. I've since change it to normal scope, and it > is included in my NAR and that error goes away. So, this > > then leans on the next error I get, which > is no class found for the WaitNotifyProtocol in > nifi.processors.standard. Is there an easy way to figure out > what NiFi class/object I'm using, and what NAR should be > listed as provided? I tried to add > nifi-standard-processors-nar, but that doesn't exist for > maven, nor does it exist in the lib directory. I looked > inside the nifi-standard-nar NAR file, and the > nifi-standard-processors is there, and inside that is > WaitNotifyProtocol, but yet, my maven won't build if I > have nifi-standard-nar listed, either with type of NAR, or > scope of provided. > > > -------------------------------------------- > > On Tue, 10/16/18, Bryan Bende <bbe...@gmail.com> > wrote: > > > > Subject: > Re: Compiling custom processor > > To: dev@nifi.apache.org, > figgie...@sbcglobal.net > > Date: Tuesday, October 16, 2018, 4:01 > PM > > > > Any reason > why the commons-lang3 > > dependency is > provided and not normal > > compile > scope? > > > > Generally > the only cases where a dependency > > > should be provided are > > when a parent > NAR > > provides it (the API scenario > earlier), or when it > > is a jar > directly in lib which means it is on > > > the classpath of > > everything (like > nifi-api > > or slf4j). > > On Tue, Oct 16, 2018 at 3:47 PM John > McGinn > > <figgie...@sbcglobal.net> > > wrote: > > > > > > I have a > > > directory /home/user/nifi-nars, which is listed in the > > nifi.properties file located under the > nifi-assembly > > directory from the > nifi-master code. That's where I > > > redropped my NAR file at. > > > > > > The error was "due to > uncaught > > Exception: > java.lang.NoClassDefFoundError: > > > org/apache/commons/lang3/StringUtils". I added a > > provided dependency of version 3.4 for > commons-lang3 in my > > processors > pom.xml. That worked initially, but with the > > redropping of the NAR file, the error > occurs. The stacktrace > > points to the > session.get() lambda function trying to use > > StringUtils.isBlank(). (Again, this is a > copy and renamed > > class of the Wait > Processor. Trying to turn it into a Yield, > > as it were, hold unless it's free to > move forward.) > > > > > > -------------------------------------------- > > > On Tue, 10/16/18, Bryan Bende > <bbe...@gmail.com> > > wrote: > > > > > > Subject: > > > Re: Compiling custom processor > > > > To: dev@nifi.apache.org > > > Date: Tuesday, October 16, 2018, > 3:34 > > PM > > > > > > > There > > shouldn't be any issue > > > doing > > > that, generally you just copy the > > > > > > new > > > > NAR into the lib directory and > > > restart. > > > > > > > What > > were the specifics of the > class not found > > > error? > > > > > > On > Tue, Oct 16, 2018 at > > > 3:24 PM > John McGinn <figgie...@sbcglobal.net> > > > wrote: > > > > > > > > > And > I'm > > > > > > back. > > > > > > > > > > So, I > had my > > > processor running, > > no errors, so things were good. > > > > > > > > > > I then add 2 if > > > statements with logger messages, > run the > > same mvn command, > > > copied the new NAR > > on top of the old one, same versionId > and > > > all, and now I get a class > not found > > error, that didn't > > > occur before I > > stopped, redeployed and started. And > I > > > can't delete the event > in the queue > > even though source > > > and destination > > are stopped. > > > > > > > > > Is there > problems when redeploying > > a > > > custom processor NAR on top of > > itself? Even if the NiFi > > > instance is > > stopped? Serialization issues? > > > > > > > > > > Frustratedly > > > yours, > > > > > John > > McGinn > > > > > > > > > > > > > > > > > > -------------------------------------------- > > > > On Tue, 10/16/18, John > McGinn > > <figgie...@sbcglobal.net> > > > wrote: > > > > > > > > > Subject: > > > > > Re: > Compiling custom processor > > > > > > > To: dev@nifi.apache.org > > > > Date: Tuesday, October 16, > 2018, > > 1:21 > > > > PM > > > > > > > > > > > Thanks > > > Bryan, > > > > > > > > > I > > > > > restarted > with a fresh maven > > > > > > > generate, and only added the API > to the > > processor pom, > > > and > > > > > then the API-NAR to the NAR > > > pom. > > > I then figured out a > > > > subsequent > > > > > dependency > (lang3), and got that in, and now my > > > > > processor shows up in local > > NiFi > > > > instance. > > > > > > > > > > > > > > > > Regarding > the copying of the standard > > > > > > > > > > processor, when I copied Wait > > over, I > renamed it from > > > Wait > > > > to MyProcessorWait to > be > > unique, > > > > and not copied. I'm > > > just > > > > > > > > > attempting > changes to Wait without modifying the actual > > > one. > > > > > Also, > > thanks on the > > > META-INF/services file > > information. > > > > > > > > > Now I can > debug and test. > > > > > > > > > > > Sincerely, > > > > John > McGinn > > > > > > > > > > > > > > > -------------------------------------------- > > > > On Tue, 10/16/18, Bryan > Bende > > <bbe...@gmail.com> > > > > wrote: > > > > > > > > > > Subject: Re: Compiling > > custom > > > > > processor > > > > To: > dev@nifi.apache.org > > > > Date: Tuesday, October > 16, > > 2018, > > > > 9:52 > > > > > > AM > > > > > > > > > > > > In general, if your processor > > > > > > > uses a controller > service then the > > > > > processors pom file needs a > > > provided > > > > dependency > > > > on > > > > the > > API of the CS, > > > > and your NAR > > pom > > > > needs > > > > > > > a NAR dependency on the NAR > > > > > > where the > > > > > CS API > > > > > is. > > > > > > > > > > > Example is > > > > > > > > shown here > > in the section > linking > > > > > > > processors and > > > > > > > controller > > > > services > - > > > https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions > > > > > > > > > > > Also, you do not > > > want to > > > > > include NiFI's standard > > > processors in > > > > > > > your > > > > own NAR, this > > will result in two > > > > copies > > of > > > > > every > > > standard processor. > > > > You will want > > > to > > > > > copy > > whatever code you need > into > > > your > > > > own NAR. > > > > > > > > > > In you last statement, > > if you > > > > > made a new processor > > in > > > > > nifi-standard-processors > > and it > > > didn't > > > > > show > > > > > > up, it is most likely > > > > > > > because you didn't > > > > > update > > META-INF/services file to > > > > > > > include the new > > > > > > > processor. > > > > > > > On Tue, Oct 16, 2018 > > > at > > > > > 9:36 AM John McGinn > > > > > <amruginn-n...@yahoo.com.invalid> > > > > wrote: > > > > > > > > > > > > > Ok, I am > > > > > > > > far from a > > maven expert, and > am > > > > > > > > struggling on this > > > > > problem. > > > > > > > > > > > > > I > > > > created > > > > a > > new project using the > > > maven > > > generate > > > > process, > and > > > compiled > > > > > > that > sample processor and > > > > > everything > > > > > was fine, and I > > could > > > > see it in my local > NiFi > > instance. I > > > > then copied over > > the > > > > > Wait processor on top of > > the > > > > MyProcessor.java class, > > changed > > > > > the package name, and > > the > class > > > name, > > > > and attempted to > > > > compile. Got errors due > to > > > dependency > > > > > > > issues. This is where > > > > > I get > > confused. > > > > > > > > > > > > > > > > > In > > > > > > > the > > > processor directory, I modify the > > > > > > > > > > > pom.xml with a > > > > > dependency line for > > > > nifi-api, > > > > > > nifi-utils, > > > > > > > > > > > > > > > nifi-distribute-cache-client-service-api and > > > > > nifi-standard-processors. > > This > > > seemed > > > > > > > to make things work, > > > > > > and I > get a 33 meg NAR file, which > > > > > > contains lots of JARs. I > > > > > > load > that up, and I get a problem > > > > with > > > > > > CalculateStats, or > > > > similar, > > because of > > > > > > > RecordReaderFactory. I go back > in and > > > > > > > include > > > > > > > > nifi-record-serialization-services, and > > it compiles, > > > > > and the JAR is > > > included in the NAR > > > > > file now, > > and I still get > > > > class not > > found. > > > > > > > > > > > > > > > > > > > > > > > > > I've > > also tried to change > the > > > > > > > dependencies to a status > > > > of > > > > > > provided, and therefore get a > > > > > > > minimal > > > 25k or so NAR file > > > > with no JARs > > > included. I get the same > > > > issue with > > > > > the class > > > > not found. > > > > > > > > > > > > > I > > > had > > > > > also attempted to copy > > > Wait.java > > > to > > > > > > > AnotherWait.java within > > > > > > > the actually NiFi source code, > and > > > > > it > > > > compiles, and shows up > > > > in the JARs and NARs > as > > I'd > > > > expect, > > > > but I > cannot get to > > > > it > within my NiFi instance. > > > > > > > > > > > > > Can > > > > someone let me know > > what > I'm > > > > > > > doing > > > > > > > wrong. > > > > > > > > > > > > > > > > > Thanks, > > > > > > John McGinn > > > > >
