Russ, I don’t know where the idea to do a “single-module project” came from. The recommended (read: working) method is to have multiple Maven modules — a parent (usually called “nifi-myprocessor-bundle”), a processor (“nifi-myprocessor-processor”), and a NAR builder (“nifi-myprocessor-nar”). This is how the Maven NIFi archetype structures a generated custom processor project [1]. Similarly, my presentation on building custom processors addresses this structure the same way [2], as does the official Developer Guide [3].
I briefly reviewed the contents of your notes, and they are nicely formatted. It appears the advantage of the “single-module” approach is a slightly cleaner presentation in your IDE, while the downside is that the processor annotations are not evaluated completely (and potentially other problems; I have not investigated setting up this non-supported structure further). I would guess that the problem is in a single-module project, the packaging type is NAR, so the bundling process of building the NAR from a compiled and built JAR doesn’t fully occur. Your notes don’t show the results of building the single module, and the contents of the target/ directory to be specific. I don’t mean to sound glib, but I don’t have the bandwidth at this time to pursue the single-module structure further, as I know of no one recommending it and have no idea why it would work other than by accident. We try to avoid recommending additional complexity unless it is required, and to the best of my knowledge, the “3 module” approach is the minimum required for successful (and complete) custom processor building and deployment. Hope this helps. [1] https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions <https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions> [2] https://github.com/alopresto/slides/tree/master/dws_barcelona_2019 <https://github.com/alopresto/slides/tree/master/dws_barcelona_2019> [3] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars <https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jan 28, 2020, at 1:49 PM, Russell Bateman <r...@windofkeltia.com> wrote: > > Andy, > > I have formulated, on the basis of posts I've seen over the years, that there > are two ways to build a NAR containing one's own, custom processors, what I > term a /single-module product/ and a /multi-module product/. mostly based on > whether I need multiple modules in IntelliJ IDEA or only one. I expose this > in a page of notes that I have maintained for some time and I just looked at > the single-module version, the one with the simple NAR build and discovered > (I had probably just never noticed before) that the simple one suffers from > the same problem as I suffer from now, to wit, that the simpler project > structure and /pom.xml/ files yields a processor whose > /@CapabilityDescription/ is ignored. > > My original projects used the more complicated project build, but I saw > someone propose the simpler one, tried it, it worked, and I have been trying > to adopt it for my new project where I plan for only one module, i.e.: the > project itself, and therefore dispence with separate /nar/ and /processor/ > subdirectories. > > This is exposed at https://www.javahotchocolate.com/notes/nifi-project.html. > > If you have time and choose to look at it, and the /single-module project/ > leaps out at you with the answer (as to why it can't be done that way or > what, of this way, prevents the documentation from working), would you point > that out? I have rebuilt using the more complex build process with /nar/ > subdirectory and three instead of one /pom.xml/ files and I now get the > documentation to work. This is the answer, but why can't I build using one > /pom.xml/ and the simpler project structure? > > Russ > > On 1/28/20 10:33 AM, Russell Bateman wrote: >> Andy, >> >> The processor is really called NoOp (not CustomProcessor), it's a straight, >> pass-through that doesn't copy let alone modify the flowfile. It's used for >> debugging and crafting flows (and is always ultimately discarded or replaced >> with a real processor doing real things). >> >> In my new NAR, I see >> >> META-INF >> bundled-dependencies >> docs >> maven >> com.windofkeltia.pipeline >> pom.properties >> pom.xml >> services >> *org.apache.nifi.processor.Processor* (of which, >> com.windofkeltia.processor.NoOp) >> com >> windofkeltia >> processor >> *NoOp.class* >> >> >> In my old, huge NAR, this is under a JAR, /standard-processors.jar/... >> >> META-INF >> maven >> com.windofkeltia.nifi.pipeline.standard-processors >> pom.properties >> pom.xml >> services >> *org.apache.nifi.processor.Processor* (of which, >> com.windofkeltia.nifi.processor.NoOp) >> com >> windofkeltia >> nifi >> processor >> *NoOp.class* >> >> >> This looks pretty much the same in both cases to me except for bundled >> dependencies. Also, this (huge) NAR is a sub NAR of a more complex, >> hierarchical way of building NARs I used to use until I stumbled upon a more >> shallow way. This (huge) NAR, called /nifi-pipeline.nar/, contains under its >> /META-INF/bundled-dependencies/ many JARs one of which is named, >> /standard-processors.jar/ which contains what I show just above. >> >> Russ >> >> >> On 1/28/20 8:52 AM, Andy LoPresto wrote: >>> When you build the Maven module which contains the custom NAR, what does >>> your output look like? Specifically under the target/ directory? If you >>> unzip the NAR file and compare that to one of the custom processors that >>> has working documentation, do you see any differences? >>> >>> >>> Andy LoPresto >>> alopre...@apache.org >>> alopresto.apa...@gmail.com >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>>> On Jan 28, 2020, at 7:02 AM, Russell Bateman<r...@windofkeltia.com> wrote: >>>> >>>> Thanks, Andy, no offense taken. I tried this in Chrome and Firefox, but am >>>> still getting that default page. My sample flow also contains an instance >>>> of GetFile feeding sample files into CustomProcessor. When I look at >>>> GetFile's usage, I see /Description: Creates FlowFiles from file in a >>>> directory/, etc. >>>> >>>> I've written a few custom processors over the years, but this is the first >>>> time I've done something inadvertently so stupid as not to stumble later >>>> upon what would make usage fail for it. The only thing different is that I >>>> have it in a totally new NAR by itself so far and am using 1.10.0. >>>> >>>> This is bizarre to say the least. >>>> >>>> Russ >>>> >>>> On 1/27/20 5:56 PM, Andy LoPresto wrote: >>>>> Hi Russ, >>>>> >>>>> I hate to suggest something so simple, but have you cleared the cache of >>>>> your browser? Sometimes this can be a result of stale caching. >>>>> >>>>> Andy LoPresto >>>>> alopre...@apache.org >>>>> alopresto.apa...@gmail.com >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>>> >>>>>> On Jan 27, 2020, at 2:30 PM, Russell Bateman<r...@windofkeltia.com> >>>>>> wrote: >>>>>> >>>>>> Addendum: the @TagsI add are not showing up in the tag field at the >>>>>> left, but when I scroll down in the processor list and see my new custom >>>>>> processor, I do see these tags listed next, to its right. >>>>>> >>>>>> On 1/27/20 3:27 PM, Russell Bateman wrote: >>>>>>> Perhaps I made my question less clear that I could have. Consider that >>>>>>> I have this annotation on the class of processor: >>>>>>> >>>>>>> @CapabilityDescription( "Custom processor usage statement..." ) >>>>>>> >>>>>>> and, because my processor is listed in the file >>>>>>> >>>>>>> >>>>>>> /src/main/resources/META-INF/services/org.apache.nifi.processor.Processor/ >>>>>>> >>>>>>> as >>>>>>> >>>>>>> /com.windofkeltia.processor.//CustomProcessor/ >>>>>>> >>>>>>> I am able to create a perfectly running and working instance of this >>>>>>> processor on the canvas through the web browser UI of NiFi 1.10.0. >>>>>>> >>>>>>> However, when I right-click the processor instance and choose View >>>>>>> usage, I do not see documentation with >>>>>>> >>>>>>> "Custom processor usage statement..." >>>>>>> >>>>>>> but, instead, I see what I assume is default page content saying >>>>>>> principally, "Apache NiFi Overview". I am looking for help/things to >>>>>>> check that are keeping NiFi from displaying documentation for my >>>>>>> capability description, properties, relationships, etc. Also, I am >>>>>>> using the @Tags annotation to add a couple of quick-find tags: neither >>>>>>> of these are showing up when I go to create an instance of the >>>>>>> processor. This is also likely a clue to what I'm doing wrong. >>>>>>> >>>>>>> Thanks, >>>>>>> Russ >>>>>>> >>>>>>> >>>>>>> On 1/24/20 3:30 PM, Russell Bateman wrote: >>>>>>>> My custom processor's usage, which should come from the >>>>>>>> @CapabilityDescription annotation of the class containing the >>>>>>>> onTrigger() method, nevertheless is nothing more than the Apache NiFi >>>>>>>> Overview. I am able to place an instance on the canvas via the >>>>>>>> processor Component Tool (so, the custom processor is there--I just >>>>>>>> can't get my usage statement): >>>>>>>> >>>>>>>> @SupportsBatching >>>>>>>> @SideEffectFree >>>>>>>> @InputRequirement( InputRequirement.Requirement.INPUT_REQUIRED ) >>>>>>>> *@CapabilityDescription( "Custom processor usage statement..**." )* >>>>>>>> public class CustomProcessor extends AbstractSessionFactory >>>>>>>> Processor >>>>>>>> { >>>>>>>> @Override >>>>>>>> public void onTrigger( final ProcessContext context, final >>>>>>>> ProcessSessionFactory sessionFactory ) >>>>>>>> throws ProcessException >>>>>>>> { >>>>>>>> ... >>>>>>>> } >>>>>>>> >>>>>>>> etc. >>>>>>>> } >>>>>>>> >>>>>>>> What thing missing should I be looking for to fix this? >>>>>>>> >>>>>>>> I'm sure it's some stupid oversight that I have not failed to provide >>>>>>>> in other (successfully documented) custom processors I've written. I >>>>>>>> have compared this processor to those others; I've structured it >>>>>>>> identically, etc. and I have pored >>>>>>>> throughhttps://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#documenting-a-component >>>>>>>> a number of times. Usage just doesn't work. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >> >
