Alan That is a great point. We talk about this a little bit in our version guide [1]. But I agree with you that we should make this a fully documented concept in the developer guide (so they know what they can/cannot do) and possibly the user guide (so they know what was done to them). We need to cover at least:
- I want to change the processor class name - I want to change the processor package name - I want to add processor relationships - I want to remove processor relationships - I want to add processor properties - I want to remove processor properties - I want to change a required property to optional - I want to change an optional property to required - I want to change the display name of a property - I want to change the behavior of a processor that could have different side effects despite the same configuration Created a ticket for this [2]. [1] https://cwiki.apache.org/confluence/display/NIFI/Version+Scheme [2] https://issues.apache.org/jira/browse/NIFI-1053 Thanks Joe On Wed, Oct 21, 2015 at 9:43 PM, Alan Jackoway <al...@cloudera.com> wrote: > We should also check similar behavior for missing properties. I was > thinking about this today while modifying a custom processor and decided it > was safest to just make a new processor with the new properties. > > I tried to find info in the developer guide but couldn't. It would be good > to document what changes in processors can be made safely unless we can > definitely say that all changes are safe. > > Alan > On Oct 21, 2015 9:31 PM, "Joe Witt" <joe.w...@gmail.com> wrote: > >> Ahhh - no need to send the logs now. Totally understand. We need to >> do something for your case for sure. We should startup and just have >> some sort of ghost/placeholder processor in the flow and prompt the >> user to select a new one. >> >> https://issues.apache.org/jira/browse/NIFI-1052 >> >> On Wed, Oct 21, 2015 at 9:19 PM, Sumanth Chinthagunta <xmlk...@gmail.com> >> wrote: >> > My case is simple and easy to reproduce: >> > 1. Create a flow with custom processor (e.g. abc.MyProcesser ) >> > 2. Make sure flow is working. Then stop the server. >> > 3. Replace nar in lib with new nar after refactoring processor's package >> ( >> > e.g., xzy.MyProcesser) >> > 4. When you try to start nifi , it fails with error in the log saying >> class >> > not found ( obviously coz deployed flow is still looking for old class) >> > >> > My expectation is system should at least start and only throw error when >> > user try to start a flow that that had reference to class(processor) >> which >> > is no more available in the newly deployed nar. >> > I can send the my error log tonight >> > >> > On Wed, Oct 21, 2015, 5:58 PM Joe Witt <joe.w...@gmail.com> wrote: >> > >> >> Sumanth, >> >> >> >> Do you happen to still have the stack traces around? Unless we do >> >> some breaking/API unfriendly thing such issues should simply not >> >> occur. Want to make sure we understand what is happening here. >> >> >> >> Thanks >> >> Joe >> >> >> >> On Wed, Oct 21, 2015 at 8:56 PM, Sumanth Chinthagunta < >> xmlk...@gmail.com> >> >> wrote: >> >> > I noticed similar error when I have my flow using older version of a >> >> custom >> >> > nar and when I deployed new version of nar with package refactoring. >> >> > My workaround to bring the server back is to redeploy old nar, start >> nifi >> >> > and delete the flow then recreate the flow after deploying refactored >> >> nar. >> >> > Nifi should start even a nar file is replaced or removed. >> >> > >> >> > On Wed, Oct 21, 2015, 9:21 AM Joe Witt <joe.w...@gmail.com> wrote: >> >> > >> >> >> Ok so in seeing this part of the stackdump we know there is a >> >> >> 'classpath thing' going on >> >> >> >> >> >> "Caused by: java.lang.ClassNotFoundException: >> >> >> org.mitre.nifi.NiFiNITFReader" >> >> >> >> >> >> And we see from your provided info that your jar does contain such a >> >> class >> >> >> >> >> >> "22243 Wed Oct 21 10:36:46 EDT 2015 >> >> org/mitre/nifi/NiFiNITFReader.class" >> >> >> >> >> >> And we see that you have a service loader manifest >> >> >> >> >> >> "808 Wed Oct 21 10:36:42 EDT 2015 >> >> >> META-INF/services/org.apache.nifi.processor.Processor" >> >> >> >> >> >> QUESTIONS: >> >> >> ------------------- >> >> >> Can you please provide content of that service manifest file? >> >> >> >> >> >> Also can you please provide the log information that writes out on >> >> >> startup which shows the processors/extensions that load? >> >> >> >> >> >> It looks like this... >> >> >> >> >> >> 2015-10-21 09:20:32,520 INFO [main] >> >> >> org.apache.nifi.nar.ExtensionManager Extension Type Mapping to >> >> >> Classloader: >> >> >> === ProvenanceEventRepository type || Classloader === >> >> >> org.apache.nifi.provenance.PersistentProvenanceRepository || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-provenance-repository-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> org.apache.nifi.provenance.VolatileProvenanceRepository || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-provenance-repository-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> === End ProvenanceEventRepository types === >> >> >> === Processor type || Classloader === >> >> >> org.apache.nifi.processors.hl7.RouteHL7 || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hl7-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> org.apache.nifi.processors.standard.MergeContent || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> org.apache.nifi.processors.standard.EncryptContent || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> org.apache.nifi.processors.aws.s3.PutS3Object || >> >> >> >> >> >> >> >> >> org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-aws-nar-0.3.1-SNAPSHOT.nar-unpacked] >> >> >> ..... >> >> >> >> >> >> >> >> >> Thanks! >> >> >> Joe >> >> >> >> >> >> On Wed, Oct 21, 2015 at 8:51 AM, Jones, Patrick L. <p...@mitre.org> >> >> wrote: >> >> >> > Howdy, >> >> >> > >> >> >> > I have created my own processor which I have been >> >> testing >> >> >> with NiFi for a while now. This stuff used to work for me, now it >> >> doesn't. >> >> >> Today nifi won't start. The nifi-app.log is below >> >> >> > The processor reads in an image type NITF and does a few things >> with >> >> >> it. I do a: >> >> >> > $ mvn install >> >> >> > >> >> >> > from ~/nifi-0.3.0/nifi-mitre-bundle/ which is where my software >> is. I >> >> >> then >> >> >> > $ cp >> >> >> >> >> >> ~/nifi-0.3.0/nifi-mitre-bundle/nifi-mitre-nar/target/nifi-mitre-nar-1.0-SNAPSHOT.nar >> >> >> ../lib/ >> >> >> > >> >> >> > When I start nifi the nifi-app.log shows the below exception with >> the >> >> >> bottom line being >> >> >> > Caused by: java.lang.ClassNotFoundException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > >> >> >> > I looked in my lib/nifi-mitre-nar-1.0-SNAPSHOT.nar it contains >> >> >> > 0 Wed Oct 21 10:36:50 EDT 2015 META-INF/ >> >> >> > 152 Wed Oct 21 10:36:48 EDT 2015 META-INF/MANIFEST.MF >> >> >> > 0 Wed Oct 21 10:36:48 EDT 2015 META-INF/bundled-dependencies/ >> >> >> > ... >> >> >> > 22201 Wed Oct 21 10:36:48 EDT 2015 >> >> >> META-INF/bundled-dependencies/nifi-mitre-processors-1.0-SNAPSHOT.jar >> >> >> > ... >> >> >> > 2963 Wed Oct 21 10:36:48 EDT 2015 META-INF/DEPENDENCIES >> >> >> > 11358 Wed Oct 21 10:36:48 EDT 2015 META-INF/LICENSE >> >> >> > 155 Wed Oct 21 10:36:48 EDT 2015 META-INF/NOTICE >> >> >> > 0 Wed Oct 21 10:36:50 EDT 2015 META-INF/maven/ >> >> >> > 0 Wed Oct 21 10:36:50 EDT 2015 META-INF/maven/mitre/ >> >> >> > 0 Wed Oct 21 10:36:50 EDT 2015 >> >> META-INF/maven/mitre/nifi-mitre-nar/ >> >> >> > 1562 Fri Oct 09 11:19:10 EDT 2015 >> >> >> META-INF/maven/mitre/nifi-mitre-nar/pom.xml >> >> >> > 111 Wed Oct 21 10:36:48 EDT 2015 >> >> >> META-INF/maven/mitre/nifi-mitre-nar/pom.properties >> >> >> > >> >> >> > I then unjarred the .nar file and looked at >> >> >> META-INF/bundled-dependencies/nifi-mitre-processors-1.0-SNAPSHOT.jar >> >> >> > >> >> >> > $ jar tvf >> >> >> >> >> >> ~/dum/META-INF/bundled-dependencies/nifi-mitre-processors-1.0-SNAPSHOT.jar >> >> >> > 0 Wed Oct 21 10:36:48 EDT 2015 META-INF/ >> >> >> > 412 Wed Oct 21 10:36:46 EDT 2015 META-INF/MANIFEST.MF >> >> >> > 0 Wed Oct 21 10:36:42 EDT 2015 META-INF/services/ >> >> >> > 0 Wed Oct 21 10:36:44 EDT 2015 org/ >> >> >> > 0 Wed Oct 21 10:36:44 EDT 2015 org/mitre/ >> >> >> > 0 Wed Oct 21 10:36:46 EDT 2015 org/mitre/nifi/ >> >> >> > 2730 Wed Oct 21 10:36:42 EDT 2015 META-INF/DEPENDENCIES >> >> >> > 11358 Wed Oct 21 10:36:42 EDT 2015 META-INF/LICENSE >> >> >> > 162 Wed Oct 21 10:36:42 EDT 2015 META-INF/NOTICE >> >> >> > 808 Wed Oct 21 10:36:42 EDT 2015 >> >> >> META-INF/services/org.apache.nifi.processor.Processor >> >> >> > 2796 Wed Oct 21 10:36:46 EDT 2015 >> >> org/mitre/nifi/NiFiNITFReader$1.class >> >> >> > 2025 Wed Oct 21 10:36:44 EDT 2015 >> >> org/mitre/nifi/NITFNiFiIoReader.class >> >> >> > 22243 Wed Oct 21 10:36:46 EDT 2015 >> org/mitre/nifi/NiFiNITFReader.class >> >> >> > 0 Wed Oct 21 10:36:48 EDT 2015 META-INF/maven/ >> >> >> > 0 Wed Oct 21 10:36:48 EDT 2015 META-INF/maven/mitre/ >> >> >> > 0 Wed Oct 21 10:36:48 EDT 2015 >> >> >> META-INF/maven/mitre/nifi-mitre-processors/ >> >> >> > 4053 Wed Oct 14 11:23:52 EDT 2015 >> >> >> META-INF/maven/mitre/nifi-mitre-processors/pom.xml >> >> >> > 118 Wed Oct 21 10:36:48 EDT 2015 >> >> >> META-INF/maven/mitre/nifi-mitre-processors/pom.properties >> >> >> > >> >> >> > I see org/mitre/nifi/NiFiNITFReader.class in the jar file. >> >> >> > >> >> >> > Any thoughts on what the problem is? >> >> >> > >> >> >> > Thank you >> >> >> > >> >> >> > Patrick Jones >> >> >> > >> >> >> > >> >> >> > >> >> >> > ------------- nifi-app.log --- >> >> >> > ... >> >> >> > 2015-10-21 10:44:44,147 INFO [main] >> >> >> o.a.n.w.c.ApplicationStartupContextListener Starting Flow >> Controller... >> >> >> > 2015-10-21 10:44:44,484 WARN [main] >> >> >> o.a.nifi.fingerprint.FingerprintFactory Unable to create Processor of >> >> type >> >> >> org.mitre.nifi.NiFiNITFReader due to >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader; its default properties will be >> >> fingerprinted >> >> >> instead of being ignored. >> >> >> > 2015-10-21 10:44:44,536 WARN [main] >> >> >> o.a.nifi.fingerprint.FingerprintFactory Unable to create Processor of >> >> type >> >> >> org.mitre.nifi.NiFiNITFReader due to >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader; its default properties will be >> >> fingerprinted >> >> >> instead of being ignored. >> >> >> > 2015-10-21 10:44:44,607 INFO [main] >> >> >> o.a.n.c.s.StandardControllerServiceProvider No Controller Services to >> >> enable >> >> >> > 2015-10-21 10:44:44,703 WARN [main] >> >> >> org.eclipse.jetty.webapp.WebAppContext Failed startup of context >> >> >> o.e.j.w.WebAppContext@59d5a6fd >> >> >> >> >> >> {/nifi-api,file:/home/plj/nifi-0.3.0/work/jetty/nifi-web-api-0.3.0.war/webapp/,STARTING}{./work/nar/framework/nifi-framework-nar-0.3.0.nar-unpacked/META-INF/bundled-dependencies/nifi-web-api-0.3.0.war} >> >> >> > org.apache.nifi.controller.FlowSynchronizationException: >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:317) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1160) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... >> >> >> > at org.apache.nifi.NiFi.main(NiFi.java:227) >> >> >> [nifi-runtime-0.3.0.jar:0.3.0] >> >> >> > Caused by: >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.FlowController.instantiateProcessor(FlowController.java:864) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... >> >> >> > ... 27 common frames omitted >> >> >> > Caused by: java.lang.ClassNotFoundException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> java.net.URLClassLoader.findClass(URLClassLoader.java:381) >> >> ~[na:1.8.0_60] >> >> >> > ... >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.FlowController.instantiateProcessor(FlowController.java:850) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... 30 common frames omitted >> >> >> > 2015-10-21 10:44:45,345 INFO [main] >> >> >> o.e.jetty.server.handler.ContextHandler Started >> >> >> o.e.j.w.WebAppContext@5fd8dd66 >> >> >> {/nifi-content-viewer,file:/home/plj/nifi-... >> >> >> > 2015-10-21 10:44:45,631 INFO [main] org.eclipse.jetty.server.Server >> >> >> Started @33273ms >> >> >> > 2015-10-21 10:44:45,644 WARN [main] >> >> >> org.apache.nifi.web.server.JettyServer Failed to start web server... >> >> >> shutting down. >> >> >> > org.apache.nifi.controller.FlowSynchronizationException: >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:317) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... >> >> >> > at org.apache.nifi.NiFi.<init>(NiFi.java:137) >> >> >> [nifi-runtime-0.3.0.jar:0.3.0] >> >> >> > at org.apache.nifi.NiFi.main(NiFi.java:227) >> >> >> [nifi-runtime-0.3.0.jar:0.3.0] >> >> >> > Caused by: >> >> >> org.apache.nifi.controller.exception.ProcessorInstantiationException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.FlowController.instantiateProcessor(FlowController.java:864) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:271) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... 27 common frames omitted >> >> >> > Caused by: java.lang.ClassNotFoundException: >> >> >> org.mitre.nifi.NiFiNITFReader >> >> >> > at >> >> >> java.net.URLClassLoader.findClass(URLClassLoader.java:381) >> >> ~[na:1.8.0_60] >> >> >> > at >> >> java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> >> >> ~[na:1.8.0_60] >> >> >> > at >> >> java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> >> >> ~[na:1.8.0_60] >> >> >> > at java.lang.Class.forName0(Native Method) >> >> ~[na:1.8.0_60] >> >> >> > at java.lang.Class.forName(Class.java:264) >> >> ~[na:1.8.0_60] >> >> >> > at >> >> >> >> >> >> org.apache.nifi.controller.FlowController.instantiateProcessor(FlowController.java:850) >> >> >> ~[nifi-framework-core-0.3.0.jar:0.3.0] >> >> >> > ... 30 common frames omitted >> >> >> > 2015-10-21 10:44:45,650 INFO [Thread-1] org.apache.nifi.NiFi >> >> Initiating >> >> >> shutdown of Jetty web server... >> >> >> >> >> >>
