James, Just to clarify... the conf directory of NiFi is not on the classpath of processors.
The behavior with core-site.xml is because the Hadoop client used by NiFi provides this behavior, and not NiFi itself. Basically if you create a new Hadoop Configuration object and you don't provide it with a list of resources, it will find any that are on the classpath, which in this case would be any that are bundled in JARs inside the given NAR, or anything in NiFI's lib directory. Putting stuff in NiFi's lib directory makes it visible to every processor and is generally dangerous, which is why the existing Hadoop processors have a property for the path to the core-site.xml file and then load it like so: String resource = ... // get the value of the property where the path is Configuration config = new Configuration(); config.addResource(new Path(resource.trim())); Hope that helps. -Bryan On Wed, Jul 5, 2017 at 4:15 PM, James Srinivasan <james.sriniva...@gmail.com> wrote: > Thanks, I ended up stracing the NiFi process to see where it was > actually trying to load my config file from. Helpfully, the list of > locations included the NiFi conf directory, which is just perfect for > me. > > I don't want to keep the config file (core-site.xml for Hadoop) in my > NAR itself since it will obviously change between different > deployments. I was inspired by: > > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java#L69-L70 > > Which says if that property isn't set, then NiFi searches the > classpath for the necessary file. > > Thanks again, > > James > > On 5 July 2017 at 00:07, Jeremy Dyer <jdy...@gmail.com> wrote: >> Hi James, >> >> NiFi doesn't do anything too crazy from what Java itself does. The "funky" >> part is the isolated classpath loaders. Really since your building a custom >> processor I would also assume your building a custom NAR. If that is the >> case you can read that resource from the "nar" just like you would any "jar" >> in java. The HDFS processor your referenced loads an external file that >> isn't bundled in the actual classpath at all. If that is the functionality >> you want I suggest following the template from the HDFS processors as you >> stated. Otherwise just carryon as usual and pretend its a regular java >> application. >> >> On Tue, Jul 4, 2017 at 4:31 PM, James Srinivasan >> <james.sriniva...@gmail.com> wrote: >>> >>> Hi, >>> >>> I'm developing a processor which needs to read some of its config from >>> the classpath. Reading the docs etc., NiFi's classpath is a little >>> funky - where's the best (least worst?) location for such files? I >>> note that the HDFS processors can read their config (core-site.xml >>> etc) from the classpath, but I can't find where that actually is. >>> >>> Thanks in advance, >>> >>> James >> >>
