Bryan, thanks so much for the pointers. I got is all working now and
ready to start on the next processor with a whole lot more confidence.
:) FWIW, maybe for someone else who needs this information, I just
added a dependency for logback-classic to get more than the default
slf4f-simple logging (which is standard error console logging only, I
believe)
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</dependency>
And then I just used the typical LoggerFactory.getLogger() to get a
logger in my helper classes and that's it.
I still get this warning in eclipse when I run unit tests but it just
happens to pick logback-classic and same when I run in production so I
do see my log messages going to the app log. I could also play around
with logback.xml now and direct my stuff where ever I want. But, at
least the traces are showing up now so that works. I did a lot of
research into how to exclude slf4j-simple from the classpath in Maven
to avoid the contention warning but nothing I read or tried works.
Anyway, it does pick the logger I wanted so I will ignore for now.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/mpetronic/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/mpetronic/.m2/repository/org/slf4j/slf4j-simple/1.7.12/slf4j-simple-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type
[ch.qos.logback.classic.util.ContextSelectorStaticBinder]
mvn dependency:tree gives this and it appears to me that that the
slf4j-simple dependency is coming from the root of that tree (test) so
maybe that is an internal dependency defined by the maven test runner
or something? Very new to maven so just guessing.
--- maven-dependency-plugin:2.9:tree (default-cli) @ hughes-bigdata-bundle ---
[INFO] com.hughes:hughes-bigdata-bundle:pom:1.0
[INFO] +- junit:junit:jar:4.12:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.mockito:mockito-core:jar:1.10.19:test
[INFO] | \- org.objenesis:objenesis:jar:2.1:test
[INFO] \- org.slf4j:slf4j-simple:jar:1.7.12:test
[INFO] \- org.slf4j:slf4j-api:jar:1.7.12:provided
On Fri, Nov 20, 2015 at 8:31 AM, Bryan Bende <bbe...@gmail.com> wrote:
> The dependencies can definitely be confusing to wrap your head around. A
> good rule of thumb is that you would generally not have a direct jar
> dependency on anything under nifi-nar-bundles in the source tree. This
> would mean no jar dependencies on processors, controller services,
> reporting tasks etc. Now there are cases where a NAR can depend on another
> NAR, this is common when a processor needs to use a controller service, a
> good description of that is here [1], another example is how we have the
> hadoop-libraries-nar so multiple NARs can depend on the same set of hadoop
> libraries and not have to duplicate all those jars. Any other dependencies
> like things from nifi-commons, such as processor utils and others, are fair
> game to include in your NAR. If you look at the nifi lib directory you can
> see the following jars:
>
> jcl-over-slf4j-1.7.12.jar
> jul-to-slf4j-1.7.12.jar
> log4j-over-slf4j-1.7.12.jar
> logback-classic-1.1.3.jar
> logback-core-1.1.3.jar
> nifi-api-0.4.0-SNAPSHOT.jar
> nifi-documentation-0.4.0-SNAPSHOT.jar
> nifi-nar-utils-0.4.0-SNAPSHOT.jar
> nifi-properties-0.4.0-SNAPSHOT.jar
> nifi-runtime-0.4.0-SNAPSHOT.jar
> slf4j-api-1.7.12.jar
>
> These are automatically available to every nar, anything else needs to be
> brought in by bundling the jars in your NAR, or by depending on another NAR.
>
> As for logging, nifi itself uses slf4j and logback, so the logger you get
> from getLogger() would be controlled from the logback.xml in the conf
> directory. I think for non-processor classes if you use the slf4j api that
> will be your best bet as I believe it will automatically use the same
> logback configuration, but others could correct me if I am wrong here. I
> believe it is also possible to use log4j directly with in your NAR, for
> example I know some third party client libraries use log4j and by having a
> dependency on log4j-over-slf4j it can somehow route all of the log4j calls
> back through the main slf4j configuration. Hope I didn't confuse things
> more here, let us know.
>
> [1]
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-LinkingProcessorsandControllerServices
>
>
>
> On Fri, Nov 20, 2015 at 1:26 AM, Mark Petronic <markpetro...@gmail.com>
> wrote:
>
>> Thanks so much Bryan. It is running now. It is starting to come together
>> but I'm still a little unclear on when to include nifi files to pick up
>> dependencies and when to directly include the upstream dependencies. Like
>> say some nifi nar like nifi-processor-utils already has a dependency on
>> commons-lang3 and my custom processor needs commons-lang3, should I satisfy
>> that by depending on nifi-processor-utils (which in know will already be in
>> the install) or should I add a dependency on commons-lang3 in my nar and
>> duplicate the libraries? I realize that there are version issues to be
>> concerned with which I believe is the genesis for nars to start with but
>> assume I need the same version, too? What't the general best practice
>> there?
>>
>> Also, now that I was able to run my processor, I have a question about
>> logging. I see that the processors use getLogger() to get a logger. But is
>> see in other places a more traditional use of LoggerFactory.getLogger().
>> What is the best approach to logging from helper classes that I use in my
>> processor that are not inner classes of the processor class by defined
>> outside in separate class files because I intend to reuse them across
>> multiple processor types? I need to debug an error that I am seeing in one
>> of those helper classes and need to add some logging.
>>
>> Mark
>>
>> On Thu, Nov 19, 2015 at 8:23 PM, Bryan Bende <bbe...@gmail.com> wrote:
>>
>> > Hi Mark,
>> >
>> > Glad to hear you were able to get started building processors, and glad
>> > that blog post helped!
>> >
>> > I pulled down your code and built and deployed it. It looks like the
>> issue
>> > is that your processors pom has a dependency on:
>> > <dependency>
>> > <groupId>org.apache.nifi</groupId>
>> > <artifactId>nifi-standard-processors</artifactId>
>> > <version>0.4.0-SNAPSHOT</version>
>> > </dependency>
>> >
>> > Which means your NAR ends up having the standard processors jar in it,
>> but
>> > they are also deployed in nifi-standard-nar which causes some problems.
>> You
>> > can see the jar is there by looking
>> > in
>> >
>> work/nar/extensions/nifi-bigdata-nar-1.0.nar-unpacked/META-INF/bundled-dependencies/.
>> >
>> > I removed that dependency from the pom and it looked like the only
>> > compilation errors were on some missing json related libraries (jackson
>> and
>> > json path). I added these to your processor pom and it seems to deploy
>> now:
>> >
>> > <dependency>
>> > <groupId>com.jayway.jsonpath</groupId>
>> > <artifactId>json-path</artifactId>
>> > <version>2.0.0</version>
>> > </dependency>
>> > <dependency>
>> > <groupId>org.codehaus.jackson</groupId>
>> > <artifactId>jackson-mapper-asl</artifactId>
>> > <version>1.9.13</version>
>> > </dependency>
>> >
>> > Hope that helps. Let us know if you have any other questions!
>> >
>> > -Bryan
>> >
>> >
>> > On Thu, Nov 19, 2015 at 7:53 PM, Mark Petronic <markpetro...@gmail.com>
>> > wrote:
>> >
>> > > Well, I finally built my first processor. Was a nice experience. Nice
>> > APIs!
>> > > This was my first work with Maven and Nifi so, if things look
>> > > wrong/strange, please go easy on me. :) The issue is when I copy the
>> NAR
>> > > file in the lib directory and restart Nifi, it does not start up. I
>> tried
>> > > removing the work directory for a clean start, no joy. If I remove my
>> > NAR,
>> > > works fine. Wondering if one of you experts would not mind helping me
>> > over
>> > > this hump because I have no clue what is happening based on the error
>> > > message in the log. Seems like some other processor, DetectDuplicate,
>> is
>> > > failing to startup when mine is in there?? I pasted the startup log
>> > traces
>> > > here: http://pastebin.com/raw.php?i=UzraAE53. My processor code is
>> here:
>> > > https://bitbucket.org/mpetronic/nificustomprocessors. I would be very
>> > > grateful for any guidance. It seems like a pretty simple processor so I
>> > was
>> > > surprised this happened. The code is commented so hope that helps you
>> > > easily understand what is going on there. Oh, I started from this very
>> > > helpful post (thanks Bryan):
>> > >
>> > >
>> >
>> http://bryanbende.com/development/2015/02/04/custom-processors-for-apache-nifi/
>> > > .
>> > > I am running a build of Nifi from the GitHub mirror,
>> > > commit 90f6830003b76204e25dec9fafec2488c8bda550.
>> > >
>> > > Thanks,
>> > > Mark
>> > >
>> >
>>
