[
https://issues.apache.org/jira/browse/NIFI-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15187877#comment-15187877
]
Alan Jackoway commented on NIFI-1529:
-------------------------------------
New and exciting datapoint.
We took the directory running our clustered nifi where this doesn't work,
copied it, removed flow.xml, and change the properties so it was standalone.
That means it was running the exact same code as our version where it fails.
Our processor works in this place.
That means one of these two things is happening:
# There is a difference in classloading between standalone and clustered mode.
# One of our processors in production is triggering the bad class load.
Hopefully it's the second one. That would explain why our development
environment which only had the new processor in the flow worked, but our
production environment with tons of processors failed. I will update when I
know more.
> hadoop-libraries-nar dependency issue for httpclient and httpcore
> -----------------------------------------------------------------
>
> Key: NIFI-1529
> URL: https://issues.apache.org/jira/browse/NIFI-1529
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Bryan Bende
> Assignee: Bryan Bende
> Fix For: 0.6.0, 0.5.1
>
>
> The following issue was reported:
> 016-02-17 15:23:11,814 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.hadoop.PutHDFS
> PutHDFS[id=0575cb57-786c-45d9-b9ef-19c15cc01290] Failed to writ
> e to HDFS due to java.lang.NoClassDefFoundError: org/apache/http/Consts
> 2016-02-17 15:23:11,816 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.hadoop.PutHDFS
> java.lang.NoClassDefFoundError: org/apache/http/Consts
> at
> org.apache.http.client.utils.URIBuilder.digestURI(URIBuilder.java:181)
> ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.http.client.utils.URIBuilder.<init>(URIBuilder.java:82)
> ~[httpclient-4.4.1.jar:4.4.1]
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.createURL(KMSClientProvider.java:368)
> ~[hadoop-common-2.6.2.jar:na]
> From looking into this, we have the following transitive dependencies from
> the hadoop libraries nar:
> - net.java.dev.jets3t:jets3t:jar:0.9.0:compile
> [INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.4.1:compile
> [INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.1.2:compile
> I believe the 4.4.1 of httpclient is coming from the top level pom which has
> a dependency management block that forces the client to 4.4.1 and doesn't for
> core to 4.4.1.
> If we look at httpcore 4.x branch it has the missing Consts class:
> https://github.com/apache/httpcore/tree/4.4.x/httpcore/src/main/java/org/apache/http
> and the 4.1.x branch does not:
> https://github.com/apache/httpcore/tree/4.1.x/httpcore/src/main/java/org/apache/http
> We either need to force both to 4.4.1, or force only the hadoop libraries nar
> to both be 4.1.2 if that is what it normally brings in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
