[
https://issues.apache.org/jira/browse/NIFI-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186049#comment-15186049
]
Alan Jackoway commented on NIFI-1529:
-------------------------------------
Is this actually fixed? We just ran into this on our nifi running 0.4.0, which
led me to this jira. However, before finding this jira I checked today's master:
{code}
$ mvn dependency:tree > /tmp/nifiDeps.txt
$ grep httpcore /tmp/nifiDeps.txt | cut -d ":" -f 4 | sort | uniq
4.1.2
4.1.3
4.2.5
4.3
4.4.1
{code}
It looks to me like we are getting bad versions from the following sources
(from my manually cut up dependency:tree):
{noformat}
[INFO] --- maven-dependency-plugin:2.9:tree (default-cli) @
nifi-hdfs-processors ---
[INFO] org.apache.nifi:nifi-hdfs-processors:jar:0.6.0-SNAPSHOT
[INFO] +- org.apache.nifi:nifi-api:jar:0.6.0-SNAPSHOT:provided
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.6.2:provided
[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.9.0:provided
[INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.4.1:provided
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.1.2:provided
[INFO] --- maven-dependency-plugin:2.9:tree (default-cli) @
nifi-hadoop-libraries-nar ---
[INFO] org.apache.nifi:nifi-hadoop-libraries-nar:nar:0.6.0-SNAPSHOT
[INFO] +-
org.apache.nifi:nifi-standard-services-api-nar:nar:0.6.0-SNAPSHOT:compile
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.6.2:compile
[INFO] | +- org.apache.hadoop:hadoop-common:jar:2.6.2:compile
[INFO] | | +- org.apache.hadoop:hadoop-auth:jar:2.6.2:compile
[INFO] | | | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] | | | | \- org.apache.httpcomponents:httpcore:jar:4.2.5:compile
[INFO] --- maven-dependency-plugin:2.9:tree (default-cli) @
nifi-solr-processors ---
[INFO] org.apache.nifi:nifi-solr-processors:jar:0.6.0-SNAPSHOT
[INFO] +- org.apache.solr:solr-solrj:jar:5.1.0:compile
[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.3.1:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.3:compile
[INFO] | +- org.apache.httpcomponents:httpmime:jar:4.3.1:compile
[INFO]
[INFO] --- maven-dependency-plugin:2.9:tree (default-cli) @ nifi-solr-nar ---
[INFO] org.apache.nifi:nifi-solr-nar:nar:0.6.0-SNAPSHOT
[INFO] +- org.apache.nifi:nifi-solr-processors:jar:0.6.0-SNAPSHOT:compile
[INFO] | +- org.apache.solr:solr-solrj:jar:5.1.0:compile
[INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.3.1:compile
[INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.3:compile
[INFO] --- maven-dependency-plugin:2.9:tree (default-cli) @
nifi-flume-processors ---
[INFO] org.apache.nifi:nifi-flume-processors:jar:0.6.0-SNAPSHOT
[INFO] +- org.apache.flume:flume-ng-sdk:jar:1.6.0:compile
[INFO] | \- org.apache.thrift:libthrift:jar:0.9.0:compile
[INFO] | \- org.apache.httpcomponents:httpcore:jar:4.1.3:compile
{noformat}
Looks like it's mostly hadoop messing us up here.
Do we know how nifi will decide which version of the jar to use if we get
different versions in different bundled-dependencies? Is it deterministic?
> hadoop-libraries-nar dependency issue for httpclient and httpcore
> -----------------------------------------------------------------
>
> Key: NIFI-1529
> URL: https://issues.apache.org/jira/browse/NIFI-1529
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Bryan Bende
> Assignee: Bryan Bende
> Fix For: 0.6.0, 0.5.1
>
>
> The following issue was reported:
> 016-02-17 15:23:11,814 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.hadoop.PutHDFS
> PutHDFS[id=0575cb57-786c-45d9-b9ef-19c15cc01290] Failed to writ
> e to HDFS due to java.lang.NoClassDefFoundError: org/apache/http/Consts
> 2016-02-17 15:23:11,816 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.hadoop.PutHDFS
> java.lang.NoClassDefFoundError: org/apache/http/Consts
> at
> org.apache.http.client.utils.URIBuilder.digestURI(URIBuilder.java:181)
> ~[httpclient-4.4.1.jar:4.4.1]
> at org.apache.http.client.utils.URIBuilder.<init>(URIBuilder.java:82)
> ~[httpclient-4.4.1.jar:4.4.1]
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.createURL(KMSClientProvider.java:368)
> ~[hadoop-common-2.6.2.jar:na]
> From looking into this, we have the following transitive dependencies from
> the hadoop libraries nar:
> - net.java.dev.jets3t:jets3t:jar:0.9.0:compile
> [INFO] | | +- org.apache.httpcomponents:httpclient:jar:4.4.1:compile
> [INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.1.2:compile
> I believe the 4.4.1 of httpclient is coming from the top level pom which has
> a dependency management block that forces the client to 4.4.1 and doesn't for
> core to 4.4.1.
> If we look at httpcore 4.x branch it has the missing Consts class:
> https://github.com/apache/httpcore/tree/4.4.x/httpcore/src/main/java/org/apache/http
> and the 4.1.x branch does not:
> https://github.com/apache/httpcore/tree/4.1.x/httpcore/src/main/java/org/apache/http
> We either need to force both to 4.4.1, or force only the hadoop libraries nar
> to both be 4.1.2 if that is what it normally brings in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
