[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-11431:
---
Fix Version/s: (was: 3.0.0-alpha3)
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Assignee: Steven Rand
>Priority: Blocker
> Labels: maven
> Fix For: 2.8.0
>
> Attachments: HDFS-11431-branch-2.8.0.001.patch,
> HDFS-11431-branch-2.8.0.002.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated HDFS-11431:
--
Fix Version/s: 3.0.0-alpha3
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Assignee: Steven Rand
>Priority: Blocker
> Labels: maven
> Fix For: 2.8.0, 3.0.0-alpha3
>
> Attachments: HDFS-11431-branch-2.8.0.001.patch,
> HDFS-11431-branch-2.8.0.002.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-11431:
--
Resolution: Fixed
Fix Version/s: 2.8.0
Release Note:
The hadoop-client POM now includes a leaner hdfs-client, stripping out all the
transitive dependencies on JARs only needed for the Hadoop HDFS daemon itself.
The specific jars now excluded are: leveldbjni-all, jetty-util, commons-daemon,
xercesImpl, netty and servlet-api.
This should make downstream projects dependent JARs smaller, and avoid version
conflict problems with the specific JARs now excluded.
Applications may encounter build problems if they did depend on these JARs, and
which didn't explicitly include them. There are two fixes for this
* explicitly include the JARs, stating which version of them you want.
* add a dependency on hadoop-hdfs. For Hadoop 2.8+, this will add the missing
dependencies. For builds against older versions of Hadoop, this will be
harmless, as hadoop-hdfs and all its dependencies are already pulled in by the
hadoop-client POM.
Target Version/s: 2.8.0 (was: 2.8.0, 3.0.0-alpha3)
Status: Resolved (was: Patch Available)
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
> Fix For: 2.8.0
>
> Attachments: HDFS-11431-branch-2.8.0.001.patch,
> HDFS-11431-branch-2.8.0.002.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Rand updated HDFS-11431:
---
Attachment: HDFS-11431-branch-2.8.0.002.patch
Attaching a new patch which excludes all the transitive dependencies not
already in hadoop-client. The difference between {{mvn -T 1C dependency:tree
-Dverbose}} pre and post patch is:
{code}
95,97c95,110
< [INFO] +- org.apache.hadoop:hadoop-hdfs-client:jar:2.8.0:compile
< [INFO] | \- com.squareup.okhttp:okhttp:jar:2.4.0:compile
< [INFO] | \- com.squareup.okio:okio:jar:1.4.0:compile
---
> [INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.8.0:compile
> [INFO] | +- org.apache.hadoop:hadoop-hdfs-client:jar:2.8.0:compile
> [INFO] | | \- com.squareup.okhttp:okhttp:jar:2.4.0:compile
> [INFO] | | \- com.squareup.okio:okio:jar:1.4.0:compile
> [INFO] | +- (com.google.guava:guava:jar:11.0.2:compile - version managed
> from 16.0.1; omitted for duplicate)
> [INFO] | +- (commons-cli:commons-cli:jar:1.2:compile - omitted for duplicate)
> [INFO] | +- (commons-codec:commons-codec:jar:1.4:compile - version managed
> from 1.9; omitted for duplicate)
> [INFO] | +- (commons-io:commons-io:jar:2.4:compile - omitted for duplicate)
> [INFO] | +- (commons-lang:commons-lang:jar:2.6:compile - version managed
> from 2.4; omitted for duplicate)
> [INFO] | +- (commons-logging:commons-logging:jar:1.1.3:compile - version
> managed from 1.1.1; omitted for duplicate)
> [INFO] | +- (log4j:log4j:jar:1.2.17:compile - version managed from 1.2.16;
> omitted for duplicate)
> [INFO] | +- (com.google.protobuf:protobuf-java:jar:2.5.0:compile - omitted
> for duplicate)
> [INFO] | +- (org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -
> version managed from 1.8.8; omitted for duplicate)
> [INFO] | +- (org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -
> version managed from 1.8.8; omitted for duplicate)
> [INFO] | +- (xmlenc:xmlenc:jar:0.52:compile - omitted for duplicate)
> [INFO] | \- (org.apache.htrace:htrace-core4:jar:4.0.1-incubating:compile -
> omitted for duplicate)
{code}
I _think_ that the dependencies marked "omitted for duplicate" have no effect
and don't need to be excluded, but please correct me if I'm misunderstanding,
or if it's still good to exclude them for other reasons.
Re: ZooKeeper and Curator, it seems that hadoop-client already depends on both
of those things via hadoop-common?
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
> Attachments: HDFS-11431-branch-2.8.0.001.patch,
> HDFS-11431-branch-2.8.0.002.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Rand updated HDFS-11431:
---
Status: Patch Available (was: Open)
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
> Attachments: HDFS-11431-branch-2.8.0.001.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Rand updated HDFS-11431:
---
Attachment: HDFS-11431-branch-2.8.0.001.patch
I've attached a patch which simply makes hadoop-client depend on hadoop-hdfs. I
tested it by publishing Hadoop locally and then building Spark against the
result of the local publish. The resulting Spark distribution is able to run
successfully against an HA HDFS cluster with no changes to Spark, which is not
the case as is.
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
> Attachments: HDFS-11431-branch-2.8.0.001.patch
>
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-11431:
---
Target Version/s: 2.8.0, 3.0.0-alpha3
Adding target versions since this looks to be possibly a real blocker.
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-11431:
Affects Version/s: 3.0.0-alpha3
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0, 3.0.0-alpha3
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11431) hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
[
https://issues.apache.org/jira/browse/HDFS-11431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-11431:
Priority: Blocker (was: Major)
> hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider
> ---
>
> Key: HDFS-11431
> URL: https://issues.apache.org/jira/browse/HDFS-11431
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: build, hdfs-client
>Affects Versions: 2.8.0
>Reporter: Steven Rand
>Priority: Blocker
> Labels: maven
>
> The {{hadoop-hdfs-client-2.8.0.jar}} file does include the
> {{ConfiguredFailoverProxyProvider}} class. This breaks client applications
> that use this class to communicate with the active NameNode in an HA
> deployment of HDFS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
