[
https://issues.apache.org/jira/browse/OOZIE-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001713#comment-16001713
]
Robert Kanter commented on OOZIE-2780:
--------------------------------------
Thanks [~dbist13] for finding those. Now it works correctly and only pulls in
Hadoop 2.6.0 jars. However, if I try to build with a newer Hadoop, there's
also Hadoop 2.6.0 jars mixed in. For instance:
{noformat}
mvn clean package assembly:single -DskipTests -Dhadoop.version=2.7.0
{noformat}
{noformat}
>> [95] 15:48 : oozie-git (master) :: find . -name hadoop*.jar | grep 2.7.0
./client/target/oozie-client-4.4.0-SNAPSHOT-client/oozie-client-4.4.0-SNAPSHOT/lib/hadoop-auth-2.7.0.jar
./server/target/dependency/hadoop-annotations-2.7.0.jar
./server/target/dependency/hadoop-auth-2.7.0.jar
./server/target/dependency/hadoop-common-2.7.0.jar
./sharelib/distcp/target/partial-sharelib/share/lib/distcp/hadoop-distcp-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-annotations-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-api-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-common-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-common-2.7.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-annotations-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-api-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-common-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-common-2.7.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/streaming/target/partial-sharelib/share/lib/mapreduce-streaming/hadoop-streaming-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/distcp/hadoop-distcp-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-annotations-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-api-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-annotations-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-api-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-common-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-resourcemanager-2.7.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/mapreduce-streaming/hadoop-streaming-2.7.0.jar
{noformat}
{noformat}
>> [96] 15:48 : oozie-git (master) :: find . -name hadoop*.jar | grep 2.6.0
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-annotations-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-api-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-common-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-common-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.6.0.jar
./distro/target/oozie-4.4.0-SNAPSHOT-distro/oozie-4.4.0-SNAPSHOT/embedded-oozie-server/webapp/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.6.0.jar
./server/target/dependency/hadoop-yarn-api-2.6.0.jar
./server/target/dependency/hadoop-yarn-common-2.6.0.jar
./server/target/dependency/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./server/target/dependency/hadoop-yarn-server-common-2.6.0.jar
./server/target/dependency/hadoop-yarn-server-resourcemanager-2.6.0.jar
./server/target/dependency/hadoop-yarn-server-web-proxy-2.6.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./sharelib/hive/target/partial-sharelib/share/lib/hive/hadoop-yarn-server-web-proxy-2.6.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./sharelib/hive2/target/partial-sharelib/share/lib/hive2/hadoop-yarn-server-web-proxy-2.6.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive/hadoop-yarn-server-web-proxy-2.6.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./sharelib/target/oozie-sharelib-4.4.0-SNAPSHOT/share/lib/hive2/hadoop-yarn-server-web-proxy-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-annotations-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-api-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-common-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-common-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-resourcemanager-2.6.0.jar
./webapp/target/oozie-webapp-4.4.0-SNAPSHOT/WEB-INF/lib/hadoop-yarn-server-web-proxy-2.6.0.jar
{noformat}
>From a quick look, I don't see where we'd be hardcoding anything to 2.6.0
>other than {{<hadoop.version>}}, which I overrode with {{2.7.0}} as a CLI
>argument.
> Upgrade minimum Hadoop version to 2.6.0
> ---------------------------------------
>
> Key: OOZIE-2780
> URL: https://issues.apache.org/jira/browse/OOZIE-2780
> Project: Oozie
> Issue Type: Improvement
> Components: core
> Affects Versions: 4.3.0
> Reporter: Artem Ervits
> Assignee: Artem Ervits
> Priority: Minor
> Labels: newbie
> Attachments: OOZIE-2780-0.patch, OOZIE-2780-1.patch,
> OOZIE-2780-2.patch
>
>
> I'd like to contribute an fs action enhancement to modify extended attributes
> to core but methods to change extended attributes is not available in
> hadoop-common < 2.6.0. Currently hadoop-common is at 2.4. I am contributing a
> patch to bump to 2.6 but so far I haven't been able to generate a clean test
> as tests in core module fail randomly. On the first run, I had a failure on
> {code} Failed tests:
> testCheckAfterActionDelete(org.apache.oozie.service.TestHAPartitionDependencyManagerService):
> Expected: <null> but was: [0000000-170123095541655-oozie-aerv-C@1]{code}
> and on second run
> {code}Failed tests: testWaitFor(org.apache.oozie.test.TestXTestCase):
> expected:<1000.0> but was:<2036.0>{code}
> I can't tell whether it is due to my patch?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
