Kengo Seki created BIGTOP-3334:
----------------------------------
Summary: Fix Oozie build after upgrading Spark to 2.4.5
Key: BIGTOP-3334
URL: https://issues.apache.org/jira/browse/BIGTOP-3334
Project: Bigtop
Issue Type: Bug
Components: build
Reporter: Kengo Seki
Assignee: Kengo Seki
After BIGTOP-3165, building Oozie came to fail as follows, because [Scala 2.10
support was removed from Spark
2.3|https://spark.apache.org/docs/2.4.5/#downloading].
{code}
$ ./gradlew oozie-pkg
(snip)
[INFO] ------------------< org.apache.oozie:oozie-examples >-------------------
[INFO] Building Apache Oozie Examples 4.3.0 [15/23]
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central:
https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
Downloading from apache.snapshots.repo:
https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
Downloading from redhat:
https://maven.repository.redhat.com/ga/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.pom
[WARNING] The POM for org.apache.spark:spark-core_2.10:jar:2.4.5 is missing, no
dependency information available
Downloading from central:
https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
Downloading from apache.snapshots.repo:
https://repository.apache.org/content/groups/snapshots/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
Downloading from redhat:
https://maven.repository.redhat.com/ga/org/apache/spark/spark-core_2.10/2.4.5/spark-core_2.10-2.4.5.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Oozie Main 4.3.0 ............................ SUCCESS [ 1.526 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 hadoop-2-4.3.0 SUCCESS [ 2.526
s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 hadoop-2-4.3.0 SUCCESS [
0.121 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test hadoop-2-4.3.0 SUCCESS [
0.275 s]
[INFO] Apache Oozie Hadoop Libs 4.3.0 ..................... SUCCESS [ 0.020 s]
[INFO] Apache Oozie Client 4.3.0 .......................... SUCCESS [ 9.910 s]
[INFO] Apache Oozie Share Lib Oozie 4.3.0 ................. SUCCESS [ 2.319 s]
[INFO] Apache Oozie Share Lib HCatalog 4.3.0 .............. SUCCESS [ 7.790 s]
[INFO] Apache Oozie Share Lib Distcp 4.3.0 ................ SUCCESS [ 0.645 s]
[INFO] Apache Oozie Core 4.3.0 ............................ SUCCESS [ 39.494 s]
[INFO] Apache Oozie Share Lib Streaming 4.3.0 ............. SUCCESS [ 4.785 s]
[INFO] Apache Oozie Share Lib Hive 4.3.0 .................. SUCCESS [ 8.152 s]
[INFO] Apache Oozie Share Lib Hive 2 4.3.0 ................ SUCCESS [ 6.853 s]
[INFO] Apache Oozie Share Lib Sqoop 4.3.0 ................. SUCCESS [ 3.060 s]
[INFO] Apache Oozie Examples 4.3.0 ........................ FAILURE [ 5.389 s]
[INFO] Apache Oozie Share Lib Spark 4.3.0 ................. SKIPPED
[INFO] Apache Oozie Share Lib 4.3.0 ....................... SKIPPED
[INFO] Apache Oozie Docs 4.3.0 ............................ SKIPPED
[INFO] Apache Oozie WebApp 4.3.0 .......................... SKIPPED
[INFO] Apache Oozie Tools 4.3.0 ........................... SKIPPED
[INFO] Apache Oozie MiniOozie 4.3.0 ....................... SKIPPED
[INFO] Apache Oozie Distro 4.3.0 .......................... SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests 4.3.0 ........ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:33 min
[INFO] Finished at: 2020-04-03T11:48:09+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project oozie-examples: Could not resolve
dependencies for project org.apache.oozie:oozie-examples:jar:4.3.0: Could not
find artifact org.apache.spark:spark-core_2.10:jar:2.4.5 in central
(https://repo1.maven.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :oozie-examples
ERROR, Oozie distro creation failed
+ MKDISTRO_RESULT=255
+ '[' 255 '!=' 0 ']'
+ echo 'ERROR: mkdistro.sh failed with error 255'
ERROR: mkdistro.sh failed with error 255
+ echo 'Build files from failed build:
/home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0'
Build files from failed build:
/home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0
+ exit 255
debian/rules:33: recipe for target 'override_dh_auto_build' failed
make[1]: *** [override_dh_auto_build] Error 255
make[1]: Leaving directory '/home/sekikn/repos/bigtop/output/oozie/oozie-4.3.0'
debian/rules:27: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1152:
dpkg-buildpackage -rfakeroot -us -uc -ui -b
failed
> Task :oozie-deb FAILED
FAILURE: Build failed with an exception.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
