AiBe Gee created HADOOP-16309: --------------------------------- Summary: Unable to build Hadoop nativelly om ARM, node.js is being downloaded for x86 Key: HADOOP-16309 URL: https://issues.apache.org/jira/browse/HADOOP-16309 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 3.3.0 Environment: export PATH="$PATH:/opt/java/bin" export M2_HOME=/opt/apache-maven-3.6.1 export "PATH=$PATH:$M2_HOME/bin" JAVA_HOME=/opt/java export JAVA_HOME export ARCH=arm export CFLAGS="-march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mvectorize-with-neon-quad -mfloat-abi=hard" export CXXFLAGS="-march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mvectorize-with-neon-quad -mfloat-abi=hard" export CPPFLAGS="-march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mvectorize-with-neon-quad -mfloat-abi=hard"
Patches applied: 1. https://stackoverflow.com/questions/50661648/spring-boot-fails-to-run-maven-surefire-plugin-classnotfoundexception-org-apache/50661649#50661649 https://maven.apache.org/surefire/maven-surefire-plugin/examples/class-loading.html - added in pom.xml: <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M3</version> <configuration> <useSystemClassLoader>false</useSystemClassLoader> </configuration> </plugin> 2. https://issues.apache.org/jira/browse/HADOOP-9320 - patch - v2.8.patch cd /kit/hadoop-3-3-0/hadoop-common-project/hadoop-common/ wget https://patch-diff.githubusercontent.com/raw/apache/hadoop/pull/224.patch patch < 224.patch Reporter: AiBe Gee I'm trying to build Hadoop 3.3.0 - git snapshot from 10 May 2019 natively on Raspberry Pi2B (ARM-armv7) and failing in the YARN Application Catalog Webapp component. Apparently the node.js version is downloaded for x86 instead of armv7 and it cannot be executed - Exec format Error Snippet from the build log: ... Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.0.0.M2a/org.eclipse.sisu.inject-0.0.0.M2a.jar (202 kB at 39 kB/s) Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.3/httpcore-4.4.3.jar (327 kB at 52 kB/s) Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.jar (264 kB at 34 kB/s) [INFO] Installing node version v8.11.3 *[INFO] Downloading https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x86.tar.gz to /root/.m2/repository/com/github/eirslett/node/8.11.3/node-8.11.3-linux-x86.tar.gz* [INFO] No proxies configured [INFO] No proxy was configured, downloading directly [INFO] Unpacking /root/.m2/repository/com/github/eirslett/node/8.11.3/node-8.11.3-linux-x86.tar.gz into /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp [INFO] Copying node binary from /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp/node-v8.11.3-linux-x86/bin/node to /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/node [INFO] Installed node locally. [INFO] Installing Yarn version v1.7.0 [INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v1.7.0/yarn-v1.7.0.tar.gz to /root/.m2/repository/com/github/eirslett/yarn/1.7.0/yarn-1.7.0.tar.gz [INFO] No proxies configured [INFO] No proxy was configured, downloading directly May 10, 2019 10:48:32 AM org.apache.http.client.protocol.ResponseProcessCookies processCookies WARNING: Invalid cookie header: "Set-Cookie: has_recent_activity=1; path=/; expires=Fri, 10 May 2019 08:48:32 -0000". Invalid 'expires' attribute: Fri, 10 May 2019 08:48:32 -0000 May 10, 2019 10:48:32 AM org.apache.http.client.protocol.ResponseProcessCookies processCookies WARNING: Invalid cookie header: "Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Tue, 10 May 2039 07:48:32 -0000; secure; HttpOnly". Invalid 'expires' attribute: Tue, 10 May 2039 07:48:32 -0000 [INFO] Unpacking /root/.m2/repository/com/github/eirslett/yarn/1.7.0/yarn-1.7.0.tar.gz into /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn [INFO] Installed Yarn locally. [INFO] [INFO] --- frontend-maven-plugin:1.6:yarn (yarn install) @ hadoop-yarn-applications-catalog-webapp --- [INFO] Running 'yarn ' in /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target *[ERROR] /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn/dist/bin/yarn: line 18: /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/node: cannot execute binary file: Exec format error* [ERROR] /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn/dist/bin/yarn: line 18: /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/node: Success [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache Hadoop Main 3.3.0-SNAPSHOT: [INFO] [INFO] Apache Hadoop Main ................................. SUCCESS [ 10.305 s] [INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 11.622 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [ 11.115 s] ..... [INFO] Apache Hadoop YARN Application Catalog ............. SUCCESS [ 1.043 s] [INFO] Apache Hadoop YARN Application Catalog Webapp ...... FAILURE [07:48 min] [INFO] Apache Hadoop YARN Application Catalog Docker Image SKIPPED [INFO] Apache Hadoop YARN Application MaWo ................ SKIPPED .... [INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 04:02 h [INFO] Finished at: 2019-05-10T10:48:40+03:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn install) on project hadoop-yarn-applications-catalog-webapp: Failed to run task: 'yarn ' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 126 (Exit value: 126) -> [Help 1] [ERROR] I was able to identify the version of node.js in: /kit/hadoop-3-3-0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/pom.xml <execution> <id>install node and yarn</id> <goals> <goal>install-node-and-yarn</goal> </goals> <phase>generate-resources</phase> <configuration> <nodeVersion>v8.11.3</nodeVersion> <yarnVersion>v1.7.0</yarnVersion> </configuration> </execution> But failed to find where it makes the actual ARM / x86 binary selection. It should download, unpack and execute: https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-armv7l.tar.gz Any help pointing me to the right file to patch/change in order to force the proper node.js version choice (ARMv7) is appreciated! Just for reference, this is the build command: nohup mvn package -Pdist,native,docs -DskipTests -Dtar 2>&1 | tee hadoop-build.log & -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org