Dear All,
I am building hadoop 0.23.1 release from source with native
support. I have already built/installed the following pre-
requisites for native support
1. gcc-c++ 4.7.1
2. protoc 2.4.1
3. autotools chain
4. JDK 1.6.0_33
5. zlib 1.2.5-6
6. lzo 2.06-2
I have also set the following variables and exported them
export LD_LIBRARY_PATH=/usr/local/lib
Other variables are also set as given below.
export LD_LIBRARY_PATH=/usr/local/lib
export JAVA_HOME=/usr/java/jdk1.6.0_33
export ANT_HOME=/home/fedora/apache-ant-1.8.4
export MAVEN_HOME=/home/fedora/apache-maven-3.0.4
export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$MAVEN_HOME/bin:$PATH
export HADOOP_COMMON_HOME=/home/fedora/hadoop/bin/release-0.23.1/
hadoop-0.23.1
export HADOOP_HDFS_HOME=$HADOOP_COMMON_HOME
export YARN_HOME=$HADOOP_COMMON_HOME
export HADOOP_MAPRED_HOME=$HADOOP_COMMON_HOME
export HADOOP_CONF_DIR=$HADOOP_COMMON_HOME/etc/hadoop
I am using the following to build the source with native support
(using root user access).
mvn clean install -Pdist -Pnative -DskipTests=true
However, after building a number of sub-projects, the build fails
at the nodemanager with the following error
[INFO] Compiling 129 source files to /home/fedora/hadoop/src/
release-0.23.1/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-
server/hadoop-yarn-server-nodemanager/target/classes
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:autoreconf (compile) @
hadoop-yarn-server-nodemanager ---
[INFO]
[INFO] --- make-maven-plugin:1.0-beta-1:configure (compile) @
hadoop-yarn-server-nodemanager ---
[INFO] checking for gcc... gcc
[INFO] checking whether the C compiler works... no
[INFO] configure: error: in `/home/fedora/hadoop/src/release-0.23.1/
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
server-nodemanager/target/native/container-executor':
[INFO] configure: error: C compiler cannot create executables
[INFO] See `config.log' for more details
The config.log mentions the following issues.
configure:2562: checking whether the C compiler works
configure:2584: gcc -DHADOOP_CONF_DIR=/etc/hadoop -m32
conftest.c >&5
/bin/ld: cannot find crt1.o: No such file or directory
/bin/ld: cannot find crti.o: No such file or directory
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
4.7.0/libgcc_s.so when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: skipping incompatible /usr/lib64/libc.so when searching
for -lc
/bin/ld: cannot find -lc
/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-redhat-linux/
4.7.0/libgcc_s.so when searching for -lgcc_s
/bin/ld: cannot find -lgcc_s
/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
configure:2588: $? = 1
configure:2626: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "linux-container-executor"
| #define PACKAGE_TARNAME "linux-container-executor"
| #define PACKAGE_VERSION "1.0.0"
| #define PACKAGE_STRING "linux-container-executor 1.0.0"
| #define PACKAGE_BUGREPORT "mapreduce-...@hadoop.apache.org"
| #define PACKAGE_URL ""
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:2631: error: in `/home/fedora/hadoop/src/release-0.23.1/
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-
server-nodemanager/target/native/container-executor':
configure:2633: error: C compiler cannot create executables
Overall build summary is as given below
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS
[7.920s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS
[1.405s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS
[6.452s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS
[0.637s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS
[3.236s]
[INFO] Apache Hadoop Auth ................................ SUCCESS
[8.805s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS
[5.105s]
[INFO] Apache Hadoop Common .............................. SUCCESS
[1:55.833s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS
[0.210s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS
[1:24.218s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS
[20.992s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS
[0.127s]
[INFO] hadoop-yarn ....................................... SUCCESS
[0.598s]
[INFO] hadoop-yarn-api ................................... SUCCESS
[57.281s]
[INFO] hadoop-yarn-common ................................ SUCCESS
[1:01.264s]
[INFO] hadoop-yarn-server ................................ SUCCESS
[0.190s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS
[19.280s]
[INFO] hadoop-yarn-server-nodemanager .................... FAILURE
[17.016s]
[INFO] hadoop-yarn-server-web-proxy ...................... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................ SKIPPED
[INFO] hadoop-yarn-server-tests .......................... SKIPPED
[INFO] hadoop-mapreduce-client ........................... SKIPPED
[INFO] hadoop-mapreduce-client-core ...................... SKIPPED
[INFO] hadoop-yarn-applications .......................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell ......... SKIPPED
[INFO] hadoop-yarn-site .................................. SKIPPED
[INFO] hadoop-mapreduce-client-common .................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED
[INFO] hadoop-mapreduce-client-app ....................... SKIPPED
[INFO] hadoop-mapreduce-client-hs ........................ SKIPPED
[INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] Apache Hadoop MapReduce Streaming ................. SKIPPED
[INFO] Apache Hadoop Distributed Copy .................... SKIPPED
[INFO] Apache Hadoop Archives ............................ SKIPPED
[INFO] Apache Hadoop Rumen ............................... SKIPPED
[INFO] Apache Hadoop Extras .............................. SKIPPED
[INFO] Apache Hadoop Tools Dist .......................... SKIPPED
[INFO] Apache Hadoop Tools ............................... SKIPPED
[INFO] Apache Hadoop Distribution ........................ SKIPPED
[INFO] Apache Hadoop Client .............................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ........................ SKIPPED
[INFO]
---
-------------------------------------------------------------------
My doubts here are that is the if C compiler was not able to create
the executables, the other projects which were build before node
manager should have also failed, since they are also using the auto
tools chain. I am able to build the entire source code if I build
without native support, but I do not want to skip the native
support build and I am trying to build the entire source code with
native libraries support.
My current OS is Fedora 17 64-bit and I have also got the same
issue on CentOS 6.2 64-bit. I dont know if I am missing something
for successful build with native libraries, and would request for
the community's help for the same.
Thanks and Regards
Pankaj Misra
________________________________
Impetus webinar: Designing a Test Automation Framework for
Interoperable Systems; July 25 (10:00am PT). http://lf1.me/0E/
Follow us on www.twitter.com/impetustech
NOTE: This message may contain information that is confidential,
proprietary, privileged or otherwise protected by law. The message
is intended solely for the named addressee. If received in error,
please destroy and notify the sender. Any use of this email is
prohibited when received in error. Impetus does not represent,
warrant and/or guarantee, that the integrity of this communication
has been maintained nor that the communication is free of errors,
virus, interception or interference.
