[ https://issues.apache.org/jira/browse/HADOOP-17708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan mercier updated HADOOP-17708: -------------------------------------- Description: Dear I tried to build hadoop from source with a vanilla fedora 34 {code:bash} dnf group install -y "Development Tools" \ && dnf install -y java-1.8.0-openjdk-devel fuse-devel snappy-java snappy-devel jansson-devel protobuf zlib-devel libzstd-devel \ maven-1:3.6.3 cmake gcc-c++ ant protobuf-compiler protobuf-java slf4j export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/ export MAVEN_OPTS="-Xms2048M -Xmx4096M" export PATH="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/bin/:$PATH" export CC=/usr/bin/gcc export CXX=/usr/bin/g++ curl -LO https://apache.mediamirrors.org/hadoop/common/hadoop-3.2.2/hadoop-3.2.2-src.tar.gz tar xf hadoop-3.2.2-src.tar.gz && cd hadoop-3.2.2-src mvn package -Pdist,native -Drequire.snappy=true -DskipTests -Dtar {code} But I have this error {code:java} at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake (CompileMojo.java:229) at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute (CompileMojo.java:98) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <args> -rf :hadoop-common {code} If we take a look to cmake log I have: {code:c} gmake[1]: *** [CMakeFiles/cmTC_82239.dir/build.make:106: cmTC_82239] Error 1 gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' gmake: *** [Makefile:140: cmTC_82239/fast] Error 2 Source file was: #include <pthread.h> static void* test_func(void* data) { return data; } int main(void) { pthread_t thread; pthread_create(&thread, NULL, test_func, NULL); pthread_detach(thread); pthread_cancel(thread); pthread_join(thread, NULL); pthread_atfork(NULL, NULL, NULL); pthread_exit(NULL); return 0; } Determining if the function pthread_create exists in the pthreads failed with the following output: Change Dir: /hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp Run Build Command(s):/usr/bin/gmake cmTC_2b9ff/fast && /usr/bin/gmake -f CMakeFiles/cmTC_2b9ff.dir/build.make CMakeFiles/cmTC_2b9ff.dir/build gmake[1]: Entering directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -c /usr/share/cmake/Modules/CheckFunctionExists.c Linking C executable cmTC_2b9ff /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2b9ff.dir/link.txt --verbose=1 /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -o cmTC_2b9ff -lpthreads /usr/bin/ld: cannot find -lpthreads collect2: error: ld returned 1 exit status gmake[1]: *** [CMakeFiles/cmTC_2b9ff.dir/build.make:106: cmTC_2b9ff] Error 1 gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' gmake: *** [Makefile:140: cmTC_2b9ff/fast] Error 2 {code} and if I create the C code shown above and build with `-lpthread` that works fine So as I use cmake 3.19.7, is it an old cmake bug ? Thanks for your help best regards was: Dear I tried to build hadoop from source with a vanilla fedora 34 {code:bash} dnf group install -y "Development Tools" \ && dnf install -y java-1.8.0-openjdk-devel fuse-devel snappy-java snappy-devel jansson-devel protobuf zlib-devel libzstd-devel \ maven-1:3.6.3 cmake gcc-c++ ant protobuf-compiler protobuf-java slf4j export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/ export MAVEN_OPTS="-Xms2048M -Xmx4096M" export PATH="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/bin/:$PATH" export CC=/usr/bin/gcc export CXX=/usr/bin/g++ curl -LO https://apache.mediamirrors.org/hadoop/common/hadoop-3.2.2/hadoop-3.2.2-src.tar.gz tar xf hadoop-3.2.2-src.tar.gz && cd hadoop-3.2.2-src mvn package -Pdist,native -Drequire.snappy=true -DskipTests -Dtar {code} But I have this error {code:java} at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake (CompileMojo.java:229) at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute (CompileMojo.java:98) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347) [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <args> -rf :hadoop-common {code} If we take a look to cmake log I have: {code:c} gmake[1]: *** [CMakeFiles/cmTC_82239.dir/build.make:106: cmTC_82239] Error 1 gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' gmake: *** [Makefile:140: cmTC_82239/fast] Error 2 Source file was: #include <pthread.h> static void* test_func(void* data) { return data; } int main(void) { pthread_t thread; pthread_create(&thread, NULL, test_func, NULL); pthread_detach(thread); pthread_cancel(thread); pthread_join(thread, NULL); pthread_atfork(NULL, NULL, NULL); pthread_exit(NULL); return 0; } Determining if the function pthread_create exists in the pthreads failed with the following output: Change Dir: /hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp Run Build Command(s):/usr/bin/gmake cmTC_2b9ff/fast && /usr/bin/gmake -f CMakeFiles/cmTC_2b9ff.dir/build.make CMakeFiles/cmTC_2b9ff.dir/build gmake[1]: Entering directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -c /usr/share/cmake/Modules/CheckFunctionExists.c Linking C executable cmTC_2b9ff /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2b9ff.dir/link.txt --verbose=1 /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -o cmTC_2b9ff -lpthreads /usr/bin/ld: cannot find -lpthreads collect2: error: ld returned 1 exit status gmake[1]: *** [CMakeFiles/cmTC_2b9ff.dir/build.make:106: cmTC_2b9ff] Error 1 gmake[1]: Leaving directory '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' gmake: *** [Makefile:140: cmTC_2b9ff/fast] Error 2 {code} and if I create the C code shown above and build with `-lpthread` that works fine So as I use cmake 3.6.3, is it an old cmake bug ? Thanks for your help best regards > Fail to build hadoop-common from source on Fedora > ------------------------------------------------- > > Key: HADOOP-17708 > URL: https://issues.apache.org/jira/browse/HADOOP-17708 > Project: Hadoop Common > Issue Type: Bug > Reporter: Jonathan mercier > Priority: Major > > Dear I tried to build hadoop from source with a vanilla fedora 34 > {code:bash} > dnf group install -y "Development Tools" \ > && dnf install -y java-1.8.0-openjdk-devel fuse-devel snappy-java > snappy-devel jansson-devel protobuf zlib-devel libzstd-devel \ > maven-1:3.6.3 cmake gcc-c++ ant protobuf-compiler > protobuf-java slf4j > export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/ > export MAVEN_OPTS="-Xms2048M -Xmx4096M" > export > PATH="/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.fc34.x86_64/bin/:$PATH" > export CC=/usr/bin/gcc > export CXX=/usr/bin/g++ > curl -LO > https://apache.mediamirrors.org/hadoop/common/hadoop-3.2.2/hadoop-3.2.2-src.tar.gz > tar xf hadoop-3.2.2-src.tar.gz && cd hadoop-3.2.2-src > mvn package -Pdist,native -Drequire.snappy=true -DskipTests -Dtar > {code} > But I have this error > {code:java} > at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runMake > (CompileMojo.java:229) > > > at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute > (CompileMojo.java:98) > > > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:210) > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > > > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:117) > > > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject > (LifecycleModuleBuilder.java:81) > > > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build > (SingleThreadedBuilder.java:56) > > > at org.apache.maven.lifecycle.internal.LifecycleStarter.execute > (LifecycleStarter.java:128) > > > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) > > > > at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) > > > > at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) > > > > at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957) > > > > at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289) > > > > at org.apache.maven.cli.MavenCli.main (MavenCli.java:193) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke > (NativeMethodAccessorImpl.java:62) > > > > at sun.reflect.DelegatingMethodAccessorImpl.invoke > (DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke (Method.java:498) > > > > at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced > (Launcher.java:282) > > > at org.codehaus.plexus.classworlds.launcher.Launcher.launch > (Launcher.java:225) > > > at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode > (Launcher.java:406) > > > at org.codehaus.plexus.classworlds.launcher.Launcher.main > (Launcher.java:347) > > > [ERROR] > > > > [ERROR] > > > > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > > > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > > > [ERROR] > > > > [ERROR] After correcting the problems, you can resume the build with the > command > > > [ERROR] mvn <args> -rf :hadoop-common > > {code} > If we take a look to cmake log > I have: > {code:c} > gmake[1]: *** [CMakeFiles/cmTC_82239.dir/build.make:106: cmTC_82239] Error 1 > > > > gmake[1]: Leaving directory > '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' > > > > gmake: *** [Makefile:140: cmTC_82239/fast] Error 2 > > > > > > > > > > > > Source file was: > > > > #include <pthread.h> > > > > > > > > static void* test_func(void* data) > > > > { > > > > return data; > > > > } > > > > > > > > int main(void) > > > > { > > > > pthread_t thread; > > > > pthread_create(&thread, NULL, test_func, NULL); > > > > pthread_detach(thread); > > > > pthread_cancel(thread); > > > > pthread_join(thread, NULL); > > > > pthread_atfork(NULL, NULL, NULL); > > > > pthread_exit(NULL); > > > > > > > > return 0; > > > > } > > > > > > > > Determining if the function pthread_create exists in the pthreads failed with > the following output: > > > Change Dir: > /hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp > > > > > > > > Run Build Command(s):/usr/bin/gmake cmTC_2b9ff/fast && /usr/bin/gmake -f > CMakeFiles/cmTC_2b9ff.dir/build.make CMakeFiles/cmTC_2b9ff.dir/build > > > gmake[1]: Entering directory > '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' > > > > Building C object CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o > > > > /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -o > CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -c > /usr/share/cmake/Modules/CheckFunctionExists.c > > > Linking C executable cmTC_2b9ff > > > > /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_2b9ff.dir/link.txt > --verbose=1 > > > /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic > CMakeFiles/cmTC_2b9ff.dir/CheckFunctionExists.c.o -o cmTC_2b9ff -lpthreads > > > /usr/bin/ld: cannot find -lpthreads > > > > collect2: error: ld returned 1 exit status > > > > gmake[1]: *** [CMakeFiles/cmTC_2b9ff.dir/build.make:106: cmTC_2b9ff] Error 1 > > > > gmake[1]: Leaving directory > '/hadoop-3.2.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeTmp' > > > > gmake: *** [Makefile:140: cmTC_2b9ff/fast] Error 2 > {code} > and if I create the C code shown above and build with `-lpthread` that works > fine > So as I use cmake 3.19.7, is it an old cmake bug ? > Thanks for your help > best regards -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org