Hi Leo, Following your instruction, the following is what I got:
[junli@mercoop-26 Linux-amd64-64]$ objdump -Tt libhadoop.so | grep -I fadvise 0000000000004d70 g F .text 000000000000006f Java_org_apache_hadoop_io_nativeio_NativeIO_posix_1fadvise 0000000000004d70 g DF .text 000000000000006f Base Java_org_apache_hadoop_io_nativeio_NativeIO_posix_1fadvise Apparently the part of "the GLIBC" is not there The libhadoop.so is the one that I downloaded as part of the tar.gz of hadoop.1.1.2 from the Hadoop Apache web site. Could you let me know how I can use the source code directory in the downloaded Hadoop Apache package, to re-compile the libhadoop.so and to make sure that the fadvise call is able to get referenced? Regards, Jun On Fri, Jun 21, 2013 at 9:19 AM, Leo Leung <lle...@ddn.com> wrote: > This looks like a compilation problem on the native hadoop libraries. > > Please locate the libhadoop.so library on your system and run > [shell] objdump -Tt libhadoop.so | grep -I fadvise > > If you don't see something like the following *the GLIBC* part (that > means the system where the share lib was compiled did not have it) > > 00000000000056a0 g F .text 00000000000000a3 > Java_org_apache_hadoop_io_nativeio_NativeIO_posix_1fadvise > 0000000000000000 F *UND* 0000000000000000 > posix_fadvise@@GLIBC_2.2.5 > 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.2.5 posix_fadvise > 00000000000056a0 g DF .text 00000000000000a3 Base > Java_org_apache_hadoop_io_nativeio_NativeIO_posix_1fadvise > > Note: objdump is from binutils- rpm (you can use yum install to install > it if you don't have it) > > > -----Original Message----- > From: Jun Li [mailto:jltz922...@gmail.com] > Sent: Friday, June 21, 2013 1:35 AM > To: user@hadoop.apache.org > Subject: how to turn on NativeIO.posixFadviseIfPossible > > Hi, > > I downloaded the current stable version from the Apache Hadoop web site, > hadoop-1.1.2. My machine is an AMD-based machine and Redhat Enterprise 6.1. > The detailed Linux kernel version is: > 2.6.32-131.0.15.el6.x86_64 > > I ran the TestNativeIO.java under the distribution directory of > "test/org/apache/hadoop/io/nativeio/TestNativeIO.java" and tried to > understand how NativeIO.posixFadviseIfPossible behaves, in particular, to > check whether "posix_fadvise" is turned on or not. I am interested in this > call as it is used in Read-Ahead-Pool to cache data to the OS's buffer > cache. The following is the test case that I ran: > > @Test > public void testPosixFadvise() throws Exception { > FileInputStream fis = new FileInputStream("/dev/zero"); > try { > NativeIO.posixFadviseIfPossible(fis.getFD(), 0, 0, > NativeIO.POSIX_FADV_SEQUENTIAL); > } catch (NativeIOException noe) { > // we should just skip the unit test on machines where we don't > // have fadvise support > assumeTrue(false); > } finally { > fis.close(); > } > > However, when I stepped into the code and reached "NativeIO.java" > under the package of "org.apache.hadoop.io.nativeio", in the particular > call below: > > public static void posixFadviseIfPossible( > FileDescriptor fd, long offset, long len, int flags) > throws NativeIOException { > > if (nativeLoaded && fadvisePossible) { > try { > posix_fadvise(fd, offset, len, flags); > } catch (UnsupportedOperationException uoe) { > fadvisePossible = false; > } catch (UnsatisfiedLinkError ule) { > fadvisePossible = false; > } > } > } > > The call to "posix_fadvise" threw the "UnsupportedOperationException" > exception. > > I further traced to the native library, and in the code "NativeIO.c", I > found > > JNIEXPORT void JNICALL > Java_org_apache_hadoop_io_nativeio_NativeIO_posix_1fadvise( > JNIEnv *env, jclass clazz, > jobject fd_object, jlong offset, jlong len, jint flags) { #ifndef > HAVE_POSIX_FADVISE > THROW(env, "java/lang/UnsupportedOperationException", > "fadvise support not available"); #else > > ... > } > > I believe that the problem of throwing the exception is because > "HAVE_POSIX_FADVISE" is not defined. I made sure that the native IO > library is loaded properly in the Java code, as I can successfully run the > other test cases in "TestNativeIO.java". > > So my question is: should I re-compile the "libhadoop" in order to get the > version of the shared library that can have "HAVE_POSIX_FADVISE" > turned on? Or by default, FADVISE is turned on already? > > Thank you! >
