Ted, you can use lzop 'myfile.txt' to produce 'myfile.txt.lzo'. The -d option will decompress. You can also write data with the LzopOutputStream.
Kevin On Fri, Jan 15, 2010 at 3:05 PM, Ted Yu <yuzhih...@gmail.com> wrote: > I was able to compile in 64 bit mode. > > One minor thing: > I noticed that small files may get bigger LZ-compressed: > [ria...@tyu-linux kevinweil-hadoop-lzo-916aeae]$ ls -l src/test/data/ > total 940 > -rw-rw-r-- 1 rialto rialto 588895 Jan 11 06:58 100000.txt > -rw-rw-r-- 1 rialto rialto 346982 Jan 11 06:58 100000.txt.lzo > -rw-rw-r-- 1 rialto rialto 292 Jan 11 06:58 100.txt > -rw-rw-r-- 1 rialto rialto 353 Jan 11 06:58 100.txt.lzo > > BTW which Linux command do I use to produce .lzo file ? > > Thanks > > On Fri, Jan 15, 2010 at 2:33 PM, Todd Lipcon <t...@cloudera.com> wrote: > > > But, I bet if you try the same with -m32, it won't work. You need the > > 32-bit > > toolchain if you want to compile the 32-bit libraries. I don't know > exactly > > how to go about setting this up. But you're better off with 64-bit JDK > > anyway these days. So, switch your JAVA_HOME to point to a 64-bit JDK and > > try again. > > > > -Todd > > > > On Fri, Jan 15, 2010 at 2:16 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > [ria...@tyu-linux kevinweil-hadoop-lzo-916aeae]$ ./a.out > > > Hello World! > > > [ria...@tyu-linux kevinweil-hadoop-lzo-916aeae]$ gcc -v > > > Using built-in specs. > > > Target: x86_64-redhat-linux > > > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > > > --infodir=/usr/share/info --enable-shared --enable-threads=posix > > > --enable-checking=release --with-system-zlib --enable-__cxa_atexit > > > --disable-libunwind-exceptions --enable-libgcj-multifile > > > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > > > --enable-java-awt=gtk > > > --disable-dssi --enable-plugin > > > --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre > > --with-cpu=generic > > > --host=x86_64-redhat-linux > > > Thread model: posix > > > gcc version 4.1.2 20080704 (Red Hat 4.1.2-46) > > > > > > > > > On Fri, Jan 15, 2010 at 1:47 PM, Alex Kozlov <ale...@cloudera.com> > > wrote: > > > > > > > Looks like your compiler is misconfigured. Can you try and compile a > > > > simple > > > > "Hello World!" program: > > > > > > > > [ale...@monster01 ~]$ cat Hello.c > > > > #include <stdio.h> > > > > int main() { > > > > printf("Hello World!\n"); > > > > } > > > > [ale...@monster01 ~]$ gcc Hello.c > > > > [ale...@monster01 ~]$ ./a.out > > > > Hello World! > > > > [ale...@monster01 ~]$ gcc -v > > > > Using built-in specs. > > > > Target: x86_64-redhat-linux > > > > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > > > > --infodir=/usr/share/info --enable-shared --enable-threads=posix > > > > --enable-checking=release --with-system-zlib --enable-__cxa_atexit > > > > --disable-libunwind-exceptions --enable-libgcj-multifile > > > > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > > > > --enable-java-awt=gtk > > > > --disable-dssi --enable-plugin > > > > --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre > > > --with-cpu=generic > > > > --host=x86_64-redhat-linux > > > > Thread model: posix > > > > gcc version 4.1.2 20080704 (Red Hat 4.1.2-46) > > > > > > > > Alex > > > > > > > > On Fri, Jan 15, 2010 at 1:34 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > > > I tried to set LD_LIBRARY_PATH to point to where crt1.o resides: > > > > > export LD_LIBRARY_PATH=/usr/lib64 > > > > > > > > > > Still the same issue. > > > > > This is snippet of config.log: > > > > > > > > > > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man > > > > > --infodir=/usr/share/info --enable-shared --enable-threads=posix > > > > > --enable-checking=release --with-system-zlib --enable-__cxa_atexit > > > > > --disable-libunwind-exceptions --enable-libgcj-multifile > > > > > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > > > > > --enable-java-awt=gtk > > > > > --disable-dssi --enable-plugin > > > > > --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre > > > > --with-cpu=generic > > > > > --host=x86_64-redhat-linux > > > > > Thread model: posix > > > > > gcc version 4.1.2 20080704 (Red Hat 4.1.2-46) > > > > > configure:2781: $? = 0 > > > > > configure:2788: gcc -V >&5 > > > > > gcc: '-V' option must have argument > > > > > configure:2791: $? = 1 > > > > > configure:2814: checking for C compiler default output file name > > > > > configure:2841: gcc -m32 conftest.c >&5 > > > > > /usr/bin/ld: crt1.o: No such file: No such file or directory > > > > > collect2: ld returned 1 exit status > > > > > configure:2844: $? = 1 > > > > > configure:2882: result: > > > > > configure: failed program was: > > > > > | /* confdefs.h. */ > > > > > | #define PACKAGE_NAME "gpl-compression" > > > > > | #define PACKAGE_TARNAME "gpl-compression" > > > > > | #define PACKAGE_VERSION "0.1.0" > > > > > | #define PACKAGE_STRING "gpl-compression 0.1.0" > > > > > | #define PACKAGE_BUGREPORT "omal...@apache.org" > > > > > | #define PACKAGE "gpl-compression" > > > > > | #define VERSION "0.1.0" > > > > > | #define _GNU_SOURCE 1 > > > > > | /* end confdefs.h. */ > > > > > | > > > > > | int > > > > > | main () > > > > > | { > > > > > | > > > > > | ; > > > > > | return 0; > > > > > | } > > > > > configure:2889: error: C compiler cannot create executables > > > > > See `config.log' for more details. > > > > > > > > > > > > > > > On Fri, Jan 15, 2010 at 1:17 PM, Todd Lipcon <t...@cloudera.com> > > > wrote: > > > > > > > > > > > Are you starting from a clean tarball of the lzo stuff? > > > > > > > > > > > > Can you make sure your /tmp/ partition isn't full? > > > > > > > > > > > > There should be a config.log file hanging around somewhere also - > > > > perhaps > > > > > > this would be helpful. > > > > > > > > > > > > -Todd > > > > > > > > > > > > On Fri, Jan 15, 2010 at 1:14 PM, Ted Yu <yuzhih...@gmail.com> > > wrote: > > > > > > > > > > > > > Todd: > > > > > > > Thanks for the continued support. > > > > > > > > > > > > > > I installed lzo-devel: > > > > > > > [r...@tyu-linux batchclient]# rpm -ivh > > > > > > > /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm > > > > > > > warning: /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm: > Header > > > V3 > > > > > DSA > > > > > > > signature: NOKEY, key ID 217521f6 > > > > > > > Preparing... > > > > ########################################### > > > > > > > [100%] > > > > > > > 1:lzo-devel > > > > ########################################### > > > > > > > [100%] > > > > > > > > > > > > > > But I still see: > > > > > > > [exec] Try `/usr/bin/install --help' for more information. > > > > > > > [exec] checking for gcc... gcc > > > > > > > [exec] checking for C compiler default output file name... > > > > > > > [exec] configure: error: C compiler cannot create > executables > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 15, 2010 at 12:09 PM, Todd Lipcon < > t...@cloudera.com > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Ted, > > > > > > > > > > > > > > > > Did you also install liblzo-devel? > > > > > > > > > > > > > > > > Here are the packages I install for LZO: > > > > > > > > > > > > > > > > lzo-2.02-2.el5.1 > > > > > > > > lzo-devel-2.02-2.el5.1 > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 4:28 PM, Ted Yu <yuzhih...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > > > > > Package gcc-4.1.2-46.el5_4.1.x86_64 already installed and > > > latest > > > > > > > version > > > > > > > > > Nothing to do > > > > > > > > > > > > > > > > > > [r...@tyu-linux batchclient]# yum install gcc-c++ > > > > > > > > > Loaded plugins: fastestmirror > > > > > > > > > Loading mirror speeds from cached hostfile > > > > > > > > > * base: mirrors.kernel.org > > > > > > > > > * updates: updates.interworx.info > > > > > > > > > * addons: mirrors.kernel.org > > > > > > > > > * extras: mirrors.netdna.com > > > > > > > > > Setting up Install Process > > > > > > > > > Parsing package install arguments > > > > > > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed > and > > > > > latest > > > > > > > > > version > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 2:32 PM, Todd Lipcon < > > > t...@cloudera.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Make sure you have gcc, g++ installed? > > > > > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 10:57 AM, Ted Yu < > > > yuzhih...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I installed > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/lzo-2.02-2.el5.1.x86_64.rpmyesterday > > > > > > > > > > > . > > > > > > > > > > > > > > > > > > > > > > [r...@tyu-linux software]# rpm -e lzo > > > > > > > > > > > [r...@tyu-linux software]# rpm -ivh > > > > > > > > > ~rialto/lzo-2.02-2.el5.1.x86_64.rpm > > > > > > > > > > > warning: /home/rialto/lzo-2.02-2.el5.1.x86_64.rpm: > Header > > > V3 > > > > > DSA > > > > > > > > > > signature: > > > > > > > > > > > NOKEY, key ID 217521f6 > > > > > > > > > > > Preparing... > > > > > > > > ########################################### > > > > > > > > > > > [100%] > > > > > > > > > > > 1:lzo > > > > > > > > ########################################### > > > > > > > > > > > [100%] > > > > > > > > > > > > > > > > > > > > > > [ria...@tyu-linux java]$ ls -l > /usr/lib64/liblzo2.so.2* > > > > > > > > > > > lrwxrwxrwx 1 root root 16 Jan 12 10:29 > > > > > > /usr/lib64/liblzo2.so.2 > > > > > > > -> > > > > > > > > > > > liblzo2.so.2.0.0 > > > > > > > > > > > -rwxr-xr-x 1 root root 123408 May 6 2007 > > > > > > > > /usr/lib64/liblzo2.so.2.0.0 > > > > > > > > > > > > > > > > > > > > > > I still see this error: > > > > > > > > > > > [exec] configure: error: C compiler cannot create > > > > > > executables > > > > > > > > > > > [exec] See `config.log' for more details. > > > > > > > > > > > > > > > > > > > > > > BUILD FAILED > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: > > > exec > > > > > > > > returned: > > > > > > > > > > 77 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 10:25 AM, Todd Lipcon < > > > > > t...@cloudera.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Ted, > > > > > > > > > > > > > > > > > > > > > > > > Please make sure you have version 2.02 of liblzo > > > installed. > > > > > > > There's > > > > > > > > > an > > > > > > > > > > > > earlier version in RHEL that doesn't fail properly at > > the > > > > > > > configure > > > > > > > > > > stage > > > > > > > > > > > > and I've seen it produce the error you pasted below. > > > > > > > > > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 12, 2010 at 10:15 AM, Ted Yu < > > > > > yuzhih...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > I followed > > > > > > > > > http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ > > > > > > > > > > > > > > > > > > > > > > > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already > > > installed > > > > > and > > > > > > > > > latest > > > > > > > > > > > > > version > > > > > > > > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul > 14 > > > > > 06:36:37 > > > > > > > EDT > > > > > > > > > > 2009 > > > > > > > > > > > > > x86_64 x86_64 x86_64 GNU/Linux > > > > > > > > > > > > > > > > > > > > > > > > > > For 32 bit build, I got: > > > > > > > > > > > > > [exec] checking for C compiler default output > > file > > > > > > name... > > > > > > > > > > > > > [exec] configure: error: C compiler cannot > create > > > > > > > executables > > > > > > > > > > > > > [exec] See `config.log' for more details. > > > > > > > > > > > > > > > > > > > > > > > > > > BUILD FAILED > > > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: > > > > > exec > > > > > > > > > > returned: > > > > > > > > > > > > 77 > > > > > > > > > > > > > > > > > > > > > > > > > > For 64 bit build, I got: > > > > > > > > > > > > > [exec] gcc -DHAVE_CONFIG_H -I. > > > > > > > > > > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native > > > > > > -I./impl > > > > > > > > > > > > > -I/usr/java/default/include > > > > > -I/usr/java/default/include/linux > > > > > > > > > > > > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl > > > > > > > > > > > > > -Isrc/com/hadoop/compression/lzo -g -Wall -fPIC -O2 > > > -m64 > > > > > -m64 > > > > > > > -MT > > > > > > > > > > > > > impl/lzo/LzoCompressor.lo -MD -MP -MF > > > > > > > > > > impl/lzo/.deps/LzoCompressor.Tpo > > > > > > > > > > > -c > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c > > > > > > > > > > > > > -fPIC -DPIC -o impl/lzo/.libs/LzoCompressor.o > > > > > > > > > > > > > [exec] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c: > > > > > > > > > > > > > In function > > > > > > > > > 'Java_com_hadoop_compression_lzo_LzoCompressor_initIDs': > > > > > > > > > > > > > [exec] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c:123: > > > > > > > > > > > > > error: expected expression before ',' token > > > > > > > > > > > > > [exec] make: *** [impl/lzo/LzoCompressor.lo] > > Error > > > 1 > > > > > > > > > > > > > > > > > > > > > > > > > > BUILD FAILED > > > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:251: > > > > > exec > > > > > > > > > > returned: > > > > > > > > > > > 2 > > > > > > > > > > > > > > > > > > > > > > > > > > Has anybody seen the above ? > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jan 11, 2010 at 3:34 PM, Todd Lipcon < > > > > > > > t...@cloudera.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Ted, > > > > > > > > > > > > > > > > > > > > > > > > > > > > You need to install liblzo from EPEL: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://fr.rpmfind.net/linux/RPM/Extras_Packages_for_Enterprise_Linux.html > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Todd > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jan 11, 2010 at 3:21 PM, Ted Yu < > > > > > > yuzhih...@gmail.com > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can someone tell me how I can install liblzo ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [r...@tyu-linux lzo-2.03]# uname -a > > > > > > > > > > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue > Jul > > > 14 > > > > > > > 06:36:37 > > > > > > > > > EDT > > > > > > > > > > > > 2009 > > > > > > > > > > > > > > > x86_64 x86_64 x86_64 GNU/Linux > > > > > > > > > > > > > > > [r...@tyu-linux lzo-2.03]# yum install > > > liblzo-devel > > > > > > > > > > > > > > > Loaded plugins: fastestmirror > > > > > > > > > > > > > > > Loading mirror speeds from cached hostfile > > > > > > > > > > > > > > > * base: mirrors.usc.edu > > > > > > > > > > > > > > > * updates: mirror.san.fastserv.com > > > > > > > > > > > > > > > * addons: centos.promopeddler.com > > > > > > > > > > > > > > > * extras: mirrors.versaweb.com > > > > > > > > > > > > > > > Setting up Install Process > > > > > > > > > > > > > > > Parsing package install arguments > > > > > > > > > > > > > > > No package liblzo-devel available. > > > > > > > > > > > > > > > Nothing to do > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jan 11, 2010 at 12:45 PM, Steve Kuo < > > > > > > > > > kuosen...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Ted, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You may want to consider LZO compression, > which > > > > > allows > > > > > > > > > > splitting > > > > > > > > > > > > for > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > comporessed file for Map jobs. On the other > > > hand, > > > > > gzip > > > > > > > is > > > > > > > > > not > > > > > > > > > > > > > > > splittable. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Check out these links. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.cloudera.com/blog/2009/11/17/hadoop-at-twitter-part-1-splittable-lzo-compression/ > > > > > > > > > > > > > > > > > > > http://wiki.apache.org/hadoop/UsingLzoCompression > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 1:13 PM, Ted Yu < > > > > > > > > yuzhih...@gmail.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The input file is in .gz format > > > > > > > > > > > > > > > > > FYI > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 11:08 AM, Ted Yu < > > > > > > > > > yuzhih...@gmail.com > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > My current project processes input file > of > > > size > > > > > > > > 333302161 > > > > > > > > > > > > bytes. > > > > > > > > > > > > > > > > > > What I plan to do is to split the file > into > > > > equal > > > > > > > size > > > > > > > > > > pieces > > > > > > > > > > > > > (and > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > blank > > > > > > > > > > > > > > > > > > line boundary) to improve performance. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I found 12 classes in 0.20.1 source code > > > which > > > > > > > > implement > > > > > > > > > > > > > > InputSplit. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > If someone has written code similar to > what > > I > > > > > plan > > > > > > to > > > > > > > > do, > > > > > > > > > > > > please > > > > > > > > > > > > > > > share > > > > > > > > > > > > > > > > > some > > > > > > > > > > > > > > > > > > hint. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 2:27 AM, Amogh > > Vasekar > > > < > > > > > > > > > > > > > am...@yahoo-inc.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> Hi, > > > > > > > > > > > > > > > > > >> The deprecation is due to the new > evolving > > > > > > mapreduce > > > > > > > ( > > > > > > > > > > > > > > > o.a.h.mapreduce > > > > > > > > > > > > > > > > ) > > > > > > > > > > > > > > > > > >> APIs. Old APIs are supported for > available > > > > > > > > > distributions. > > > > > > > > > > > The > > > > > > > > > > > > > > > > equivalent > > > > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > >> TextInputFormat is available in new API > : > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Thanks, > > > > > > > > > > > > > > > > > >> Amogh > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> On 1/8/10 3:47 AM, "Ted Yu" < > > > > > yuzhih...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> According to: > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/TextInputFormat.html#isSplitable%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path%29 > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> isSplitable() is deprecated. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Which method should I use to replace it > ? > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Thanks > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >