Hi Ted,
Looks like you may be running a 32-bit JDK, so it's trying to compile 32-bit
GPL compression libs. But, you're probably missing some parts of the 32-bit
toolchain.
-Todd
On Fri, Jan 15, 2010 at 1:34 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> I tried to set LD_LIBRARY_PATH to point to where crt1.o resides:
> export LD_LIBRARY_PATH=/usr/lib64
>
> Still the same issue.
> This is snippet of config.log:
>
> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
> --infodir=/usr/share/info --enable-shared --enable-threads=posix
> --enable-checking=release --with-system-zlib --enable-__cxa_atexit
> --disable-libunwind-exceptions --enable-libgcj-multifile
> --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
> --enable-java-awt=gtk
> --disable-dssi --enable-plugin
> --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic
> --host=x86_64-redhat-linux
> Thread model: posix
> gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
> configure:2781: $? = 0
> configure:2788: gcc -V >&5
> gcc: '-V' option must have argument
> configure:2791: $? = 1
> configure:2814: checking for C compiler default output file name
> configure:2841: gcc -m32 conftest.c >&5
> /usr/bin/ld: crt1.o: No such file: No such file or directory
> collect2: ld returned 1 exit status
> configure:2844: $? = 1
> configure:2882: result:
> configure: failed program was:
> | /* confdefs.h. */
> | #define PACKAGE_NAME "gpl-compression"
> | #define PACKAGE_TARNAME "gpl-compression"
> | #define PACKAGE_VERSION "0.1.0"
> | #define PACKAGE_STRING "gpl-compression 0.1.0"
> | #define PACKAGE_BUGREPORT "omal...@apache.org"
> | #define PACKAGE "gpl-compression"
> | #define VERSION "0.1.0"
> | #define _GNU_SOURCE 1
> | /* end confdefs.h. */
> |
> | int
> | main ()
> | {
> |
> | ;
> | return 0;
> | }
> configure:2889: error: C compiler cannot create executables
> See `config.log' for more details.
>
>
> On Fri, Jan 15, 2010 at 1:17 PM, Todd Lipcon <t...@cloudera.com> wrote:
>
> > Are you starting from a clean tarball of the lzo stuff?
> >
> > Can you make sure your /tmp/ partition isn't full?
> >
> > There should be a config.log file hanging around somewhere also - perhaps
> > this would be helpful.
> >
> > -Todd
> >
> > On Fri, Jan 15, 2010 at 1:14 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >
> > > Todd:
> > > Thanks for the continued support.
> > >
> > > I installed lzo-devel:
> > > [r...@tyu-linux batchclient]# rpm -ivh
> > > /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm
> > > warning: /opt/kindsight/lzo-devel-2.02-2.el5.1.x86_64.rpm: Header V3
> DSA
> > > signature: NOKEY, key ID 217521f6
> > > Preparing... ###########################################
> > > [100%]
> > > 1:lzo-devel ###########################################
> > > [100%]
> > >
> > > But I still see:
> > > [exec] Try `/usr/bin/install --help' for more information.
> > > [exec] checking for gcc... gcc
> > > [exec] checking for C compiler default output file name...
> > > [exec] configure: error: C compiler cannot create executables
> > >
> > >
> > > On Fri, Jan 15, 2010 at 12:09 PM, Todd Lipcon <t...@cloudera.com>
> wrote:
> > >
> > > > Hi Ted,
> > > >
> > > > Did you also install liblzo-devel?
> > > >
> > > > Here are the packages I install for LZO:
> > > >
> > > > lzo-2.02-2.el5.1
> > > > lzo-devel-2.02-2.el5.1
> > > >
> > > > -Todd
> > > >
> > > > On Tue, Jan 12, 2010 at 4:28 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > > >
> > > > > Package gcc-4.1.2-46.el5_4.1.x86_64 already installed and latest
> > > version
> > > > > Nothing to do
> > > > >
> > > > > [r...@tyu-linux batchclient]# yum install gcc-c++
> > > > > Loaded plugins: fastestmirror
> > > > > Loading mirror speeds from cached hostfile
> > > > > * base: mirrors.kernel.org
> > > > > * updates: updates.interworx.info
> > > > > * addons: mirrors.kernel.org
> > > > > * extras: mirrors.netdna.com
> > > > > Setting up Install Process
> > > > > Parsing package install arguments
> > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed and
> latest
> > > > > version
> > > > >
> > > > >
> > > > > On Tue, Jan 12, 2010 at 2:32 PM, Todd Lipcon <t...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > Make sure you have gcc, g++ installed?
> > > > > >
> > > > > > -Todd
> > > > > >
> > > > > > On Tue, Jan 12, 2010 at 10:57 AM, Ted Yu <yuzhih...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > I installed
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/lzo-2.02-2.el5.1.x86_64.rpmyesterday
> > > > > > > .
> > > > > > >
> > > > > > > [r...@tyu-linux software]# rpm -e lzo
> > > > > > > [r...@tyu-linux software]# rpm -ivh
> > > > > ~rialto/lzo-2.02-2.el5.1.x86_64.rpm
> > > > > > > warning: /home/rialto/lzo-2.02-2.el5.1.x86_64.rpm: Header V3
> DSA
> > > > > > signature:
> > > > > > > NOKEY, key ID 217521f6
> > > > > > > Preparing...
> > > > ###########################################
> > > > > > > [100%]
> > > > > > > 1:lzo
> > > > ###########################################
> > > > > > > [100%]
> > > > > > >
> > > > > > > [ria...@tyu-linux java]$ ls -l /usr/lib64/liblzo2.so.2*
> > > > > > > lrwxrwxrwx 1 root root 16 Jan 12 10:29
> > /usr/lib64/liblzo2.so.2
> > > ->
> > > > > > > liblzo2.so.2.0.0
> > > > > > > -rwxr-xr-x 1 root root 123408 May 6 2007
> > > > /usr/lib64/liblzo2.so.2.0.0
> > > > > > >
> > > > > > > I still see this error:
> > > > > > > [exec] configure: error: C compiler cannot create
> > executables
> > > > > > > [exec] See `config.log' for more details.
> > > > > > >
> > > > > > > BUILD FAILED
> > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: exec
> > > > returned:
> > > > > > 77
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jan 12, 2010 at 10:25 AM, Todd Lipcon <
> t...@cloudera.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi Ted,
> > > > > > > >
> > > > > > > > Please make sure you have version 2.02 of liblzo installed.
> > > There's
> > > > > an
> > > > > > > > earlier version in RHEL that doesn't fail properly at the
> > > configure
> > > > > > stage
> > > > > > > > and I've seen it produce the error you pasted below.
> > > > > > > >
> > > > > > > > -Todd
> > > > > > > >
> > > > > > > > On Tue, Jan 12, 2010 at 10:15 AM, Ted Yu <
> yuzhih...@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > I followed
> > > > > http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ
> > > > > > > > >
> > > > > > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed
> and
> > > > > latest
> > > > > > > > > version
> > > > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14
> 06:36:37
> > > EDT
> > > > > > 2009
> > > > > > > > > x86_64 x86_64 x86_64 GNU/Linux
> > > > > > > > >
> > > > > > > > > For 32 bit build, I got:
> > > > > > > > > [exec] checking for C compiler default output file
> > name...
> > > > > > > > > [exec] configure: error: C compiler cannot create
> > > executables
> > > > > > > > > [exec] See `config.log' for more details.
> > > > > > > > >
> > > > > > > > > BUILD FAILED
> > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243:
> exec
> > > > > > returned:
> > > > > > > > 77
> > > > > > > > >
> > > > > > > > > For 64 bit build, I got:
> > > > > > > > > [exec] gcc -DHAVE_CONFIG_H -I.
> > > > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native
> > -I./impl
> > > > > > > > > -I/usr/java/default/include
> -I/usr/java/default/include/linux
> > > > > > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl
> > > > > > > > > -Isrc/com/hadoop/compression/lzo -g -Wall -fPIC -O2 -m64
> -m64
> > > -MT
> > > > > > > > > impl/lzo/LzoCompressor.lo -MD -MP -MF
> > > > > > impl/lzo/.deps/LzoCompressor.Tpo
> > > > > > > -c
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c
> > > > > > > > > -fPIC -DPIC -o impl/lzo/.libs/LzoCompressor.o
> > > > > > > > > [exec]
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c:
> > > > > > > > > In function
> > > > > 'Java_com_hadoop_compression_lzo_LzoCompressor_initIDs':
> > > > > > > > > [exec]
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c:123:
> > > > > > > > > error: expected expression before ',' token
> > > > > > > > > [exec] make: *** [impl/lzo/LzoCompressor.lo] Error 1
> > > > > > > > >
> > > > > > > > > BUILD FAILED
> > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:251:
> exec
> > > > > > returned:
> > > > > > > 2
> > > > > > > > >
> > > > > > > > > Has anybody seen the above ?
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > >
> > > > > > > > > On Mon, Jan 11, 2010 at 3:34 PM, Todd Lipcon <
> > > t...@cloudera.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Ted,
> > > > > > > > > >
> > > > > > > > > > You need to install liblzo from EPEL:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://fr.rpmfind.net/linux/RPM/Extras_Packages_for_Enterprise_Linux.html
> > > > > > > > > >
> > > > > > > > > > -Todd
> > > > > > > > > >
> > > > > > > > > > On Mon, Jan 11, 2010 at 3:21 PM, Ted Yu <
> > yuzhih...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Can someone tell me how I can install liblzo ?
> > > > > > > > > > >
> > > > > > > > > > > [r...@tyu-linux lzo-2.03]# uname -a
> > > > > > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14
> > > 06:36:37
> > > > > EDT
> > > > > > > > 2009
> > > > > > > > > > > x86_64 x86_64 x86_64 GNU/Linux
> > > > > > > > > > > [r...@tyu-linux lzo-2.03]# yum install liblzo-devel
> > > > > > > > > > > Loaded plugins: fastestmirror
> > > > > > > > > > > Loading mirror speeds from cached hostfile
> > > > > > > > > > > * base: mirrors.usc.edu
> > > > > > > > > > > * updates: mirror.san.fastserv.com
> > > > > > > > > > > * addons: centos.promopeddler.com
> > > > > > > > > > > * extras: mirrors.versaweb.com
> > > > > > > > > > > Setting up Install Process
> > > > > > > > > > > Parsing package install arguments
> > > > > > > > > > > No package liblzo-devel available.
> > > > > > > > > > > Nothing to do
> > > > > > > > > > >
> > > > > > > > > > > Thanks
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Jan 11, 2010 at 12:45 PM, Steve Kuo <
> > > > > kuosen...@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Ted,
> > > > > > > > > > > >
> > > > > > > > > > > > You may want to consider LZO compression, which
> allows
> > > > > > splitting
> > > > > > > > for
> > > > > > > > > a
> > > > > > > > > > > > comporessed file for Map jobs. On the other hand,
> gzip
> > > is
> > > > > not
> > > > > > > > > > > splittable.
> > > > > > > > > > > >
> > > > > > > > > > > > Check out these links.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://www.cloudera.com/blog/2009/11/17/hadoop-at-twitter-part-1-splittable-lzo-compression/
> > > > > > > > > > > > http://wiki.apache.org/hadoop/UsingLzoCompression
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Jan 8, 2010 at 1:13 PM, Ted Yu <
> > > > yuzhih...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > The input file is in .gz format
> > > > > > > > > > > > > FYI
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Jan 8, 2010 at 11:08 AM, Ted Yu <
> > > > > yuzhih...@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > My current project processes input file of size
> > > > 333302161
> > > > > > > > bytes.
> > > > > > > > > > > > > > What I plan to do is to split the file into equal
> > > size
> > > > > > pieces
> > > > > > > > > (and
> > > > > > > > > > on
> > > > > > > > > > > > > blank
> > > > > > > > > > > > > > line boundary) to improve performance.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I found 12 classes in 0.20.1 source code which
> > > > implement
> > > > > > > > > > InputSplit.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If someone has written code similar to what I
> plan
> > to
> > > > do,
> > > > > > > > please
> > > > > > > > > > > share
> > > > > > > > > > > > > some
> > > > > > > > > > > > > > hint.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 2:27 AM, Amogh Vasekar <
> > > > > > > > > am...@yahoo-inc.com
> > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Hi,
> > > > > > > > > > > > > >> The deprecation is due to the new evolving
> > mapreduce
> > > (
> > > > > > > > > > > o.a.h.mapreduce
> > > > > > > > > > > > )
> > > > > > > > > > > > > >> APIs. Old APIs are supported for available
> > > > > distributions.
> > > > > > > The
> > > > > > > > > > > > equivalent
> > > > > > > > > > > > > of
> > > > > > > > > > > > > >> TextInputFormat is available in new API :
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Thanks,
> > > > > > > > > > > > > >> Amogh
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On 1/8/10 3:47 AM, "Ted Yu" <
> yuzhih...@gmail.com>
> > > > > wrote:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> According to:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/TextInputFormat.html#isSplitable%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path%29
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> isSplitable() is deprecated.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Which method should I use to replace it ?
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Thanks
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
