Package gcc-4.1.2-46.el5_4.1.x86_64 already installed and latest version Nothing to do
[r...@tyu-linux batchclient]# yum install gcc-c++ Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.kernel.org * updates: updates.interworx.info * addons: mirrors.kernel.org * extras: mirrors.netdna.com Setting up Install Process Parsing package install arguments Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed and latest version On Tue, Jan 12, 2010 at 2:32 PM, Todd Lipcon <t...@cloudera.com> wrote: > Make sure you have gcc, g++ installed? > > -Todd > > On Tue, Jan 12, 2010 at 10:57 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > I installed > > > > > ftp://fr.rpmfind.net/linux/EPEL/5/x86_64/lzo-2.02-2.el5.1.x86_64.rpmyesterday > > . > > > > [r...@tyu-linux software]# rpm -e lzo > > [r...@tyu-linux software]# rpm -ivh ~rialto/lzo-2.02-2.el5.1.x86_64.rpm > > warning: /home/rialto/lzo-2.02-2.el5.1.x86_64.rpm: Header V3 DSA > signature: > > NOKEY, key ID 217521f6 > > Preparing... ########################################### > > [100%] > > 1:lzo ########################################### > > [100%] > > > > [ria...@tyu-linux java]$ ls -l /usr/lib64/liblzo2.so.2* > > lrwxrwxrwx 1 root root 16 Jan 12 10:29 /usr/lib64/liblzo2.so.2 -> > > liblzo2.so.2.0.0 > > -rwxr-xr-x 1 root root 123408 May 6 2007 /usr/lib64/liblzo2.so.2.0.0 > > > > I still see this error: > > [exec] configure: error: C compiler cannot create executables > > [exec] See `config.log' for more details. > > > > BUILD FAILED > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: exec returned: > 77 > > > > > > On Tue, Jan 12, 2010 at 10:25 AM, Todd Lipcon <t...@cloudera.com> wrote: > > > > > Hi Ted, > > > > > > Please make sure you have version 2.02 of liblzo installed. There's an > > > earlier version in RHEL that doesn't fail properly at the configure > stage > > > and I've seen it produce the error you pasted below. > > > > > > -Todd > > > > > > On Tue, Jan 12, 2010 at 10:15 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > I followed http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ > > > > > > > > Package gcc-c++-4.1.2-46.el5_4.1.x86_64 already installed and latest > > > > version > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14 06:36:37 EDT > 2009 > > > > x86_64 x86_64 x86_64 GNU/Linux > > > > > > > > For 32 bit build, I got: > > > > [exec] checking for C compiler default output file name... > > > > [exec] configure: error: C compiler cannot create executables > > > > [exec] See `config.log' for more details. > > > > > > > > BUILD FAILED > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:243: exec > returned: > > > 77 > > > > > > > > For 64 bit build, I got: > > > > [exec] gcc -DHAVE_CONFIG_H -I. > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native -I./impl > > > > -I/usr/java/default/include -I/usr/java/default/include/linux > > > > -I/home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl > > > > -Isrc/com/hadoop/compression/lzo -g -Wall -fPIC -O2 -m64 -m64 -MT > > > > impl/lzo/LzoCompressor.lo -MD -MP -MF > impl/lzo/.deps/LzoCompressor.Tpo > > -c > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c > > > > -fPIC -DPIC -o impl/lzo/.libs/LzoCompressor.o > > > > [exec] > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c: > > > > In function 'Java_com_hadoop_compression_lzo_LzoCompressor_initIDs': > > > > [exec] > > > > > > > > > > > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/src/native/impl/lzo/LzoCompressor.c:123: > > > > error: expected expression before ',' token > > > > [exec] make: *** [impl/lzo/LzoCompressor.lo] Error 1 > > > > > > > > BUILD FAILED > > > > /home/rialto/kevinweil-hadoop-lzo-916aeae/build.xml:251: exec > returned: > > 2 > > > > > > > > Has anybody seen the above ? > > > > > > > > Thanks > > > > > > > > On Mon, Jan 11, 2010 at 3:34 PM, Todd Lipcon <t...@cloudera.com> > > wrote: > > > > > > > > > Hi Ted, > > > > > > > > > > You need to install liblzo from EPEL: > > > > > > > > > > > > > > > > > > > > http://fr.rpmfind.net/linux/RPM/Extras_Packages_for_Enterprise_Linux.html > > > > > > > > > > -Todd > > > > > > > > > > On Mon, Jan 11, 2010 at 3:21 PM, Ted Yu <yuzhih...@gmail.com> > wrote: > > > > > > > > > > > Can someone tell me how I can install liblzo ? > > > > > > > > > > > > [r...@tyu-linux lzo-2.03]# uname -a > > > > > > Linux tyu-linux 2.6.18-128.2.1.el5 #1 SMP Tue Jul 14 06:36:37 EDT > > > 2009 > > > > > > x86_64 x86_64 x86_64 GNU/Linux > > > > > > [r...@tyu-linux lzo-2.03]# yum install liblzo-devel > > > > > > Loaded plugins: fastestmirror > > > > > > Loading mirror speeds from cached hostfile > > > > > > * base: mirrors.usc.edu > > > > > > * updates: mirror.san.fastserv.com > > > > > > * addons: centos.promopeddler.com > > > > > > * extras: mirrors.versaweb.com > > > > > > Setting up Install Process > > > > > > Parsing package install arguments > > > > > > No package liblzo-devel available. > > > > > > Nothing to do > > > > > > > > > > > > Thanks > > > > > > > > > > > > On Mon, Jan 11, 2010 at 12:45 PM, Steve Kuo <kuosen...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > Ted, > > > > > > > > > > > > > > You may want to consider LZO compression, which allows > splitting > > > for > > > > a > > > > > > > comporessed file for Map jobs. On the other hand, gzip is not > > > > > > splittable. > > > > > > > > > > > > > > Check out these links. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.cloudera.com/blog/2009/11/17/hadoop-at-twitter-part-1-splittable-lzo-compression/ > > > > > > > http://wiki.apache.org/hadoop/UsingLzoCompression > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 1:13 PM, Ted Yu <yuzhih...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > The input file is in .gz format > > > > > > > > FYI > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 11:08 AM, Ted Yu <yuzhih...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > > > > > My current project processes input file of size 333302161 > > > bytes. > > > > > > > > > What I plan to do is to split the file into equal size > pieces > > > > (and > > > > > on > > > > > > > > blank > > > > > > > > > line boundary) to improve performance. > > > > > > > > > > > > > > > > > > I found 12 classes in 0.20.1 source code which implement > > > > > InputSplit. > > > > > > > > > > > > > > > > > > If someone has written code similar to what I plan to do, > > > please > > > > > > share > > > > > > > > some > > > > > > > > > hint. > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jan 8, 2010 at 2:27 AM, Amogh Vasekar < > > > > am...@yahoo-inc.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >> Hi, > > > > > > > > >> The deprecation is due to the new evolving mapreduce ( > > > > > > o.a.h.mapreduce > > > > > > > ) > > > > > > > > >> APIs. Old APIs are supported for available distributions. > > The > > > > > > > equivalent > > > > > > > > of > > > > > > > > >> TextInputFormat is available in new API : > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html > > > > > > > > >> > > > > > > > > >> Thanks, > > > > > > > > >> Amogh > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> On 1/8/10 3:47 AM, "Ted Yu" <yuzhih...@gmail.com> wrote: > > > > > > > > >> > > > > > > > > >> According to: > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/TextInputFormat.html#isSplitable%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path%29 > > > > > > > > >> > > > > > > > > >> isSplitable() is deprecated. > > > > > > > > >> > > > > > > > > >> Which method should I use to replace it ? > > > > > > > > >> > > > > > > > > >> Thanks > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
