Thanks for the responses. Saurabh, Thanks for the response. I got LZO installed without the packager, but I do appreciate the help as the rpm method seems like a better way to go.
Pete, The clusters are all 64-bit machines. That doesn't matter at this point because I was able to get this to work without the packager by reading many, many web pages. I had wanted to use the packager because I assumed it would resolve open questions like: if the code evolved from GoogleCode->KevinWeil->ToddLipconn, and the GoogleCode says to use branch 0.1 because the trunk is for the new Hadoop APIs (which Hive doesn't use yet), then what's the right thing to install. I do have LZO working now with Hive. Tables can now be compressed as LZO, with Indexes, with data validated via lzop directly, and I can use intermediate LZO compression. That took more work than I anticipated, so I'm still interested in the packager. To sum up what I've done: I replaced "wget" with "wget --no-check-certificate" in run.sh. I did replace the string in the file you mentioned, but the only file with that name is in the build directory [root@adhocmaster01an hadoop-lzo-packager]# find ./ -name "*toddlipcon-hadoop-lzo.spec*" ./build/topdir/SPECS/toddlipcon-hadoop-lzo.spec Unsurprisingly, executing "run.sh" again yielded the original error message. It seems I should be able to change the version tag "upstream" somewhere, but I don't have enough time now to research SPEC files and how deep this rabbit hole might go. I do appreciate the body of work that this is all built on and your responses, but I am surprised that there isn't a more comprehensive set of How-Tos. I'll assist in writing up something like that when I get a chance. Are other people using CDH3B3 with 64-bit CentOS/RedHat, or is there something I'm missing? Thanks for the assistance. On Sun, Mar 6, 2011 at 9:25 PM, Pete Haidinyak <javam...@cox.net> wrote: > Are you doing a 64bit build or doesn't it matter? > > Thanks > > -Pete > > > On Sat, 05 Mar 2011 21:35:30 -0800, Saurabh Dutta < > saurabh.du...@impetus.co.in> wrote: > > Hi, >> >> Try and open the file "toddlipcon-hadoop-lzo.spec" in your SPECS directory >> and replace the '-' with '_' in line number 3. See if this works for you. >> >> Thanks, >> Saurabh >> ________________________________________ >> From: phil young [phil.wills.yo...@gmail.com] >> Sent: Sunday, March 06, 2011 4:27 AM >> To: hadoop-common-user >> Subject: LZO Packager on Centos 5.5 fails >> >> I'm trying to install LZO on CDH3B3, using the LZO Packager tool here ( >> https://github.com/toddlipcon/hadoop-lzo-packager) >> I'm running the following on Centos 5.5. >> >> I'm not an expert on RPM building. Does anyone have any suggestions about >> how to debug this? >> >> hadoop-lzo-packager]# ./run.sh >> + ANT_VERSION=1.8.1 >> + ANT_TARBALL=apache-ant-1.8.1-bin.tar.gz >> + ANT_TARBALL_URL= >> >> http://www.gtlib.gatech.edu/pub/apache/ant/binaries/apache-ant-1.8.1-bin.tar.gz >> + '[' -n '' ']' >> + '[' -n '' ']' >> + SRC_PROJECT=github >> + RELEASE=1 >> ++ getent passwd root >> ++ cut -d: -f5 >> ++ cut -d, -f1 >> + PACKAGER=root >> ++ hostname -f >> + HOST=pyoung-dev.tripadvisor.com >> + PACKAGER_EMAIL=r...@pyoung-dev.tripadvisor.com >> + HADOOP_HOME=/usr/lib/hadoop >> +++ dirname ./run.sh >> ++ readlink -f . >> + BINDIR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager >> + mkdir -p build >> + setup_github >> + GITHUB_ACCOUNT=toddlipcon >> + GITHUB_BRANCH=master >> + PACKAGE_HOMEPAGE=http://github.com/toddlipcon/hadoop-lzo >> + TARURL=http://github.com/toddlipcon/hadoop-lzo/tarball/master >> ++ ls >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz >> + '[' -z >> >> '/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz' >> ']' >> ++ ls -1 >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174723.0.4.9-3-g2bd0d5b.tar.gz >> ++ head -1 >> + >> >> ORIG_TAR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> ++ expr match >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> '.*hadoop-lzo-\(.*\).tar.gz' >> + GIT_HASH=0.4.9-3-g2bd0d5b >> + echo 'Git hash: 0.4.9-3-g2bd0d5b' >> Git hash: 0.4.9-3-g2bd0d5b >> + NAME=toddlipcon-hadoop-lzo >> ++ date +%Y%m%d%H%M%S >> + VERSION=20110305174749.0.4.9-3-g2bd0d5b >> + pushd /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/ >> + mkdir toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/ >> + tar -C toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/ >> --strip-components=1 -xzf >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-0.4.9-3-g2bd0d5b.tar.gz >> + tar czf toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz >> toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b/ >> + popd >> + >> >> CHECKOUT_TAR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz >> + >> TOPDIR=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir >> + >> >> CHECKOUT=/root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b >> + checkout_github >> + echo -n >> + '[' '!' -e >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz >> ']' >> + '[' -n '' ']' >> + '[' -z '' ']' >> + rm -Rf >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir >> + mkdir -p >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir >> + cd /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/ >> + mkdir SOURCES BUILD SPECS SRPMS RPMS BUILDROOT >> + cat /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/template.spec >> + do_substs >> + sed ' >> s,@PACKAGE_NAME@,toddlipcon-hadoop-lzo,g; >> s,@PACKAGE_HOMEPAGE@,http://github.com/toddlipcon/hadoop-lzo,g; >> s,@VERSION@,20110305174749.0.4.9-3-g2bd0d5b,g; >> s,@RELEASE@,1,g; >> s,@PACKAGER@,root,g; >> s,@PACKAGER_EMAIL@,r...@pyoung-dev.tripadvisor.com,g; >> s,@HADOOP_HOME@,/usr/lib/hadoop,g; >> ' >> + cp >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/toddlipcon-hadoop-lzo-20110305174749.0.4.9-3-g2bd0d5b.tar.gz >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SOURCES >> + pushd >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS >> ++ pwd >> ++ pwd >> + rpmbuild --buildroot >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS/../BUILDROOT >> --define '_topdir >> >> /root/pwy/lzo_research/toddlipcon/hadoop-lzo-packager/build/topdir/SPECS/..' >> -ba toddlipcon-hadoop-lzo.spec >> error: line 3: Illegal char '-' in version: Version: >> 20110305174749.0.4.9-3-g2bd0d5b >> >> ________________________________ >> >> Are you exploring a Big Data Strategy ? Listen to this recorded webinar on >> Planning your Hadoop/ NoSQL projects for 2011 at >> www.impetus.com/featured_webinar?eventid=37 >> >> Follow us on www.twitter.com/impetuscalling or visit www.impetus.com to >> know more. >> >> >> NOTE: This message may contain information that is confidential, >> proprietary, privileged or otherwise protected by law. The message is >> intended solely for the named addressee. If received in error, please >> destroy and notify the sender. Any use of this email is prohibited when >> received in error. Impetus does not represent, warrant and/or guarantee, >> that the integrity of this communication has been maintained nor that the >> communication is free of errors, virus, interception or interference. >> > >
