HBase 0.1.2 is pegged against 0.16.3, not hadoop 0.17.0. I don't think
the two will work together.
Also, be sure to pick up the new 0.1.2 candidate (trying to put it up
now). May improvements over the first candidate.
St.Ack
Jim R. Wilson wrote:
you would need to build new images anyway since you need HBase installed
and started at boot time.
Right-o. I'll be setting up a local dev environment to build fresh
fedora-8 AMI's with hadoop-0.17-pre and hbase-0.1.2-pre starting in
the morning.
Of course, I'll probably be creating an ec2 contrib dir in hbase with
hbase-init script etc. I'm tempted to try and make my image
multi-versioned so that it has both the latest releases hadoop/hbase
as well as pre-released versions, then make it up to user-data params
to decide which will be started.
The idea of course being that the running the the latest bleeding edge
hbase/hadoop or latest stable could be done with the same image.
-- Jim
On Wed, May 7, 2008 at 5:22 PM, Chris K Wensel <[EMAIL PROTECTED]> wrote:
sorry if I wasn't clear that the new scripts and old scripts were not
compatible. thus their being in 0.17.0, not 0.16.4.
you would need to build new images anyway since you need HBase installed
and started at boot time.
On May 7, 2008, at 3:15 PM, Jim R. Wilson wrote:
I've come to the conclusion that using the contrib/ec2 scripts from
hadoop 0.17 is incompatible with the prebuilt hadoop-0.16.1 image
currently available in the hadoop-ec2-images bucket (ami-461df82f
hadoop-ec2-images/hadoop-0.16.1.manifest.xml to be precise).
The problem is that the user-data passed in by 0.17 has a different
format than what is expected by the hadoop-init script packaged with
0.16.1. Specifically, 0.17's user -data is meant to be a comma
delimited list of bash var settings of the form KEY=VAL, whereas
0.16.x seems to expect just a comma delimited list of values whose
keys are known by their ordinal placement (that is, the first value is
the number of instances, the second value is the name of the master
node).
So now I'm back to the idea that I'm going to have to build myself an
ec2 AMI with hadoop 0.17 from "scratch" (using the create-instance
scripts of course). This isn't /too/ much more work than I'd have to
do anyway. I plan on running hbase on my cluster as well as python
hadoop-streaming jobs which was going to require other libraries (like
SQLAlchemy and Thrift). These items were going to necessitate
creating my own images anyway :/
-- Jim
On Wed, May 7, 2008 at 3:51 PM, Jim R. Wilson <[EMAIL PROTECTED]>
wrote:
keep the questions coming. will be glad to see HBase running on ec2,
maybe
we can put your changes back into the tree.
Thanks :)
Just prior to this excursion, I found some things needing tweaking to
work with my hosting provider. Now that I'm moving to ec2, I'll be on
the lookout for similar issues. I'll submit any patches I end up
making.
-- Jim
On Wed, May 7, 2008 at 3:39 PM, Chris K Wensel <[EMAIL PROTECTED]> wrote:
In the aforementioned EC2 wiki page, it has a configuration section
labeled "(Pre 0.17) Hadoop cluster variables (GROUP, MASTER_HOST,
NO_INSTANCES)". I had assumed that I would actually need
hadoop-0.17
on my ec2 instances in order to forgo those instructions. It sounds
like you're telling me that all the "pre 0.17" or "0.17" specific
instructions in the wiki page refer only to the ec2 creation
scripts,
not the actual running version in the cluster, is that correct?
correct.
the only reason these scripts are in the 0.17 branch is because they
are
not backward compatible with themselves.
Also, how safe is it to run a different version of the ec2 scripts
from the actual running hadoop instance? I'm guessing it's pretty
safe since you suggested it :)
there are no dependencies between the EC2 scripts and Hadoop core. you
can
use them with any version, as long as you build EC2 Images for the
versions
of Hadoop you are after with the 'new' scripts.
Thanks again for all the help - still wrapping my mind around this
stuff.
keep the questions coming. will be glad to see HBase running on ec2,
maybe
we can put your changes back into the tree.
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
http://www.cascading.org/
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
http://www.cascading.org/
