Hi,
I am starting a cluster of Apache Hadoop distributions, like .18 and also
.19. This all works fine, then I log in. I see that the Hadoop daemons are
already working. However, when I try
# which hadoop
/usr/local/hadoop-0.19.0/bin/hadoop
# jps
1355 Jps
1167 NameNode
1213 JobTracker
# hadoop fs
Hi,
I was trying to set-up hadoop/hbase cluster on ec2 which took me few hours
to set-up from scratch on bundled image from s3. I am curious to know, what
is the best way to setting hadoop/hbase cluster on amazon ec2? How do we do
it fast?
Thanks in advance!
regards,
Shah
of these.
Kind regards
Steve Watt
From:
Mark Kerzner
To:
core-u...@hadoop.apache.org
Date:
11/24/2009 03:02 PM
Subject:
Hadoop on EC2
Hi,
I am starting a cluster of Apache Hadoop distributions, like .18 and also
.19. This all works fine, then I log in. I see that the Hadoop daemons are
rectory of each ec2 image that these values are being passed into that
> does the work of starting the tasktracker/datanode processes on each one
> of these.
>
> Kind regards
> Steve Watt
>
>
>
> From:
> Mark Kerzner
> To:
> core-u...@hadoop.apache.org
> Date:
&g
Well, maybe I found what I was doing wrong:
I was always using hdfs://localhost, and it works just as well with /
instead
Mark
On Tue, Nov 24, 2009 at 3:01 PM, Mark Kerzner wrote:
> Hi,
>
> I am starting a cluster of Apache Hadoop distributions, like .18 and also
> .19. This all works fine, th
If you use hadoop fs -ls hdfs:// that will work for your intent. Thanks!
On 11/25/09 2:31 AM, "Mark Kerzner" wrote:
Hi,
I am starting a cluster of Apache Hadoop distributions, like .18 and also
.19. This all works fine, then I log in. I see that the Hadoop daemons are
already working. However,
It did! thank you
hadoop fs -ls hdfs://
Found 1 items
drwxr-xr-x - root supergroup 0 2009-11-24 23:04 /mnt
On Tue, Nov 24, 2009 at 11:37 PM, Rekha Joshi wrote:
> If you use hadoop fs -ls hdfs:// that will work for your intent. Thanks!
>
> On 11/25/09 2:31 AM, "Mark Kerzner" wrote:
>
You are looking for the Apache Whirr project: http://whirr.apache.org/
Here's a great article at Phil Whelan's site that covers getting HBase
up in a jiffy on ec2:
http://www.philwhln.com/run-the-latest-whirr-and-deploy-hbase-in-minutes
On Wed, Sep 7, 2011 at 4:48 PM, Shahnawaz Saifi wrote:
> Hi
I second that. Whirr is an invaluable resource for automagically spinning
up resources on EC2
On Wed, Sep 7, 2011 at 4:28 AM, Harsh J wrote:
> You are looking for the Apache Whirr project: http://whirr.apache.org/
>
> Here's a great article at Phil Whelan's site that covers getting HBase
> up i
Interested in this topic. We have experienced plenty of difficulties
running hadoop in Eucalyptus based virtual instance clusters. Typical
issues like
java.net.SocketTimeoutException: 69000 millis timeout while waiting for
channel to be ready for read. ch : java.nio.channels.SocketChannel
k
Thanks a lot, I will definitely try this. But there are so many blogs about
configuring hadoop/hbase and bundling images to s3 bucket. Whirr is faster
or smoother than this concept?
regards,
Shah
On Wed, Sep 7, 2011 at 8:28 PM, John Conwell wrote:
> I second that. Whirr is an invaluable resour
I dont know about configuring hbase, but whirr definitely makes spinning up
a hadoop cluster very very easy. Checkout the quickstart guide. As long as
you have your EC2 creds, its very simple to ramp up a hadoop cluster.
http://whirr.apache.org/docs/0.6.0/quick-start-guide.html
On Wed, Sep 7, 20
Hi,
I am planning on running my MapReduce app on Amazon's EC2. I had a look at
the public Hadoop images in the hadoop-images bucket and there is no image
for the stable 0.18.3 release. The most recent Hadoop versions I see are
0.18.1 and 0.19.0. Which of those would be better to use? Or should I t
I can recommend the cloudera EC2 images. I am not sure what version
they are built on right now, but I think they pick stable ones and
apply critical patches I believe.
http://www.cloudera.com/hadoop-ec2
Cheers,
Tim
On Mon, Sep 7, 2009 at 7:08 AM, John Clarke wrote:
> Hi,
>
> I am planning on ru
Hi,
The EC2 scripts will boot Cloudera's distribution for Hadoop. Currently they
boot our distribution of 0.18.3, but 0.20 support should be ready pretty
soon now. Here's a list of what patches are in our newest 0.18.3
distribution:
http://archive.cloudera.com/cdh/testing/hadoop-0.18.3+70.CHANGES
Thanks for the replies, I've been developing against 0.18.3 under Windows XP
and testing on Ubuntu.
That seems like a long list of changes from 0.18.3! Should I expect any
specific issues if I try Cloudera's version on EC2 seeing as I have only
tested against the stock 0.18.3?
Cheers,
John
2
On Mon, Sep 7, 2009 at 8:14 AM, John Clarke wrote:
> Thanks for the replies, I've been developing against 0.18.3 under Windows
> XP
> and testing on Ubuntu.
>
> That seems like a long list of changes from 0.18.3! Should I expect any
> specific issues if I try Cloudera's version on EC2 seeing as I
Thanks Todd, when you said "The EC2 scripts will boot Cloudera's
distribution for Hadoop." were you referring to the EC2 scripts that come
with Hadoop or Cloudera's python scripts?
Are there advantages to using Cloudera's scripts over the stock ones for a
fairly standard job - i.e. up to 20 nodes
On Mon, Sep 7, 2009 at 9:25 AM, John Clarke wrote:
> Thanks Todd, when you said "The EC2 scripts will boot Cloudera's
> distribution for Hadoop." were you referring to the EC2 scripts that come
> with Hadoop or Cloudera's python scripts?
>
>
Sorry that wasn't clear -- the Cloudera scripts are ver
Thanks Todd. I got my EC2 cluster running without too much bother and my app
seems to work as expected.
John
2009/9/7 Todd Lipcon
> On Mon, Sep 7, 2009 at 9:25 AM, John Clarke wrote:
>
> > Thanks Todd, when you said "The EC2 scripts will boot Cloudera's
> > distribution for Hadoop." were you r
20 matches
Mail list logo
