@hadoop.apache.org
Sent: Mon, December 14, 2009 10:32:44 AM
Subject: Re: Starting HBase in fully distributed mode...
Yes, 'slaves' and 'regionservers' files cannot be generated as until the
instances start their identities are unknown. Some user script which runs
subsequent to the cluste
er@hadoop.apache.org
Sent: Mon, December 14, 2009 9:38:11 AM
Subject: Re: Starting HBase in fully distributed mode...
Andrew,
One thing I noticed with the ec2 scripts is that they don't update 'slaves'
file for hadoop as well as 'regionservers' file for hbase. As a result,
On Mon, Dec 14, 2009 at 9:38 AM, Something Something <
mailinglist...@gmail.com> wrote:
> Also, I believe this has been pointed out before, but it will be nice if
> hbase jar & zookeeper jar get copied automatically to hadoop lib because
> they are needed by MapReduce.
>
>
Or can just add referenc
d the changes will persist in your security group ACLs.
>
> - Andy
>
>
>
> ____________
> From: Something Something
> To: hbase-user@hadoop.apache.org
> Sent: Fri, December 11, 2009 9:50:30 AM
> Subject: Re: Starting HBase in fully distributed mode..
ty group ACLs.
- Andy
From: Something Something
To: hbase-user@hadoop.apache.org
Sent: Fri, December 11, 2009 9:50:30 AM
Subject: Re: Starting HBase in fully distributed mode...
1) Yes, I used the same cluster name. Okay, let me try again tonight, but
in any case, I
> - Andy
>
>
>
>
> From: Something Something
> To: hbase-user@hadoop.apache.org
> Sent: Thu, December 10, 2009 8:21:10 PM
> Subject: Re: Starting HBase in fully distributed mode...
>
> Andy,
>
> Thanks for the tips. It&#x
EC2 also?
Running mapreduce jobs on the HBase cluster itself is a work in progress.
- Andy
From: Something Something
To: hbase-user@hadoop.apache.org
Sent: Thu, December 10, 2009 8:21:10 PM
Subject: Re: Starting HBase in fully distributed mode...
Andy,
Th
Andy,
Thanks for the tips. It's all working now. I was using a different KeyPair
for EC2_ROOT_SSH_KEY. Once I changed this to use the root.pem it started
working. I was able to ssh to the 'master' instance and get into hbase
shell etc. This script is VERY helpful! Thank you so much.
A few q
We have HBase running on EC2 with starting Zookeeper within HBase. We have
it up since July 2009. No problems so far on Zookeeper front.
Regards,
Vaibhav Puranik
Gumgum
On Thu, Dec 10, 2009 at 8:12 AM, Something Something <
mailinglist...@gmail.com> wrote:
> Finally, I was able to get HBase runn
rom: Something Something
To: hbase-user@hadoop.apache.org
Sent: Wed, December 9, 2009 8:25:41 PM
Subject: Re: Starting HBase in fully distributed mode...
When I run:
hbase-ec2 launch-cluster testcluster 3 3
I keep getting 'lost connection' messages (See
Finally, I was able to get HBase running on EC2 in fully distributed mode.
I started ZooKeeper quorum myself and pointed HBase to it. I was able to
create tables using HBase shell, ran a Mapreduce job that writes to these
tables, and run queries against these tables. I used HBase shell from all
When I run:
hbase-ec2 launch-cluster testcluster 3 3
I keep getting 'lost connection' messages (See below). Tried this 4 times.
Please help. Thanks.
-
Creating/checking security groups
Security group testcluster-master exists, ok
Se
> Sounds like others have used Andrew's script successfully. The only
> difference seems to be that it starts a *dedicated* ZooKeeper quorum.
> Should have listened to Mark when he suggested that 4 days ago :)
>
> Anyway, I will try Andrew's script tomorrow.
I can vouch that the scripts in svn tr
> From: Patrick Hunt
> To: hbase-user@hadoop.apache.org
> Cc: Jean-Daniel Cryans
> Sent: Tue, December 8, 2009 3:31:53 PM
> Subject: Re: Starting HBase in fully distributed mode...
>
> I consulted with a ZK user who runs exclusively on EC2. He mentioned
> that one poten
hbase-user@hadoop.apache.org
Cc: Jean-Daniel Cryans
Sent: Tue, December 8, 2009 3:31:53 PM
Subject: Re: Starting HBase in fully distributed mode...
I consulted with a ZK user who runs exclusively on EC2. He mentioned
that one potential problem ZK users may be seeing is the following:
EC2 instanc
anned.
- Andy
From: Something Something
To: hbase-user@hadoop.apache.org
Sent: Tue, December 8, 2009 12:40:17 PM
Subject: Re: Starting HBase in fully distributed mode...
Cool. Didn't know there were scripts in the trunk for HBase. Will
definitely try those.
Should I first run t
o the config
and distributing it to the slaves, etc.
See subtasks on HBASE-1961 for further developments planned.
- Andy
From: Something Something
To: hbase-user@hadoop.apache.org
Sent: Tue, December 8, 2009 12:40:17 PM
Subject: Re: Starting HBase in fully distribut
application specific
importer-exporter map reduce job.
- Andy
From: Vaibhav Puranik
To: hbase-user@hadoop.apache.org
Sent: Tue, December 8, 2009 2:32:29 PM
Subject: Re: Starting HBase in fully distributed mode...
Something,
You can now boot your instance
; See subtasks on HBASE-1961 for further developments planned.
> >
> > - Andy
> >
> >
> >
> >
> >
> > From: Something Something
> > To: hbase-user@hadoop.apache.org
> > Sent: Tue, December 8, 2009 12:40:17 PM
> > Subject: Re: Starting HBase
s, etc.
>
> See subtasks on HBASE-1961 for further developments planned.
>
> - Andy
>
>
>
>
>
> From: Something Something
> To: hbase-user@hadoop.apache.org
> Sent: Tue, December 8, 2009 12:40:17 PM
> Subject: Re: Starting HBase in
distributing it to the slaves, etc.
See subtasks on HBASE-1961 for further developments planned.
- Andy
From: Something Something
To: hbase-user@hadoop.apache.org
Sent: Tue, December 8, 2009 12:40:17 PM
Subject: Re: Starting HBase in fully distributed mode...
Cool
Cool. Didn't know there were scripts in the trunk for HBase. Will
definitely try those.
Should I first run the scripts for Hadoop (from /src/contrib/ec2)
and then run scripts for HBase (from /src/contrib/ec2)? Or would the
HBase scripts install/format HDFS automatically for me so that I don't n
>> Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ --
>> do, and so I can say it works.
>
> I don't think there are scripts for HBase on EC2, are there?
We (HBase) have EC2 scripts -- src/contrib/ec2/, in SVN, in trunk and on
the 0.20 branch.
- Andy
009 at 7:58 AM, Andrew Purtell wrote:
> Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ -- do, and
> so I can say it works.
>
> - Andy
>
>
>
>
>
> From: Vaibhav Puranik
> To: hbase-user@hadoop.apache.org
> Sent
in fully distributed mode...
Hmm.. not sure what you mean by "Add hbase into hbase"
I added security group 'hbase' using AWS Console. The screen has following
columns at the bottom:
Connection Method, Protocol, From Port, To Port, Source, Actions
Please let me know. Tha
Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ -- do, and so
I can say it works.
- Andy
From: Vaibhav Puranik
To: hbase-user@hadoop.apache.org
Sent: Mon, December 7, 2009 2:12:55 PM
Subject: Re: Starting HBase in fully distributed mode
Good advice. This is what our EC2 scripts -- in src/contrib/ec2/ -- do, and so
I can say it works.
- Andy
From: Vaibhav Puranik
To: hbase-user@hadoop.apache.org
Sent: Mon, December 7, 2009 2:12:55 PM
Subject: Re: Starting HBase in fully distributed mode
I do see this, but it's pretty old:
http://markmail.org/message/vvb2i2brxezwrxtt
It's now clear to me how we would work around that though.
Also, there's an interesting comment in the discussion section of the
link you sent me to EIP:
"This doc need to explicitly point out that EIP DNS names
Or if you have all of your instances in default group. YOu can add default
group into default group.
Regards,
Vaibhav
On Mon, Dec 7, 2009 at 1:43 PM, Vaibhav Puranik wrote:
> Changing connection method to custom might be ok. I don't remember it
> exactly.
> Unfortunately, there is no way to add
Changing connection method to custom might be ok. I don't remember it
exactly.
Unfortunately, there is no way to add an instance to a security group once
it's booted. You have to specify the security group at launch.
(check this FAQ from Amazon -
http://developer.amazonwebservices.com/connect/entry
After doing that.. "Connection Method" was changed automatically to
"Custom...". Is that correct?
Next step is... "Add all the machines on which hbase is running - master and
slaves to hbase
group"
Does that mean -
Connection Method: SSH
Protocol: TCP
>From & To Ports: ?
Source: Master IP
Con
Select the following fields:
Connection Method:All
Leave Protocal, From Port and To Port empty (or default)
Type 'hbase' (or the same group name) in the source field. Notice that the
source field says IP or Group. You can type any group name there.
Regards,
Vaibhav Puranik
Gumgum
On Mon, Dec 7,
Hmm.. not sure what you mean by "Add hbase into hbase"
I added security group 'hbase' using AWS Console. The screen has following
columns at the bottom:
Connection Method, Protocol, From Port, To Port, Source, Actions
Please let me know. Thanks.
On Mon, Dec 7, 2009 at 11:12 AM, Vaibhav Pura
Here is what I suggest:
Make a security group - say hbase.
Add hbase into hbase.
Add all the machines on which hbase is running - master and slaves to hbase
group.
And use private names that start with domU- in configuration
files.
This should work.
Regards,
Vaibhav
On Sun, Dec 6,
Hi J-D, any insight on why I am not seeing what you are seeing? (my
questions below?) I'd like to address this but I won't be able to unless
I can understand fully, and hopefully replicate, the environment that
causes this to happen.
Regards,
Patrick
Patrick Hunt wrote:
That is weird because
After using internal IPs on EC2, Hadoop started cleanly, with no errors in
any of the 4 logs (on Master) & 2 logs (on each Slave).
But when I start HBase, I get this...
java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChann
That is weird because it works for me. I just tried your example (eth0
vs ath0) and I was able to "echo stat |nc 2181" as well
as connect a ZK client successfully using either IP address.
netstat -a shows this:
tcp6 0 0 [::]:2181 [::]:*LISTEN
What do you see for netstat?
I'
It seems not... For example on my dev machine I have an interface for
wired network and another one for wireless. When I start ZK it binds
on only one interface so if I connect to the other IP it doesn't work.
J-D
On Fri, Dec 4, 2009 at 2:35 PM, Patrick Hunt wrote:
> Sorry, but I'm still not abl
Sorry, but I'm still not able to grok this issue. Perhaps you can shed
more light: here's the exact code from our server to bind to the client
port:
ss.socket().bind(new InetSocketAddress(port));
my understanding from the java docs is this:
public InetSocketAddress(int port)
"
Yep those hostnames are the private ones.
J-D
On Fri, Dec 4, 2009 at 12:52 PM, Something Something
wrote:
> Thanks, Jean. My bad. I will try with the private hostnames later. I
> believe under EC2 they look something like this
>
> domU-12-31-38-00-9D-E3
>
>
>
> On Fri, Dec 4, 2009 at 12:4
The first two definitions here is what I'm talking about
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1346
So by default it usually doesn't listen on the interface associated
with the hostname ec2-IP-compute-1.amazonaws.com but on the other one
(IIRC starts with dom-).
J-D
Thanks, Jean. My bad. I will try with the private hostnames later. I
believe under EC2 they look something like this
domU-12-31-38-00-9D-E3
On Fri, Dec 4, 2009 at 12:41 PM, Patrick Hunt wrote:
> I'm not familiar with ec2, when you say "listen on private hostname" what
> does that mean?
I'm not familiar with ec2, when you say "listen on private hostname"
what does that mean? Do you mean "by default listen on an interface with
a non-routable (localonly) ip"? Or something else. Is there an aws page
you can point me to?
Patrick
Jean-Daniel Cryans wrote:
When you saw:
org.apac
When you saw:
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete
/ebs1/mapred/system,/ebs2/mapred/system. Name node is in safe mode.
The ratio of reported blocks 0. has not reached the threshold 0.9990. *Safe
mode will be turned off automatically*.
It means that HDFS is b
o away
after a while if you look at previous threads...
I'm going to defer to the experts here. Good luck!
-Original Message-
From: Something Something [mailto:mailinglist...@gmail.com]
Sent: Friday, December 04, 2009 2:23 PM
To: hbase-user@hadoop.apache.org
Subject: Re: Start
Hadoop: 0.20.1
HBase: 0.20.2
Zookeeper: The one which gets started by default by HBase.
HBase logs:
1) Master log shows this WARN message, but then it says 'connection
successful'
2009-12-04 07:07:37,149 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.selec
When I first started my hbase cluster, it too gave me the nonode for
/hbase/master several times before it started working, and I believe this is a
common beginner's error (I've seen it in a few emails in the past 2 weeks).
What versions of HBase, Hadoop and ZooKeeper are you using?
Also, take
47 matches
Mail list logo
