yes so here is what happens. the region server reports its address to
the zookeeper ( it's been a while ago and it might be the hbase master
not zookeeper, but never the less the point stands ), now the
regionserver gets its address from looking up the hostname, and if in
your /etc/hosts, on that server, the hostname is associated with
loopback ( 127.0.0.1 ) then the ip addresss that would be reported to
the zookeeper( or hbase master) would be 127.0.0.1 which means that they
won't be able to connect back to the regionserver.
another situation that may cause the error message that you mentioned
would be the max number of concurrent connection on the zookeeper. I
don't remember what the default value was. I think 30 or 100. you
probably want about 10 * (number of regionserver). once connections
exceed that number zookeeper starts to reject new connection and you get
that error.
that being said, I don't recommend hbase for nutch since the latest
version of hbase is not supported by gora. and the version that is
supported is not really stable under heavy load. I am experimenting with
cassandra right now and the jury is still out on that, but it looks
promising. never the less, if the number of sites that you want to crawl
is anywhere close to the number of sites in DMOZ , then you probably
want to start replacing mysql asap.
On 08/13/2013 11:49 AM, Ralf R. Kotowski wrote:
You mean NOT to point to 127.0.0.1? I thought it was supossed to do that
from what I read.
Anyway, I'm using MySQL for now, I'll work on Hbase later when this is all
functional
-----Original Message-----
From: kaveh minooie [mailto:[email protected]]
Sent: Monday, August 12, 2013 10:52 PM
To: [email protected]
Subject: Re: Hbase is able to connect to Zookeeper but the connection closes
immediatly
I don't know if this is your problem but it would cause the same error
message as well. you have to make sure that your hostnames do not
return the loopback address (127.0.0.1) so your hostname either should
not be in /etc/hosts or it should have a proper IP address which is also
the same on that your region servers are binding to.
On 08/09/2013 10:01 AM, Ralf R. Kotowski wrote:
Thnx,
You're right, I just have been able to reproduce the error via the Hbase
shell without having Nutch in the loop.
-----Original Message-----
From: Lewis John Mcgibbney [mailto:[email protected]]
Sent: Friday, August 09, 2013 5:39 PM
To: [email protected]
Subject: Re: Hbase is able to connect to Zookeeper but the connection
closes
immediatly
Hi Ralf,
AFAICS this would be much better suited to hbase user list.
Sorry I can't help more
On Friday, August 9, 2013, Ralf R. Kotowski <[email protected]> wrote:
Nutch 2.2.1
Hbase 0.90.4
Solr 4.4.0
Fedora Core 19
Sun Java (latest)
Error Msg: Hbase is able to connect to Zookeeper but the connection
closes
immediately
Sanned web/mailing lists, about to make a new install of everything,
could
this be a Firewall issue?
--
Kaveh Minooie
