Hello,
I'm trying to install hama-0.2.0 in pseudo-distributed mode. As I have
already HBase, which I'm going to read from, installed on my system, I'm
basically trying to get hama to work with CDH3. So basically:
- hbase 0.90.1-cdh3u0
- hadoop 0.20.2-cdh3u0
- zookeeper 3.3.3-cdh3u0
I configured hadoop in pseudo-distributed mode and launched the namenode
and the datanode. I've configured zookpeeper with default configs. I'm
also running hbase and from the client shell i can connect to ZK.
This is hbase's shell zkdump:
base(main):004:0> zk_dump
HBase is rooted at /hbase
Master address: localhost:60000
Region server holding ROOT: localhost:60020
Region servers:
localhost:60020
Quorum Server Statistics:
localhost:2181
Zookeeper version: 3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT
Clients:
/127.0.0.1:35176[1](queued=0,recved=190,sent=199)
/0:0:0:0:0:0:0:1:57645[1](queued=0,recved=163,sent=164)
/127.0.0.1:35163[1](queued=0,recved=201,sent=213)
/127.0.0.1:53414[0](queued=0,recved=1,sent=0)
/0:0:0:0:0:0:0:1:38233[1](queued=0,recved=14,sent=14)
/0:0:0:0:0:0:0:1:57647[1](queued=0,recved=161,sent=162)
Latency min/avg/max: 0/0/47
Received: 731
Sent: 753
Outstanding: 0
Zxid: 0x28
Mode: standalone
Node count: 12
So apparently ZK is up and running. I've started hama with these
hama-site.xml:
<configuration>
<property>
<name>hama.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
</configuration>
But i get these errors at startup of groom:
STARTUP_MSG: build = -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14;
compiled by 'hudson' on Fri Mar 25 19:56:23 PDT 2011
************************************************************/
2011-07-05 17:09:49,566 INFO org.apache.hama.bsp.GroomServer: groom start
2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer:
[Ljava.lang.String;@20b9b538
2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer:
/tmp/hadoop-cmartella/bsp/local
2011-07-05 17:09:49,742 INFO org.apache.hadoop.ipc.Server: Starting
Socket Reader #1 for port 61000
2011-07-05 17:09:49,744 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=BSPPeer, port=61000
2011-07-05 17:09:49,775 INFO
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
Metrics with hostName=BSPPeer, port=61000
2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2011-07-05 17:09:49,778 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 61000: starting
2011-07-05 17:09:49,778 INFO org.apache.hama.bsp.BSPPeer: BSPPeer
address:rose.inf.unibz.it port:61000
2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 61000: starting
2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT
2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client
environment:host.name=rose.inf.unibz.it
2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_22
2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.22/jre
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.class.path=/data/cmartella/graffiti/hama-0.2.0/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-examples.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-test.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/GroboTestingJUnit-1.2.1-core.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-cli-1.2.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-api-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/hadoop-core-0.20.2-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/junit-3.8.1.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/log4j-1.2.15.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/zookeeper-3.3.3-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/jsp-2.1/*.jar
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.name=Linux
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.arch=amd64
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-27-server
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.name=cmartella
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.home=/home/students/cmartella
2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.dir=/data/cmartella/graffiti/hama-0.2.0
2011-07-05 17:09:49,809 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, connectString=localhost:21810 sessionTimeout=3000
watcher=org.apache.hama.bsp.BSPPeer@217f242c
2011-07-05 17:09:49,820 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:21810
2011-07-05 17:09:49,824 WARN org.apache.zookeeper.ClientCnxn: Session
0x0 for server null, unexpected error, closing socket connection and
attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer:
2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/0:0:0:0:0:0:0:1:21810
2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session
0x0 for server null, unexpected error, closing socket connection and
attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer:
2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server localhost/0:0:0:0:0:0:0:1:21810
2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session
0x0 for server null, unexpected error, closing socket connection and
attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2011-07-05 17:09:50,338 ERROR org.apache.hama.bsp.BSPPeer:
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /bsp
2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.Server: Starting
Socket Reader #1 for port 50000
2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=GroomServer, port=50000
2011-07-05 17:09:50,341 INFO
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
Metrics with hostName=GroomServer, port=50000
2011-07-05 17:09:50,342 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 50000: starting
2011-07-05 17:09:50,343 INFO org.apache.hama.bsp.GroomServer: Worker rpc
server --> rose.inf.unibz.it:50000
2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 50000: starting
2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.Server: Starting
Socket Reader #1 for port 47963
2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=GroomServer, port=47963
2011-07-05 17:09:50,345 INFO
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
Metrics with hostName=GroomServer, port=47963
2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 47963: starting
2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 47963: starting
2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 47963: starting
2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 47963: starting
2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 47963: starting
2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 47963: starting
2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer:
GroomServer up at: localhost/127.0.0.1:47963
2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: Starting
groom: groomd_rose.inf.unibz.it_61000
2011-07-05 17:09:50,351 INFO org.apache.hama.GroomServerRunner:
SHUTDOWN_MSG:
As there were compatibility problems, I substituted the hadoop-core and
zookeper jars in lib/ with those shipped by CDH.
Apparenty groom is trying to connect to ZK on (hama's default) 21810
although I've set the port to ZK's default 2181.
Any idea why this is happening?
Best,
Claudio
--
Claudio Martella
Free Software & Open Technologies
Analyst
TIS innovation park
Via Siemens 19 | Siemensstr. 19
39100 Bolzano | 39100 Bozen
Tel. +39 0471 068 123
Fax +39 0471 068 129
[email protected] http://www.tis.bz.it
Short information regarding use of personal data. According to Section 13 of
Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we
process your personal data in order to fulfil contractual and fiscal
obligations and also to send you information regarding our services and events.
Your personal data are processed with and without electronic means and by
respecting data subjects' rights, fundamental freedoms and dignity,
particularly with regard to confidentiality, personal identity and the right to
personal data protection. At any time and without formalities you can write an
e-mail to [email protected] in order to object the processing of your personal
data for the purpose of sending advertising materials and also to exercise the
right to access personal data and other rights referred to in Section 7 of
Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige,
Siemens Street n. 19, Bolzano. You can find the complete information on the web
site www.tis.bz.it.
