Ok, apparently there are two options to set the ZK client port, i set it correctly through zookeeper.client.port.
Now I'm getting this error at groom start: localhost: Exception in thread "regionservernull" java.lang.NoSuchMethodError: org.apache.hadoop.filecache.DistributedCache.purgeCache(Lorg/apache/hadoop/conf/Configuration;)V localhost: at org.apache.hama.bsp.GroomServer.initialize(GroomServer.java:202) localhost: at org.apache.hama.bsp.GroomServer.run(GroomServer.java:541) localhost: at java.lang.Thread.run(Thread.java:662) I'm afraid there are mismatches between hadoop 0.20.2 as hama is built on (and shipped with) and the hadoop from CDH I'm using right now. The method has been probably removed. Any suggestion? I can easily comment the call out, apparently it's bad practice for the user to call the purgeCache directly. On 7/5/11 5:14 PM, Claudio Martella wrote: > Hello, > > I'm trying to install hama-0.2.0 in pseudo-distributed mode. As I have > already HBase, which I'm going to read from, installed on my system, I'm > basically trying to get hama to work with CDH3. So basically: > > - hbase 0.90.1-cdh3u0 > - hadoop 0.20.2-cdh3u0 > - zookeeper 3.3.3-cdh3u0 > > > I configured hadoop in pseudo-distributed mode and launched the namenode > and the datanode. I've configured zookpeeper with default configs. I'm > also running hbase and from the client shell i can connect to ZK. > > This is hbase's shell zkdump: > > base(main):004:0> zk_dump > HBase is rooted at /hbase > Master address: localhost:60000 > Region server holding ROOT: localhost:60020 > Region servers: > localhost:60020 > Quorum Server Statistics: > localhost:2181 > Zookeeper version: 3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT > Clients: > /127.0.0.1:35176[1](queued=0,recved=190,sent=199) > /0:0:0:0:0:0:0:1:57645[1](queued=0,recved=163,sent=164) > /127.0.0.1:35163[1](queued=0,recved=201,sent=213) > /127.0.0.1:53414[0](queued=0,recved=1,sent=0) > /0:0:0:0:0:0:0:1:38233[1](queued=0,recved=14,sent=14) > /0:0:0:0:0:0:0:1:57647[1](queued=0,recved=161,sent=162) > > Latency min/avg/max: 0/0/47 > Received: 731 > Sent: 753 > Outstanding: 0 > Zxid: 0x28 > Mode: standalone > Node count: 12 > > > So apparently ZK is up and running. I've started hama with these > hama-site.xml: > > <configuration> > <property> > <name>hama.zookeeper.property.clientPort</name> > <value>2181</value> > </property> > <property> > <name>fs.default.name</name> > <value>hdfs://localhost:8020</value> > </property> > </configuration> > > But i get these errors at startup of groom: > > STARTUP_MSG: build = -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14; > compiled by 'hudson' on Fri Mar 25 19:56:23 PDT 2011 > ************************************************************/ > 2011-07-05 17:09:49,566 INFO org.apache.hama.bsp.GroomServer: groom start > 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: > [Ljava.lang.String;@20b9b538 > 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: > /tmp/hadoop-cmartella/bsp/local > 2011-07-05 17:09:49,742 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 61000 > 2011-07-05 17:09:49,744 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=BSPPeer, port=61000 > 2011-07-05 17:09:49,775 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=BSPPeer, port=61000 > 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:49,778 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 61000: starting > 2011-07-05 17:09:49,778 INFO org.apache.hama.bsp.BSPPeer: BSPPeer > address:rose.inf.unibz.it port:61000 > 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 61000: starting > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:zookeeper.version=3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:host.name=rose.inf.unibz.it > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.version=1.6.0_22 > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.vendor=Sun Microsystems Inc. > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.22/jre > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.class.path=/data/cmartella/graffiti/hama-0.2.0/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-examples.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0-test.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../hama-0.2.0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/GroboTestingJUnit-1.2.1-core.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-cli-1.2.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-logging-api-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/hadoop-core-0.20.2-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/junit-3.8.1.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/log4j-1.2.15.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/zookeeper-3.3.3-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/jsp-2.1/*.jar > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.io.tmpdir=/tmp > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.compiler=<NA> > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.name=Linux > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.arch=amd64 > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.version=2.6.32-27-server > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.name=cmartella > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.home=/home/students/cmartella > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.dir=/data/cmartella/graffiti/hama-0.2.0 > 2011-07-05 17:09:49,809 INFO org.apache.zookeeper.ZooKeeper: Initiating > client connection, connectString=localhost:21810 sessionTimeout=3000 > watcher=org.apache.hama.bsp.BSPPeer@217f242c > 2011-07-05 17:09:49,820 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/127.0.0.1:21810 > 2011-07-05 17:09:49,824 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) > 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: > 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/0:0:0:0:0:0:0:1:21810 > 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) > 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: > 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/0:0:0:0:0:0:0:1:21810 > 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) > 2011-07-05 17:09:50,338 ERROR org.apache.hama.bsp.BSPPeer: > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /bsp > 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 50000 > 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=GroomServer, port=50000 > 2011-07-05 17:09:50,341 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=GroomServer, port=50000 > 2011-07-05 17:09:50,342 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 50000: starting > 2011-07-05 17:09:50,343 INFO org.apache.hama.bsp.GroomServer: Worker rpc > server --> rose.inf.unibz.it:50000 > 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 50000: starting > 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 47963 > 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=GroomServer, port=47963 > 2011-07-05 17:09:50,345 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=GroomServer, port=47963 > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 47963: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 47963: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 1 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 2 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 3 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 4 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 5 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 6 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 7 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 8 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 9 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: > GroomServer up at: localhost/127.0.0.1:47963 > 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: Starting > groom: groomd_rose.inf.unibz.it_61000 > 2011-07-05 17:09:50,351 INFO org.apache.hama.GroomServerRunner: > SHUTDOWN_MSG: > > As there were compatibility problems, I substituted the hadoop-core and > zookeper jars in lib/ with those shipped by CDH. > Apparenty groom is trying to connect to ZK on (hama's default) 21810 > although I've set the port to ZK's default 2181. > > Any idea why this is happening? > > Best, > Claudio > -- Claudio Martella Free Software & Open Technologies Analyst TIS innovation park Via Siemens 19 | Siemensstr. 19 39100 Bolzano | 39100 Bozen Tel. +39 0471 068 123 Fax +39 0471 068 129 [email protected] http://www.tis.bz.it Short information regarding use of personal data. According to Section 13 of Italian Legislative Decree no. 196 of 30 June 2003, we inform you that we process your personal data in order to fulfil contractual and fiscal obligations and also to send you information regarding our services and events. Your personal data are processed with and without electronic means and by respecting data subjects' rights, fundamental freedoms and dignity, particularly with regard to confidentiality, personal identity and the right to personal data protection. At any time and without formalities you can write an e-mail to [email protected] in order to object the processing of your personal data for the purpose of sending advertising materials and also to exercise the right to access personal data and other rights referred to in Section 7 of Decree 196/2003. The data controller is TIS Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can find the complete information on the web site www.tis.bz.it.
