OK, so ZooKeeper works now and the client can obviously connect. After the
cluster is running can you start the HBase shell and see if you can scan meta
or root? Simply try a "> scan '-ROOT-'" in the shell.
Do you have anything blocking access to the server hosting those regions? Is
port 60020 accessible by the client?
Lars
On Dec 7, 2010, at 11:25, exception qin <exceptionq...@gmail.com> wrote:
> thanks Geogre.
>
> I have already shut down the Flume instance. So there should be no zookeeper
> conflict.
>
> I wrote a shell script to run the java program. This is the script:
>
> #!/bin/bash
>
> HADOOPHOME="/root/hadoop/";
> HBASEHOME="/root/hbase";
> ZOOKEEPERHOME="/root/zookeeper";
>
> RUNLIB="${HADOOPHOME}/hadoop-0.20.2-core.jar:${HADOOPHOME}/lib/log4j-1.2.15.jar:${HADOOPHOME}/lib/commons-logging-1.0.4.jar:$
> {HBASEHOME}/hbase-0.20.6.jar:${ZOOKEEPERHOME}/zookeeper-3.3.1.jar:.";
>
> sudo java -classpath ${RUNLIB} org/taomee/HBaseClient;
>
> For the config problem, I set them manually in the code. Now the log looks
> like:
>
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14 GMT
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:host.name
> =dev32
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.version=1.6.0_12
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.vendor=Sun Microsystems Inc.
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:os.version=2.6.26-2-amd64
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client environment:user.name
> =root
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:user.home=/root
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Client
> environment:user.dir=/root/eqin/hbaseclient/bin
> 10/12/07 18:08:09 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=dev32:2181 sessionTimeout=60000
> watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@bbe0f0a
> 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Opening socket connection to
> server dev32/10.1.1.32:2181
> 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Socket connection established
> to dev32/10.1.1.32:2181, initiating session
> 10/12/07 18:08:09 INFO zookeeper.ClientCnxn: Session establishment complete
> on server dev32/10.1.1.32:2181, sessionid = 0x12cc00271d00013, negotiated
> timeout = 40000
>
> As you can see, it can connect to zookeeper but still block.
>
> I dig into the code a little bit and find the program blocked at this line:
> this.connection.locateRegion(tableName, HConstants.EMPTY_START_ROW);
> at HTable.java
>
> This problem should be related to the configuration. Is there anything else
> I need to do?
>
> thanks.
>
> On Tue, Dec 7, 2010 at 4:32 PM, Lars George <lars.geo...@gmail.com> wrote:
>
>> Hi Exception,
>>
>> This is up to you to set up properly. If you run them on the same
>> cluster/network then you either share the same ZooKeeper or make sure they
>> use different ports (as per the zoo.cfg). Also make sure you have the proper
>> ZooKeeper quorum set and your client being able to "see" it. If you see
>> something like this on you client
>>
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
>> connection,
>>>>> connectString=localhost:2181 sessionTimeout=60000
>>
>> It means that your client does *not* read the proper config but uses the
>> defaults. What is the classpath you use on the client?
>>
>> Lars
>>
>> On Dec 7, 2010 at 2:34, exception qin <exceptionq...@gmail.com> wrote:
>>
>>> Hi George,
>>>
>>> thanks for you reply and sorry for the silly mistake.
>>>
>>> I change the hbase-site.xml to this:
>>>
>>> <configuration>
>>> <configuration>
>>> <property>
>>> <name>hbase.cluster.distributed</name>
>>> <value>true</value>
>>> </property>
>>> <property>
>>> <name>hbase.rootdir</name>
>>> <value>hdfs://dev32:9000/hbase</value>
>>> </property>
>>> <property>
>>> <name>hbase.zookeeper.quorum</name>
>>> <value>dev32</value>
>>> </property>
>>> </configuration>
>>> </configuration>
>>>
>>> and sync it with slave nodes. but I am still getting the same problem.
>>>
>>> I also run the flume instance on the same cluster. Do the flume and hbase
>>> share the same zookeeper? Is this the reason why I get this problem?
>>>
>>>
>>>
>>>
>>> On Mon, Dec 6, 2010 at 7:27 PM, Lars George <lars.geo...@gmail.com>
>> wrote:
>>>
>>>> Hi Exception,
>>>>
>>>> For starters the logs say you are trying the wrong ZooKeeper node to
>>>> get the HBase details (localhost) and you config has:
>>>>
>>>> <property>
>>>> <name>hbase.zookeeper.quorum</name>
>>>> <value>dev32</value>
>>>> </property>
>>>> <property>
>>>> <name>hbase.zookeeper.quorum</name>
>>>> <value>localhost</value>
>>>> </property>
>>>> <property>
>>>>
>>>> You are declaring it twice and the last one wins. Remove the second
>>>> definition.
>>>>
>>>> Lars
>>>>
>>>> On Mon, Dec 6, 2010 at 8:55 AM, exception qin <exceptionq...@gmail.com>
>>>> wrote:
>>>>> hello,
>>>>>
>>>>> I have written a simple java program to load data and scan table in
>>>> hbase,
>>>>> but get problems.
>>>>>
>>>>> this is my code snap:
>>>>>
>>>>> HBaseConfiguration config = new HBaseConfiguration();
>>>>> HTable table = new HTable(config, "logtable");
>>>>> ......
>>>>> //do something with the table
>>>>>
>>>>> this program blocked when trying to construct the HTable. here is the
>>>> log:
>>>>>
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
>> GMT
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
>> host.name
>>>>> =dev32
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:java.version=1.6.0_12
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:java.vendor=Sun Microsystems Inc.
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.12/jre
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>>
>>>>
>> environment:java.class.path=/root/hadoop//hadoop-0.20.2-core.jar:/root/hadoop//lib/log4j-1.2.15.jar:/root/hadoop//lib/commons-logging-1.0.4.jar:/root/hbase/hbase-0.20.6.jar:/root/zookeeper/zookeeper-3.3.1.jar:.
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>>
>>>>
>> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.12/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:java.io.tmpdir=/tmp
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:java.compiler=<NA>
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:os.name
>>>> =Linux
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>> environment:os.arch=amd64
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:os.version=2.6.26-2-amd64
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client environment:
>> user.name
>>>>> =root
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:user.home=/root
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Client
>>>>> environment:user.dir=/root/eqin/hbaseclient/bin
>>>>> 10/12/06 14:33:41 INFO zookeeper.ZooKeeper: Initiating client
>> connection,
>>>>> connectString=localhost:2181 sessionTimeout=60000
>>>>>
>>>>
>> watcher=org.apache.hadoop.hbase.client.hconnectionmanager$clientzkwatc...@47df280b
>>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Opening socket connection
>> to
>>>>> server localhost/127.0.0.1:2181
>>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Socket connection
>>>> established
>>>>> to localhost/127.0.0.1:2181, initiating session
>>>>> 10/12/06 14:33:41 INFO zookeeper.ClientCnxn: Session establishment
>>>> complete
>>>>> on server localhost/127.0.0.1:2181, sessionid = 0x12cba604b020005,
>>>>> negotiated timeout = 40000
>>>>>
>>>>>
>>>>> I am on hbase 0.20.6 and I have set up a fully distributed cluster with
>>>>> three nodes.(dev32,dev_26,vmtest).
>>>>> dev32 is the master node, on which the zookeeper and hmaster located.
>>>>> there are two region servers running on dev_26 and vmtest. I tested the
>>>>> hbase shell, it works fine. probably, there is something wrong with my
>>>>> config. so I print some of my config file.
>>>>>
>>>>> zoo.cfg
>>>>>
>>>>> # The number of milliseconds of each tick
>>>>> tickTime=2000
>>>>> # The number of ticks that the initial
>>>>> # synchronization phase can take
>>>>> initLimit=10
>>>>> # The number of ticks that can pass between
>>>>> # sending a request and getting an acknowledgement
>>>>> syncLimit=5
>>>>> # the directory where the snapshot is stored.
>>>>> dataDir=/var/zookeeper
>>>>> # the port at which the clients will connect
>>>>> clientPort=2181
>>>>> server.0=dev32:2888:3888
>>>>>
>>>>> hbase-site.xnl
>>>>>
>>>>> ......
>>>>> <property>
>>>>> <name>hbase.zookeeper.quorum</name>
>>>>> <value>dev32</value>
>>>>> </property>
>>>>> <property>
>>>>> <name>hbase.zookeeper.quorum</name>
>>>>> <value>localhost</value>
>>>>> </property>
>>>>> <property>
>>>>> <name>hbase.zookeeper.property.clientPort</name>
>>>>> <value>2181</value>
>>>>> </property>
>>>>> ......
>>>>>
>>>>> I have found a lot of sample code. But it doesn't help. What should I
>> do
>>>> to
>>>>> let my program run? Any help on these issues would be greatly
>>>> appreciated.
>>>>>
>>>>>
>>>>>
>>>>> Cheers
>>>>> Exception
>>>>>
>>>>
>>
