I tried setting up using hdfs, still I have the same issue. On Fri, Oct 23, 2015 at 11:28 AM, Pankil Doshi <forpan...@gmail.com> wrote:
> One other thing which is different in my setup is I am using filesystem > for hbase rather hbase-indexer needs hdfs setup to be fully functional. So > I need to change that. > > Also, do you have local hbase setup or hbase cluster mode ? > > Pankil > > On Fri, Oct 23, 2015 at 11:23 AM, Pankil Doshi <forpan...@gmail.com> > wrote: > >> Hi Beeshma, >> >> Thanks for your response. >> >> I am running zookeeper locally. but I am not managing it with hbase i.e I >> have this set: >> export HBASE_MANAGES_ZK=false >> >> and also: >> <property> >> <name>hbase.cluster.distributed</name> >> <value>true</value> >> </property> >> >> even though I have everything running locally in standalone mode. >> >> If I dont set "hbase.cluster.distributed" I am seeing zookeeper being >> started with start of my hbase. I am not sure if there is any other good >> way not to start or stop zookeeper with hbase as by only setting >> (HBASE_MANAGES_ZK=false) it doesnt work. >> >> Were you able to setup hbase-indexer at all ? >> >> Pankil >> >> >> On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <beeshm...@gmail.com> wrote: >> >>> Hi Pankil, >>> >>> Are you sure your hbase is running with external zookeeper ensemble ? >>> >>> As per documentation on Hbase Replication >>> >>> >>> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html >>> >>> zookeeper must not be managed by HBase,.But i havent tried this >>> >>> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi < >>> ashish.singhi.apa...@gmail.com> wrote: >>> >>> > Hi Pankil. >>> > >>> > A similar issue was reported few days back ( >>> > >>> > >>> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication >>> > ). >>> > >>> > May be this is due to hbase-indexer code ? >>> > One more Q, did you upgrade hbase from 0.94 and you see this issue ? >>> > >>> > Regards, >>> > Ashish Singhi >>> > >>> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <forpan...@gmail.com> >>> wrote: >>> > >>> > > Hi, >>> > > >>> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily ( >>> > > http://ngdata.github.io/hbase-indexer/). >>> > > >>> > > I am seeing below error when I add my indexer: >>> > > >>> > > >>> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread] >>> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode >>> > > expired, triggering peerListChanged event >>> > > >>> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread] >>> > > regionserver.ReplicationSourceManager: Error while adding a new peer >>> > > >>> > > org.apache.hadoop.hbase.replication.ReplicationException: Error >>> adding >>> > peer >>> > > with id=Indexer_newtest2 >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) >>> > > >>> > > at >>> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) >>> > > >>> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException: >>> > Error >>> > > starting the peer state tracker for peerId=Indexer_newtest2 >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384) >>> > > >>> > > ... 6 more >>> > > >>> > > Caused by: >>> > org.apache.zookeeper.KeeperException$DataInconsistencyException: >>> > > KeeperErrorCode = DataInconsistency >>> > > >>> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452) >>> > > >>> > > ... 7 more >>> > > >>> > > Caused by: >>> org.apache.hadoop.hbase.exceptions.DeserializationException: >>> > > Missing pb magic PBUF prefix >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90) >>> > > >>> > > at >>> > > >>> > > >>> > >>> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83) >>> > > >>> > > ... 8 more >>> > > >>> > > >>> > > >>> > > My Hbase-site.xml: >>> > > >>> > > >>> > > <?xml version="1.0"?> >>> > > >>> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >>> > > >>> > > <!-- >>> > > >>> > > /** >>> > > >>> > > * Copyright 2010 The Apache Software Foundation >>> > > >>> > > * >>> > > >>> > > * Licensed to the Apache Software Foundation (ASF) under one >>> > > >>> > > * or more contributor license agreements. See the NOTICE file >>> > > >>> > > * distributed with this work for additional information >>> > > >>> > > * regarding copyright ownership. The ASF licenses this file >>> > > >>> > > * to you under the Apache License, Version 2.0 (the >>> > > >>> > > * "License"); you may not use this file except in compliance >>> > > >>> > > * with the License. You may obtain a copy of the License at >>> > > >>> > > * >>> > > >>> > > * http://www.apache.org/licenses/LICENSE-2.0 >>> > > >>> > > * >>> > > >>> > > * Unless required by applicable law or agreed to in writing, >>> software >>> > > >>> > > * distributed under the License is distributed on an "AS IS" BASIS, >>> > > >>> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >>> > implied. >>> > > >>> > > * See the License for the specific language governing permissions >>> and >>> > > >>> > > * limitations under the License. >>> > > >>> > > */ >>> > > >>> > > --> >>> > > >>> > > <configuration> >>> > > >>> > > <!--<property> >>> > > >>> > > <name>zookeeper.znode.parent</name> >>> > > >>> > > <value>/hbase-unsecure</value> >>> > > >>> > > </property>--> >>> > > >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.cluster.distributed</name> >>> > > >>> > > <value>true</value> >>> > > >>> > > </property> >>> > > >>> > > //Here you have to set the path where you want HBase to store its >>> files. >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.rootdir</name> >>> > > >>> > > <value>file:/tmp/HBase/HFiles</value> >>> > > >>> > > </property> >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.zookeeper.property.clientPort</name> >>> > > >>> > > <value>2181</value> >>> > > >>> > > <description>Property from ZooKeeper's config zoo.cfg. >>> > > >>> > > The port at which the clients will connect. >>> > > >>> > > </description> >>> > > >>> > > </property> >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.zookeeper.quorum</name> >>> > > >>> > > <value>localhost</value> >>> > > >>> > > <description>Comma separated list of servers in the ZooKeeper >>> > Quorum. >>> > > >>> > > For example, "host1.mydomain.com,host2.mydomain.com, >>> > > host3.mydomain.com >>> > > ". >>> > > >>> > > By default this is set to localhost for local and >>> > pseudo-distributed >>> > > modes >>> > > >>> > > of operation. For a fully-distributed setup, this should be >>> set to >>> > a >>> > > full >>> > > >>> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in >>> > > hbase-env.sh >>> > > >>> > > this is the list of servers which we will start/stop ZooKeeper >>> on. >>> > > >>> > > </description> >>> > > >>> > > </property> >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.zookeeper.property.dataDir</name> >>> > > >>> > > <value>/tmp/zookeeper</value> >>> > > >>> > > <description>Property from ZooKeeper config zoo.cfg. >>> > > >>> > > The directory where the snapshot is stored. >>> > > >>> > > </description> >>> > > >>> > > </property> >>> > > >>> > > // FOR HBASE INDEXER >>> > > >>> > > >>> > > <!-- SEP is basically replication, so enable it --> >>> > > >>> > > <property> >>> > > >>> > > <name>hbase.replication</name> >>> > > >>> > > <value>true</value> >>> > > >>> > > </property> >>> > > >>> > > <!-- Source ratio of 100% makes sure that each SEP consumer is >>> actually >>> > > >>> > > used (otherwise, some can sit idle, especially with small >>> > clusters) >>> > > --> >>> > > >>> > > <property> >>> > > >>> > > <name>replication.source.ratio</name> >>> > > >>> > > <value>1.0</value> >>> > > >>> > > </property> >>> > > >>> > > <!-- Maximum number of hlog entries to replicate in one go. If >>> this is >>> > > >>> > > large, and a consumer takes a while to process the events, the >>> > > >>> > > HBase rpc call will time out. --> >>> > > >>> > > <property> >>> > > >>> > > <name>replication.source.nb.capacity</name> >>> > > >>> > > <value>1000</value> >>> > > >>> > > </property> >>> > > >>> > > <!-- A custom replication source that fixes a few things and adds >>> > > >>> > > some functionality (doesn't interfere with normal replication >>> > > >>> > > usage). --> >>> > > >>> > > <property> >>> > > >>> > > <name>replication.replicationsource.implementation</name> >>> > > >>> > > <value>com.ngdata.sep.impl.SepReplicationSource</value> >>> > > >>> > > </property> >>> > > >>> > > </configuration> >>> > > >>> > > >>> > > All research so far shows me one can face problem while upgrading >>> hbase. >>> > > But in my case, I am not upgrading. >>> > > >>> > > Any ideas/suggestions will be very helpful. >>> > > >>> > > Thanks, >>> > > Pankil >>> > > >>> > >>> >>> >>> >>> -- >>> >> >> >
