Are you running a distributed cluster?
If yes, do you have localhost in /etc/hosts file?
You are getting reference to localhost in hbck output:
ERROR: Region { meta => null, hdfs =>
hdfs://localhost:9000/hbase/test2/b0d4a5f294809c94fccb3d4ce10c3b23,
deployed => } on HDFS, but not listed in META or deployed on any region
server
~Anil
On Fri, Aug 10, 2012 at 3:08 PM, Marco Gallotta <ma...@gallotta.co.za>wrote:
> Here's the output from hbck -details: http://pastebin.com/ZxVZEctY
>
> Extract:
>
> 6 inconsistencies detected.
> Status: INCONSISTENT
>
> 6 is the number of tables that appear in "list" but cannot be operated on
> (which btw, includes not being able to run disable/drop on them - both ops
> say table not found). I also just noticed "foo" does not occur in a table
> list, although I did create it at one point but was able to clear it from
> .META. when it also was reporting table not found when trying to
> disable/drop it. All these come from when I ^C'ed (i.e. killed) table
> creation when I was trying to get lzo compression working and table
> creation was hanging.
>
> Is there any way to repair this? I see hbck has repair options, but I want
> to proceed with caution.
>
> --
> Marco Gallotta | Mountain View, California
> Software Engineer, Infrastructure | Loki Studios
> fb.me/marco.gallotta | twitter.com/marcog
> ma...@gallotta.co.za | +1 (650) 417-3313
>
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
>
> On Friday 10 August 2012 at 2:49 PM, anil gupta wrote:
>
> > Hi Marco,
> >
> > Did anything disastrous happen to cluster?
> > Can you try using hbck utility of HBase.
> > Run: 'hbase hbck -help' to get all the available options.
> >
> > ~Anil
> >
> > On Fri, Aug 10, 2012 at 2:22 PM, Marco Gallotta
> > <ma...@gallotta.co.za(mailto:
> ma...@gallotta.co.za)>wrote:
> >
> > > Hi there
> > >
> > > I have a few tables which show up in a "list" in the shell, but produce
> > > "table not found" when performing any operation on them. There is no
> > > reference of them in the .META. table. It seems to be resulting in
> some of
> > > the hbase services being killed every so often.
> > >
> > > Here are some logs from master (foo is one of the tables not found):
> > >
> > > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster:
> > > Master server abort: loaded coprocessors are: []
> > > 2012-08-09 20:40:44,301 FATAL org.apache.hadoop.hbase.master.HMaster:
> > > Unexpected state : foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485.
> > > state=PENDING_OPEN, ts=1344570044277,
> > > server=ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110
> ..
> > > Cannot transit it to OFFLINE.
> > >
> > >
> > > There are also a number of the following types of error logs:
> > >
> > > 2012-08-09 20:10:04,308 ERROR
> > > org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment in:
> > > ip-10-170-150-10.us-west-1.compute.internal,60020,1344559455110 due to
> > >
> org.apache.hadoop.hbase.regionserver.RegionAlreadyInTransitionException:
> > > Received:OPEN for the
> > > region:foo,,1343175078663.527bb34f4bb5e40dd42e82054d7c5485. ,which we
> are
> > > already trying to OPEN.
> > >
> > > Any ideas how to find and remove any references to these non-existent
> > > tables?
> > >
> > > --
> > > Marco Gallotta | Mountain View, California
> > > Software Engineer, Infrastructure | Loki Studios
> > > fb.me/marco.gallotta (http://fb.me/marco.gallotta) |
> twitter.com/marcog (http://twitter.com/marcog)
> > > ma...@gallotta.co.za (mailto:ma...@gallotta.co.za) | +1 (650) 417-3313
> > >
> > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
> >
> >
>
>
>
--
Thanks & Regards,
Anil Gupta
