On Thu, Jan 17, 2013 at 8:21 PM, Jean-Marc Spaggiari <
jean-m...@spaggiari.org> wrote:
> Hi,
>
*below is snapshot of webui,* tabel 'cdrtestsnap' and
'cdrtestsnappy' is on the list , as well as their regions.
cdrtestGZ <http://192.168.250.106:60010/table.jsp?name=cdrtestGZ> {NAME =>
'cdrtestGZ', FAMILIES => [{NAME => 'cf', COMPRESSION => 'GZ', MIN_VERSIONS
=> '0'}]} cdrtestsnap<http://192.168.250.106:60010/table.jsp?name=cdrtestsnap>
{NAME
=> 'cdrtestsnap', FAMILIES => [{NAME => 'cf', COMPRESSION => 'SNAPPY',
MIN_VERSIONS => '0'}]}
cdrtestsnappy<http://192.168.250.106:60010/table.jsp?name=cdrtestsnappy>
{NAME
=> 'cdrtestsnappy', FAMILIES => [{NAME => 'cf', COMPRESSION => 'SNAPPY',
MIN_VERSIONS => '0'}]}
region information of table 'cdrtestsnap' show below:
Table Attributes Attribute Name Value Description Enabled true Is
the table enabled Compaction
------------------------------
Actions:
Region Key (optional): This action will force a compaction of all
regions of the table, or, if a key is supplied, only the region containing
the given key. Region Key (optional): This action will force a split of
all eligible regions of the table, or, if a key is supplied, only the
region containing the given key. An eligible region is one that does not
contain any references to other regions. Split requests for noneligible
regions will be ignored.
*below is excerption hbase hbck:*
Summary:
-ROOT- is okay.
Number of regions: 1
Deployed on: hadoop2,60020,1358418770574
.META. is okay.
Number of regions: 1
Deployed on: hadoop2,60020,1358418770574
cdrtest is okay.
Number of regions: 122
Deployed on: hadoop3,60020,1358418770520
cdrtest1 is okay.
Number of regions: 1
Deployed on: hadoop3,60020,1358418770520
cdrtestGZ is okay.
Number of regions: 1
Deployed on: hadoop2,60020,1358418770574
filelog is okay.
Number of regions: 1
Deployed on: hadoop3,60020,1358418770520
merge_bigtable is okay.
Number of regions: 1
Deployed on: hadoop1,60020,1358418784993
zlytest is okay.
Number of regions: 1
Deployed on: hadoop1,60020,1358418784993
zlytest1 is okay.
Number of regions: 1
Deployed on: hadoop3,60020,1358418770520
2 inconsistencies detected.
Status: INCONSISTENT
[hadoop@hadoop2 ~]$ d
* these two table are in the transition state.*
Regions in Transition Region State
89d3b8af7181ad8b857ef7f55c3abe46
cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=PENDING_OPEN, ts=Thu Jan 17 20:19:13 CST 2013 (945s ago),
server=hadoop3,60020,1358418770520 c56109758acea3619fb9c656b8b45aa3
cdrtestsnap,,1358321463618.c56109758acea3619fb9c656b8b45aa3.
state=PENDING_OPEN, ts=Thu Jan 17 20:15:59 CST 2013 (1138s ago), server=null
masterlog:
2013-01-17 20:17:13,630 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_FAILED_OPEN, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,630 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,630 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=CLOSED, ts=1358425018648, server=hadoop3,60020,1358418770520
2013-01-17 20:17:13,630 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x33c48075af00003 Creating (or updating) unassigned node for
89d3b8af7181ad8b857ef7f55c3abe46 with OFFLINE state
2013-01-17 20:17:13,639 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan
for cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
destination server is hadoop3,60020,1358418770520
2013-01-17 20:17:13,639 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE, server=hadoop1,60000,1358418784101,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,639 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan
for region cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.;
plan=hri=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.,
src=, dest=hadoop3,60020,1358418770520
2013-01-17 20:17:13,639 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46. to
hadoop3,60020,1358418770520
2013-01-17 20:17:13,639 DEBUG org.apache.hadoop.hbase.master.ServerManager:
New connection to hadoop3,60020,1358418770520
2013-01-17 20:17:13,648 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,656 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_FAILED_OPEN, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,656 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,656 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=CLOSED, ts=1358425018687, server=hadoop3,60020,1358418770520
2013-01-17 20:17:13,656 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x33c48075af00003 Creating (or updating) unassigned node for
89d3b8af7181ad8b857ef7f55c3abe46 with OFFLINE state
2013-01-17 20:17:13,664 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE, server=hadoop1,60000,1358418784101,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,664 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan
for cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
destination server is hadoop3,60020,1358418770520
2013-01-17 20:17:13,664 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan
for region cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.;
plan=hri=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.,
src=, dest=hadoop3,60020,1358418770520
2013-01-17 20:17:13,664 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46. to
hadoop3,60020,1358418770520
2013-01-17 20:17:13,664 DEBUG org.apache.hadoop.hbase.master.ServerManager:
New connection to hadoop3,60020,1358418770520
2013-01-17 20:17:13,675 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,683 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_FAILED_OPEN, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,684 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,684 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=CLOSED, ts=1358425018714, server=hadoop3,60020,1358418770520
2013-01-17 20:17:13,684 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x33c48075af00003 Creating (or updating) unassigned node for
89d3b8af7181ad8b857ef7f55c3abe46 with OFFLINE state
2013-01-17 20:17:13,692 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan
for cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
destination server is hadoop3,60020,1358418770520
2013-01-17 20:17:13,692 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan
for region cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.;
plan=hri=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.,
src=, dest=hadoop3,60020,1358418770520
2013-01-17 20:17:13,692 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46. to
hadoop3,60020,1358418770520
2013-01-17 20:17:13,692 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE, server=hadoop1,60000,1358418784101,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,692 DEBUG org.apache.hadoop.hbase.master.ServerManager:
New connection to hadoop3,60020,1358418770520
2013-01-17 20:17:13,700 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,708 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_FAILED_OPEN, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,709 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,709 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=CLOSED, ts=1358425018739, server=hadoop3,60020,1358418770520
2013-01-17 20:17:13,709 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x33c48075af00003 Creating (or updating) unassigned node for
89d3b8af7181ad8b857ef7f55c3abe46 with OFFLINE state
2013-01-17 20:17:13,717 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Found an existing plan
for cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
destination server is hadoop3,60020,1358418770520
2013-01-17 20:17:13,717 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan
for region cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.;
plan=hri=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.,
src=, dest=hadoop3,60020,1358418770520
2013-01-17 20:17:13,717 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46. to
hadoop3,60020,1358418770520
2013-01-17 20:17:13,717 DEBUG org.apache.hadoop.hbase.master.ServerManager:
New connection to hadoop3,60020,1358418770520
2013-01-17 20:17:13,717 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE, server=hadoop1,60000,1358418784101,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,725 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,733 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_FAILED_OPEN, server=hadoop3,60020,1358418770520,
region=89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,733 DEBUG
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler: Handling CLOSED
event for 89d3b8af7181ad8b857ef7f55c3abe46
2013-01-17 20:17:13,734 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE;
was=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=CLOSED, ts=1358425018764, server=hadoop3,60020,1358418770520
2013-01-17 20:17:13,734 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:60000-0x33c48075af00003 Creating (or updating) unassigned node for
89d3b8af7181ad8b857ef7f55c3abe46 with OFFLINE state
2013-01-17 20:17:13,742 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=M_ZK_REGION_OFFLINE, server=hadoop1,60000,1358418784101,
region=89d3b8af718
[root@hadoop1 logs]# tail -f hbase-hadoop-master-hadoop1.log
2013-01-17 20:38:25,958 DEBUG
org.apache.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@7decd1e2
2013-01-17 20:38:25,962 INFO org.apache.zookeeper.ZooKeeper: Session:
0x23c4806bfd50012 closed
2013-01-17 20:38:25,962 INFO org.apache.zookeeper.ClientCnxn: EventThread
shut down
2013-01-17 20:39:35,138 DEBUG org.apache.hadoop.hbase.master.HMaster: Not
running balancer because 2 region(s) in transition:
{89d3b8af7181ad8b857ef7f55c3abe46=cdrtestsnappy,,1358316641849.89d3b8af7181ad8b857ef7f55c3abe46.
state=PENDING_OPEN, ts=1358425153007, server=hadoop3,60020,1358418770520,
c56109758acea3619fb9c656b8b45aa3=cdrtestsnap,,1358321463618.c56109758acea3619fb9c6...
2013-01-17 20:39:35,139 DEBUG
org.apache.hadoop.hbase.client.HTable$ClientScanner: Creating scanner over
.META. starting at key ''
2013-01-17 20:39:35,139 DEBUG
org.apache.hadoop.hbase.client.HTable$ClientScanner: Advancing internal
scanner to startKey at ''
2013-01-17 20:39:35,212 DEBUG
org.apache.hadoop.hbase.client.HTable$ClientScanner: Finished with scanning
at {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED =>
1028785192,}
2013-01-17 20:39:35,212 DEBUG
org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 130 catalog row(s)
and gc'd 0 unreferenced parent region(s)
2013-01-17 20:40:11,469 INFO org.apache.hadoop.hbase.master.HMaster:
Balance=false
2013-01-17 20:40:13,028 INFO org.apache.hadoop.hbase.master.HMaster:
Balance=true
use hbase zkcli to delete the /hbase/unassigned, still not works.
Thanks!
beatls
> What do you mean by "the file system is ok"? Have you run both the
> HBase and the Hadoop checks and both of them aren't returning any
> error?
>
> Also, what do you have on the webui? Can yo usee the table? The regions?
>
> When you are trying to delete the table, what do you havae on you
> master logs? Can you share this log?
>
> Thanks,
>
> JM
>
> 2013/1/17, hua beatls <bea...@gmail.com>:
> > Hi,
> > the file system is ok.
> >
> > any other idea?
> >
> > beatls
> >
> > On Thu, Jan 17, 2013 at 7:38 PM, Jean-Marc Spaggiari <
> > jean-m...@spaggiari.org> wrote:
> >
> >> Hi Beat,
> >>
> >> Have you tried to check the filesystem using fsck to see if there is
> >> no inconsistencies?
> >>
> >> JM
> >>
> >> 2013/1/17, hua beatls <bea...@gmail.com>:
> >> > hi,
> >> > A table mistakenly created cannot be dropped?
> >> > first we create it with 'hbase shell', below:
> >> > [hadoop@hadoop3 ~]$
> >> > [hadoop@hadoop3 ~]$ hbase shell
> >> > 13/01/17 15:49:35 WARN conf.Configuration: hadoop.native.lib is
> >> deprecated.
> >> > Instead, use io.native.lib.available
> >> > HBase Shell; enter 'help<RETURN>' for list of supported commands.
> >> > Type "exit<RETURN>" to leave the HBase Shell
> >> > Version 0.92.1-cdh4.1.2, r, Thu Nov 1 17:54:35 PDT 2012
> >> > hbase(main):001:0> create 't111',{NAME=>'CF111',COMPRESSION=>'snappy'}
> >> > SLF4J: Class path contains multiple SLF4J bindings.
> >> > SLF4J: Found binding in
> >> >
> >>
> [jar:file:/app/hbase/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >> > SLF4J: Found binding in
> >> >
> >>
> [jar:file:/app/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> >> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> >> > explanation.
> >> > ^C[hadoop@hadoop3 ~]$
> >> > (this command hang after i issue "ctrl-c". (please be noted that
> >> 'snappy'
> >> > is not installed/config in this hbase.)
> >> > second, we 'describe' this table, below,
> >> > hbase(main):003:0> describe 't111'
> >> > DESCRIPTION
> >> > ENABLED
> >> > {NAME => 't111', FAMILIES => [{NAME => 'CF111', BLOOMFILTER =>
> 'NONE',
> >> > REPLICATION_SCOPE => '0', VERSIONS
> >> > true
> >> > => '3', COMPRESSION => 'SNAPPY', MIN_VERSIONS => '0', TTL =>
> >> '2147483647',
> >> > BLOCKSIZE => '65536',
> >> > IN_MEMORY
> >> > => 'false', BLOCKCACHE =>
> >> > 'true'}]}
> >> >
> >> > 1 row(s) in 0.0470 seconds
> >> > 3rd, but this table is in the 'transition' state, and remains in
> >> this
> >> > state.
> >> > 4th, i want to 'drop' this table, but 'drop' hang and failed.
> >> > hbase(main):002:0> disable 't111'
> >> > 0 row(s) in 1.1560 seconds
> >> > hbase(main):005:0> drop 't111'
> >> >
> >> > 'drop' is hang until i 'ctrl-c'.
> >> >
> >> > is there any other way to drop this table forcefully?
> >> >
> >> >
> >> > Beat R.
> >> >
> >> > beatls
> >> >
> >>
> >
>
