[
https://issues.apache.org/jira/browse/HBASE-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096880#comment-13096880
]
Ted Yu edited comment on HBASE-4301 at 9/4/11 2:42 PM:
-------------------------------------------------------
Here is the tail of /hbase/.META./.tableinfo:
{code}
{NAME => '.META.', IS_META => 'true', FAMILIES => [{NAME => 'info', BLOOMFILTER
=> 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '10',
TTL => '2147483647', MIN_VERSIONS => '0', BLOCKSIZE => '8192', IN_MEMORY =>
'true', BLOCKCACHE => 'true'}]}
{code}
Table name was written.
The stack trace just indicated that
HConnectionImplementation.locateRegionInMeta() got an HRegionInfo which didn't
have table name in it.
I added some more debug logging and restarted the cluster.
But distributed log splitting seemed to be stuck:
{code}
Doing distributed log split in
[hdfs://sjc1-hadoop0.sjc1.carrieriq.com:9000/hbase/.logs/sjc1-hadoop8.sjc1.carrieriq.com,60020,1315063940619]
Waiting for distributed tasks to finish. scheduled=2 done=1 error=0
840s
{code}
Here is the tail of master log:
{code}
2011-09-04 14:23:26,992 INFO org.apache.zookeeper.server.NIOServerCnxn:
Accepted socket connection from /10.202.50.78:24013
2011-09-04 14:23:26,993 INFO org.apache.zookeeper.server.NIOServerCnxn: Client
attempting to establish new session at /10.202.50.78:24013
2011-09-04 14:23:26,994 INFO org.apache.zookeeper.server.NIOServerCnxn:
Established session 0x13234d287c00003 with negotiated timeout 490000 for client
/10.202.50.78:24013
2011-09-04 14:23:45,368 INFO org.apache.zookeeper.server.PrepRequestProcessor:
Got user-level KeeperException when processing sessionid:0x13234d287c00000
type:create cxid:0x23 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error
Path:/hbase/splitlog/hdfs%3A%2F%2Fsjc1-hadoop0.sjc1.carrieriq.com%3A9000%2Fhbase%2F.logs%2Fsjc1-hadoop8.sjc1.carrieriq.com%2C60020%2C1315063940619%2Fsjc1-hadoop8.sjc1.carrieriq.com%252C60020%252C1315063940619.1315067555219
Error:KeeperErrorCode = NodeExists for
/hbase/splitlog/hdfs%3A%2F%2Fsjc1-hadoop0.sjc1.carrieriq.com%3A9000%2Fhbase%2F.logs%2Fsjc1-hadoop8.sjc1.carrieriq.com%2C60020%2C1315063940619%2Fsjc1-hadoop8.sjc1.carrieriq.com%252C60020%252C1315063940619.1315067555219
2011-09-04 14:31:36,002 INFO org.apache.zookeeper.server.ZooKeeperServer:
Expiring session 0x1322feb3d8b0005, timeout of 490000ms exceeded
2011-09-04 14:31:36,003 INFO org.apache.zookeeper.server.PrepRequestProcessor:
Processed session termination for sessionid: 0x1322feb3d8b0005
{code}
Region server log doesn't indicate much activity.
was (Author: yuzhih...@gmail.com):
Here is the tail of /hbase/-ROOT-/.tableinfo:
{code}
{NAME => '.META.', IS_META => 'true', FAMILIES => [{NAME => 'info', BLOOMFILTER
=> 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '10',
TTL => '2147483647', MIN_VERSIONS => '0', BLOCKSIZE => '8192', IN_MEMORY =>
'true', BLOCKCACHE => 'true'}]}
{code}
Table name was written.
The stack trace just indicated that
HConnectionImplementation.locateRegionInMeta() got an HRegionInfo which didn't
have table name in it.
I added some more debug logging and restarted the cluster.
But distributed log splitting seemed to be stuck:
{code}
Doing distributed log split in
[hdfs://sjc1-hadoop0.sjc1.carrieriq.com:9000/hbase/.logs/sjc1-hadoop8.sjc1.carrieriq.com,60020,1315063940619]
Waiting for distributed tasks to finish. scheduled=2 done=1 error=0
840s
{code}
Here is the tail of master log:
{code}
2011-09-04 14:23:26,992 INFO org.apache.zookeeper.server.NIOServerCnxn:
Accepted socket connection from /10.202.50.78:24013
2011-09-04 14:23:26,993 INFO org.apache.zookeeper.server.NIOServerCnxn: Client
attempting to establish new session at /10.202.50.78:24013
2011-09-04 14:23:26,994 INFO org.apache.zookeeper.server.NIOServerCnxn:
Established session 0x13234d287c00003 with negotiated timeout 490000 for client
/10.202.50.78:24013
2011-09-04 14:23:45,368 INFO org.apache.zookeeper.server.PrepRequestProcessor:
Got user-level KeeperException when processing sessionid:0x13234d287c00000
type:create cxid:0x23 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error
Path:/hbase/splitlog/hdfs%3A%2F%2Fsjc1-hadoop0.sjc1.carrieriq.com%3A9000%2Fhbase%2F.logs%2Fsjc1-hadoop8.sjc1.carrieriq.com%2C60020%2C1315063940619%2Fsjc1-hadoop8.sjc1.carrieriq.com%252C60020%252C1315063940619.1315067555219
Error:KeeperErrorCode = NodeExists for
/hbase/splitlog/hdfs%3A%2F%2Fsjc1-hadoop0.sjc1.carrieriq.com%3A9000%2Fhbase%2F.logs%2Fsjc1-hadoop8.sjc1.carrieriq.com%2C60020%2C1315063940619%2Fsjc1-hadoop8.sjc1.carrieriq.com%252C60020%252C1315063940619.1315067555219
2011-09-04 14:31:36,002 INFO org.apache.zookeeper.server.ZooKeeperServer:
Expiring session 0x1322feb3d8b0005, timeout of 490000ms exceeded
2011-09-04 14:31:36,003 INFO org.apache.zookeeper.server.PrepRequestProcessor:
Processed session termination for sessionid: 0x1322feb3d8b0005
{code}
Region server log doesn't indicate much activity.
> META migration from 0.90 to trunk fails
> ---------------------------------------
>
> Key: HBASE-4301
> URL: https://issues.apache.org/jira/browse/HBASE-4301
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.92.0
> Reporter: Todd Lipcon
> Assignee: Ted Yu
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: 4301.txt, 4301_v2.txt, logs.tar.gz, master-log.txt
>
>
> I started a trunk cluster as an upgrade from 0.90.4ish, and now my META table
> is screwed. I can't scan it, etc, and other operations fail.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
