[jira] [Updated] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[ https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxiang Sun updated HBASE-27251: - Release Note: Before upgrading HBase cluster to 2.5.0, it is strongly recommended to upgrade the cluster to HBase-2.4.14 or later first. This is to guarantee a smooth rollback in case. Otherwise, it may run into HBASE-27251. Resolution: Fixed Status: Resolved (was: Patch Available) > Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo` > --- > > Key: HBASE-27251 > URL: https://issues.apache.org/jira/browse/HBASE-27251 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.5.0 >Reporter: Nick Dimiduk >Assignee: Huaxiang Sun >Priority: Critical > Fix For: 2.4.14 > > > I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13 > and the master won't start. Stack trace ends in, > {noformat} > java.io.FileNotFoundException: File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo > > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86) > >at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76) > at > org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156) > >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089) > > > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762) > > > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458) > > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971) > at > java.base/java.security.AccessController.doPrivileged(AccessController.java:712) > > > at java.base/javax.security.auth.Subject.doAs(Subject.java:439) > > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976) > {noformat} > When I examine the on-disk file system, I see, > {noformat} > nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/ > Found 3 items > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.initialized > drwxr-xr-x - nonroot supergroup 0 2022-07-19
[jira] [Updated] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[ https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27251: Assignee: Huaxiang Sun Status: Patch Available (was: Open) > Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo` > --- > > Key: HBASE-27251 > URL: https://issues.apache.org/jira/browse/HBASE-27251 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.5.0 >Reporter: Nick Dimiduk >Assignee: Huaxiang Sun >Priority: Critical > Fix For: 2.4.14 > > > I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13 > and the master won't start. Stack trace ends in, > {noformat} > java.io.FileNotFoundException: File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo > > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86) > >at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76) > at > org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156) > >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089) > > > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762) > > > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458) > > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971) > at > java.base/java.security.AccessController.doPrivileged(AccessController.java:712) > > > at java.base/javax.security.auth.Subject.doAs(Subject.java:439) > > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976) > {noformat} > When I examine the on-disk file system, I see, > {noformat} > nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/ > Found 3 items > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.initialized > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.tabledesc > drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25 > /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 > nonroot@namenode-0:~$ hdfs dfs
[jira] [Updated] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[ https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27251: Fix Version/s: 2.4.14 (was: 2.5.0) (was: 3.0.0-alpha-4) > Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo` > --- > > Key: HBASE-27251 > URL: https://issues.apache.org/jira/browse/HBASE-27251 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.5.0 >Reporter: Nick Dimiduk >Priority: Critical > Fix For: 2.4.14 > > > I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13 > and the master won't start. Stack trace ends in, > {noformat} > java.io.FileNotFoundException: File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo > > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86) > >at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76) > at > org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156) > >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089) > > > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762) > > > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458) > > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971) > at > java.base/java.security.AccessController.doPrivileged(AccessController.java:712) > > > at java.base/javax.security.auth.Subject.doAs(Subject.java:439) > > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976) > {noformat} > When I examine the on-disk file system, I see, > {noformat} > nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/ > Found 3 items > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.initialized > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.tabledesc > drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25 > /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 > nonroot@namenode-0:~$ hdfs dfs
[jira] [Updated] (HBASE-27251) Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: /hbase/MasterData/data/master/store/.initialized/.regioninfo`
[ https://issues.apache.org/jira/browse/HBASE-27251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-27251: Fix Version/s: 2.5.0 3.0.0-alpha-4 Priority: Critical (was: Major) > Rolling back from 2.5.0-SNAPSHOT to 2.4.13 fails due to `File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo` > --- > > Key: HBASE-27251 > URL: https://issues.apache.org/jira/browse/HBASE-27251 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 2.5.0 >Reporter: Nick Dimiduk >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-4 > > > I was doing some perf testing with builds of 2.5.0. I rolled back to 2.4.13 > and the master won't start. Stack trace ends in, > {noformat} > java.io.FileNotFoundException: File does not exist: > /hbase/MasterData/data/master/store/.initialized/.regioninfo > > at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:86) > >at > org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:76) > at > org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:156) > >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2089) > > > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:762) > > > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:458) > > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > > > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043) > > > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971) > at > java.base/java.security.AccessController.doPrivileged(AccessController.java:712) > > > at java.base/javax.security.auth.Subject.doAs(Subject.java:439) > > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976) > {noformat} > When I examine the on-disk file system, I see, > {noformat} > nonroot@namenode-0:~$ hdfs dfs -ls /hbase/MasterData/data/master/store/ > Found 3 items > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.initialized > drwxr-xr-x - nonroot supergroup 0 2022-07-19 17:37 > /hbase/MasterData/data/master/store/.tabledesc > drwxr-xr-x - nonroot supergroup 0 2022-07-27 16:25 > /hbase/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 > nonroot@namenode-0:~$