Vladimir Rodionov created HBASE-22826: -----------------------------------------
Summary: Wrong FS: recovered.edits goes to wrong file system Key: HBASE-22826 URL: https://issues.apache.org/jira/browse/HBASE-22826 Project: HBase Issue Type: New Feature Affects Versions: 2.0.5 Reporter: Vladimir Rodionov Assignee: Vladimir Rodionov When WAL is attached to a separate file system, recovered.edits are going to hbase root directory. PROBLEM * Customer environment HBase root directory : On WASB hbase.wal.dir : On HDFS Customer is creating and HBase table and running VIEW DDL on top of the Hbase table. The recovered.edits are going to hbase root directory in WASB and region assignments getting failed. Customer is on HBase 2.0.4. {code:java}if (RegionReplicaUtil.isDefaultReplica(getRegionInfo())) { LOG.debug("writing seq id for {}", this.getRegionInfo().getEncodedName()); WALSplitter.writeRegionSequenceIdFile(fs.getFileSystem(), getWALRegionDir(), nextSeqId); //WALSplitter.writeRegionSequenceIdFile(getWalFileSystem(), getWALRegionDir(), nextSeqId - 1);{code} {code:java}2019-08-05 22:07:31,940 ERROR [RS_OPEN_META-regionserver/c47-node3:16020-0] handler.OpenRegionHandler: Failed open of region=hbase:meta,,1.1588230740 java.lang.IllegalArgumentException: Wrong FS: hdfs://c47-node2.squadron-labs.com:8020/hbasewal/hbase/meta/1588230740/recovered.edits, expected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:730) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:86) at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:460) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1868) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1910) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:678) at org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:270) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1868) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1910) at org.apache.hadoop.hbase.wal.WALSplitter.getSequenceIdFiles(WALSplitter.java:647) at org.apache.hadoop.hbase.wal.WALSplitter.writeRegionSequenceIdFile(WALSplitter.java:680) at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:984) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:881) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7149) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7108) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7080) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7038) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6989) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:283) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:108) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016)