[
https://issues.apache.org/jira/browse/HBASE-15109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pankaj Kumar updated HBASE-15109:
---------------------------------
Description:
HMaster failed to start during installing ShutdownHook when
"fs.hdfs.impl.disable.cache" is set to true.
{noformat}
2016-10-10 10:33:28,367 FATAL [master/HOST-10-18-106-146/10.18.106.146:16000]
master.HMaster: Unhandled: Failed suppression of fs shutdown hook:
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
java.lang.RuntimeException: Failed suppression of fs shutdown hook:
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
at
org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:204)
at
org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:84)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:950)
at java.lang.Thread.run(Thread.java:745)
{noformat}
And RegionServer is in blocking state until a master is available,
{noformat}
"regionserver/HOST-10-18-106-146/10.18.106.146:16020" #17 prio=5 os_prio=0
tid=0x0000000000e4e800 nid=0xdc38 in Object.wait() [0x00007f329dac6000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:159)
- locked <0x00000000ae25c0c8> (a
org.apache.hadoop.hbase.zookeeper.MasterAddressTracker)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:870)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:809)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:783)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:943)
at java.lang.Thread.run(Thread.java:745)
{noformat}
During installation, first time it will try to suppress the HDFS shutdownhook
by removing the hdfsclientfinalizer from HDFS. Since fs.hdfs.impl.disable.cache
is enabled, so removal will be unsuccessful from HDFS (FS is not cached) and
RuntimeException will be thrown with "Failed suppression of fs shutdown hook"
message.
In ShutdownHook,
{code}
if (!fsShutdownHooks.containsKey(hdfsClientFinalizer) &&
!ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
throw new RuntimeException("Failed suppression of fs shutdown hook: " +
hdfsClientFinalizer);
}
{code}
was:
Region server failed to start during installing ShutdownHook when
"fs.hdfs.impl.disable.cache" is set to true in core-site.xml at HBase side.
{code}
2016-01-14 15:30:56,358 | FATAL | regionserver/ds2/192.168.152.230:21302 |
ABORTING region server ds2,21302,1452756654352: Unhandled: Failed suppression
of fs shutdown hook:
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@571527d6 |
org.apache.hadoop.hbase.regionserver.HRegionServer.abort(HRegionServer.java:2054)
java.lang.RuntimeException: Failed suppression of fs shutdown hook:
org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@571527d6
at
org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:204)
at
org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:84)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745)
{code}
During installation, first time it will try to suppress the HDFS shutdownhook
by removing the hdfsclientfinalizer from HDFS. Since fs.hdfs.impl.disable.cache
is enabled, so removal will be unsuccessful from HDFS (FS is not cached) and
RuntimeException will be thrown with "Failed suppression of fs shutdown hook"
message.
In ShutdownHook,
{code}
if (!fsShutdownHooks.containsKey(hdfsClientFinalizer) &&
!ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
throw new RuntimeException("Failed suppression of fs shutdown hook: " +
hdfsClientFinalizer);
}
{code}
> HM/RS failed to start when "fs.hdfs.impl.disable.cache" is set to true
> ----------------------------------------------------------------------
>
> Key: HBASE-15109
> URL: https://issues.apache.org/jira/browse/HBASE-15109
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 2.0.0
> Reporter: Pankaj Kumar
> Assignee: Pankaj Kumar
> Priority: Critical
> Attachments: HBASE-15109-branch-1.patch, HBASE-15109.patch,
> HBASE-15109.patch
>
>
> HMaster failed to start during installing ShutdownHook when
> "fs.hdfs.impl.disable.cache" is set to true.
> {noformat}
> 2016-10-10 10:33:28,367 FATAL [master/HOST-10-18-106-146/10.18.106.146:16000]
> master.HMaster: Unhandled: Failed suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
> java.lang.RuntimeException: Failed suppression of fs shutdown hook:
> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@19a003b4
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:204)
> at
> org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:84)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:950)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> And RegionServer is in blocking state until a master is available,
> {noformat}
> "regionserver/HOST-10-18-106-146/10.18.106.146:16020" #17 prio=5 os_prio=0
> tid=0x0000000000e4e800 nid=0xdc38 in Object.wait() [0x00007f329dac6000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.blockUntilAvailable(ZooKeeperNodeTracker.java:159)
> - locked <0x00000000ae25c0c8> (a
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.blockAndCheckIfStopped(HRegionServer.java:870)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:809)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:783)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:943)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> During installation, first time it will try to suppress the HDFS shutdownhook
> by removing the hdfsclientfinalizer from HDFS. Since
> fs.hdfs.impl.disable.cache is enabled, so removal will be unsuccessful from
> HDFS (FS is not cached) and RuntimeException will be thrown with "Failed
> suppression of fs shutdown hook" message.
> In ShutdownHook,
> {code}
> if (!fsShutdownHooks.containsKey(hdfsClientFinalizer) &&
> !ShutdownHookManager.deleteShutdownHook(hdfsClientFinalizer)) {
> throw new RuntimeException("Failed suppression of fs shutdown hook: " +
> hdfsClientFinalizer);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
