[
https://issues.apache.org/jira/browse/HBASE-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905791#action_12905791
]
stack commented on HBASE-2952:
------------------------------
To be clear, this would be a complement to the patch up in HBASE-2669 which
removes the shutdown handler altogether. The refcount would also be used to
let go of shared zk sessions.
> HConnectionManager's shutdown hook interferes with client's operations
> ----------------------------------------------------------------------
>
> Key: HBASE-2952
> URL: https://issues.apache.org/jira/browse/HBASE-2952
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.20.0
> Reporter: Prakash Khemani
>
> My HBase client calls incrementColValue() in pairs. If someone kills the
> client (SIGINT or SIGTERM) I want my client's increment threads to gracefully
> exit. If a thread has already done one of the incrementColValue() then I want
> that thread to complete the other incrementColValue() and then exit.
> For this purpose I installed my own shutdownHook(). My shitdownHook() thread
> 'sugnals' all the threads in my process that it is time to exit and then
> waits for them to complete.
> The problem is that HConnectionManager's shutdownHook thread also runs and
> shuts down all connections and IPC threads.
> My increment thread keeps waiting to increment and then times out after 240s.
> Two problems with this - the incrementColValiue() didn't go through which
> will increase the chances of inconsistency in my HBase data. And it too 240s
> to exit. I am pasting some of the messages that the client thread outputs
> while it tries contact the HBase server.
> Signalled. Exiting ...
> 2010-09-01 12:11:14,769 DEBUG [HCM.shutdownHook]
> zookeeper.ZooKeeperWrapper(787):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Closed
> connection with ZooKeeper; /hbase/root-region-server
> flushing after 7899
> 2010-09-01 12:11:19,669 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(903): Cache hit for row <> in
> tableName .META.: location server hadoop2205.snc3.facebook.com:60020,
> location region name .META.,,1.1028785192
> 2010-09-01 12:11:19,671 INFO [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(206): Reconnecting to zookeeper
> 2010-09-01 12:11:19,671 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(212):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Connected
> to zookeeper again
> 2010-09-01 12:11:24,679 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(964): Removed .META.,,1.1028785192 for
> tableName=.META. from cache because of content_action_url_metrics,\x080r&
> B\xF7\x81_T\x07\x08\x16uOrcom.gigya 429934274290948,99999999999999
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(857): locateRegionInMeta attempt 0 of
> 4 failed; retrying after sleep of 5000 because: The client is stopped
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(470):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Trying to
> read /hbase/root-region-server
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0]
> zookeeper.ZooKeeperWrapper(489):
> <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Read
> ZNode /hbase/root-region-server got 10.26.119.190:60020
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0]
> client.HConnectionManager$TableServers(1116): Root region location changed.
> Sleeping.
> ===
> It might be a good idea to only run the HCM shutdown code when all the
> HTables referring to it have been closed. That way the client can control
> when the shutdown actually happens.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
