[
https://issues.apache.org/jira/browse/HBASE-22457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846258#comment-16846258
]
Lars Hofhansl edited comment on HBASE-22457 at 5/22/19 10:17 PM:
-----------------------------------------------------------------
> No, but this is no more or less fast then the close then open we do for
> 'alter' processing. It would be implemented the same way, ideally.
We came to that conclusion as well in a discussion in the office. Just alter
some minor thing on the Table/ColumnDescriptor so that all regions are
closed/reopened resulting "Wouldn't it be nice if we had a tool that could do
that without forcing us to change something." :)
> Scanner wrapping is a key thing. Without it I don't think Phoenix works.
Oh totally agree. Though perhaps it is structurally somehow possible to ensure
that passed scanner is either wrapped or closed. I can't think of anything,
though.
(I.e. we can check after the hook invocation whether the returned scanner is
different from the passed one... but of course we cannot tell whether it
wrapped the passed scanner and will eventually close it.)
was (Author: lhofhansl):
> No, but this is no more or less fast then the close then open we do for
> 'alter' processing. It would be implemented the same way, ideally.
We came to that conclusion as well in a discussion in the office. Just alter
some minor thing on the Table/ColumnDescriptor so that all regions are
closed/reopened resulting "Wouldn't it be nice if we had a tool that could do
that without forcing us to change something." :)
> Scanner wrapping is a key thing. Without it I don't think Phoenix works.
Oh totally agree. Though perhaps it is structurally somehow possible to ensure
that passed scanner is either wrapped or closed. I can't think of anything,
though.
> Harden the HBase HFile reader reference counting
> ------------------------------------------------
>
> Key: HBASE-22457
> URL: https://issues.apache.org/jira/browse/HBASE-22457
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Priority: Major
> Attachments: 22457-random-1.5.txt
>
>
> The problem that any coprocessor hook that replaces a passed scanner without
> closing it can cause an incorrect reference count.
> This was bad and wrong before of course, but now it has pretty bad
> consequences, since an incorrect reference could will prevent HFiles from
> being archived indefinitely.
> All hooks that are passed a scanner and return a scanner are suspect, since
> the returned scanner may or may not close the passed scanner:
> * preCompact
> * preCompactScannerOpen
> * preFlush
> * preFlushScannerOpen
> * preScannerOpen
> * preStoreScannerOpen
> * preStoreFileReaderOpen...? (not sure about this one, it could mess with the
> reader)
> I sampled the Phoenix and also Tephra code, and found a few instances where
> this is happening.
> And for those I filed issued: TEPHRA-300, PHOENIX-5291
> (We're not using Tephra)
> The Phoenix ones should be rare. In our case we are seeing readers with
> refCount > 1000.
> Perhaps there are other issues, a path where not all exceptions are caught
> and scanner is left open that way perhaps. (Generally I am not a fan of
> reference counting in complex systems - it's too easy to miss something. But
> that's a different discussion. :) ).
> Let's brainstorm some way in which we can harden this.
> [~ram_krish], [~anoop.hbase], [~apurtell]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
