[ https://issues.apache.org/jira/browse/HBASE-22457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16847806#comment-16847806 ]
Andrew Purtell commented on HBASE-22457: ---------------------------------------- [~lhofhansl] do you want to really submit that patch? put this in PA status etc? > Harden the HBase HFile reader reference counting > ------------------------------------------------ > > Key: HBASE-22457 > URL: https://issues.apache.org/jira/browse/HBASE-22457 > Project: HBase > Issue Type: Brainstorming > Reporter: Lars Hofhansl > Assignee: Lars Hofhansl > Priority: Major > Attachments: 22457-random-1.5.txt > > > The problem that any coprocessor hook that replaces a passed scanner without > closing it can cause an incorrect reference count. > This was bad and wrong before of course, but now it has pretty bad > consequences, since an incorrect reference could will prevent HFiles from > being archived indefinitely. > All hooks that are passed a scanner and return a scanner are suspect, since > the returned scanner may or may not close the passed scanner: > * preCompact > * preCompactScannerOpen > * preFlush > * preFlushScannerOpen > * preScannerOpen > * preStoreScannerOpen > * preStoreFileReaderOpen...? (not sure about this one, it could mess with the > reader) > I sampled the Phoenix and also Tephra code, and found a few instances where > this is happening. > And for those I filed issued: TEPHRA-300, PHOENIX-5291 > (We're not using Tephra) > The Phoenix ones should be rare. In our case we are seeing readers with > refCount > 1000. > Perhaps there are other issues, a path where not all exceptions are caught > and scanner is left open that way perhaps. (Generally I am not a fan of > reference counting in complex systems - it's too easy to miss something. But > that's a different discussion. :) ). > Let's brainstorm some way in which we can harden this. > [~ram_krish], [~anoop.hbase], [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)