[ https://issues.apache.org/jira/browse/HBASE-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522698#comment-16522698 ]
Andrew Purtell edited comment on HBASE-20704 at 6/25/18 7:13 PM: ----------------------------------------------------------------- bq. For 1.x in the small window the region hasn't be set a closed by the RS yet the client will get an NPE when the scan tries to access the fs stream. The client will retry. My opinion is a NPE is always a bug. Ideally we do something more graceful. Even simply substituting an IOException with suitably descriptive message would be better then have NullPointerException appear in a log somewhere. Guaranteed to draw attention/concern. was (Author: apurtell): bq. For 1.x in the small window the region hasn't be set a closed by the RS yet the client will get an NPE when the scan tries to access the fs stream. The client will retry. My opinion is a NPE is always a bug. Ideally we do something more graceful than propagate it to the client for a retry. Even simply substituting an IOException (with suitably descriptive message) would be better. > Sometimes some compacted storefiles are not archived on region close > -------------------------------------------------------------------- > > Key: HBASE-20704 > URL: https://issues.apache.org/jira/browse/HBASE-20704 > Project: HBase > Issue Type: Bug > Components: Compaction > Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0 > Reporter: Francis Liu > Assignee: Francis Liu > Priority: Critical > Attachments: HBASE-20704.001.patch, HBASE-20704.002.patch > > > During region close compacted files which have not yet been archived by the > discharger are archived as part of the region closing process. It is > important that these files are wholly archived to insure data consistency. ie > a storefile containing delete tombstones can be archived while older > storefiles containing cells that were supposed to be deleted are left > unarchived thereby undeleting those cells. > On region close a compacted storefile is skipped from archiving if it has > read references (ie open scanners). This behavior is correct for when the > discharger chore runs but on region close consistency is of course more > important so we should add a special case to ignore any references on the > storefile and go ahead and archive it. > Attached patch contains a unit test that reproduces the problem and the > proposed fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)