[ https://issues.apache.org/jira/browse/HBASE-22582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Hu resolved HBASE-22582. ------------------------------ Resolution: Fixed Hadoop Flags: Reviewed > The Compaction writer may access the lastCell whose memory has been released > when appending fileInfo in the final > ----------------------------------------------------------------------------------------------------------------- > > Key: HBASE-22582 > URL: https://issues.apache.org/jira/browse/HBASE-22582 > Project: HBase > Issue Type: Bug > Components: Compaction > Reporter: Zheng Hu > Assignee: Zheng Hu > Priority: Major > Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.7 > > > Copy the comment from [~javaman_chen] under HBASE-21879: > https://issues.apache.org/jira/browse/HBASE-21879?focusedCommentId=16862244&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16862244 > In Compactor#compact, we have the following: > {code} > protected List<Path> compact(final CompactionRequest request... > ... > try { > ... > } finally { > Closeables.close(scanner, true); > if (!finished && writer != null) { > abortWriter(writer); > } > } > assert finished : "We should have exited the method on all error paths"; > assert writer != null : "Writer should be non-null if no error"; > return commitWriter(writer, fd, request); > } > {code} > should we call writer#beforeShipped() before Closeables.close(scanner, true); > In order to copy some cell's data out of the ByteBuff before it released, or > commitWriter may be wrong in the following call stack > {code} > Compactor#commitWriter > -> HFileWriterImpl#close > -> HFileWriterImpl#writeFileInfo > -> HFileWriterImpl#finishFileInfo > {code} > {code} > protected void finishFileInfo() throws IOException { > if (lastCell != null) { > // Make a copy. The copy is stuffed into our fileinfo map. Needs a clean > // byte buffer. Won't take a tuple. > byte [] lastKey = > PrivateCellUtil.getCellKeySerializedAsKeyValueKey(this.lastCell); > fileInfo.append(FileInfo.LASTKEY, lastKey, false); > } > ... > } > {code} > Because the lastCell may refer to a reused ByteBuff. > Checked the code, It's a bug and will need to fix in all 2.x & master branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)