[
https://issues.apache.org/jira/browse/ACCUMULO-4692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114613#comment-16114613
]
Christopher Tubbs commented on ACCUMULO-4692:
---------------------------------------------
[~milleruntime], when I took a look at the code, it looks like this affects 1.7+
I spoke to [~kturner] about this and he indicated that it would be hard to know
exactly where to end the scan, because you don't know the split points in
advance for the given table range. So, you don't know how many tablets metadata
to read for that table. We talked about the possibility of implementing the
"close" method for scanner so that such scans could be terminated once you've
read all the data you want from the scanner.
> CompactionDriver leaves abandoned metadata scans
> ------------------------------------------------
>
> Key: ACCUMULO-4692
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4692
> Project: Accumulo
> Issue Type: Bug
> Components: fate
> Reporter: Adam Fuchs
>
> We wrote a tool to kick off tablet compactions in the background while
> minimizing compaction load per-server. The tool uses range compaction on one
> tablet per call. We're seeing a high number of scans on the metadata table
> (~7,000 on a ~100 node cluster).
> The metadata query in the isReady() method of CompactionDriver that is used
> to see if the compaction has completed uses a range that goes to the end of
> the metadata entries for the given table, but it stops consuming the results
> of the scanner at the end of the compaction range. isReady gets called in a
> pretty tight loop, especially with hundreds of compactions running
> concurrently. Seems like we should limit the scan to the metadata range
> associated with the compaction so that the scan can get cleaned up quickly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
