[
https://issues.apache.org/jira/browse/ACCUMULO-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated ACCUMULO-665:
--------------------------------
Attachment: ACCUMULO-665.patch
Update to SortedKeyValueIterator#seek javadoc. Changes to IntersectingIterator,
OrIterator, and IndexedDocIterator to avoid confusion about the columnFamilies
argument to the seek() method.
> large values, complex iterator stacks, and RFile readers can consume a
> surprising amount of memory
> --------------------------------------------------------------------------------------------------
>
> Key: ACCUMULO-665
> URL: https://issues.apache.org/jira/browse/ACCUMULO-665
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.5.0, 1.4.0
> Environment: large cluster
> Reporter: Eric Newton
> Assignee: Eric Newton
> Priority: Minor
> Fix For: 1.4.1
>
> Attachments: ACCUMULO-665.patch
>
>
> On a production cluster, with a complex iterator tree, a large value (~350M)
> was causing a 4G tserver to fail with out-of-memory.
> There were several factors contributing to the problem:
> # a bug: the query should not have been looking to the big data
> # complex iterator tree, causing many copies of the data to be held at the
> same time
> # RFile doubles the buffer it uses to load values, and continues to use that
> large buffer for future values
> This ticket is for the last point. If we know we're not even going to look
> at the value, we can read past it without storing it in memory. It is
> surprising that skipping past a large value would cause the server to run out
> of memory, especially since it should fit into memory enough times to be
> returned to the caller.
> The provided iterators inside core/org/apache/accumulo/iterators should be
> revisited to ensure that they properly set the seekColumnFamilies where
> necessary, specifically the IntersectingIterator.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
