Eric Newton created ACCUMULO-665:
------------------------------------
Summary: large values, complex iterator stacks, and RFile readers
can consume a surprising amount of memory
Key: ACCUMULO-665
URL: https://issues.apache.org/jira/browse/ACCUMULO-665
Project: Accumulo
Issue Type: Bug
Components: tserver
Affects Versions: 1.5.0, 1.4.0
Environment: large cluster
Reporter: Eric Newton
Assignee: Eric Newton
Priority: Minor
On a production cluster, with a complex iterator tree, a large value (~350M)
was causing a 4G tserver to fail with out-of-memory.
There were several factors contributing to the problem:
# a bug: the query should not have been looking to the big data
# complex iterator tree, causing many copies of the data to be held at the same
time
# RFile doubles the buffer it uses to load values, and continues to use that
large buffer for future values
This ticket is for the last point. If we know we're not even going to look at
the value, we can read past it without storing it in memory. It is surprising
that skipping past a large value would cause the server to run out of memory,
especially since it should fit into memory enough times to be returned to the
caller.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
