[
https://issues.apache.org/jira/browse/ACCUMULO-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793047#comment-13793047
]
Eric Newton edited comment on ACCUMULO-1770 at 10/11/13 9:10 PM:
-----------------------------------------------------------------
I created a slightly different test.
Endlessly ingest a random key that has a of length 2048-4096. The table ran
with out a WALog, and the DevNull iterator for minor compactions. Measure the
resident size of the tserver over time. With the native map, the resident
memory grows at about 40M/hour. With the Java InMemoryMap, resident size grows
more slowly (but takes up more memory initially).
I ran both for two hours. I've attached graphs of the data collected.
was (Author: ecn):
I created a slightly different test.
Endlessly ingest a random key that has a of length 2048-4096. Measure the
resident size of the tserver over time. With the native map, the resident
memory grows at about 40M/hour. With the Java InMemoryMap, resident size grows
more slowly (but takes up more memory initially).
I ran both for two hours. I've attached graphs of the data collected.
> out of memory error on very long running tablet server
> ------------------------------------------------------
>
> Key: ACCUMULO-1770
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1770
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
> Reporter: Eric Newton
> Assignee: Eric Newton
> Attachments: FragmentTest.java, javamap.png, memory-usage.png,
> nativemap.png
>
>
> On a large cluster it was noticed that a few of the tablet servers had been
> pushed into swap. This didn't effect the performance of the server until it
> ran out of memory, and the process was killed. The gc reports in the debug
> log showed the system had plenty of heap space for the JVM. The number of
> threads in the server were not excessive (dozens). This cluster ingests some
> large values (megabytes). The tablet server had been up for a month prior to
> running out of memory. MALLOC_ARENA_MAX had already been set to 1.
> * Investigate the effect of fragmentation on memory usage for large value
> inserts.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
