[ https://issues.apache.org/jira/browse/ACCUMULO-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793047#comment-13793047 ]
Eric Newton edited comment on ACCUMULO-1770 at 10/11/13 9:10 PM: ----------------------------------------------------------------- I created a slightly different test. Endlessly ingest a random key that has a of length 2048-4096. The table ran with out a WALog, and the DevNull iterator for minor compactions. Measure the resident size of the tserver over time. With the native map, the resident memory grows at about 40M/hour. With the Java InMemoryMap, resident size grows more slowly (but takes up more memory initially). I ran both for two hours. I've attached graphs of the data collected. was (Author: ecn): I created a slightly different test. Endlessly ingest a random key that has a of length 2048-4096. Measure the resident size of the tserver over time. With the native map, the resident memory grows at about 40M/hour. With the Java InMemoryMap, resident size grows more slowly (but takes up more memory initially). I ran both for two hours. I've attached graphs of the data collected. > out of memory error on very long running tablet server > ------------------------------------------------------ > > Key: ACCUMULO-1770 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1770 > Project: Accumulo > Issue Type: Bug > Components: tserver > Reporter: Eric Newton > Assignee: Eric Newton > Attachments: FragmentTest.java, javamap.png, memory-usage.png, > nativemap.png > > > On a large cluster it was noticed that a few of the tablet servers had been > pushed into swap. This didn't effect the performance of the server until it > ran out of memory, and the process was killed. The gc reports in the debug > log showed the system had plenty of heap space for the JVM. The number of > threads in the server were not excessive (dozens). This cluster ingests some > large values (megabytes). The tablet server had been up for a month prior to > running out of memory. MALLOC_ARENA_MAX had already been set to 1. > * Investigate the effect of fragmentation on memory usage for large value > inserts. -- This message was sent by Atlassian JIRA (v6.1#6144)