While running a Solr-based Web application on Tomcat 6, we have been
repeatedly running into Out of Memory issues. However, these OOM errors are
not related to the Java heap. A snapshot of our Solr dashboard just before
the OOM error reported:
Physical memory: 7.13/7.29 GB
JVM-Memory: 57.90 MB - 3.05 GB - 3.56 GB
In addition, the top command displays:
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
25382 tomcat20 0 9716m 6.8g 175m S 47.9 92.6 196:13.70 java
We're unsure as to why the physical memory usage is so much higher than the
JVM usage, especially given that the size of our index is roughly 500 MB.
We were originally using OpenJDK, and we tried switching to Oracle JDK with
no luck.
Is it normal for physical memory usage to be this high? We do not want to
upgrade our RAM if the problem is really just an error in the configuration.
I've attached environment info below, as well as an excerpt of the latest
OOM error report.
Thank you very much in advance.
Kind regards,
Michael
Additional info about our application:
We index documents from a remote location by retrieving them via a REST API.
The entire remote repository is crawled at regular intervals by our
application. Twenty-five documents are loaded at a time (the page size
provided by the API), and we manually commit each set of twenty-five
documents. We do have auto-commit (but not auto-soft-commit) enabled with a
time of 60s, but an auto-commit has never actually occurred.
Solr Info:
Solr 4.8.0
524 MB Index Size
31 Fields
Just under 3000 documents
Directory factory is MMapDirectory
Caches enabled with default settings/size limits
Selected JVM Arguments:
-XX:MaxPermSize=128m
-Dorg.apache.pdfbox.baseParser.pushBackSize=524288
-Xmx4096m
-Xms1024m
Environment:
64-bit AWS EC2 running CentOS 6.5
Tomcat 6.0.24
7.5 GB RAM
Tried using both Oracle JDK 1.7.0_60 and Open JDK
OOM Log Entry:
OpenJDK 64-Bit Server VM warning: INFO:
os::commit_memory(0x000773c8, 366477312, 0) failed; error='Cannot
allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 366477312 bytes for
committing reserved memory.
# An error report file with more information is saved as:
# /tmp/jvm-18372/hs_error.log
OOM Error Report Snippet:
# Native memory allocation (malloc) failed to allocate 366477312 bytes for
committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2769), pid=18372, tid=140031038150400
#
# JRE version: OpenJDK Runtime Environment (7.0_55-b13) (build
1.7.0_55-mockbuild_2014_04_16_12_11-b00)
# Java VM: OpenJDK 64-Bit Server VM (24.51-b03 mixed mode linux-amd64
compressed oops)
--
View this message in context:
http://lucene.472066.n3.nabble.com/Non-Heap-OOM-Error-with-Small-Index-Size-tp4141175.html
Sent from the Solr - User mailing list archive at Nabble.com.
