Hi,
we're experimenting with loading larger amounts of data into ML9 using mlcp
and we're experiencing OOM kills of the MarkLogic server process even
though we have configured the system according to what we could find
regarding swap space and huge pages. The overall amount of data that we
want to load amounts to about 500GB worth of XML. We don't even see
unusually high resident set sizes of any process, but there still seems to
be a shortage in system memory that causes the OOM killer to hit us.
Here's some top output of the system running:
[image: Inline image 1]
ErrorLog.txt seems to contain nothing suspicious, except for maybe:
2017-07-25 07:54:01.100 Info: Linux Huge Pages: detected 55000, using
49042, recommend 40832 to 50349
[1888722.818954] MarkLogic invoked oom-killer: gfp_mask=0x3000d0, order=2,
oom_score_adj=0
[1888722.819209] MarkLogic cpuset=/ mems_allowed=0
[1888722.819341] CPU: 7 PID: 16313 Comm: MarkLogic Not tainted
3.10.0-514.21.1.el7.x86_64 #1
...
[1888722.838675] [16294] 0 16294 81228644 2660319 77204 6838413
0 MarkLogic
...
[1888722.839651] [ 3658] 10011 3658 9125024 1366314 5727 1478105
0 java
I'm not an expert in reading OOM killer output, but it seems to me that the
system killed MarkLogic based on ~81GB memory usage with mlcp at ~9GB. On
a system with 128GB physical memory, this strikes me as odd.
Is there any way to limit the overall memory usage of MarkLogic server so
that its virtual and resident set size always stay below a certain limit?
Any pointers would be greatly appreciated.
Thanks,
Hans
--
LambdaWerk GmbH
Oranienburger Straße 87/89
10178 Berlin
Phone: +49 30 555 7335 0
Fax: +49 30 555 7335 99
HRB 169991 B Amtsgericht Charlottenburg
USt-ID: DE301399951
Geschäftsführer: Hans Hübner
http://lambdawerk.com/
_______________________________________________
General mailing list
[email protected]
Manage your subscription at:
http://developer.marklogic.com/mailman/listinfo/general
