Hi Elton,
Sounds like
https://lists.apache.org/thread/dmmkndmy2ds8pf95zvqbcxpv84bj7cz6
where heap size is set too big.
But maybe host is configured to limit container size?
On 10/03/2023 23:50, Elton Soares wrote:
Dear Jena Community,
I'm running Jena Fuseki Version 4.7.0 as a container on an OpenShift Cluster.
OS Version Info (cat /etc/os-release):
NAME="Red Hat Enterprise Linux"
VERSION="8.7 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.7"
...
Hardware Info (from Jena Fuseki initialization log):
[2023-03-10 18:34:36] Server INFO Memory: 32.0 GiB
[2023-03-10 18:34:36] Server INFO Java: 11.0.18
[2023-03-10 18:34:36] Server INFO OS: Linux
3.10.0-1160.76.1.el7.x86_64 amd64
[2023-03-10 18:34:36] Server INFO PID: 1
Disk Info (df -h):
Filesystem Size Used
Avail Use% Mounted on
overlay 99G 77G
17G 83% /
tmpfs 64M 0
64M 0% /dev
tmpfs 63G 0
63G 0% /sys/fs/cgroup
shm 64M 0
64M 0% /dev/shm
/docker_data 99G 77G 17G
83% /config
/data 1.0T 364G
661G 36% /usr/app/run
…
For some unknown reason, the memory consumption of the container keeps
exceeding the threshold specified at the server initialization command:
java -Xmx32G -jar fuseki-server.jar --port 3030
At our latest experiment, we’ve set the maximum amount of memory available to
the container to 48GB, which leaves 16GB (about 50% of Jena’s maximum expected
memory allocation)
What's the estimate based on?
What sort of databases are configured?
free for any other processes that might be triggered in the background. This was based on a recommendation found at https://www.ibm.com/support/pages/understanding-memory-requirements-32-and-64-bit-systems
(Not related to your report)
If you are running Fuseki in a container with nothing else, that's not a
good balance. TDB2, per database needs about 2G heap, and the rest is
non-heap memory (memory map files) which are managed by the OS and doe
not need to be set in anyway.
which suggested that:
“When allocating memory to a JVM, 10% to 25% of the memory is for the JVM
process itself and the remainder is for the code running in the JVM. This means
that a JVM that has been allocated 4 GB may only have 3.5 GB available to the
code running in the JVM.”
Although we expected that this would be sufficient to avoid any issues with
regards to memory consumption, the container end’s up trying to allocate more
than 48GB which results in an OOMKilled error.
That is a container being killed by the host, not a heap issue.
Hwo long does it take before it happens? (And what's the workload?)
Provided that, would there be any specific recommendations or known strategies
to deal with this issue besides increasing the memory available to the
container? If not, would there be a better way to determine the ideal amount of
memory that should be made available to the container based on the value of
-Xmx parameter (currently 32G)?
Depends on what's configured.
TDB2, per database needs about 2G heap - add an bit for the JVM. And
make sure the host configuration allows the container to grow.
Andy
Thanks for your help!
Best Regards,
Elton Figueiredo de Souza Soares
Research Software Engineer
E-mail: [email protected]<mailto:[email protected]>
