There is an issue with memory growth in fuseki, though it's growth
outside of normal java heap and non-heap space.
See https://www.mail-archive.com/users@jena.apache.org/msg20362.html
For that scale of data and scale of machine suggest setting the heap
smaller -Xmx1G or -Xmx500M. Empirically the process growth seems to
largely levels off at around 4x the given heap size (though this very
much depends on the usage model and have no clear explanation for this).
You might also try -XX:MaxDirectMemorySize=1G or less though exactly
what size to set will depend on how much data is involved in your
queries. If the process dies with an exception about unable to allocate
new direct memory then increase it.
If this is not a public service liable to security issues and you are
able to use a 3.17.0 (or earlier) version of fuseki then those are not
subject to this growth issue. Or at least not to the version of the
issue that we are seeing in our own usage.
Dave
On 09/07/2023 20:33, Laura Morales wrote:
I'm running a job that is submitting a lot of queries to a Fuseki server, in
parallel. My problem is that Fuseki is OOM-killed and I don't know how to fix
this. Some details:
- Fuseki is queried as fast as possible. Queries take around 50-100ms to
complete so I think it's serving 10s of queries each second
- Fuseki 4.8. OS is Debian 12 (minimal installation with only OS, Fuseki, no
desktop environments, uses only ~100MB of RAM)
- all the queries are read queries. No updates, inserts, or other write queries
- all the queries are over HTTP to the Fuseki endpoint
- database is TDB2 (created with tdb2.tdbloader)
- database contains around 2.5M triples
- the machine has 8GB RAM. I've tried on another PC with 16GB and it completes
the job. On 8GB though, it won't
- with -Xmx6G it's killed earlier. With -Xmx2G it's killed later. Either way
it's always killed.
Is there anything that I can tweak to avoid Fuseki getting killed? Something that isn't
"just buy more RAM".
Thank you
