Re: Out of memory

2019-01-29 Thread Andy Seaborne
TDB uses the OS file cache via mmap files. The files appear as part of the process address space but of course they are not part of the heap. It also flex up and down as needed (unlike the heap). Some sys tools report the total address - and that is not the amount of RAM the process is

Re: Out of memory

2019-01-29 Thread Dan Pritts
It's often misunderstood, but Java programs use memory in addition to the configured heap. Fuseki in my experience sometimes uses a LOT more, more than I could explain. Some of the folks here (Andy for sure) spent some time looking at it with me and weren't able to come to any conclusions. You

Re: Out of memory

2019-01-29 Thread Andy Seaborne
On 29/01/2019 14:28, Rob Vesse wrote: This may be partly a case of a simple looking query having unexpected execution semantics. Strictly speaking your query says select all triples in the specific graph then join them with these list of values for ?s. Now the optimiser should, and does

Re: Out of memory

2019-01-29 Thread Andy Seaborne
This case should be optimized to be the flipped join(VALUES, BGP) (prefix ((lsr: )) (sequence (table (vars ?s) (row [?s lsr:10609f75-5cf3-4544-8fc1-c361778c3bd8]) (row [?s lsr:88d0bb8c-35d8-4051-a27d-a0d93af77985]) (row [?s lsr:fc7b2c65-453e-469b-9c5d-8c7ee4ee6902])

Re: Export named graph from TDB to several ntriples files

2019-01-29 Thread vincent ventresque
Hi Andy, Thanks again for your idea to modify the s-get script, it helped me understand ruby utilities and http requests (I often use the ruby scripts but never really looked inside). Don't know how to submit a pull request, and I'm not a ruby expert! Therefore I've put a small test file

Re: Out of memory

2019-01-29 Thread Mikael Pesonen
On 29/01/2019 16:28, Rob Vesse wrote: This may be partly a case of a simple looking query having unexpected execution semantics. Strictly speaking your query says select all triples in the specific graph then join them with these list of values for ?s. Now the optimiser should, and does

Re: Out of memory

2019-01-29 Thread Rob Vesse
This may be partly a case of a simple looking query having unexpected execution semantics. Strictly speaking your query says select all triples in the specific graph then join them with these list of values for ?s. Now the optimiser should, and does appear, to do the right thing and flip the

Re: Out of memory

2019-01-29 Thread Mikael Pesonen
In addition to reply to Andy, all queries are run in serial. On 29/01/2019 15:52, Rob Vesse wrote: Comments inline: On 29/01/2019, 11:34, "Mikael Pesonen" wrote: I'm not able to run a basic read-only script without running out of memory on the server.

Re: Out of memory

2019-01-29 Thread Rob Vesse
Comments inline: On 29/01/2019, 11:34, "Mikael Pesonen" wrote: I'm not able to run a basic read-only script without running out of memory on the server. Consumption goes to 7+gigs (VM 10+ gigs), then system kills Fuseki when running out of memory. All I'm

Re: Out of memory

2019-01-29 Thread Mikael Pesonen
Server: /usr/bin/java -Dlog4j.configuration=file:/home/text/tools/apache-jena-fuseki-3.9.0/log4j.properties -Xmx5600M -jar fuseki-server.jar --update --port 3030 --loc=/home/text/tools/jena_data_test/ /ds No custom configs, default installation package. Sparql similar to this (returns

Re: Out of memory

2019-01-29 Thread Andy Seaborne
Mikael, There aren't enough details except to mention the suspects like sorting. With all the questions on the list, I personally don't track the details of each installation so please also remind me of your current setup. Andy On 29/01/2019 11:32, Mikael Pesonen wrote: I'm not able

Out of memory

2019-01-29 Thread Mikael Pesonen
I'm not able to run a basic read-only script without running out of memory on the server. Consumption goes to 7+gigs (VM 10+ gigs), then system kills Fuseki when running out of memory. All I'm running is simple sparql query getting few triples of resource. This is run for about 50k times.

Out of memory

2019-01-29 Thread Mikael Pesonen
I'm not able to run a basic read-only script without running out of memory on the server. Consumption goes to 7+gigs (VM 10+ gigs), then system kills Fuseki when running out of memory. All I'm running is simple sparql query getting few triples of resource. This is run for about 50k times.