On 14/11/11 17:51, Camel Christophe wrote:
Hi,
I’m facing an already discussed issue about the memory usage of Fuseki.
I’m using version 0.2.0 of Fuseky, with a DatasetTDB database
(unionDefaultGraph to true) running on a 32 bits JVM on windows Seven,
started with “-Xmx1200M” java option.
The test is pretty simple. It consists in sending an rdf document of
about 1.5Mb using the file endpoint of Fuski, then performing a removal
of all the database (clear) content via the sparql update endpoint of
Fuseki. The test is repeated many times (x1500).
If you are interested, here is the code:
public class Test {
public static void main(String[] args) {
File f = new File("C:\\rdf\\test.rdf");
WebResource uploadWebResource =
Client.create().resource("http://localhost:3030/my-dataset/upload";);
WebResource updateWebResource =
Client.create().resource("http://localhost:3030/ my-dataset /update");
ClientResponse response;
for (int i = 0; i < 1500; ++i) {
System.out.println(i + "\t\t ingesting");
// Upload
FormDataMultiPart fdmp = new FormDataMultiPart();
fdmp.bodyPart(new FileDataBodyPart("test.rdf", f,
MediaType.valueOf("application/rdf+xml")));
What's in test.rdf?
fdmp.bodyPart(new FormDataBodyPart("graph", "http://graph/exemple";));
FormDataMultiPart form = new FormDataMultiPart().field("UNSET FILE
NAME", f, MediaType.MULTIPART_FORM_DATA_TYPE).field("graph",
"http://graph/exemple";);
response =
uploadWebResource.type(MediaType.MULTIPART_FORM_DATA_TYPE).post(ClientResponse.class,
fdmp);
System.out.println(i + "\t\t deleting");
// Delete
MultivaluedMap formData = new MultivaluedMapImpl();
formData.add("update", "clear all");
response =
updateWebResource.type("application/x-www-form-urlencoded").post(ClientResponse.class,
formData);
This does not seem get the statuc code - does it close the connection
afterwards? Otherwise your client is holding http: connections open and
the server needs space at it's end.
System.out.println(response.getEntity(String.class));
}
}
}
After a while, Fuseki hangs and throws a Java.lang.OutOfMemoryError:
Java heap space.
The following graph is the capture of the memory consumption during the test
The graph didn't make it through email - could you fwd the original
message to me directly please (andy AT apache.org)
Linear growth upto the JVM heap limit isbn't unexpected.
As you can see, the memory consumption is growing up linearly until it
reaches a threshold. At this stage (I don’t know exactly when), Fuseki
starts to throw out of memory exceptions.
I wonder how to fix this issue. Will the increase of the maximum heap
size of the jvm (xmx setting) be sufficient ?
Unlikely:
1/ There is a Java limitation of1.5Gb on 32 bit regardless
2/ I would expect GC to trgigger
Is there something else to do in order to master the memory consumption
of Fuseki ?
Use jvisualvm to see which objects are holding on to the memory.
Andy
