Hi Brice Writing SPARQL/XML should be entirely streaming and in principal has minimal memory overhead.
The most likely culprit of a OOM is in the query execution itself since depending on the query the system may need to hold a lot of intermediate data in memory. You haven't said anything about what your query is or what you data is e.g. how large, whether it is in-memory or in some persistent triple store. Also I am guessing from your variable names you are querying an OntModel/InfModel which may itself cause lots of memory usage during query execution if rules have to be evaluated. The ResultSet object you get when you call execSelect() is essentially a thin wrapper over a QueryIterator which is itself essentially a wrapper over the query plan for answering the query. Until you start iterating over the ResultSet (and thus the underlying QueryIterator) the query engine does not actually do any work, when you call ResultSetFormatter.outputAsXML() it starts iterating and thus evaluating the query which is why you get the OOM at this line (you didn't say so directly in your email but I will assume this from your assertion that outputting the XML is the problem). If you need further help with this it is much easier for us to help you if you provide a complete minimal example I.e. one that includes code, query and data plus the error message w/ stack trace. Rob On 6/5/13 9:13 AM, "Brice Sommacal" <[email protected]> wrote: >Hello everyone, > >I'm facing a "java.lang.OutOfMemoryError: GC overhead limit exceeded" >error >and I would like an advice about how I could optimize my code. > >The aim of this method is to run a SPARQL query, convert it on a XML >format >and then apply a XSL stylesheet[1] to write a JSON format (readable by >Exhibit - Scripted [2]). > > My piece of code was working well untill today. (I have been trying to >query a big model and the query returns too much results). >This makes my program break. > ><quote> >Query queryToExec = QueryFactory.create(query, Syntax.syntaxARQ); >QueryExecution qexec = QueryExecutionFactory.create(queryToExec, ontoIn); > ResultSet result = null; >BufferedOutputStream buf; >try{ > result = qexec.execSelect(); >buf = new BufferedOutputStream(new FileOutputStream(new File(root + >"XML/JSON_XML/"+qNameClass+".xml"))); > //Serialization of the resultSet >ResultSetFormatter.outputAsXML(buf, result); >buf.close(); > } >catch (Exception e) { >e.printStackTrace(); > } >finally{ >qexec.close(); >} ></quote> > >I know that writing XML file use loads memory.... > >I was thinking of: > - creating several XML files by tracing the ResullSetFormatter memory >usage. (is there possible?) > - avoiding XML intermediate format and write directly in one or several >JSON file... > - ... > > >> Is there someone whom find a way to avoid this kind of error (without >increasing Xms Xmx) ?? > >Thanks in advance, > > >Brice > >[1] http://data-gov.tw.rpi.edu/wiki/Sparqlxml2exhibitjson.xsl >[2] http://www.simile-widgets.org/exhibit3/
