Brice,
What's the query?
Andy
On 07/06/13 08:52, Brice Sommacal wrote:
Hello,
The preceding error (XSLTransformation) was occuring in my Eclipse
environment (set with Xmx and Xms at 1024M).
When I move my code in a web server environment (set with Xmx and Xms at
6000M), the XSL transformation goes well, but I keep tracking a Java Heap
Space error:
java.lang.OutOfMemoryError: Java heap space
at java.util.ArrayList.<init>(ArrayList.java:112)
at java.util.ArrayList.<init>(ArrayList.java:119)
at org.apache.jena.atlas.lib.DS.list(DS.java:54)
at
org.apache.jena.atlas.iterator.IteratorConcat.<init>(IteratorConcat.java:34)
at
org.apache.jena.atlas.iterator.IteratorConcat.concat(IteratorConcat.java:45)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:77)
at
com.hp.hpl.jena.sparql.engine.binding.BindingProjectBase.actualVars(BindingProjectBase.java:79)
at
com.hp.hpl.jena.sparql.engine.binding.BindingProjectBase.vars1(BindingProjectBase.java:71)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:75)
at
com.hp.hpl.jena.sparql.engine.binding.BindingProjectBase.actualVars(BindingProjectBase.java:79)
at
com.hp.hpl.jena.sparql.engine.binding.BindingProjectBase.vars1(BindingProjectBase.java:71)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.vars(BindingBase.java:75)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:199)
at
com.hp.hpl.jena.sparql.engine.binding.BindingBase.hashCode(BindingBase.java:185)
at java.util.HashMap.put(HashMap.java:372)
at java.util.HashSet.add(HashSet.java:200)
at org.apache.jena.atlas.data.SortedDataBag.add(SortedDataBag.java:114)
at
org.apache.jena.atlas.data.DistinctDataNet.netAdd(DistinctDataNet.java:58)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterDistinct.isFreshSighting(QueryIterDistinct.java:66)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIterDistinctReduced.hasNextBinding(QueryIterDistinctReduced.java:61)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
at
com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
Definetely, the XML serialization is not good enough for my use case.
What should be the best solution?
<quote>
- Read data from a RDF Store (Jena TBD, Sesame) and return data with a
SPARQL end point (and apply the XSL on the fly [streaming])
- Convert data from OWL files into an Exhibit table (staged mode). So
let's directly parameter the Exhibit storage mode.
(by the way, I didn't succeed to set up Exhibit 3 staged in
a windows environment yet)
- Read data from a RDF Store and create a specific connector with Exhibit
API?
</quote>
Regards,
Brice
2013/6/6 Brice Sommacal <[email protected]>
Hi Andy,
I was using Jena 2.6.4 and I have just upgraded to 2.10.1..
The logs are:
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Vector.ensureCapacityHelper(Unknown Source)
at java.util.Vector.addElement(Unknown Source)
at
com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.startElement(Unknown
Source)
at
com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.startElement(Unknown
Source)
at
com.sun.org.apache.xalan.internal.xsltc.trax.TransformerHandlerImpl.startElement(Unknown
Source)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown
Source)
at
org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown
Source)
at com.sun.org.apache.xalan.internal.xsltc.trax.TrAXFilter.parse(Unknown
Source)
at
com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(Unknown
Source)
at
com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown
Source)
at
com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown
Source)
at
com.pcoinnovation.genericbrowser.json.FiltreXSL.transformer(FiltreXSL.java:47)
So, from now, it's not because of the ResultSetFormatter but from the XSL
Transformation with SAX.
Thanks Andy for pointing this out.
There is no parralel requests because I'm executing them one by one, and
close the query every time.
2013/6/6 Andy Seaborne <[email protected]>
On 06/06/13 13:52, Brice Sommacal wrote:
The XML processing is inside the class ResultSetFormatter available from
Jena API. I'm not sure if it's parse with XML DOM or SAX.
Logs are here :
at
org.openjena.atlas.io.**IndentedWriter.write(**IndentedWriter.java:128)
at
org.openjena.atlas.io.**IndentedWriter.printOneChar(**
IndentedWriter.java:123)
at org.openjena.atlas.io.**IndentedWriter.print(**
IndentedWriter.java:87)
at
com.hp.hpl.jena.sparql.**resultset.XMLOutputResultSet.**printLiteral(**
XMLOutputResultSet.java:182)
at
com.hp.hpl.jena.sparql.**resultset.XMLOutputResultSet.**
printBindingValue(**XMLOutputResultSet.java:148)
at
com.hp.hpl.jena.sparql.**resultset.XMLOutputResultSet.**
binding(XMLOutputResultSet.**java:132)
Jena API provide way to add the stylesheet inside the XML (xsl:reference)
but not to directly run the XML with the XSL.
That's wy I firstly write the XML file (a result set serialization) , and
then run a SAX processor with a stylesheet. The output is a JSON file.
(version? it's not the current one)
The ResultSet writing is streaming and not RAM limited. It does not use
SAX or DOM, it just writes direct output. The query may be consuming
space, some queries do, especially if inferencing is involved (ontoIn
suggests it might be) and this just happens to be where the heap limit is
hit.
Processing the XML output may well be memory consuming but that's not
Jena.
Are there parallel requests going on? They all compete for RAM.
Andy
Brice
2013/6/6 Claude Warren <[email protected]>
I have not followed this discussion very closely so please excuse any
items
that have already been discussed.
You state you are serializing the result set to XML apply a style sheet
and
output as json.
Does your XML processing use the XML Dom or SAX processor? (DOM
results in
a memory footprint of approx 3x document size) You can run the Style
sheet
processing directly agains the SAX processor and have a minimal
footprint.
Does your stylesheet output the JSON or do you use an XML to JSON
converter? If the latter, is does it use or can it use streaming like
the
SAX parser does?
Claude
On Thu, Jun 6, 2013 at 1:28 PM, Brice Sommacal <
[email protected]
wrote:
Hi Olivier,
Thanks for the tips for using your library.
It may be useful one day.
Can I have a look at it? I'm wondering how the n3 graph is read (from a
file?)
Is it possible to manage an other data source from? like a RDF Store?
For my case, my code is inside a java servlet and I don't manage to set
up
the application with data from a IHM. So there is no way to use a
javascript library (not yet ;-))
Thanks anyway,
Brice
2013/6/5 Olivier Rossel <[email protected]>
i have a small javascript that converts a n3 graph into a javascript
graph
of objects.
if your problem is related to XML stuff and such a lib could help, let
me
know.
(it might be interesting to contribute it directly to exhibit, btw)
On Wed, Jun 5, 2013 at 6:13 PM, Brice Sommacal <
[email protected]
wrote:
Hello everyone,
I'm facing a "java.lang.OutOfMemoryError: GC overhead limit exceeded"
error
and I would like an advice about how I could optimize my code.
The aim of this method is to run a SPARQL query, convert it on a XML
format
and then apply a XSL stylesheet[1] to write a JSON format (readable
by
Exhibit - Scripted [2]).
My piece of code was working well untill today. (I have been trying
to
query a big model and the query returns too much results).
This makes my program break.
<quote>
Query queryToExec = QueryFactory.create(query, Syntax.syntaxARQ);
QueryExecution qexec = QueryExecutionFactory.create(**queryToExec,
ontoIn);
ResultSet result = null;
BufferedOutputStream buf;
try{
result = qexec.execSelect();
buf = new BufferedOutputStream(new FileOutputStream(new File(root +
"XML/JSON_XML/"+qNameClass+".**xml")));
//Serialization of the resultSet
ResultSetFormatter.**outputAsXML(buf, result);
buf.close();
}
catch (Exception e) {
e.printStackTrace();
}
finally{
qexec.close();
}
</quote>
I know that writing XML file use loads memory....
I was thinking of:
- creating several XML files by tracing the ResullSetFormatter
memory
usage. (is there possible?)
- avoiding XML intermediate format and write directly in one or
several
JSON file...
- ...
Is there someone whom find a way to avoid this kind of error
(without
increasing Xms Xmx) ??
Thanks in advance,
Brice
[1]
http://data-gov.tw.rpi.edu/**wiki/Sparqlxml2exhibitjson.xsl<http://data-gov.tw.rpi.edu/wiki/Sparqlxml2exhibitjson.xsl>
[2]
http://www.simile-widgets.org/**exhibit3/<http://www.simile-widgets.org/exhibit3/>
--
I like: Like Like - The likeliest place on the web<
http://like-like.xenei.com>
Identity:
https://www.identify.nu/user.**[email protected]<https://www.identify.nu/[email protected]>
LinkedIn:
http://www.linkedin.com/in/**claudewarren<http://www.linkedin.com/in/claudewarren>
