Maybe I should evaluate, if I need an InfModel there in the first place...
On Thu, Jun 23, 2016 at 10:38 PM, Martynas Jusevičius <marty...@graphity.org> wrote: > Hey again, > > I have profiled the CPU time, and it seems that a lot of it (93.5% > after some 22500 HTTP requests) is spent in the following methods: > > com.hp.hpl.jena.rdf.model.ModelFactory.createInfModel > (com.hp.hpl.jena.reasoner.Reasoner, com.hp.hpl.jena.rdf.model.Model, > com.hp.hpl.jena.rdf.model.Model) > com.hp.hpl.jena.reasoner.rulesys.GenericRuleReasoner.bindSchema > (com.hp.hpl.jena.graph.Graph) > com.hp.hpl.jena.reasoner.rulesys.FBRuleInfGraph.prepare () > com.hp.hpl.jena.reasoner.rulesys.impl.RETEEngine.fastInit > (com.hp.hpl.jena.reasoner.Finder) > > Probably not so smart to create an InfModel with every > request/response. But in my case it is created using HTTP response > body and metadata only: Model from response body, and schema OntModel > from headers metadata, so I'm not sure how it could be cached. Here is > the code: > https://github.com/AtomGraph/Processor/blob/master/src/main/java/org/graphity/processor/filter/response/HypermediaFilter.java#L107 > > I would appreciate suggestions on how to improve performance. > > Martynas > > On Tue, Jun 21, 2016 at 10:28 AM, Dave Reynolds > <dave.e.reyno...@gmail.com> wrote: >> Hi Martynas, >> >> On 20/06/16 22:18, Martynas Jusevičius wrote: >>> >>> Hey, >>> >>> after using GenericRuleReasoner and InfModel more extensively, we >>> started experiencing memory leaks that eventually kill our webapp >>> because it runs out of heap space. Jena version is 2.11.0. >>> >>> After some profiling, it seems that RETEEngine.clauseIndex and/or >>> RETEEngine.infGraph are retaining a lot of references. It might be >>> related to this report, but I'm not sure: >>> >>> https://mail-archives.apache.org/mod_mbox/jena-users/201403.mbox/%3c5319b4e0.4060...@gmail.com%3E >> >> >> If it is related to that then it is not a leak it is "just" memory use. >> >> A leak implies that when you turn over data then unused internal state >> objects are not reclaimed. Are you continuously adding and deleting data? If >> so then the delete should release the whole of the RETEEngine state and >> start over. If that isn't happening then that's a bug but you could work >> around with an explicit reset() or even delete and recreate your InfGraph at >> that stage. A delete loses all the state anyway. >> >>> The suggestion was to use use backward rules instead of forward rules. >>> I have read the following: >>> https://jena.apache.org/documentation/inference/#rules >>> >>> But still I fail to understand in which situations backward rules >>> can/should be used instead of forward rules? >> >> >> Forward rules are generally faster because they keep all that partially >> matched state. So if you have stable data or just add triples monotonically, >> and have a lot of queries, then generally use forward rules for performance. >> >> Backward rules (without tabling) keep no state so there's less memory >> overhead and no cost for delete but they are slow and have to redo the work >> for every query. >> >> Strictly the performance trade-off is a bit more subtle than that. Forward >> rules will try to work out all the entailments whereas backward rules are >> just responding to specific queries. So if your queries only touch a small >> part of the possible space then backward rules could be more efficient. >> However in practice RDF rules seem involve a lot of unground terms and lots >> of rules match nearly every query. >> >> Tabling allows you to selectively cache certain predicates which can enable >> you to get more reasonable performance while keeping memory use under >> control. You can also do some tuning of how the rules execute by testing if >> variables are bound or not and using different clause orderings for >> different query patterns. >> >>> I guess simply replacing >>> -> with <- will not be enough? >> >> >> Unless you use non-monotonic predicates (which, sadly, you do) then that >> would be enough to get something working. In fact you don't even need to do >> that. If you create a pure backward reasoner instances (as opposed to the >> hybrid) reasoner it'll read forward syntax rules but treat them as backward. >> >>> The actual rules in question look like >>> this: >>> >>> [gp: (?class rdf:type >>> <http://www.w3.org/2000/01/rdf-schema#Class>), (?class ?p ?o), (?p >>> rdf:type owl:AnnotationProperty), (?p rdfs:isDefinedBy >>> <http://graphity.org/gp#>), (?subClass rdfs:subClassOf ?class), >>> (?subClass rdf:type <http://www.w3.org/2000/01/rdf-schema#Class>), >>> noValue(?subClass ?p) -> (?subClass ?p ?o) ] >> >> >> That's a horrible rule from the engine's point of view. The head is >> completely ungrounded so when running backwards then it will need to run for >> *every* triple pattern. [It also makes no sense to me as a use of >> owl:AnnotationProperty but whatever.] You could try it backwards but put the >> clauses in a more efficient order: >> >> (?subClass ?p ?o) <- >> (?p rdf:type owl:AnnotationProperty), >> (?p rdfs:isDefinedBy <http://graphity.org/gp#>), >> (?subClass rdfs:subClassOf ?class), (?class ?p ?o) . >> >> The rdf:type rdfs:Class constraints are pointless since those are implied by >> rdfs:subClassOf anyway. The noValue check is probably best avoided for both >> cases. >> >> Alternatively, depending on the nature of your space leak you could use >> hybrid rules: >> >> (?p rdf:type owl:AnnotationProperty), >> (?p rdfs:isDefinedBy <http://graphity.org/gp#>) >> -> >> [ (?subClass ?p ?o) <- (?subClass rdfs:subClassOf ?class), >> (?class ?p ?o) ] >> >> That way the forward engine is only looking at your annotations and the >> backward engine then has rules that have grounded predicates. You could also >> table those predicates: >> >> (?p rdf:type owl:AnnotationProperty), >> (?p rdfs:isDefinedBy <http://graphity.org/gp#>) >> -> >> table(?p), >> [ (?subClass ?p ?o) <- (?subClass rdfs:subClassOf ?class), >> (?class ?p ?o) ] >> >>> [gcdm: (?template rdf:type <http://graphity.org/gp#Template>), >>> (?template <http://graphity.org/gc#defaultMode> ?o), (?subClass >>> rdfs:subClassOf ?template), (?subClass rdf:type >>> <http://graphity.org/gp#Template>), noValue(?subClass >>> <http://graphity.org/gc#defaultMode>) -> (?subClass >>> <http://graphity.org/gc#defaultMode> ?o) ] >>> [gcsm: (?template rdf:type <http://graphity.org/gp#Template>), >>> (?template <http://graphity.org/gc#supportedMode> ?supportedMode), >>> (?subClass rdfs:subClassOf ?template), (?subClass rdf:type >>> <http://graphity.org/gp#Template>) -> (?subClass >>> <http://graphity.org/gc#supportedMode> ?supportedMode) ] >> >> >> These two are more reasonable and could be used backwards or hybrid. >> >>> [rdfs9: (?x rdfs:subClassOf ?y), (?a rdf:type ?x) -> (?a rdf:type ?y)] >> >> >> That would work backwards. Depending on the scale of your data you might >> want to table rdf:type for performance/space tradeoff. >> >>> Can these be rewritten as backward rules instead? >> >> >> Sure, the challenge is performance tuning as noted above. >> >>> Does it involve code changes, such as calling reset() etc? >> >> Shouldn't do. >> >> Dave
