Hi,
There is a short discussion on the former Jena Development/Support mailing list related to memory usage of Jena OntModel using different versions of Jena. For your information I have included a segment of the correspondence at the end of the message. Since I had a problem that looks much like memory usage limit overrun, I got interested in the problem. I repeated the same experiment but with different conclusions? I compared Jena 2.6.0 with Jena 2.6.4. I couldn't notice the difference between Jena versions as reported in your previous experiment (though I used the same code). Tried with several variations of the OntModel creations styles but with minor differences in memory usage only (within 10%). I run the experiment from within Eclipse with JVM set to jre6 using compiler compliance level 1.5 (as well as 1.6). The total memory usage was around 27975k. Than, I changed the JVM to jdk 1.5. There was similar relative difference between the Jena 2.6.0 and Jena 2.6.4 (within 10%). However, the absolute amount of used memory dropped to about 3400k. Than I again changed the JVM to jdk 1.6. As in all previous testing configurations, there was similar relative difference between the Jena 2.6.0 and Jena 2.6.4 (within 10%). However, the absolute amount of used memory was about 6501k. Consequently, I have two problems now: 1) I couldn't repeat you experiment with the same results. What I am doing wrong? 2) What should I do to put the memory usage under my control? Namely, 5 times larger memory usage cause my application to broke on machines with limited memory usage. Thanks, Milorad Tosic > > --- In [email protected], Dave Reynolds <dave.e.reynolds@...> wrote: > > > > Rick, > > > > Thanks for letting us know, thanks for the clear report. > > > > Dave > > > > rspates2 wrote: > > > Dave, > > > > > > I grabbed OWLProfile.java and reran my tests. It looks fixed to me in terms of speed and memory use. We should be able to use 2.6.2 now. > > > > > > Thanks for your swift resolution to the problem. > > > > > > Rick > > > > > > --- In [email protected], Dave Reynolds <dave.e.reynolds@> wrote: > > >> Hi Rick, > > >> > > >> A follow up on my earlier off-list response. > > >> > > >> Thanks again for giving us a test case for this. It turns out not to be > > >> a reasoner issue as such. There was a change in the internals of the > > >> Ontology API which worked inefficiently when a reasoner is present. Ian > > >> has checked in some patches for this which brings the memory use down a > > >> lot and gives a 5x speed improvement. > > >> > > >> It is not quite back to 2.6.0 timings but is a significant improvement. > > >> > > >> We'll investigate a little more but this may be as close as we can get > > >> for now. Let us know if this is enough to make it usable for you. > > >> > > >> Cheers, > > >> Dave > > >> > > >> rspates2 wrote: > > >>> Dave, > > >>> > > >>> The difference seems to be between 2.6.0 and 2.6.2, > > >>> > > >>> OntModel.listClasses(): > > >>> 2.6.2 time: 2187, totMem: 19948k, usedMem: 13345k > > >>> 2.6.0 time: 110, totMem: 5056k, usedMem: 3681k > > >>> 2.5.7 time: 78, totMem: 5056k, usedMem: 3726k > > >>> > > >>> listHierarchyRootClasses(): > > >>> 2.6.2 time: 2312, totMem: 20152k, usedMem: 13859k > > >>> 2.6.0 time: 172, totMem: 5056k, usedMem: 4312k > > >>> 2.5.7 time: 188, totMem: 5056k, usedMem: 4078k > > >>> > > >>> Let me know if you need anything else. > > >>> > > >>> Thanks, > > >>> Rick > > >>> > > >>> --- In [email protected], Dave Reynolds <dave.e.reynolds@> wrote: > > >>>> rspates2 wrote: > > >>>>> Hi, > > >>>>> > > >>>>> We switched our application from Jena 2.5.7 to 2.6.2. For an operation on OntModel.listHierarchyRootClasses() with an OntModelSpec of OWL_MEM_RDFS_INF, we saw a very significant increase in the use of memory and processing time, including out-of-memory condition for our full ontology. Using a subset of our ontology and the code below to demonstrate, we measured processing times and memory use: > > >>>>> > > >>>>> 2.6.2 time: 2063, totMem: 20008k, usedMem: 14539k > > >>>>> 2.5.7 time: 172, totMem: 5056k, usedMem: 3974k > > >>>> Certainly a huge jump :( > > >>>> > > >>>>> The specifications OWL_MEM_RDFS_INF and OWL_DL_MEM_RDFS_INF produced comparable results, while OWL_DL_MEM_TRANS_INF at 2.6.2 was comparable to our 2.5.7 run with OWL_MEM_RDFS_INF. > > >>>>> > > >>>>> I can post the subset ontology if you need it (170 k), but thought there might be some known explanation you could provide without running the demo code. > > >>>>> > > >>>>> Any idea what is causing this? > > >>>> No immediate ideas. The RFDS rule file itself hasn't changed in that > > >>>> time frame. It is tricky to trace back code changes over that period > > >>>> because we did a code reorg which hides a lot of the history. However, > > >>>> from memory (and the release notes) I can't think of anything that > > >>>> should have had an effect like that. There were changes around > > >>>> handling > > >>>> validation of data types but that isn't relevant here. We generalized > > >>>> handling of non-legal RDF triples but again that shouldn't be relevant. > > >>>> > > >>>> Could you try two experiments? > > >>>> > > >>>> First, would it be possible to time a simpler operation, such as > > >>>> listing > > >>>> all classes, to eliminate listHierarchyRootClasses as a factor? > > >>>> > > >>>> Secondly, is it possible to check with Jena 2.6.0 - that would > > >>>> separate > > >>>> the Java 5 changes and the datatype validation from more recent tweaks? > > >>>> > > >>>> If that's a problem then send me your ontology subset and I'll take a > > >>>> look when I can. > > >>>> > > >>>> Dave > > >>>> > > >>>>> Thanks for your help, > > >>>>> Rick > > >>>>> > > >>>>> import java.io.InputStream; > > >>>>> import org.junit.Test; > > >>>>> > > >>>>> import com.hp.hpl.jena.ontology.OntModel; > > >>>>> import com.hp.hpl.jena.ontology.OntModelSpec; > > >>>>> import com.hp.hpl.jena.rdf.model.Model; > > >>>>> import com.hp.hpl.jena.rdf.model.ModelFactory; > > >>>>> import com.hp.hpl.jena.util.iterator.ExtendedIterator; > > >>>>> > > >>>>> public class TestReasonerProblem { > > >>>>> > > >>>>> @Test > > >>>>> public void test() throws Exception { > > >>>>> > > >>>>> Model model = ModelFactory.createDefaultModel(); > > >>>>> String file="clinical-factDCMapVent.owl"; > > >>>>> InputStream fis = this.getClass().getResourceAsStream(file); > > >>>>> model.read(fis, null, "RDF/XML"); > > >>>>> OntModelSpec spec = OntModelSpec.OWL_MEM_RDFS_INF; > > >>>>> // OntModelSpec spec = OntModelSpec.OWL_DL_MEM_RDFS_INF; > > >>>>> // OntModelSpec spec = OntModelSpec.OWL_DL_MEM_TRANS_INF; > > >>>>> OntModel ontModel = ModelFactory.createOntologyModel(spec, > > >>>>>model); > > >>>>> > > >>>>> long ctms = System.currentTimeMillis(); > > >>>>> for (ExtendedIterator j = > > >>>>>ontModel.listHierarchyRootClasses(); j.hasNext(); ){ > > >>>>> j.next(); > > >>>>> } > > >>>>> System.out.printf("time: %d, totMem: %dk, usedMem: %dk\n", > > >>>>> (System.currentTimeMillis()-ctms), > > >>>>> (Runtime.getRuntime().totalMemory()/1024), > > >>>>> > > >>>>>((Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory())/1024)); > > >>>>> } > > >>>>> > > >>>>> } > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> ------------------------------------ > > >>>>> > > >>>>> Yahoo! Groups Links > > >>>>> > > >>>>> > > >>>>> > > >>> > > >>> > > >>> > > >>> ------------------------------------ > > >>> > > >>> Yahoo! Groups Links > > >>> > > >>> > > >>> > > > > > > > > > > > > > > > ------------------------------------ > > > > > > Yahoo! Groups Links > > > > > > > > > > > > >
