Hi Dave, I created simple test just to be sure that eventual additional dependencies are eliminated. Hence, I am now able to answer you questions:
>Is it the identical war file (assuming this is a war) in both cases? Yes. >Is it an identical web container? No. One is Tomcat the other is Resin. >Might you have multiple versions of Jena on the classpath (e.g. due to >putting something in an endorsed directory)? No. The Jena packaged within the WAR is the only one. >Is it the same jvm in each case? I do not know! I can not know jvm version running on the shared rented server. >Your error trace is from when the Jena Runtime starts up - it is loading >a metadata file which should be in the Jena jar and accessible via the >classloader. It looks like the classloader can't find the resource which >suggests a broken set of jars or just maybe a duff jvm. Yes. It definitely looks like a problem with classloader. Broken set of jars: I copied all jars from lib directory from Jena installation (C:\Install\Jena\Jena-2.6.4\lib). The only additional jar included in WAR is json.jar. I doubt that broken set of jar may be in question here? Duff jvm: Though I can not tell what jvm is it, this is the first time that something like this error happens. Assuming the jvm is duff, but I do not have administration rights over it, what are my options to fix the problem? Thanks, Milorad >________________________________ > From: Dave Reynolds <[email protected]> >To: Milorad Tosic <[email protected]> >Cc: "[email protected]" <[email protected]> >Sent: Saturday, December 17, 2011 6:45 PM >Subject: Re: comparison of JVM and Jena versions by memory usage > >Hi Milorad, > >> Memory limit was my best theory about what is going on with my application. >> Namely, I am trying to migrate an application using Jena from dedicated >> server to a shared server. The exactly same code is running without any >> problems until it is ported on the dedicated server. I paste my error trace >> at the end of the message for your reference. > >That looks likely to be an inconsistent set of jars. > >Is it the identical war file (assuming this is a war) in both cases? >Is it an identical web container? >Might you have multiple versions of Jena on the classpath (e.g. due to >putting something in an endorsed directory)? >Is it the same jvm in each case? > >> The application simply diminish without any exception caught. I traced it >> down to the single line instantiating OntModel (default reasoner, no >> ontology loaded initially): >> >> OntModel metaMCQ = null; >> >> metaMCQ = ModelFactory.createOntologyModel(); > >I suspect that's just because it is the first call which causes Jena to >do anything non trivial. > >Your error trace is from when the Jena Runtime starts up - it is loading >a metadata file which should be in the Jena jar and accessible via the >classloader. It looks like the classloader can't find the resource which >suggests a broken set of jars or just maybe a duff jvm. > >Dave > >> >> Exactly the same code works on the dedicated server while keeps crashing on >> the shared server? >> >> Thank you in advance, >> Milorad >> >> >> My error trace follows: >> >> java.lang.ExceptionInInitializerError >> at >> com.hp.hpl.jena.rdf.model.impl.RDFReaderFImpl.<clinit>(RDFReaderFImpl.java:85) >> at com.hp.hpl.jena.rdf.model.impl.ModelCom.<clinit>(ModelCom.java:42) >> at >> com.hp.hpl.jena.rdf.model.ModelFactory.createDefaultModel(ModelFactory.java:122) >> at >> com.hp.hpl.jena.rdf.model.ModelFactory.createDefaultModel(ModelFactory.java:116) >> at com.hp.hpl.jena.vocabulary.OWL.<clinit>(OWL.java:37) >> at >> com.hp.hpl.jena.ontology.ProfileRegistry.<clinit>(ProfileRegistry.java:48) >> at >> com.hp.hpl.jena.rdf.model.ModelFactory.createOntologyModel(ModelFactory.java:344) >> at >> com.virtuona.service.semcqgenerator.SeMCQgeneratorService.runApp(SeMCQgeneratorService.java:182) >> at >> com.virtuona.service.semcqgenerator.SeMCQgeneratorService.runService(SeMCQgeneratorService.java:380) >> at com.virtuona.service.ServiceCaller.callService(ServiceCaller.java:28) >> at >> com.virtuona.service.ServletServiceCaller.callService(ServletServiceCaller.java:397) >> at >> com.virtuona.service.ServletServiceCaller.procesRequest(ServletServiceCaller.java:258) >> at >> com.virtuona.service.ServletServiceCaller.doPost(ServletServiceCaller.java:107) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:165) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:103) >> at >> com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96) >> at com.caucho.server.http.Invocation.service(Invocation.java:315) >> at com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135) >> at com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346) >> at >> com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274) >> at com.caucho.server.TcpConnection.run(TcpConnection.java:139) >> at java.lang.Thread.run(Thread.java:595) >> Caused by: java.lang.NullPointerException >> at java.util.XMLUtils.load(XMLUtils.java:62) >> at java.util.Properties.loadFromXML(Properties.java:701) >> at com.hp.hpl.jena.util.Metadata.read(Metadata.java:67) >> at com.hp.hpl.jena.util.Metadata.addMetadata(Metadata.java:41) >> at com.hp.hpl.jena.util.Metadata.<init>(Metadata.java:35) >> at com.hp.hpl.jena.JenaRuntime.<clinit>(JenaRuntime.java:25) >> ... 22 more >> >> >> >> >> >________________________________ >> > From: Dave Reynolds <[email protected]> >> >To: Milorad Tosic <[email protected]> >> >Cc: "[email protected]" <[email protected]> >> >Sent: Friday, December 16, 2011 4:41 PM >> >Subject: Re: comparison of JVM and Jena versions by memory usage >> > >> >Hi, >> > >> >On Fri, 2011-12-16 at 05:39 -0800, Milorad Tosic wrote: >> >> Hi, >> >> >> >> >> >> There is a short discussion on the former Jena Development/Support >> >> mailing list related to memory usage of Jena OntModel using different >> >> versions of Jena. For your information I have included a segment of the >> >> correspondence at the end of the message. Since I had a problem that >> >> looks much like memory usage limit overrun, I got interested in the >> >> problem. >> >> >> >> >> >> I repeated the same experiment but with >> >> different conclusions? I compared Jena 2.6.0 with Jena 2.6.4. I couldn't >> >> notice the difference between Jena versions as reported in your >> >> previous experiment (though I used the same code). Tried with several >> >> variations of the OntModel creations styles but with minor differences >> >> in memory usage only (within 10%). I run the experiment from within >> >> Eclipse with JVM set to jre6 using compiler compliance level 1.5 (as >> >> well as 1.6). The total memory usage was around 27975k. >> >> >> >> Than, I changed the JVM to jdk 1.5. There was similar relative >> >> difference between the Jena 2.6.0 and Jena 2.6.4 (within 10%). However, >> >> the absolute amount of used memory dropped to about 3400k. >> >> >> >> Than I again changed the JVM to jdk 1.6. As in all previous testing >> >> configurations, there was similar relative difference between the Jena >> >> 2.6.0 and Jena 2.6.4 (within 10%). However, the absolute amount of used >> >> memory was about 6501k. >> >> >> >> Consequently, I have two problems now: >> >> >> >> 1) I couldn't repeat you experiment with the same results. What I am >> >> doing wrong? >> > >> >Assuming you are running the code that Rick originally posted then the >> >problem is that that uses a naive way to measure memory usage. >> >If I run with java 1.6 (sun-java on Linux) and the current Jena snapshot >> >and run in eclipse I get: >> > >> >time: 19, totMem: 122688k, usedMem: 4859k >> > >> >If I run identical code from the command line I get: >> >time: 12, totMem: 122688k, usedMem: 28548k >> > >> >The difference is down to whether a garbage collect has kicked in. If I >> >force a System.gc() in that code then I get: >> > >> >time: 86, totMem: 122688k, usedMem: 4006k >> > >> >where the extra time comes from the garbage collect. >> > >> > >> >> 2) What should I do to put the memory usage under my control? Namely, 5 >> >> times larger memory usage cause my application to broke on machines >> >> with limited memory usage. >> > >> >I don't think there's any evidence that the problem in the thread that >> >you are referring to is still an issue. >> > >> >You would need to explain more about what you are actually do in your >> >application before we could help with that. However, if you are >> >currently using inference and are having memory problems then avoiding >> >inference, or using a weaker inference setting, would be a good place to >> >start. >> > >> >Dave >> > >> >> >> >> Thanks, >> >> Milorad Tosic >> >> > >> >> > --- In [email protected], Dave Reynolds <dave.e.reynolds@...> >> >> > wrote: >> >> > > >> >> > > Rick, >> >> > > >> >> > > Thanks for letting us know, thanks for the clear report. >> >> > > >> >> > > Dave >> >> > > >> >> > > rspates2 wrote: >> >> > > > Dave, >> >> > > > >> >> > >> >> > > I grabbed OWLProfile.java and reran my tests. It looks fixed >> >> to me in terms of speed and memory use. We should be able to use 2.6.2 >> >> now. >> >> > > > >> >> > > > Thanks for your swift resolution to the problem. >> >> > > > >> >> > > > Rick >> >> > > > >> >> > > > --- In [email protected], Dave Reynolds <dave.e.reynolds@> >> >> > > > wrote: >> >> > > >> Hi Rick, >> >> > > >> >> >> > > >> A follow up on my earlier off-list response. >> >> > > >> >> >> > > >> Thanks again for giving us a test case for this. It turns out not >> >> > > >> to be >> >> > > >> a reasoner issue as such. There was a change in the internals of >> >> > > >> the >> >> > > >> Ontology API which worked inefficiently when a reasoner is >> >> > > >> present. Ian >> >> > > >> has checked in some patches for this which brings the memory use >> >> > > >> down a >> >> > > >> lot and gives a 5x speed improvement. >> >> > > >> >> >> > > >> It is not quite back to 2.6.0 timings but is a significant >> >> > > >> improvement. >> >> > > >> >> >> > > >> We'll investigate a little more but this may be as close as we can >> >> > > >> get >> >> > > >> for now. Let us know if this is enough to make it usable for you. >> >> > > >> >> >> > > >> Cheers, >> >> > > >> Dave >> >> > > >> >> >> > > >> rspates2 wrote: >> >> > > >>> Dave, >> >> > > >>> >> >> > > >>> The difference seems to be between 2.6.0 and 2.6.2, >> >> > > >>> >> >> > > >>> OntModel.listClasses(): >> >> > > >>> 2.6.2 time: 2187, totMem: 19948k, usedMem: 13345k >> >> > > >>> 2.6.0 time: 110, totMem: 5056k, usedMem: 3681k >> >> > > >>> 2.5.7 time: 78, totMem: 5056k, usedMem: 3726k >> >> > > >>> >> >> > > >>> listHierarchyRootClasses(): >> >> > > >>> 2.6.2 time: 2312, totMem: 20152k, usedMem: 13859k >> >> > > >>> 2.6.0 time: 172, totMem: 5056k, usedMem: 4312k >> >> > > >>> 2.5.7 time: 188, totMem: 5056k, usedMem: 4078k >> >> > > >>> >> >> > > >>> Let me know if you need anything else. >> >> > > >>> >> >> > > >>> Thanks, >> >> > > >>> Rick >> >> > > >>> >> >> > > >>> --- In [email protected], Dave Reynolds <dave.e.reynolds@> >> >> > > >>> wrote: >> >> > > >>>> rspates2 wrote: >> >> > > >>>>> Hi, >> >> > > >>>>> >> >> > >> >> > >>>>> We switched our application from Jena 2.5.7 >> >> to 2.6.2. For an operation on OntModel.listHierarchyRootClasses() with >> >> an OntModelSpec of OWL_MEM_RDFS_INF, we saw a very significant increase >> >> in the use of memory and processing time, including out-of-memory >> >> condition for our full ontology. Using a subset of our ontology and the >> >> code below to demonstrate, we measured processing times and memory use: >> >> > > >>>>> >> >> > > >>>>> 2.6.2 time: 2063, totMem: 20008k, usedMem: 14539k >> >> > > >>>>> 2.5.7 time: 172, totMem: 5056k, usedMem: 3974k >> >> > > >>>> Certainly a huge jump :( >> >> > > >>>> >> >> > >> >> > >>>>> The specifications OWL_MEM_RDFS_INF and >> >> OWL_DL_MEM_RDFS_INF produced comparable results, while >> >> OWL_DL_MEM_TRANS_INF at 2.6.2 was comparable to our 2.5.7 run with >> >> OWL_MEM_RDFS_INF. >> >> > > >>>>> >> >> > > >> >> >>>>> I can post the subset ontology if you need it (170 >> >> k), but thought there might be some known explanation you could provide >> >> without running the demo code. >> >> > > >>>>> >> >> > > >>>>> Any idea what is causing this? >> >> > > >>>> No immediate ideas. The RFDS rule file itself hasn't changed in >> >> > > >>>> that >> >> > > >>>> time frame. It is tricky to trace back code changes over that >> >> > > >>>> period >> >> > > >>>> because we did a code reorg which hides a lot of the history. >> >> > > >>>> However, >> >> > > >>>> from memory (and the release notes) I can't think of anything >> >> > > >>>> that >> >> > > >>>> should have had an effect like that. There were changes around >> >> > > >>>> handling >> >> > > >>>> validation of data types but that isn't relevant here. We >> >> > > >>>> generalized >> >> > > >>>> handling of non-legal RDF triples but again that shouldn't be >> >> > > >>>> relevant. >> >> > > >>>> >> >> > > >>>> Could you try two experiments? >> >> > > >>>> >> >> > > >>>> First, would it be possible to time a simpler operation, such as >> >> > > >>>> listing >> >> > > >>>> all classes, to eliminate listHierarchyRootClasses as a factor? >> >> > > >>>> >> >> > > >>>> Secondly, is it possible to check with Jena 2.6.0 - that would >> >> > > >>>> separate >> >> > > >>>> the Java 5 changes and the datatype validation from more recent >> >> > > >>>> tweaks? >> >> > > >>>> >> >> > > >>>> If that's a problem then send me your ontology subset and I'll >> >> > > >>>> take a >> >> > > >>>> look when I can. >> >> > > >>>> >> >> > > >>>> Dave >> >> > > >>>> >> >> > > >>>>> Thanks for your help, >> >> > > >>>>> Rick >> >> > > >>>>> >> >> > > >>>>> import java.io.InputStream; >> >> > > >>>>> import org.junit.Test; >> >> > > >>>>> >> >> > > >>>>> import com.hp.hpl.jena.ontology.OntModel; >> >> > > >>>>> import com.hp.hpl.jena.ontology.OntModelSpec; >> >> > > >>>>> import com.hp.hpl.jena.rdf.model.Model; >> >> > > >>>>> import com.hp.hpl.jena.rdf.model.ModelFactory; >> >> > > >>>>> import com.hp.hpl.jena.util.iterator.ExtendedIterator; >> >> > > >>>>> >> >> > > >>>>> public class TestReasonerProblem { >> >> > > >>>>> >> >> > > >>>>> @Test >> >> > > >>>>> public void test() throws Exception { >> >> > > >>>>> >> >> > > >>>>> Model model = ModelFactory.createDefaultModel(); >> >> > > >>>>> String file="clinical-factDCMapVent.owl"; >> >> > > >>>>> InputStream fis = >> >> > > >>>>>this.getClass().getResourceAsStream(file); >> >> > > >>>>> model.read(fis, null, "RDF/XML"); >> >> > > >>>>> OntModelSpec spec = OntModelSpec.OWL_MEM_RDFS_INF; >> >> > > >>>>> // OntModelSpec spec = OntModelSpec.OWL_DL_MEM_RDFS_INF; >> >> > > >>>>> // OntModelSpec spec = OntModelSpec.OWL_DL_MEM_TRANS_INF; >> >> > > >>>>> OntModel ontModel = >> >> > > >>>>>ModelFactory.createOntologyModel(spec, model); >> >> > > >>>>> >> >> > > >>>>> long ctms = System.currentTimeMillis(); >> >> > > >>>>> for (ExtendedIterator j = >> >> > > >>>>>ontModel.listHierarchyRootClasses(); j.hasNext(); ){ >> >> > > >>>>> j.next(); >> >> > > >>>>> } >> >> > > >>>>> System.out.printf("time: %d, totMem: %dk, usedMem: >> >> > > >>>>>%dk\n", >> >> > > >>>>> (System.currentTimeMillis()-ctms), >> >> > > >>>>> (Runtime.getRuntime().totalMemory()/1024), >> >> > > >>>>> >> >> > > >>>>>((Runtime.getRuntime().totalMemory()-Runtime.getRuntime().freeMemory())/1024)); >> >> > > >>>>> } >> >> > > >>>>> >> >> > > >>>>> } >> >> > > >>>>> >> >> > > >>>>> >> >> > > >>>>> >> >> > > >>>>> >> >> > > >>>>> ------------------------------------ >> >> > > >>>>> >> >> > > >>>>> Yahoo! Groups Links >> >> > > >>>>> >> >> > > >>>>> >> >> > > >>>>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> ------------------------------------ >> >> > > >>> >> >> > > >>> Yahoo! Groups Links >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > ------------------------------------ >> >> > > > >> >> > > > Yahoo! Groups Links >> >> > > > >> >> > > > >> >> > > > >> >> > > >> >> > >> >> > >> > >> > >> > >> > >> > >> > > > > > >
