Hey Dave, another case of this came up. When calling imports.hasNext() on
ExtendedIterator<OntResource> imports = ontology.listImports(); I consistently get ConcurrentModificationException: at org.apache.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkCME(LPTopGoalIterator.java:248) at org.apache.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:222) at org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at org.apache.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:55) at org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at org.apache.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:55) at org.apache.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:49) at org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) at org.apache.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:49) Is there something special about listImports()? Is that considered a WRITE operation? I am creating a new OntModel with each request to avoid concurrent access, so I am quite sure (not 100% certain though) that nothing else is modifying this OntModel at the same time. As I had understood, in that case it should not be necessary to lock the model explicitly? I am attempting to implement polymorphism support, not sure if it could be related. On Fri, Jul 22, 2016 at 9:24 AM, Dave Reynolds <dave.e.reyno...@gmail.com> wrote: > On 21/07/16 22:26, Martynas Jusevičius wrote: >> >> Thanks Dave. Does the following code look reasonable? >> >> >> OntModel ontModel = >> OntDocumentManager.getInstance().getOntology(ontologyURI, >> ontModelSpec); >> ontModel.enterCriticalSection(Lock.READ); >> try >> { >> OntModel clonedModel = >> ModelFactory.createOntologyModel(ontModelSpec); >> clonedModel.add(ontModel); >> return clonedModel; >> } >> finally >> { >> ontModel.leaveCriticalSection(); >> } > > > Seems reasonable so long as all the code that uses ontModel is similarly > wrapped in critical sections. > > Dave > > >> On Wed, Jul 20, 2016 at 10:46 AM, Dave Reynolds >> <dave.e.reyno...@gmail.com> wrote: >>> >>> So that's the reasoner in which case you need to lock the OntModel. >>> >>> Dave >>> >>> >>> On 19/07/16 17:04, Martynas Jusevičius wrote: >>>> >>>> >>>> Hey Andy, >>>> >>>> I am not sure yet what is it that I need to lock - is it the OntModel, >>>> or >>>> the OntDocumentManager instance, or maybe both. >>>> >>>> But here are 2 actual exceptions: >>>> >>>> java.util.ConcurrentModificationException >>>> at >>>> >>>> >>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkCME(LPTopGoalIterator.java:247) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:221) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> at >>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> at >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> at >>>> >>>> >>>> org.graphity.client.filter.response.ConstructorBase.construct(ConstructorBase.java:130) >>>> >>>> >>>> >>>> >>>> java.util.ConcurrentModificationException: Due to closed iterator >>>> >>>> >>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkClosed(LPTopGoalIterator.java:256) >>>> >>>> >>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.moveForward(LPTopGoalIterator.java:95) >>>> >>>> >>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:222) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54) >>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48) >>>> >>>> >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90) >>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeRDFStatements(Basic.java:85) >>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeRDFStatements(Basic.java:74) >>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeBody(Basic.java:48) >>>> >>>> >>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.writeXMLBody(BaseXMLWriter.java:492) >>>> >>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:464) >>>> >>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:450) >>>> com.hp.hpl.jena.rdf.model.impl.ModelCom.write(ModelCom.java:340) >>>> >>>> >>>> com.hp.hpl.jena.ontology.impl.OntModelImpl.writeAll(OntModelImpl.java:2650) >>>> >>>> On Tue, 19 Jul 2016 at 14:04, Andy Seaborne <a...@apache.org> wrote: >>>> >>>>> On 17/07/16 13:54, Martynas Jusevičius wrote: >>>>>> >>>>>> >>>>>> But then again, I cannot lock what I don't have: locks work on models, >>>>> >>>>> >>>>> and >>>>>> >>>>>> >>>>>> I only get OntModel instance when getOntology() is called. >>>>>> >>>>>> Maybe synchronized is the solution here? >>>>> >>>>> >>>>> >>>>> You can lock how you like - the Jena lock code is helper code, to >>>>> promote proper critical sections and make app writers aware that >>>>> concurrency matters. It is not fundamental in any way - it's normally a >>>>> a java ReentrantReadWriteLock with some tracking code to catch >>>>> programming errors. >>>>> >>>>> If you are observing actual exceptions, it would be good to get the >>>>> traces. The caches are based on Guava's thread safe cache code although >>>>> (legacy) the cache setter is outside the get-or-fill operation. From >>>>> reading the code, it looks like the worse that can happen is to read a >>>>> remote resource more than once. But it's possible I've missed what >>>>> happening. >>>>> >>>>> As a general system point : notional read operations that internally >>>>> cache should make themselves multi-thread-read safe. >>>>> >>>>> Andy >>>>> >>>>> >>>>>> On Sun, 17 Jul 2016 at 15:32, Martynas Jusevičius >>>>>> <marty...@graphity.org >>>>>> >>>>>> wrote: >>>>>> >>>>>>> Thanks Andy. >>>>>>> >>>>>>> All of our code except OntDocumentManager access uses imutable >>>>>>> objects, >>>>> >>>>> >>>>> so >>>>>>> >>>>>>> >>>>>>> I hope to keep lock usage to an absolute minimum. Transactions look >>>>> >>>>> >>>>> like an >>>>>>> >>>>>>> >>>>>>> overkill at this point. >>>>>>> >>>>>>> So if getOntology() is a write operation, if I wrap only this piece >>>>>>> of >>>>>>> code where it is called into a WRITE critical section, wouldn't that >>>>>>> be >>>>>>> enough? >>>>>>> >>>>>>> I think the docs lack an explanation on which ops are read and which >>>>>>> are >>>>>>> write. >>>>>>> >>>>>>> On Sun, 17 Jul 2016 at 12:22, Andy Seaborne <a...@apache.org> wrote: >>>>>>> >>>>>>>> Better: transactions (3.1.0) >>>>>>>> >>>>>>>> http://jena.staging.apache.org/documentation/txn/transactions.html >>>>>>>> >>>>>>>> We now have a fully ACID transactional in-memory graph (TIM) as well >>>>>>>> as >>>>>>>> TDB. >>>>>>>> >>>>>>>> The "Txn" highlevel API is not in the released codebase. >>>>>>>> >>>>>>>> http://jena.staging.apache.org/documentation/txn/txn.html >>>>>>>> >>>>>>>> Planning now for transactions should make things a lot easier later. >>>>>>>> >>>>>>>> Andy >>>>>>>> >>>>>>>> On 16/07/16 16:03, Martynas Jusevičius wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On a second thought, since our code is only reading from the >>>>>>>>> models, >>>>> >>>>> >>>>> it >>>>>>>> >>>>>>>> >>>>>>>> is >>>>>>>>> >>>>>>>>> >>>>>>>>> probably easier and safer just to clone the shared model into a >>>>>>>> >>>>>>>> >>>>>>>> per-request >>>>>>>>> >>>>>>>>> >>>>>>>>> copy, and keep the code as it is. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> If the shared model is only ever read, you can share it. >>>>>>>> >>>>>>>> (getOntology isn't a read operation.) >>>>>>>> >>>>>>>>> >>>>>>>>> Is the following code the fastest way to do it? Is add() expensive? >>>>>>>>> >>>>>>>>> >>>>>>>>> OntModel ontModel = >>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI, >>>>> >>>>> >>>>> ontModelSpec); >>>>>>>>> >>>>>>>>> >>>>>>>>> OntModel clonedModel = >>>>>>>>> ModelFactory.createOntologyModel(ontModelSpec); >>>>>>>>> clonedModel.add(ontModel); >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jul 16, 2016 at 3:47 PM, Martynas Jusevičius < >>>>>>>> >>>>>>>> >>>>>>>> marty...@graphity.org> >>>>>>>>> >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hey, >>>>>>>>>> >>>>>>>>>> in our webapp, all objects are created per-request, except >>>>>>>>>> ontologies >>>>>>>>>> stored in a shared OntDocumentManager, that are accessed like >>>>>>>>>> this: >>>>>>>>>> >>>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI, >>>>>>>> >>>>>>>> >>>>>>>> ontModelSpec); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This has caused some ConcurrentModificationExceptions. >>>>>>>>>> I have read the >>>>>>>>>> https://jena.apache.org/documentation/notes/concurrency-howto.html >>>>>>>>>> document, and wanted to check if I get this right. >>>>>>>>>> >>>>>>>>>> Basically, every operation and iterator on a shared OntModel has >>>>>>>>>> to >>>>> >>>>> >>>>> be >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> in a critical section? Even if it's indirect and called on an >>>>> >>>>> >>>>> OntClass >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> from the OntModel: >>>>>>>>>> >>>>>>>>>> OntModel ontModel = >>>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI, >>>>>>>>>> ontModelSpec); >>>>>>>>>> ontModel.enterCriticalSection(Lock.READ); >>>>>>>>>> try >>>>>>>>>> { >>>>>>>>>> OntClass ontClass = >>>>>>>>>> ontModel.getOntClass("http://some/class"); >>>>>>>>>> NodeIterator it = >>>>> >>>>> >>>>> ontClass.listPropertyValues(GC.supportedMode); >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> try >>>>>>>>>> { >>>>>>>>>> while (it.next) >>>>>>>>>> ... >>>>>>>>>> } >>>>>>>>>> finally >>>>>>>>>> { >>>>>>>>>> it.close(); >>>>>>>>>> } >>>>>>>>>> } >>>>>>>>>> finally >>>>>>>>>> { >>>>>>>>>> model.leaveCriticalSection() ; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> Is that correct? >>>>>>>>>> >>>>>>>>>> Martynas >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >