Sorry, false alarm: this turned out to be a simpler case of Model
modification during iteration, obscured by a recursive call.
On Fri, Aug 26, 2016 at 8:04 PM, Martynas Jusevičius
<marty...@graphity.org> wrote:
> Hey Dave,
>
> another case of this came up. When calling imports.hasNext() on
>
> ExtendedIterator<OntResource> imports = ontology.listImports();
>
> I consistently get ConcurrentModificationException:
>
> at
> org.apache.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkCME(LPTopGoalIterator.java:248)
> at
> org.apache.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:222)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
> at
> org.apache.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:55)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
> at
> org.apache.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:55)
> at org.apache.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:49)
> at
> org.apache.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
> at org.apache.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:49)
>
> Is there something special about listImports()? Is that considered a
> WRITE operation?
>
> I am creating a new OntModel with each request to avoid concurrent
> access, so I am quite sure (not 100% certain though) that nothing else
> is modifying this OntModel at the same time. As I had understood, in
> that case it should not be necessary to lock the model explicitly?
>
> I am attempting to implement polymorphism support, not sure if it
> could be related.
>
> On Fri, Jul 22, 2016 at 9:24 AM, Dave Reynolds
> <dave.e.reyno...@gmail.com> wrote:
>> On 21/07/16 22:26, Martynas Jusevičius wrote:
>>>
>>> Thanks Dave. Does the following code look reasonable?
>>>
>>>
>>> OntModel ontModel =
>>> OntDocumentManager.getInstance().getOntology(ontologyURI,
>>> ontModelSpec);
>>> ontModel.enterCriticalSection(Lock.READ);
>>> try
>>> {
>>> OntModel clonedModel =
>>> ModelFactory.createOntologyModel(ontModelSpec);
>>> clonedModel.add(ontModel);
>>> return clonedModel;
>>> }
>>> finally
>>> {
>>> ontModel.leaveCriticalSection();
>>> }
>>
>>
>> Seems reasonable so long as all the code that uses ontModel is similarly
>> wrapped in critical sections.
>>
>> Dave
>>
>>
>>> On Wed, Jul 20, 2016 at 10:46 AM, Dave Reynolds
>>> <dave.e.reyno...@gmail.com> wrote:
>>>>
>>>> So that's the reasoner in which case you need to lock the OntModel.
>>>>
>>>> Dave
>>>>
>>>>
>>>> On 19/07/16 17:04, Martynas Jusevičius wrote:
>>>>>
>>>>>
>>>>> Hey Andy,
>>>>>
>>>>> I am not sure yet what is it that I need to lock - is it the OntModel,
>>>>> or
>>>>> the OntDocumentManager instance, or maybe both.
>>>>>
>>>>> But here are 2 actual exceptions:
>>>>>
>>>>> java.util.ConcurrentModificationException
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkCME(LPTopGoalIterator.java:247)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:221)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> at
>>>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> at
>>>>>
>>>>>
>>>>> org.graphity.client.filter.response.ConstructorBase.construct(ConstructorBase.java:130)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> java.util.ConcurrentModificationException: Due to closed iterator
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.checkClosed(LPTopGoalIterator.java:256)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.moveForward(LPTopGoalIterator.java:95)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.reasoner.rulesys.impl.LPTopGoalIterator.hasNext(LPTopGoalIterator.java:222)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:54)
>>>>> com.hp.hpl.jena.util.iterator.Map1Iterator.hasNext(Map1Iterator.java:48)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
>>>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeRDFStatements(Basic.java:85)
>>>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeRDFStatements(Basic.java:74)
>>>>> com.hp.hpl.jena.xmloutput.impl.Basic.writeBody(Basic.java:48)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.writeXMLBody(BaseXMLWriter.java:492)
>>>>>
>>>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:464)
>>>>>
>>>>> com.hp.hpl.jena.xmloutput.impl.BaseXMLWriter.write(BaseXMLWriter.java:450)
>>>>> com.hp.hpl.jena.rdf.model.impl.ModelCom.write(ModelCom.java:340)
>>>>>
>>>>>
>>>>> com.hp.hpl.jena.ontology.impl.OntModelImpl.writeAll(OntModelImpl.java:2650)
>>>>>
>>>>> On Tue, 19 Jul 2016 at 14:04, Andy Seaborne <a...@apache.org> wrote:
>>>>>
>>>>>> On 17/07/16 13:54, Martynas Jusevičius wrote:
>>>>>>>
>>>>>>>
>>>>>>> But then again, I cannot lock what I don't have: locks work on models,
>>>>>>
>>>>>>
>>>>>> and
>>>>>>>
>>>>>>>
>>>>>>> I only get OntModel instance when getOntology() is called.
>>>>>>>
>>>>>>> Maybe synchronized is the solution here?
>>>>>>
>>>>>>
>>>>>>
>>>>>> You can lock how you like - the Jena lock code is helper code, to
>>>>>> promote proper critical sections and make app writers aware that
>>>>>> concurrency matters. It is not fundamental in any way - it's normally a
>>>>>> a java ReentrantReadWriteLock with some tracking code to catch
>>>>>> programming errors.
>>>>>>
>>>>>> If you are observing actual exceptions, it would be good to get the
>>>>>> traces. The caches are based on Guava's thread safe cache code although
>>>>>> (legacy) the cache setter is outside the get-or-fill operation. From
>>>>>> reading the code, it looks like the worse that can happen is to read a
>>>>>> remote resource more than once. But it's possible I've missed what
>>>>>> happening.
>>>>>>
>>>>>> As a general system point : notional read operations that internally
>>>>>> cache should make themselves multi-thread-read safe.
>>>>>>
>>>>>> Andy
>>>>>>
>>>>>>
>>>>>>> On Sun, 17 Jul 2016 at 15:32, Martynas Jusevičius
>>>>>>> <marty...@graphity.org
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks Andy.
>>>>>>>>
>>>>>>>> All of our code except OntDocumentManager access uses imutable
>>>>>>>> objects,
>>>>>>
>>>>>>
>>>>>> so
>>>>>>>>
>>>>>>>>
>>>>>>>> I hope to keep lock usage to an absolute minimum. Transactions look
>>>>>>
>>>>>>
>>>>>> like an
>>>>>>>>
>>>>>>>>
>>>>>>>> overkill at this point.
>>>>>>>>
>>>>>>>> So if getOntology() is a write operation, if I wrap only this piece
>>>>>>>> of
>>>>>>>> code where it is called into a WRITE critical section, wouldn't that
>>>>>>>> be
>>>>>>>> enough?
>>>>>>>>
>>>>>>>> I think the docs lack an explanation on which ops are read and which
>>>>>>>> are
>>>>>>>> write.
>>>>>>>>
>>>>>>>> On Sun, 17 Jul 2016 at 12:22, Andy Seaborne <a...@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Better: transactions (3.1.0)
>>>>>>>>>
>>>>>>>>> http://jena.staging.apache.org/documentation/txn/transactions.html
>>>>>>>>>
>>>>>>>>> We now have a fully ACID transactional in-memory graph (TIM) as well
>>>>>>>>> as
>>>>>>>>> TDB.
>>>>>>>>>
>>>>>>>>> The "Txn" highlevel API is not in the released codebase.
>>>>>>>>>
>>>>>>>>> http://jena.staging.apache.org/documentation/txn/txn.html
>>>>>>>>>
>>>>>>>>> Planning now for transactions should make things a lot easier later.
>>>>>>>>>
>>>>>>>>> Andy
>>>>>>>>>
>>>>>>>>> On 16/07/16 16:03, Martynas Jusevičius wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On a second thought, since our code is only reading from the
>>>>>>>>>> models,
>>>>>>
>>>>>>
>>>>>> it
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> probably easier and safer just to clone the shared model into a
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> per-request
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> copy, and keep the code as it is.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> If the shared model is only ever read, you can share it.
>>>>>>>>>
>>>>>>>>> (getOntology isn't a read operation.)
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Is the following code the fastest way to do it? Is add() expensive?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OntModel ontModel =
>>>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI,
>>>>>>
>>>>>>
>>>>>> ontModelSpec);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OntModel clonedModel =
>>>>>>>>>> ModelFactory.createOntologyModel(ontModelSpec);
>>>>>>>>>> clonedModel.add(ontModel);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Jul 16, 2016 at 3:47 PM, Martynas Jusevičius <
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> marty...@graphity.org>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hey,
>>>>>>>>>>>
>>>>>>>>>>> in our webapp, all objects are created per-request, except
>>>>>>>>>>> ontologies
>>>>>>>>>>> stored in a shared OntDocumentManager, that are accessed like
>>>>>>>>>>> this:
>>>>>>>>>>>
>>>>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ontModelSpec);
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This has caused some ConcurrentModificationExceptions.
>>>>>>>>>>> I have read the
>>>>>>>>>>> https://jena.apache.org/documentation/notes/concurrency-howto.html
>>>>>>>>>>> document, and wanted to check if I get this right.
>>>>>>>>>>>
>>>>>>>>>>> Basically, every operation and iterator on a shared OntModel has
>>>>>>>>>>> to
>>>>>>
>>>>>>
>>>>>> be
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> in a critical section? Even if it's indirect and called on an
>>>>>>
>>>>>>
>>>>>> OntClass
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> from the OntModel:
>>>>>>>>>>>
>>>>>>>>>>> OntModel ontModel =
>>>>>>>>>>> OntDocumentManager.getInstance().getOntology(ontologyURI,
>>>>>>>>>>> ontModelSpec);
>>>>>>>>>>> ontModel.enterCriticalSection(Lock.READ);
>>>>>>>>>>> try
>>>>>>>>>>> {
>>>>>>>>>>> OntClass ontClass =
>>>>>>>>>>> ontModel.getOntClass("http://some/class";);
>>>>>>>>>>> NodeIterator it =
>>>>>>
>>>>>>
>>>>>> ontClass.listPropertyValues(GC.supportedMode);
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> try
>>>>>>>>>>> {
>>>>>>>>>>> while (it.next)
>>>>>>>>>>> ...
>>>>>>>>>>> }
>>>>>>>>>>> finally
>>>>>>>>>>> {
>>>>>>>>>>> it.close();
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>> finally
>>>>>>>>>>> {
>>>>>>>>>>> model.leaveCriticalSection() ;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Is that correct?
>>>>>>>>>>>
>>>>>>>>>>> Martynas
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
