An update on this problem: when I retry the experiments with the patch I produced for https://issues.apache.org/jira/browse/JENA-131, I am not able to see an issue anymore if I do not execute an abort. So, somehow, the problem below (the "No such block") appears related to the problem I noticed in Jena-131.
however, even with this patch, if I permit query aborts, I get all the issues reported at the beginning of this thread. So, I suspect that there is still an issue with query abortion and transactions. Simon From: Simon Helsen/Toronto/IBM To: [email protected] Date: 10/19/2011 04:46 PM Subject: Re: queryExecution.abort() and TxTDB transactions Hmmm, when I remove the concurrent abort, I am still seeing issues. This time the first error I see is the following: 16:22:28,017 [1332957043@qtp-1797810984-15] ERROR com.ibm.team.jfs - Originating Exception: com.hp.hpl.jena.tdb.base.block.BlockException: No such block at com.hp.hpl.jena.tdb.base.recordbuffer.RecordRangeIterator.iterator( RecordRangeIterator.java:39) at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator( BPlusTree.java:378) at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(BPlusTree.java) at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findWorker( TupleIndexRecord.java:164) at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findOrScan( TupleIndexRecord.java:84) at com.hp.hpl.jena.tdb.index.TupleIndexRecord.performFind( TupleIndexRecord.java:78) at com.hp.hpl.jena.tdb.index.TupleIndexBase.find( TupleIndexBase.java:88) at com.hp.hpl.jena.tdb.index.TupleTable.find(TupleTable.java:138) at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find( NodeTupleTableConcrete.java:169) at com.hp.hpl.jena.tdb.solver.StageMatchTuple.makeNextStage( StageMatchTuple.java:101) at com.hp.hpl.jena.tdb.solver.StageMatchTuple.makeNextStage( StageMatchTuple.java:44) at org.openjena.atlas.iterator.RepeatApplyIterator.hasNext( RepeatApplyIterator.java:42) at org.openjena.atlas.iterator.Iter$4.hasNext(Iter.java:274) at com.hp.hpl.jena.sparql.engine.iterator.QueryIterPlainWrapper.hasNextBinding( QueryIterPlainWrapper.java:54) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.iterator.QueryIterProcessBinding.hasNextBinding( QueryIterProcessBinding.java:60) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding( QueryIterConvert.java:65) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding( QueryIterSlice.java:76) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding( QueryIteratorWrapper.java:40) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding( QueryIteratorWrapper.java:40) at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext( QueryIteratorBase.java:107) at com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext( ResultSetStream.java:70) at com.ibm.team.jfs.rdf.internal.jenatdb.InternalResultSet.retrieveAllBindings( InternalResultSet.java:46) It must be something more fundamental :-( From: Simon Helsen/Toronto/IBM To: [email protected] Date: 10/19/2011 03:26 PM Subject: Re: queryExecution.abort() and TxTDB transactions Let me give me more detail how I set up the code. The main thread just sets up the transaction, so, something like this: DatasetGraphTxn dsGraph = null; try { dsGraph = StoreConnection.make(this.location ).begin(ReadWrite.READ); Dataset ds = dsGraph.toDataset(); ... QueryExecution qe = null; ... try { results = qe.execSelect(); ... } finally { if (qe != null) { qe.close(); } } } catch (QueryCancelledException e) { if (dsGraph != null) { dsGraph.abort(); } } finally { if (dsGraph != null) { dsGraph.close(); } } A parallel thread may decide that the given query needs to be cancelled, so it has access to the QueryExecution and may decide to call this.queryExecution.abort(); The question is whether I am just coding this wrong or whether there is possible flow which does not yet work in TxTDB thanks Simon From: Simon Helsen/Toronto/IBM To: [email protected] Date: 10/19/2011 03:18 PM Subject: queryExecution.abort() and TxTDB transactions Hi guys, is the interaction between queryExecution.abort() and TxTDB transactions tested? When I use it, it seems that the store corrupts again. Note that when the QueryCancellationException is thrown, I actively abort the DatasetGraphTxn object, but I am seeing 14:59:20,580 [477961341@qtp-1709008349-8] WARN hpl.jena.sparql.engine.iterator.QueryIteratorCheck - Open iterator: QueryIterFilterExpr/37159 and then shortly after: 15:00:34,691 [jazz.jfs.indexer.jfs_tests_default_consumer_name.triple] ERROR com.ibm.team.jfs - Originating Exception: com.hp.hpl.jena.tdb.base.file.FileException: ObjectFile.read(8072)[12980][12980]: Impossibly large object : 1936010863 bytes at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read( ObjectFileStorage.java:294) at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage$ObjectIterator.next( ObjectFileStorage.java:409) etc. Note that I call queryExecution.abort() in a separate thread and I wonder if it runs in trouble if the main thread (executing the query) is just calling queryExecution.close() Simon
