Andy,
We use TDB using the default mode (we don't set anything special). This I
think is "mapped" on a 64-bit JVM.
Thanks
Steve Groeger
From: Andy Seaborne <a...@apache.org>
To: users@jena.apache.org,
Date: 24/01/2014 12:59
Subject: Re: BlockException: No such block
Steve,
the exception is the system detecting an inconsistency. It's symptom,
not cause. The cause is somewhere else.
How do you run TDB? Some groups in IBM run in direct mode because of
the need t delete databases from inside a JVM, which you can't do in MS
Windows when using memory mapped files.
Andy
On 24/01/14 07:18, Steve Groeger wrote:
> Andy,
>
> Just for completeness here is the stack trace from running the posted
> code.
>
> com.hp.hpl.jena.tdb.base.block.BlockException: No such block
> at
> com.hp.hpl.jena.tdb.base.recordbuffer.RecordRangeIterator.iterator(
> RecordRangeIterator.java:39)
> at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
> BPlusTree.java:383)
> at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
> BPlusTree.java:366)
> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findWorker(
> TupleIndexRecord.java:164)
> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findOrScan(
> TupleIndexRecord.java:84)
> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.performFind(
> TupleIndexRecord.java:78)
> at com.hp.hpl.jena.tdb.index.TupleIndexBase.find(
> TupleIndexBase.java:91)
> at
com.hp.hpl.jena.tdb.index.TupleTable.find(TupleTable.java:197)
> at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
> NodeTupleTableConcrete.java:169)
> at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
> NodeTupleTableConcrete.java:157)
> at
> com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.findAsNodeIds(
> NodeTupleTableConcrete.java:146)
> at com.hp.hpl.jena.tdb.store.QuadTable.find(QuadTable.java:87)
> at
> com.hp.hpl.jena.tdb.store.DatasetGraphTDB.findInSpecificNamedGraph(
> DatasetGraphTDB.java:89)
> at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.findNG(
> DatasetGraphBaseFind.java:59)
> at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.find(
> DatasetGraphBaseFind.java:48)
> at com.hp.hpl.jena.sparql.core.GraphView.graphBaseFind(
> GraphView.java:110)
> at
com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:287)
> at com.hp.hpl.jena.rdf.model.impl.ModelCom.findTriplesFrom(
> ModelCom.java:1308)
> at com.hp.hpl.jena.rdf.model.impl.ModelCom.listStatements(
> ModelCom.java:1279)
> at com.ibm.jenatest.ReaderThread.test(ReaderThread.java:61)
> at com.ibm.jenatest.GenericTestThread.run(
> GenericTestThread.java:39)
>
> It took 16690 iterations of the writer thread to obtain this error but
as
> you can see the reader thread gets the exception when trying to execute
> this line:
> StmtIterator found = model.listStatements(new
> SimpleSelector((Resource)null, (Property)null, (RDFNode)null));
>
> Thanks
> Steve Groeger
>
>
>
>
> From: Andy Seaborne <a...@apache.org>
> To: users@jena.apache.org,
> Date: 23/01/2014 18:49
> Subject: Re: BlockException: No such block
>
>
>
> Steve,
>
> The stacktrace does not correspond to the test code posted - there's no
> class CreateThread nor methods .checkModel or .getModelCopy.
>
> I have fixed up the code in your message and run it and it has been OK
> for me over a couple of extended runs.
>
> Andy
>
>
> On 23/01/14 13:04, Steve Groeger wrote:
>> We are using Jena 2.11.0 and trying to do some concurrency stress
> testing.
>> When running our tests, after a while we get the following error.
>>
>> com.hp.hpl.jena.tdb.base.block.BlockException: No such block
>> at
>> com.hp.hpl.jena.tdb.base.recordbuffer.RecordRangeIterator.iterator(
>> RecordRangeIterator.java:39)
>> at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
>> BPlusTree.java:383)
>> at com.hp.hpl.jena.tdb.index.bplustree.BPlusTree.iterator(
>> BPlusTree.java:366)
>> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findWorker(
>> TupleIndexRecord.java:164)
>> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.findOrScan(
>> TupleIndexRecord.java:84)
>> at com.hp.hpl.jena.tdb.index.TupleIndexRecord.performFind(
>> TupleIndexRecord.java:78)
>> at com.hp.hpl.jena.tdb.index.TupleIndexBase.find(
>> TupleIndexBase.java:91)
>> at
> com.hp.hpl.jena.tdb.index.TupleTable.find(TupleTable.java:197)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
>> NodeTupleTableConcrete.java:169)
>> at com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.find(
>> NodeTupleTableConcrete.java:157)
>> at
>> com.hp.hpl.jena.tdb.nodetable.NodeTupleTableConcrete.findAsNodeIds(
>> NodeTupleTableConcrete.java:146)
>> at
com.hp.hpl.jena.tdb.store.QuadTable.find(QuadTable.java:87)
>> at
>> com.hp.hpl.jena.tdb.store.DatasetGraphTDB.findInSpecificNamedGraph(
>> DatasetGraphTDB.java:89)
>> at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.findNG(
>> DatasetGraphBaseFind.java:59)
>> at com.hp.hpl.jena.sparql.core.DatasetGraphBaseFind.find(
>> DatasetGraphBaseFind.java:48)
>> at com.hp.hpl.jena.sparql.core.GraphView.graphBaseFind(
>> GraphView.java:110)
>> at com.hp.hpl.jena.sparql.core.GraphView.graphBaseFind(
>> GraphView.java:104)
>> at
> com.hp.hpl.jena.graph.impl.GraphBase.find(GraphBase.java:268)
>> at
com.hp.hpl.jena.graph.GraphUtil.findAll(GraphUtil.java:128)
>> at
com.hp.hpl.jena.graph.GraphUtil.addInto(GraphUtil.java:183)
>> at
> com.hp.hpl.jena.rdf.model.impl.ModelCom.add(ModelCom.java:208)
>> at
> com.hp.hpl.jena.rdf.model.impl.ModelCom.add(ModelCom.java:202)
>> at com.ibm.jenatest.CreateThread.getModelCopy(
>> CreateThread.java:154)
>> at
> com.ibm.jenatest.CreateThread.checkModel(CreateThread.java:125)
>> at com.ibm.jenatest.CreateThread.test(CreateThread.java:86)
>> at com.ibm.jenatest.CreateThread.run(CreateThread.java:52)
>>
>> Can anyone please explain why we are getting this error, ie if this is
a
>> bug or we are doing something wrong in our test.
>>
>> Occasionally we get other issues in the reader thread when we retrieve
> the
>> statements from the resource, the number that are returned do not match
>> the number that are expected (ie 1000) but if we get the statements
> again
>> all 1000 are there. Hopefully there is an explanation for each of these
>> issues.
>>
>> We are running out test on a RHEL system using the IBM JVM, but have
> also
>> used other JVMs and we get the same issues.
>>
>> Our test has a writer thread and several (currently 2) reader threads.
> The
>> number of iterations necessary for the test to fail vary greatly but it
>> does fail at some point.
>>
>> Here are the classes for our test and the classes for the 2 threads:
>>
>> public class ReaderWriterTest {
>>
>> public static void main(String[] args) {
>> int i = 1;
>> GenericTestThread writer = null;
>> writer = new WriterThread("writer" + (i++), args[0] +
> "jenatest");
>> List<ReaderThread> readers = new ArrayList<ReaderThread>();
>> // setup the reader threads
>> readers.add(new ReaderThread("reader" + (i++), args[0] +
>> "jenatest"));
>> readers.add(new ReaderThread("reader" + (i++), args[0] +
>> "jenatest"));
>>
>> if (writer!=null)
>> writer.setReaderThreads(readers);
>>
>> for (GenericTestThread reader : readers) {
>> if (writer!=null)
>> reader.setWriterThread(writer);
>>
>> // create the specific data for each of the reader
threads
> to
>> read
>> reader.setReaderThreads(readers);
>> Dataset createSession = TDBFactory.createDataset
>> (((ReaderThread)reader).location);
>> createSession.begin(ReadWrite.WRITE);
>> String uri = "http://www.ibm.com/"; + reader.getName() +
"/"
> +
>> 0;
>>
>> // Create Jena model with a number of properties
>> Model model = createSession.getNamedModel(uri);
>> Resource res = model.createResource(uri);
>> for (int j = 0; j < 1000; j++) {
>> Property p = model.createProperty("
>> http://www.ibm.com/prop/"; + reader.getName() + "/" + j);
>> res.addProperty(p,
model.createTypedLiteral("Property"
> +
>> reader.getName() + "/" + j));
>> }
>> model.close();
>> createSession.commit();
>> createSession.end();
>> createSession.close();
>> }
>> // start the writer thread
>> if (writer!=null)
>> writer.start();
>>
>> // start the reader threads
>> for (GenericTestThread reader : readers) {
>> reader.start();
>> }
>> }
>> }
>>
>>
>> public class WriterThread extends GenericTestThread {
>>
>> private static final int NUM_PROPS = 1000;
>> private String location;
>>
>> public WriterThread(String name, String location) {
>> super(name);
>> this.location = location;
>> }
>>
>> @Override
>> protected void test() {
>> for (int i = 0; true; i++) {
>> String uri = "http://www.ibm.com/"; + getName() + "/" + i;
>> checkStop(uri);
>>
>> Dataset createSession =
TDBFactory.createDataset(location);
>> createSession.begin(ReadWrite.WRITE);
>> info(""+i+"
size="+createSession.asDatasetGraph().size());
>>
>> // Create Jena model with a number of properties
>> Model model = createSession.getNamedModel(uri);
>> Resource res = model.createResource(uri);
>> for (int i1 = 0; i1 < NUM_PROPS; i1++) {
>> Property p = model.createProperty("
>> http://www.ibm.com/prop/"; + getName() + "/" + i1);
>> res.addProperty(p,
model.createTypedLiteral("Property"
> +
>> getName() + "/" + i1));
>> }
>> createSession.commit();
>> createSession.end();
>> model.close();
>>
>> createSession.begin(ReadWrite.WRITE);
>> model = createSession.getNamedModel(uri);
>> model.removeAll();
>> createSession.removeNamedModel(uri);
>> createSession.commit();
>> createSession.end();
>> model.close();
>> createSession.close();
>> }
>> }
>> }
>>
>> public class ReaderThread extends GenericTestThread {
>>
>> private static final int NUM_PROPS = 1000;
>> protected String location;
>>
>> public ReaderThread(String name, String location) {
>> super(name);
>> this.location = location;
>> }
>>
>> @Override
>> protected void test() {
>> String uri = "http://www.ibm.com/"; + getName() + "/" + 0;
>> int iteration = 0;
>> while (true) {
>> if (((iteration++) % 500) == 0) {
>> info("check: "+iteration);
>> // need to add a sleep to the thread otherwise
> jena
>> can never write the data and the journal.jml file
>> // gets excessively large and we run out of
> memory
>> in the JVM
>> try {
>> Thread.sleep(1000);
>> } catch (InterruptedException e) {
>> // TODO Auto-generated catch block
>> e.printStackTrace();
>> }
>> }
>> Dataset checkSession =
TDBFactory.createDataset(location);
>> checkSession.begin(ReadWrite.READ);
>> Model model = checkSession.getNamedModel(uri);
>> StmtIterator found = model.listStatements(new
>> SimpleSelector((Resource)null, (Property)null, (RDFNode)null));
>> Int count = found.toList().size();
>> if (count != NUM_PROPS) {
>> System.out.println("Models do not compare expected
> size: "
>> + NUM_PROPS + " was: " + count);
>> }
>> found.close();
>> checkSession.end();
>> // model.close(); // removed as causes errors with jena
>> saying we have multiple writers !!!!!!
>> checkSession.close();
>> }
>> }
>> }
>>
>> public abstract class GenericTestThread extends Thread {
>>
>> private GenericTestThread writerThread;
>> private volatile boolean stopped = false;
>> private List<ReaderThread> readers;
>>
>> public GenericTestThread(String name) {
>> setName(name);
>> }
>>
>> protected abstract void test();
>>
>> public void run() {
>> try {
>> info("Running thread");
>> test();
>> info("Thread done");
>> } catch (Exception e) {// STYLE reason: fault barrier
>> synchronized (WriterThread.class) {
>> snapshotWriterThread();
>> for (GenericTestThread reader : readers) {
>> reader.shutdown();
>> }
>> e.printStackTrace(System.out);// STYLE reason: unit
> test
>> // System.exit(0); // STYLE reason: TODO
>> }
>> }
>> }
>>
>> protected void snapshotWriterThread() {
>> if (writerThread != null) {
>> info(writerThread.getStackTrace());
>> writerThread.shutdown();
>> }
>> }
>>
>> private void shutdown() {
>> stopped = true;
>> }
>>
>> private void info(StackTraceElement[] stackTraceElements) {
>> for (StackTraceElement trace : stackTraceElements) {
>> info(" " + trace);
>> }
>> }
>>
>> protected void info(Exception e) {
>> System.out.println(getName() + ": " + e.toString());// STYLE
>> reason: unit test
>> }
>>
>> protected void info(String message) {
>> System.out.println(getName() + ": " + message);// STYLE
reason:
>> unit test
>> }
>>
>> protected void checkStop(String message) {
>> if (stopped) {
>> throw new RuntimeException("thread stopped: " + message);
>> }
>> }
>>
>> public void setWriterThread(GenericTestThread other) {
>> this.writerThread = other;
>> }
>>
>> public GenericTestThread getWriterThread() {
>> return this.writerThread;
>> }
>>
>> public void setReaderThreads(List<ReaderThread> readers) {
>> this.readers = readers;
>> }
>>
>> public List<ReaderThread> getReaderThreads() {
>> return this.readers;
>> }
>> }
>>
>> Thanks
>> Steve Groeger
>>
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with
number
>> 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
>>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
