Hi Andy, All the latest tests I was running were setting SystemTDB.NullOut = true;
I will add the code you specified to our tests and re-run them and will send you our results shortly. Thanks Steve Groeger IBM WebSphere Service Registry Development MP 211, Hursley Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 Fax (44) 1962 816800 Lotus Notes: Steve Groeger/UK/IBM Internet: groe...@uk.ibm.com From: Andy Seaborne <a...@apache.org> To: users@jena.apache.org, Date: 27/02/2014 13:36 Subject: Re: BlockException: No such block Hi Steve, Thanks for the update. On 25/02/14 15:15, Steve Groeger wrote: > Hi Andy, > > I picked up a snapshot of 2.11.2 and tried running our test using these > libraries. > > Adding the changes you suggested seems to have stopped the exception " > BlockException: No such block", the but we are now seeing other symptoms, > which we saw previously but not as frequently as we are now. > The change in behaviour seems to be result of the SystemTDB.NullOut = true > ; as I also tried various version of the listStatements and this gave the > same results, What does the SystemTDB.NullOut=true do? NullOut makes the B+Tree code overwrite areas of blocks that are now unused because of a delete with some value (FillByte = 0xFF). In normal operation, it just leaves it as junk, but of NullOut is set it goes and sets the bytes to a specific value, and one which does not make sense. I would not expect it to change the frequency of errors, but it would change the symptom. > What we are now seeing is data not being returned correctly. > We initially populate the database with a Resource with 1000 Properties, > then in the ReaderThread we query the database, using the following: > > StmtIterator found = model.listStatements((Resource)null, > (Property)null, (RDFNode)null); > > we are expecting to get back 1000 statements when we perform the > > count = found.toList().size(); > > however we occasionally get a batch of results that show 0 or non 1000 > statements being returned, then after several more reads this goes back to > getting 1000 statements and everything continues as normal. I'm not clear whether that is with or without NullOut set true. > > We need to determine why we occasionally dont get the 1000 statements > returned that we are expecting. Any advice you caould provide would be > great. It looks as if under some unusual conditions, either the B+Tree branch node or leaf node is not being updated correctly for deletions. ("unusual" as this code is not new and for anyone doing deletes, heavily used). That gives us something to go on. Could you please add some code that not only counts the StmtIterator but also touches every part of the statement. Execute with SystemTDB.NullOut=true There is some delayed evaluation under the covers and this forces that evaluation. StmtIterator found = .... long count = 0 ; while(found.hasNext()) { count++ ; Statement s = found.next(); try { Node subj = s.getSubject().asNode() ; Node pred = s.getPredicate().asNode() ; Node obj = s.getObject().asNode() ; } catch (Exception ex) { ex.printStackTrace(System.err) ; System.exit(99) ; } } Warning - this may slow the execution down, or not, depending on cache efficiencies. Andy > > Thanks > Steve Groeger > IBM WebSphere Service Registry Development > MP 211, Hursley > Tel: (44) 1962 816911 Mobex: 279990 Mobile: 07718 517 129 > Fax (44) 1962 816800 > Lotus Notes: Steve Groeger/UK/IBM > Internet: groe...@uk.ibm.com Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU