At least... I have fixed the multi-threaded in-memory test !
Having had a calm day ad some time to focus on the problem, something I as expecting for more than a week, the solution is quite clear. I haven't yet committed anything. In a few words, there are a lot of modification needed to get it working : - the BtreeHeader will now contain way less informations (most of the common parameters are stored in the associated Btree) - we now have a map of existing revisions whch is cleaned when we don't need them anymore. This Map is a ConcurentHashMap, as it's used by the reading threads to get the current revision and by the writing thread to add a new revision. - there is one element which is used by the read or write threads, this is the currentRevision (it's an AtomicLong). It won't be possible to corrupt it. I hope to be able to clean the code tomorrow, has I will have a few more calm hours. Le 1/22/14 11:53 PM, Emmanuel Lécharny a écrit : > I'm dumb... And this is plain normal, I currently don't have time to > focus on this part of the code, so I'm just trying to fix it the wrong > way (ie, doig this and that quite randomly). > > We do have a ReadTransaction class which *must* be use for any read of a > b-tree, and i'm not using it. The very same for the update methods. > > > > Le 1/21/14 1:44 AM, Emmanuel Lécharny a écrit : >> I'm still fighting with the multi-threaded in-memory B-tree. >> >> The way we handle the rootpage is not correct : we use it in search >> operations while it can be modified in insert/delete operations. I tried >> to protect it by using two different variable (a currentRootPage and a >> newRootPage, which get swaped whe the update operation is committed) but >> it's not enough, I still have some bad insertions quite frenquently. >> This is typical to a concurrent pb. >> >> I also realized that the variable to protect is not the rootPage, but >> the btreeHeader, which should give access to the current rootPage. In >> fact, anything that can change during an update should be copied, and >> insolated from any search operation. >> >> Also the in-memory B-tree does not support multi-version atm, and this >> is something we must fix. >> >> A lot to do in this area... >> >> >> Le 1/19/14 2:28 PM, Kiran Ayyagari a écrit : >>> thanks Emmanuel for the heads up, I will give it a try tonight >>> >>> >>> On Fri, Jan 17, 2014 at 11:19 PM, Emmanuel Lécharny >>> <[email protected]>wrote: >>> >>>> Next step : completed. I'm now able to get data from many revisions of a >>>> B-tree. >>>> >>>> The insert and delete operations are now fully functional. >>>> >>>> Most of the tests are now passing, I still have 2 errors and one failure : >>>> testManagedBTreeBuilding is in error >>>> testBTreesDuplicateKeys is in error >>>> and >>>> testInsertMultiThreads is in failure >>>> >>>> This last failure is certainly due to a concurrent access I have to >>>> investigate. >>>> >>>> Anyway, as soon as the test will pass, I will commit my changes. >>>> >>>> Note that the grabage collection of old pages is still to be done, but >>>> we now have saved the pages to be freed in the file. >>>> >>>> -- >>>> Regards, >>>> Cordialement, >>>> Emmanuel Lécharny >>>> www.iktek.com >>>> >>>> > -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
