Hi !
What a week ! Ok I was'nt able to work as much as I would have liked on Mavibot, but still, there is some progress. Addition of values have been fixed. I was able to conduct some performance tests, with various number of elements being added within a transaction (from 1 to 100 000 per trasactions). I have already mentinned that gathering updates within a transaction speeds up the process of adding elements. Here are the numbers : Delta add : time it takes to inject 100 000 elements in a B-tree Delta browse : time it takes to browse the 100 000 elements 1000 times (so 100 000 000 loops) File size : The resulting file size when the 100 000 elements has been added Increment : The number of addition withing one single transaction (ie, if it's 5, that means we will add 5 elements before committing the transaction) 100 000 elements, increment 1. Delta add : 36.590 (2 732/s) Delta browse : 12.489 File size : 1 034 376 704 bytes 100 000 elements, increment 2. Delta add : 18.081 (5 530/s) Delta browse : 11.207 File size : 508 566 016 bytes 100 000 elements, increment 4. Delta : 9.066 (11 030/s) Delta browse : 11.446 File size : 252 087 808 bytes 100 000 elements, increment 8. Delta add : 4.798 (20 842/s) Delta browse : 11.379 File size : 123 848 192 bytes 100 000 elements, increment 10. Delta add : 3.879 (25 779/s) Delta browse : 12.585 File size : 99 265 024 bytes 100 000 elements, increment 20. Delta add : 2.237 (44 702/s) Delta browse : 11.347 File size : 51 596 800 bytes 100 000 elements, increment 50. Delta add : 1.199 (83 402/s) Delta browse : 10.961 File size : 24 276 992 bytes 100 000 elements, increment 100. Delta add : 0.899 (111 234/s) Delta browse : 11.303 File size : 15 324 160 bytes 100 000 elements, increment 200. Delta add : 0.795 (125 786/s) Delta browse : 10.773 File size : 11 107 840 bytes 100 000 elements, increment 500. Delta add : 0.730 (136 986/s) File size : 8 688 640 bytes 100 000 elements, increment 1 000. Delta add : 0.795 (126 000/s) Delta browse : 8 million/s File size : 7 903 744 bytes 100 000 elements, increment 2 000. Delta add : 1.221 (/s) File size : 7 536 640 bytes 100 000 elements, increment 5 000. Delta add : 2.444 (/s) File size : 7 326 208 bytes As we can see, we max at around 500 update per transaction, and then it starts to slow down. I haven't spent time on studying what's going on, but I suspect the GC is teh culprit. Anyway, addition is just working fine now. I'm now working on the deletion, which needs some love : currently, we process pages merge from the bottom, which requires some extra informations about parents. It's way better to let the parent deal with merge, and that is what I'm refactoring atm. It will take a bit fo time ;-) As soon as add/delete operations are completed, the next step will be to manage concurrent reads, ie concurrent versions. We need to keep a track of all the used versions, in order to be able to harvest the dead versions later on. I'll keep you posted, of course ! PS : I have committed the code in a branch (directory/mavibot/branches/single-value/mavibot) -- Emmanuel Lecharny Symas.com directory.apache.org
