I have tagged and released version 8. However all changes relate to the new index format, and almost all to on-the-fly merging. This will be vital in the next stage - making XMLSpider generate new format indexes, progressively, so that 1) XMLSpider no longer takes a week to write an index from scratch, and 2) it can actually insert the index.
Going backwards in time... 0e5af601a81b3daca6a79953f92372a5d7c38cf6 - this commit removes a bunch of distributed indexing design docs - are there up to date docs re distributed indexing? are these all going to have to be severely rewritten anyway? eb90053d6ae081d9a6b9d6bcd67ba58a808edbc3 - random unit tests are good, but unless you have a fixed seed, it is essential that you print it out, ideally on failure but if necessary on starting the tests. c3d26271bea79189e836c56fd9df8d5bb584dc0e - BTreeMap.subSet(). - last line: how can k2 be null? surely if it is null then there is nothing left, we can return empty? 9724ad257b76473a6fa94d7dcd5f1b65cd963f14 BTreeMap.subEntries() - if there is only one key in the tail set, we return entries.subMap(lk, lk), which is empty at best or a contradiction at worst; changing to subMap(lk, rk) doesn't seem to fix it either, afaics this line should just be removed. cc8a478986b6039b8c6d68cbd33108db175a700c (t = timeout) == timeout eh?? bad idea to Thread.sleep while holding a lock. Use wait/notify not sleep/interrupt, it is more reliable. f34dd12004afceb7fb332af642d6b98ea81500bd Integers.allocateEvenlyRandom (todo only atm) - shouldn't it be deterministic, to maximise overlap when inserting the same data etc? or would it only be used in unit tests? (in which case must pass a random in) 71bd3b2f74fef2f74c0cf6684b9d96de6fb09304 ObjectProcessor.update() - what if it's already run? Looks racy - or you expect this to happen before we start _any_ jobs? 0d96f8101002ce4b127ff3028858b4738e95bf54 - use NativeThread? would need a priority passing in ... 3a28caa359a30d5e8361c94118002fb5135936fc - Sorted.keySet(). - instanceof SortedMap? SortedSet surely. - i guess this doesn't exist any more? 6aac0ad832dcc829ef0e3e7769818d5d6bf3be1d - new GhostNode(null, null, null, en.getValue()) - when do the keys get set? 1a65974565ccfe809230a8b3abd954c23477e66d - merge and split seem to be swapped around? looking forward to promised javadocs! ParallelSerialiser: - // TODO: toad - if it fails, we won't necessarily know until all of the + // TODO NORM: toad - if it fails, we won't necessarily know until all of the dunno ... fast failure is definitely best for UI ... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100501/12d7d7af/attachment.pgp>
