On Wednesday 30 April 2008 13:19, Michael Rogers wrote: > On Apr 30 2008, Matthew Toseland wrote: > > Keys to block number. Block numbers to keys is handled by the on disk > > structure. So we can actually pick a random block number to dump - but at > > the cost of having to keep a key index. > > Cool, I see what you mean now - I'll simulate that too. > > > I'm surprised that hashing works so well, it has some big disadvantages > > e.g. once the datastore is say half full, half of all new incoming keys > > will overwrite old data rather than being added to the end. So we end up > > storing less data: it takes a much longer time for the datastore to fill > > up. > > Hmm, good point. On the other hand filling the store (or 99% filling it) > would typically only take a few days, so maybe it's more important to > optimise the steady state behaviour than the startup behaviour?
Depends on how big it is. > > > What is the approximate ratio of store filling rates for the same size > > store on LRU versus on a direct hashing implementation? Can you simulate > > this? > > So far I've been allowing the simulations to reach a steady state before > making any measurements, but it shouldn't be a problem to simulate it. Ok. > > > IMHO most of it will be filesharing, just as a massive chunk of the total > > internet bandwidth is filesharing. > > OK, I'll simulate filesharing two popularity distributions, uniform and > Zipf. Each file will contain a lognormally distributed number of blocks, > and the downloader will randomly choose 2/3 of them to request. I won't > bother with splitfile healing, inserts, churn, congestion, swapping, phase > of the moon, etc. > > > SSK polling for messages obviously > > will also be huge, right now we have 2.5 SSKs for every CHK (but SSKs are > > ~ 10x than CHKs). That should reduce a bit in future with some new > > measures such as RecentlyFailed ... but it will increase as FMS is more > > widely adopted... So no idea really... I do know that if we spend all our > > bandwidth on SSK polling, filesharing will not work well. :| Also, SSKs > > are kept in a separate store from CHKs, this is not likely to change. > > I'll stick to simulating CHKs for the moment - RecentlyFailed and ULPRs > will affect the way SSKs are cached, but I don't have time to dig into the > code to find out how they work (and into Frost and FMS to find out what > kind of traffic patterns they produce). Sensible imho. > > Cheers, > Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20080430/62c149a0/attachment.pgp>
