I am way behind on the Freenet mailing list, but I figured I should
reply to these 3-April messages before it gets any later.
> Date: Mon, 03 Apr 2000 01:38:08 +0100
> From: Ian Clarke <I.Clarke at dynamicblue.com>
> Subject: Re: [Freenet-dev] Wanting to change fundamental functionality
> (wasBugs)
>
> > However, I've come up with a new idea that might work better.
>
> I have been thinking about a test-harness which will allow us to test
> out theories by simulating several thousand nodes efficiently on one
> machine. This would be kind-of like the original simulation I wrote,
> but would use as much of the current code as possible (so that
> enhancements to the simulation will also be enhancements to the actual
> server code). Obviously this is for the future.
and
> Date: Sun, 2 Apr 2000 19:49:09 -0500 (CDT)
> From: Brandon <blanu at uts.cc.utexas.edu>
> Subject: Re: [Freenet-dev] Wanting to change fundamental functionality
> (wasBugs)
>
> ...
> I think the most difficult part of this will be to simulate real usage
> patterns. Getting a realistic topology and a realistic patterns of
> inserts and requests coming from different places for different keys
> at different times will be hard to to do, mainly because we don't know
> what the real usage patterns will be.
I have also been thinking (but only very slowly coding) about a Freenet
simulator. I was planning to implement the protocol from Lee Crocker's
spec[1], but I've made zero progress on that. Most of my thinking has
actually been going into the reference pattern question. Here is the
strategy I'm currently thinking about.
The simulator is populated by documents and peers. Each is identified
by a N-bit random number (N is 128 or 160 bits). The id is used to
derive various properties of the object. These properties include a
priority and a category vector. The category vector has a length of
1..16 where each element specifies a category id (8 bits) and a
magnitude (1..c bits). The probability that a peer will request a
document depends on its priority, which maps to its activity level. The
document it will fetch depends on its interests which is given by the
category vector. So each peer can be interested in up to about 10
categories, and the relative strength of interest in each is given by
the category magnitude. The probability that a document will be
selected depends on its priority, which maps to its popularity. The
topics that the document covers is given by category vector as with a
peer. To decide if a peer is interested in a document we compute the
dot product of the two category vectors and use that as a probability.
So the likelihood that a particular peer (P) will fetch a particular
document (D) is:
P.pri * (P.cat [dot] D.cat) * D.pri
where all the priorities are normalized to [0..1) and each category is
normalized to be a unit vector.
The remaining trick is to map bits from the document or peer id into
these various numbers. There are numerous approaches, but my plan is to
assign a fixed size to the priority (8..10 bits) and look it up in a doc
or peer table to get the actual popularity or activity value. Using a
table allows an arbitrary mapping between the uniformly distributed
input (random bits from the id) and the desired value. For instance,
documents could follow the 90% of everything is crap maxim. For the
category vector, 4 bits give the number of categories (again through a
mapping table), followed by 8 bits for each category id, followed by the
remaining bits divided evenly between each catid giving the magnitude of
each. This provides more dynamic range for an object that concentrates
on fewer categories.
I still have lingering doubts about running a CPU and memory intensive
simulation in Java. But I suppose porting the code to C, once it is all
designed and tested wouldn't be that big a job.
Ted
[1] http://www.piclab.com/freenet/spec.html
_______________________________________________
Freenet-dev mailing list
Freenet-dev at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/freenet-dev
