Ian: thanks for your message! Ian Clarke wrote: > > Thanks for your comments - they are extremely interesting and suggest > that there is increasing convergence between the two projects.
Yep. Since Mnet has (temporarily) lost economic mechanisms, since Freenet is adding erasure coding, and since both projects are currently developing new routing schemes, the two designs have grown closer. However I think (and hope) that this similarity will be temporary -- the new routing schemes will probably be substantially different, and each project will continue to add new features that the other project doesn't. > I look > forward to continuing to work together to our mutual benefit, however > before continuing, I do want to say that as the former Nigerian Minister > for Penis Enlargement I think it is quite a shame that this email won't > make it to the Mnet mailing list. Heh heh heh. Fortunately, I found your letter in my "special offers" folder while I was searching for a way to buy a green card application. > Anyway, one high-level point that I think is worth making at the outset > is that it is important to draw a distinction between NGrouting, which > is more of a general principal, and the specifics of our current > implementation. Ah! That *is* interesting. The abstraction of NGrouting requires only that the decision of which node to send to is based on past history, including both performance and keyspace. I think you also want a sort of "convergence" property -- you want it to be the case that even if node A and node B use *different* NGrouting implementations to evaluate node C, that they tend to come to similar conclusions about which part of the keyspace node C specializes in. This would seem to be a natural result of almost any reasonable NGrouting scheme, wouldn't it? Hm. The longer I think about it, the more ways I come up with for two different NGrouting schemes to develop divergent views of which peers handle which keyspace. That would be bad. Perhaps the convergence property should be written down in the NGrouting doc -- it is very important for the behavior of a large-scale network. > Well, I am not exactly sure what you mean by this - but perhaps your > question will be answered by the fact that we plan to introduce some > random salt in the decision as to where a request should be routed, and > also the fact that the more DNFs a node returns - the worse future > estimates for its routing time will be. M-hm. Interesting. Thanks for the answer to my question. > > So Mnet combines all kinds of failure (stored in hit rate) and > > performance (stored in average turnaround time) into a single scalar: > > (1/latency) * hitrate. > > Very interesting indeed, and quite a different way to look at it. If > you were to describe what this number is an estimate of - how would you > characterize it? Something like "the average amount of data retrieved > assuming continuous requests for data"? I realize that this isn't it - > but something similar? No I think that's it -- average amount of data retrieved per time, assuming continuous requests for data. I think it is really just the inverse of your "expected time to get the data by hook or by crook". Using the inverse notion of rate instead of latency makes it easy to factor in failures by simply multiplying in the failure probability. There is definitely some information lost by this though. Suppose peer A and peer B have identical chance of returning data versus returning DNF, and it takes them an identical amount of time to return the data if they return it, but peer A returns DNFs twice as fast as peer B. Well, peer A is definitely better, and the more frequent DNFs are the better peer A becomes, but the Mnet measure gives peer A and peer B identical scores. Does the Freenet measure distinguish between peer A and peer B? I'm not sure. > Yes, comparisons of Freenet, particularly post-NGrouting, with DHT > routing algorithms are *very* interesting to me - since they represent > different approaches, which rely on a similar vague principal (get > closer to the data at each hop), but where Freenet's approach is more > "organic" or "heuristic", whereas DHTs are quite rigorous. This gives > DHTs the benefit of being able to make performance guarantees, which are > essential in many situations, but my suspicion is that a more organic > approach might be better-suited to tolerate the unreliability and > unpredictability of the Internet (particularly when relying on peers > that can vanish at any moment). Yep. I wish there were some really good simulations of NGrouting -- it seems hard to answer these questions analytically. > Well, I am not so sure - we can observe the accuracy of the estimates > produced by this NGrouting implementation for a single node, and based > on that accuracy we should be able to make inferences about the > network's performance as a whole. Hm. Better performance from the perspective of a single node *does* suggest better performance of the network as a whole, but only assuming fairly convergent routing. That is: if people inserting data into the network are choosing different nodes to store the data than the people requesting the data from the network choose to query, then the network as a whole will suffer even though from any individual node's perspective his own strategy is locally optimal. A similar argument goes for two different queriers using different routing, because if their routes are divergent they lose caching, and maybe they even get DNFs. > In the case where a node received most of its requests within 0.004 of > its keyspace - then isn't it desirable that most of the points be within > that segment to more accurately represent the variations in request > times within that segment of keyspace for which the rest of the network > obviously considers this node to be an expert? Yes, but wouldn't it take an infinitely long time for the points to get dragged into a small space? I guess I misunderstood how the running average is computed. > Perhaps we are working to different definitions of "scale-free". Hm. Well, the constant number "10" interacts with the size of the network, at least at *first*, if not indefinitely. If you start up a fresh new network of 5 Freenet nodes, then the computation will tend to influence around four of the points at first, whereas if you start up a fresh new network of 50,000 Freenet nodes, then the computation will tend to influence only two of the nodes at first. It isn't clear to me how long this difference would persist as the Freenet network matures. Regards, Zooko http://zooko.com/ _______________________________________________ devl mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
