On Friday 02 Sep 2011 18:20:02 Ian Clarke wrote: > On Wed, Aug 31, 2011 at 8:00 AM, Matthew Toseland <t...@amphibian.dyndns.org > > wrote: > > > WE NEED MORE DATA. > > Well, my gut tells me that our existing scheme is likely too complicated to > fix unless we are extremely fortuitous, however I'm happy to be wrong about > that if others think that they have a good understanding of why we're having > problems and how to fix them. > > So this is fine with me provided that its not just data for data's sake, but > *actionable* data. By this I mean that the data we collect should give us > clear information about what is and isn't working, and hopefully tell us how > to fix it.
As a matter of principle, empiricism requires experimental data. But it's true that sometimes the metrics which are easiest to gather, and most closely related to our expected usage (i.e. good benchmarks), are hardest to theorise about. > > But a bunch of metrics we have no idea how to interpret won't get us > anywhere. Well, the following appear to be justifiable: - We can detect whether the theory about network topology being messed up by local requests is correct. This is directly actionable: If a large proportion of nodes have this problem, we could try the "no path folding above htl 16" rule. Of course, this might slow down local requests, and might even make it harder to bootstrap new nodes ... - Throughput data ... I guess some of our existing tests probably give us this, e.g. the daily timings for insert/retrieve, which lately succeed but take rather a long time. I guess we probably have enough here, although arguably we should have a big-file inter-day test (current tests are small files for multiple days or big files for one day). - Build to build comparisons: If we are going to deploy changes to the network we need to be able to compare one build to another. Otherwise, no matter how good our simulations (and odds are they won't be very good, getting the right level of accuracy is hard), we won't know whether it works in practice.
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
