> -----Original Message----- > From: jrandom [mailto:jrandom at cryptomail.org] > Sent: Thursday, March 13, 2003 6:20 PM > To: devl at freenetproject.org > Subject: [freenet-dev] re: Data propagation, node > assimilation, freenet > capacity (thoughts) > > > Hmm, interesting ideas Costas > > This would help in situations where certain freenet pages are > repeatedly searched for (such as in the case of big index pages, > flogs, and the like), as even though this wouldn't get rid of DNFs, > it could come back and reduce the likelyhood of later DNFs. > > It would also have positive - not negative - implications for > anonymity, as intermediary nodes (if in "assimilation mode") would > be firing off these requests too, not just the real request > initiator. This, coupled with the time delay, would make requests > a little more "mix"-ish.
True. > While implementation within Fred wouldn't be too hard, implementing > OUTSIDE Fred would be preferable (e.g. standalone java app that > queries the local node's status & failed key table, firing off > requests to that node to tell it to retry those failed queries when > the load & bandwidth allows). > > That standalone app could even be given an additional set of DBRs > to "prefetch" every day (like TFE, TFEE, TII, YoYo, etc), as well > as some edition based sites (updating the edition #s to retrieve > the latest). It could even retrieve this set of keys to search > for FROM a key (or set of keys) so that you for instance could > make a DBR with a list of pages you want to be heavily propogated, > and anyone using this app could 'subscribe' to your list. > > By default, that app could even come with a preset list of keys to > search for (either of the developer's choosing or a full dump from > one of the big index pages). > > Some of the data points that might be useful to such an app may not > currently be exposed via FCP or FProxy clearly at the moment, but > adding them should be pretty easy (a lot easier than implementing all > of the "assimilation mode" logic into Fred, imho). No. That's not assimilation mode, that's something user-controled. Assimilation mode is for the core freenet engine (is that Fred?). FCP has also nothing to do with it (if I understand things correctly, that's just the node-to-node communication language). I'm talking about an internal and automatic mechanism. What you want is a stand-alone app, or something that is deemed a crucial enough service to be included into FProxy. See further down. > So, when do you start coding? :) Trying to be funny, eh? :) Although I am a programmer (employed as one anyway) i don't consider myself very good, especially in Java. After a year of getting a feel for Java, I'm only now starting to do stuff with it. > -jrandom Brainstorm alert!!! Reading Panamerica's post and thinking of pulling data from deep into the network... Let's assume I'm an intermediary node. A request for data x originates somewhere on freenet. I'm passed the request for x with an HTL of y which is supposedly lower than that used by the originating node. What if I don't find the data (my request forward DNFs) but I wanted to help by being able to find it or even have it later? Should I schedule a request with a higher HTL? The answer is probably no. First, I don't really know if the originating node got a DNF; maybe it found the data via another route. Second, if every node thought like this we'd overload the network via feedback in a matter of hours. In the end, how do I really know I'm not the originating node? Does FProxy behave as a node, or is it accessing Fred directly? "Need more input..." :) Please share the knowledge... On another note... Key types may be an obstacle. I'm not sure yet how keys are handled on freenet. I'm under the impression that all types of non-CHK keys get mangled into a CHK by Fred(?) or FProxy (which is worse). FProxy gets to decode non-CHK data (meta-data) it requested as whatever needs to be fetched next until it gets a real CHK to request. After that, FProxy responds to the browser with a header having a MIME-type description from the metadata and the actual data that came at the end. That would mean that Fred doesn't really know anything more than how to mangle non-CHKs into CHKs (maybe not even that), thus being unable to distinguish data from meta-data and thus being unable to cache a request to actual data (i.e. not meta-data) which is the hot potato in our case. It's the data we want, meta-data is just directions on what kind of data the target data is and how to get it. Right? Doc _______________________________________________ devl mailing list devl at freenetproject.org http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
