Re: [freenet-dev] Specialization in NGR
Toad wrote: Implemented in unstable 6348. Cool. I just tried this with a fresh routing table and DS. Looking at the small graphs in nodestatus.html, I noticed that most nodes were initialized with a peak (ie. one key that gets especially bad estimates, if I am reading the graph correctly). One would assume that this should be a trough rather than a peak given the changes in 6348 - am I wrong? Ian. ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] weird crash with 6348
After several hours of uptime but no interaction with the user (i.e. node was left overnight) when I decided to check out the web interface in the morning fred suddenly hang. No cpu whatsoever was used, and when I tried to get a stack dump I got: Fatal: Stack size too small. Use 'java -Xss' to increase default stack size. kernel is 2.4 nptl enabled (fc1) and LD_ASSUME_KERNEL=2.2.5 pgp0.pgp Description: PGP signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Re: High 'Data waiting to be transfered'
On Fri, 2003-11-21 at 04:23, Ian Clarke wrote: Edward J. Huff wrote: It seems to me that NGR can't possibly do certain things. When the standard deviation exceeds the mean, you can't even predict the sign of your random variable. The standard deviation exceeding the mean, if it does occur, isn't the fatal problem you think it is. NGR's data is likely to be extremely noisy, but provided that it is able to extract some generalization it should work, irrespective of the std dev. The question at issue is whether NGR can be used to distinguish between nodes based on bandwidth, while the bandwidth available to a transfer is allowed to vary randomly depending on requests from other nodes. The issue is complicated: any node can have its marginal available bandwidth fall near zero, and there is a positive feedback loop through the estimators leading to oscillation and inefficient use of resources. I think this is a valid and important question. Your response does not address it. -- Ed Huff ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] 6342 Status
On November 21, 2003 06:20 pm, Mike Stump wrote: Uncle. It is curious that 92 connections transmitting and a 31 MB backlog (from ocm) can only get 49.2% of the upstream (from Current upstream bandwidth usage). Either, this number doesn't contain all the upstream used, or something is really broken. It is consuming all the CPU (98%) to send out not much data, less than 4KB a second for a 400 MHz CPU. Or much of that 32M is for only a few boxes that cannot accept data very fast but have asked for lots. It felt better using 50% of the CPU to send out 80% (100%) of the upstream when we did upstream usage based QRs. Yes, with this I agree. Ed My config: maxNodeConnections=200 successfulDelayCutoff=1000 successfulSendTimeCutoff=1000 requestSendTimeCutoff=100 requestDelayCutoff=100 maximumThreads=80 outputBandwidthLimit=8000 Could we have our node communicate the backlog/current upstream used figure to other nodes in some way so that we ensure the entire network is performing as badly as my node? That's 2.8 hours of backlog. Also, could someone try using gcj to try and compile freenet and let us know if that works and how to do it. I've tried, and it dies with the bytecode, I've not tried the actual source. If that works, maybe we can get a bit more performance out of it that way. Also, this might help with some of the jvm problems by providing a known good baseline that is portable to oodles of machines. ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Re: High 'Data waiting to be transfered'
On Sat, Nov 22, 2003 at 09:35:45AM -0500, Ed Tomlinson wrote: On November 22, 2003 09:12 am, Edward J. Huff wrote: On Fri, 2003-11-21 at 04:23, Ian Clarke wrote: Edward J. Huff wrote: It seems to me that NGR can't possibly do certain things. When the standard deviation exceeds the mean, you can't even predict the sign of your random variable. The standard deviation exceeding the mean, if it does occur, isn't the fatal problem you think it is. NGR's data is likely to be extremely noisy, but provided that it is able to extract some generalization it should work, irrespective of the std dev. The question at issue is whether NGR can be used to distinguish between nodes based on bandwidth, while the bandwidth available to a transfer is allowed to vary randomly depending on requests from other nodes. The issue is complicated: any node can have its marginal available bandwidth fall near zero, and there is a positive feedback loop through the estimators leading to oscillation and inefficient use of resources. This is a very very important point. The first NGR implementation did not use transfer rates in its estimators - maybe, exploitable as it was, it made more sense. It really depends on if transfer rate is predictable for a given node. Given that outbound bandwidth is more often than not the limiting constraint on a node I suspect that its not all that predictable... I dunno, it may well be transferring the same approximate number of trailers over a long period... Ed I think this is a valid and important question. Your response does not address it. -- Ed Huff -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] 6348 - sharp specialization
On Sat, Nov 22, 2003 at 01:52:41AM -0500, Zlatin Balevsky wrote: After starting a brand new node with 6348 the outbound requests become sharply specialized in a matter of minutes. The incoming requests are How do you determine this? of course in disaray, but hopefully when more nodes start running this code we'll finally see the coveted specialization *coughmergecough* -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Re: Specialization in NGR
On Fri, 21 Nov 2003 22:47:21 +, Toad wrote: On Fri, Nov 21, 2003 at 03:20:58PM -0500, Ed Tomlinson wrote: On November 21, 2003 01:45 pm, Toad wrote: On Fri, Nov 21, 2003 at 12:01:27PM -0500, Ed Tomlinson wrote: On November 21, 2003 06:41 am, Ian Clarke wrote: It seems that we aren't seeing the hoped-for specialization in NGR. Rather than futzing about with all sorts of ideas while trying to test them in the chaotic real network, we need a simple simulator to help us test these ideas. One idea to test on that simulator is are we specializing by key or for fast connections. It seems to me that, by far, the largest chunks of the NG estimator are from transmission time... When I did the first stabs at NG I was only basing it on search times and, if memory serves, it was getting much better numbers. Maybe we should be normallizing to a size that does not give a number much greater than transmission time? Why? The search time is usually a number averaging around 15,000 ms The time to recieve a normalized file is over 150,000 ms. This means the 15000 (or 2000 or 3) gets lost in the noise and NGR just looks for the fastest node. Those are not the only factors involved. The transfer time is multiplied by the probability of a transfer occurring. I must ask... Doesn't this mean that the worse the node's chances of finding a given key, the better (lower) the estimate ? Shouldn't you multiply by the (1 - probability of transfer occurring) ? Unfortunately, I was unable to find the formula of estimate in the CVS source, so I couldn't check... Anyway, if this is true, it would explain the despecialization. Anyway, here's my datastore and inbound request stats. I think they have started getting a little less flat in the past hours, so maybe the pessimistic initial estimators are working... Running 6348. BTW. There seems to be something wrong with uploads (very very slow), but it's hard to say if it's a Frost or Freenet problem... Histogram of keys in in fred's data store These are the keys to the data in your node's local cache (DataStore) Nov 22, 2003 6:20:47 PM keys: 3278 scale factor: 0.28828829526901245 (This is used to keep lines 64 characters) 0 | 1 |== 2 |== 3 |= 4 | 5 |=== 6 |= 7 |== 8 |=== 9 |== a |== b |== c |= d | e |=== f |= peaks (count/mean) 0 -- (1.0835875) 3 -- (1.0347773) 7 -- (1.0640634) a -- (1.0640634) e -- (1.0152532) * Histogram of successful externally requested keys. This count has nothing to do with keys in your datastore Nov 22, 2003 6:20:55 PM keys: 302 scale factor: 1.0 (This is used to keep lines 64 characters) 0 |== 1 |=== 2 |=== 3 |=== 4 | 5 |== 6 |= 7 | 8 |== 9 |=== a | b | c |=== d |== e | f |== peaks (count/mean) 1 -- (1.218543) 4 -- (0.8476821) 6 -- (1.5364238) 9 -- (1.642384) d -- (1.3774835) * Histogram of requested keys. This count has nothing to do with keys in your datastore Nov 22, 2003 6:20:59 PM keys: 27183 scale factor: 0.03528114780783653 (This is used to keep lines 64 characters) 0 |=== 1 |== 2 |= 3 |= 4 |= 5 |== 6 | 7 | 8 | 9
Re: [freenet-dev] Re: Specialization in NGR
On Sat, Nov 22, 2003 at 06:22:57PM +0200, Jusa Saari wrote: On Fri, 21 Nov 2003 22:47:21 +, Toad wrote: On Fri, Nov 21, 2003 at 03:20:58PM -0500, Ed Tomlinson wrote: On November 21, 2003 01:45 pm, Toad wrote: On Fri, Nov 21, 2003 at 12:01:27PM -0500, Ed Tomlinson wrote: On November 21, 2003 06:41 am, Ian Clarke wrote: It seems that we aren't seeing the hoped-for specialization in NGR. Rather than futzing about with all sorts of ideas while trying to test them in the chaotic real network, we need a simple simulator to help us test these ideas. One idea to test on that simulator is are we specializing by key or for fast connections. It seems to me that, by far, the largest chunks of the NG estimator are from transmission time... When I did the first stabs at NG I was only basing it on search times and, if memory serves, it was getting much better numbers. Maybe we should be normallizing to a size that does not give a number much greater than transmission time? Why? The search time is usually a number averaging around 15,000 ms The time to recieve a normalized file is over 150,000 ms. This means the 15000 (or 2000 or 3) gets lost in the noise and NGR just looks for the fastest node. Those are not the only factors involved. The transfer time is multiplied by the probability of a transfer occurring. I must ask... Doesn't this mean that the worse the node's chances of finding a given key, the better (lower) the estimate ? Shouldn't you multiply by the (1 - probability of transfer occurring) ? No. The possibility of failure is also taken into account. Further, a node that is better at a given key subspace should also transfer data faster in that area. Unfortunately, I was unable to find the formula of estimate in the CVS source, so I couldn't check... Anyway, if this is true, it would explain the despecialization. StandardNodeEstimator.longEstimate(). -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] More bugs in NGRouting
While debugging the new changes to NGRouting (and finding several bugs in that code), I came across this: SEC 11 22, 2003 17:19:24:349 (freenet.node.rt.StandardNodeEstimator, QThread-74, MINOR): [EMAIL PROTECTED]: tcp/62.155.178.40:28146, sessions=1, presentations=1, ID=DSA(04a1 4b6d ca19 d8fd 5cd6 521a 9079 ec6b 1aa9 1192).estimate(): rFT=1218121.0, needConn=false, size=1049900, key=bb37b4535999ce668b9f2206ca7b0c2ed268b442140302, htl=15, pTransferFailed=0.8, tTransferFailed=0.0, pDNF=0.0, tDNF=2.7E7, pLegitDNF=0.0, tSuccess=1.6799845822E10,transferRate=6.25E-5, estimate=1.68008203188E10ms Okay, the obvious, and hopefully easy to fix thing, is tTransferFailed=0. The far more interesting item is rFT=1218121.0 , but tSuccess=1.6799845822E10 Translation: rFT is requestFailTime. It's the estimated time for a request for that key to succeed. It's a lot lower than tSuccess - the estimated time for a successful transfer. transferRate here is at the minimum value, to which it is pessimistically initialized; the estimator will rapidly recover when it gets some requests, this is quite intentional (see the thread on specialization). But tSuccess being much greater than rFT is really damaging on large files... The fix is to have separate global estimators for successful search time and transfer rate, and combine them (with the file size) to get requestFailTime. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] More bugs in NGRouting
On Sat, Nov 22, 2003 at 05:30:11PM +, Toad wrote: While debugging the new changes to NGRouting (and finding several bugs in that code), I came across this: SEC 11 22, 2003 17:19:24:349 (freenet.node.rt.StandardNodeEstimator, QThread-74, MINOR): [EMAIL PROTECTED]: tcp/62.155.178.40:28146, sessions=1, presentations=1, ID=DSA(04a1 4b6d ca19 d8fd 5cd6 521a 9079 ec6b 1aa9 1192).estimate(): rFT=1218121.0, needConn=false, size=1049900, key=bb37b4535999ce668b9f2206ca7b0c2ed268b442140302, htl=15, pTransferFailed=0.8, tTransferFailed=0.0, pDNF=0.0, tDNF=2.7E7, pLegitDNF=0.0, tSuccess=1.6799845822E10,transferRate=6.25E-5, estimate=1.68008203188E10ms Okay, the obvious, and hopefully easy to fix thing, is tTransferFailed=0. The far more interesting item is rFT=1218121.0 , but tSuccess=1.6799845822E10 Translation: rFT is requestFailTime. It's the estimated time for a request for that key to succeed. It's a lot lower than tSuccess - the estimated time for a successful transfer. transferRate here is at the minimum value, to which it is pessimistically initialized; the estimator will rapidly recover when it gets some requests, this is quite intentional (see the thread on specialization). But tSuccess being much greater than rFT is really damaging on large files... Maybe it's not a big issue in this particular instance - this node is not supposed to be routed to, that's why it has pessimistic estimators. So routing is doing it's job. However, we do need separate rate and searchtime global estimators, otherwise we end up with a major problem when routing requests for files above the standard size. Which I will implement ASAP. The fix is to have separate global estimators for successful search time and transfer rate, and combine them (with the file size) to get requestFailTime. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-CVS] freenet/src/freenet Version.java,1.544,1.545
Update of /cvsroot/freenet/freenet/src/freenet In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet Modified Files: Version.java Log Message: 6349: Major debugging on NGRouting recent changes. Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical HTLs and therefore not introduce bias. Transients should not accept requests; include a flag that they are transient in their references, and don't put transients into the routing table. Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT. Accept out of band references if less than 40 out of 50 nodes in RT and not backed off. Add NGRT.countUnbackedOffNodes() Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no key to start from. Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF because of that bug. Initialize pSearchFailed pessimistically to 1.0. Logging. Index: Version.java === RCS file: /cvsroot/freenet/freenet/src/freenet/Version.java,v retrieving revision 1.544 retrieving revision 1.545 diff -u -w -r1.544 -r1.545 --- Version.java22 Nov 2003 02:47:51 - 1.544 +++ Version.java22 Nov 2003 17:56:01 - 1.545 @@ -20,7 +20,7 @@ public static String protocolVersion = 1.47; /** The build number of the current revision */ -public static final int buildNumber = 6348; +public static final int buildNumber = 6349; public static final int ignoreBuildsAfter = 6500; ___ cvs mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node NodeReference.java, 1.32, 1.33 Node.java, 1.254, 1.255 NewNodeContactor.java, 1.1, 1.2 Main.java, 1.312, 1.313
Update of /cvsroot/freenet/freenet/src/freenet/node
In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet/node
Modified Files:
NodeReference.java Node.java NewNodeContactor.java Main.java
Log Message:
6349:
Major debugging on NGRouting recent changes.
Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical
HTLs and therefore not introduce bias.
Transients should not accept requests; include a flag that they are transient in their
references, and don't put transients into the routing table.
Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working
non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT.
Accept out of band references if less than 40 out of 50 nodes in RT and not backed off.
Add NGRT.countUnbackedOffNodes()
Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no
key to start from.
Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF
because of that bug.
Initialize pSearchFailed pessimistically to 1.0.
Logging.
Index: NodeReference.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/NodeReference.java,v
retrieving revision 1.32
retrieving revision 1.33
diff -u -w -r1.32 -r1.33
--- NodeReference.java 31 Oct 2003 19:21:19 - 1.32
+++ NodeReference.java 22 Nov 2003 17:56:00 - 1.33
@@ -54,6 +54,7 @@
private long ARKrevision; // ARK revision number
private byte[] ARKcrypt;// ARK data encryption key
private DSASignature signature;
+private boolean isTransient = false;
private String version;
// kill me quickly
@@ -136,6 +137,12 @@
public NodeReference(FieldSet ref, boolean verify, boolean slack, Identity ident)
throws BadReferenceException {
+ String transientFlag = ref.get(transient);
+ if(transientFlag != null) {
+ if(transientFlag.equals(true))
+ isTransient = true;
+ }
+
if (!ref.isSet(identity) ident != null) {
// It's sort of ugly, but we'll actually insert the identity
// into the fieldset.
@@ -355,11 +362,13 @@
public NodeReference ( Identity identity, Address[] addr,
long[] sessions, long[] presentations,
String version, long ARKrevision,
- byte[] ARKcrypt ) {
+ byte[] ARKcrypt, boolean isTransient ) {
this(identity, parsePhysical(addr), sessions, presentations, version);
hasARK = true;
this.ARKrevision = ARKrevision;
this.ARKcrypt = ARKcrypt;
+ this.isTransient = isTransient;
+
}
/**
@@ -462,6 +471,13 @@
}
/**
+ * @return true if the reference is transient.
+ */
+public boolean isTransient() {
+ return isTransient;
+}
+
+/**
* Checks that all addresses for transports that we support are
* correct.
*/
@@ -663,6 +679,9 @@
fs.put(sessions, Fields.numberList(sessions));
fs.put(presentations,Fields.numberList(presentations));
+
+if(isTransient)
+ fs.put(transient, true);
if (version != null) fs.put(version, version);
Index: Node.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/Node.java,v
retrieving revision 1.254
retrieving revision 1.255
diff -u -w -r1.254 -r1.255
--- Node.java 21 Nov 2003 21:12:43 - 1.254
+++ Node.java 22 Nov 2003 17:56:00 - 1.255
@@ -127,7 +127,8 @@
config.addOption(cacheProbPerHop, 1, 0.8F, 1412);
config.addOption(minStoreFullPCache, 1, 0.9F, 1413);
config.addOption(minRTFullPRef, 1, 0.3F, 1414);
-config.addOption(maxLog2DataSize, 1, 20, 1415);
+config.addOption(minRTNodesPRef, 1, 0.8F, 1415);
+config.addOption(maxLog2DataSize, 1, 20, 1416);
// network resource limiting options
config.addOption(bandwidthLimit, 1, 0, 1200);
@@ -507,7 +508,14 @@
// minRTFullPRef
config.setExpert (minRTFullPRef, true);
config.argDesc (minRTFullPRef, number between 0 and 1);
-config.longDesc (minRTFullPRef, Minimium proportion of the routing table
that must be filled before probabilistic referencing kicks in.);
+config.longDesc (minRTFullPRef, Minimium proportion of the routing table
(classic mode) that must be filled before probabilistic ,
+ referencing kicks in.);
+
+// minRTNodesPRef
+config.setExpert (minRTNodesPRef, true);
+config.argDesc (minRTNodesPRef, number between 0 and 1);
+config.longDesc(minRTNodesPRef, Minimum
[freenet-CVS] freenet/src/freenet/node/states/FNP NewRequest.java, 1.47, 1.48 NewDataRequest.java, 1.9, 1.10
Update of /cvsroot/freenet/freenet/src/freenet/node/states/FNP
In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet/node/states/FNP
Modified Files:
NewRequest.java NewDataRequest.java
Log Message:
6349:
Major debugging on NGRouting recent changes.
Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical
HTLs and therefore not introduce bias.
Transients should not accept requests; include a flag that they are transient in their
references, and don't put transients into the routing table.
Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working
non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT.
Accept out of band references if less than 40 out of 50 nodes in RT and not backed off.
Add NGRT.countUnbackedOffNodes()
Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no
key to start from.
Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF
because of that bug.
Initialize pSearchFailed pessimistically to 1.0.
Logging.
Index: NewRequest.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FNP/NewRequest.java,v
retrieving revision 1.47
retrieving revision 1.48
diff -u -w -r1.47 -r1.48
--- NewRequest.java 21 Nov 2003 00:26:20 - 1.47
+++ NewRequest.java 22 Nov 2003 17:56:01 - 1.48
@@ -69,7 +69,18 @@
throw new RequestAbortException(null);
}
+
try {
+ if(n.isTransient()) {
+ String reason = Transient node;
+ Core.logger.log(this,
+ Rejecting query from host (we are transient):
+reason,
+ Logger.NORMAL);
+ Message m = new QueryRejected(id, mo.hopsToLive,
+ reason, mo.otherFields);
+ n.sendMessageAsync(m, ref, PeerHandler.EXPENDABLE,
+ Core.hopTime(1), null);
+ }
// enforce version aloofness
String vers = mo.getSource().getVersion();
if (vers != null !Version.checkGoodVersion(vers)) {
Index: NewDataRequest.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FNP/NewDataRequest.java,v
retrieving revision 1.9
retrieving revision 1.10
diff -u -w -r1.9 -r1.10
--- NewDataRequest.java 21 Nov 2003 21:12:44 - 1.9
+++ NewDataRequest.java 22 Nov 2003 17:56:01 - 1.10
@@ -42,6 +42,8 @@
RequestInitiator ri = new RequestInitiator(id,
drmo.getReceivedTime());
Node.recentKeys.add(drmo.searchKey);
+ Core.diagnostics.occurrenceContinuous(startedRequestHTL,
+ drmo.hopsToLive);
Pending p = new DataPending(id, (int) drmo.hopsToLive,
drmo.searchKey, origRec, ft, ri, false);
Core.diagnostics.occurrenceCounting(incomingRequests, 1);
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/states/FCP NewClientGet.java, 1.13, 1.14
Update of /cvsroot/freenet/freenet/src/freenet/node/states/FCP
In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet/node/states/FCP
Modified Files:
NewClientGet.java
Log Message:
6349:
Major debugging on NGRouting recent changes.
Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical
HTLs and therefore not introduce bias.
Transients should not accept requests; include a flag that they are transient in their
references, and don't put transients into the routing table.
Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working
non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT.
Accept out of band references if less than 40 out of 50 nodes in RT and not backed off.
Add NGRT.countUnbackedOffNodes()
Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no
key to start from.
Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF
because of that bug.
Initialize pSearchFailed pessimistically to 1.0.
Logging.
Index: NewClientGet.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FCP/NewClientGet.java,v
retrieving revision 1.13
retrieving revision 1.14
diff -u -w -r1.13 -r1.14
--- NewClientGet.java 21 Nov 2003 21:12:44 - 1.13
+++ NewClientGet.java 22 Nov 2003 17:56:01 - 1.14
@@ -52,6 +52,9 @@
Key k = ckey.getKey();
if (k == null) throw new KeyException(got null Key);
Node.recentKeys.add(k);
+ Core.diagnostics.occurrenceContinuous(startedRequestHTL,
+ cgmo.getHTL());
+
if(k.log2size() Node.maxLog2DataSize) {
sendMessage(new URIError(id, Key too big));
return null;
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/client InternalClient.java, 1.36, 1.37
Update of /cvsroot/freenet/freenet/src/freenet/client In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet/client Modified Files: InternalClient.java Log Message: 6349: Major debugging on NGRouting recent changes. Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical HTLs and therefore not introduce bias. Transients should not accept requests; include a flag that they are transient in their references, and don't put transients into the routing table. Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT. Accept out of band references if less than 40 out of 50 nodes in RT and not backed off. Add NGRT.countUnbackedOffNodes() Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no key to start from. Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF because of that bug. Initialize pSearchFailed pessimistically to 1.0. Logging. Index: InternalClient.java === RCS file: /cvsroot/freenet/freenet/src/freenet/client/InternalClient.java,v retrieving revision 1.36 retrieving revision 1.37 diff -u -w -r1.36 -r1.37 --- InternalClient.java 21 Nov 2003 21:12:44 - 1.36 +++ InternalClient.java 22 Nov 2003 17:56:01 - 1.37 @@ -507,6 +507,9 @@ } } +Core.diagnostics.occurrenceContinuous(startedRequestHTL, + ft.req.htl); + Node.recentKeys.add(k); ClientEvent evt = ft.state(Request.PREPARED); ___ cvs mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/rt NGRoutingTable.java, 1.47, 1.48 FilterRoutingTable.java, 1.11, 1.12 StandardNodeStats.java, 1.1, 1.2 StandardNodeEstimator.java, 1.49, 1.50 RoutingTable.java,
Update of /cvsroot/freenet/freenet/src/freenet/node/rt
In directory sc8-pr-cvs1:/tmp/cvs-serv30192/src/freenet/node/rt
Modified Files:
NGRoutingTable.java FilterRoutingTable.java
StandardNodeStats.java StandardNodeEstimator.java
RoutingTable.java CPAlgoRoutingTable.java
Log Message:
6349:
Major debugging on NGRouting recent changes.
Add diagnostic startedRequestHTL, use it to make the maintenance requests use typical
HTLs and therefore not introduce bias.
Transients should not accept requests; include a flag that they are transient in their
references, and don't put transients into the routing table.
Reinstate probabilistic referencing in NGRouting - under strict conditions (40 working
non-backed off nodes out of 50). New param: minRTNodesPRef. Move shouldReference to RT.
Accept out of band references if less than 40 out of 50 nodes in RT and not backed off.
Add NGRT.countUnbackedOffNodes()
Fix pessimistic estimators initializing pDNF to min pDNF rather than max pDNF if no
key to start from.
Increase estimators Version to 3 to avoid pollution by estimators intted to 0.0 pDNF
because of that bug.
Initialize pSearchFailed pessimistically to 1.0.
Logging.
Index: NGRoutingTable.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/NGRoutingTable.java,v
retrieving revision 1.47
retrieving revision 1.48
diff -u -w -r1.47 -r1.48
--- NGRoutingTable.java 22 Nov 2003 02:47:51 - 1.47
+++ NGRoutingTable.java 22 Nov 2003 17:56:01 - 1.48
@@ -19,6 +19,7 @@
import freenet.Identity;
import freenet.Key;
import freenet.Version;
+import freenet.message.StoreData;
import freenet.node.Main;
import freenet.node.Node;
import freenet.node.NodeReference;
@@ -151,10 +152,13 @@
if(!Version.checkGoodVersion(ref.getVersion(), true)) {
Core.logger.log(this, Rejecting reference +ref+
- too old (+ref.getVersion()+
-) in loadEstimators,
-new Exception(debug), Logger.NORMAL);
+) in loadEstimators, Logger.NORMAL);
continue;
}
+if(ref.isTransient()) {
+ Core.logger.log(this, Rejecting reference +ref+
+- transient!, Logger.NORMAL);
+}
ne = factory.create(mem, ref, e, false);
} catch (IOException ex) {
Core.logger.log(this, Caught +ex+ deserializing a
NodeEstimator for +mem, ex, Logger.ERROR);
@@ -172,7 +176,8 @@
public boolean wantUnkeyedReference(NodeReference ref) {
Identity id = ref.getIdentity();
-if(estimators.size() = (maxNodes*4/5)) return false;
+if(ref.isTransient()) return false;
+if(countUnbackedOffNodes() = (maxNodes*4/5)) return false;
if(estimators.get(id) != null) return false;
if(ref.noPhysical()) return false;
if(!Version.checkGoodVersion(ref.getVersion(), true)) return false;
@@ -185,7 +190,8 @@
Core.logger.log(this, updateReference(+nr+), Logger.DEBUG);
if(mem != null) {
NodeEstimator e = (NodeEstimator)(estimators.get(i));
-if(nr.supersedes(mem.getNodeReference())) {
+if(nr.supersedes(mem.getNodeReference())
+ (!nr.isTransient())) {
((DataObjectRoutingMemory)mem).noderef = nr;
if(e == null)
Core.logger.log(this, Got +mem+ but not estimator!,
@@ -214,6 +220,7 @@
public synchronized void reference(Key k, NodeReference nr, FieldSet estimator) {
Identity i = nr.getIdentity();
if(i == null) return;
+if(nr.isTransient()) return;
if(i.equals(Main.id)) return;
// Not just in FilterRT: serialization and updateRef - FIXME
RoutingMemory mem = routingStore.getNode(i);
@@ -645,4 +652,51 @@
return hl;
}
+ /* (non-Javadoc)
+* @see
freenet.node.rt.RoutingTable#shouldReference(freenet.node.NodeReference,
freenet.message.StoreData)
+*/
+ public boolean shouldReference(NodeReference nr, StoreData sd) {
+ if(nr == null) {
+ Core.logger.log(this, shouldReference returning false because
+
+ null ref, Logger.DEBUG);
+ return false;
+ }
+ int x = countUnbackedOffNodes();
+ Core.logger.log(this, Integer.toString(x)+ elements in RoutingTable,
+ Logger.DEBUG);
+ if(x (maxNodes * (double)Node.minRTNodesPRef)) {
+ Core.logger.log(this, shouldReference because RT less than
required size,
+
[freenet-dev] Unstable 6349
Freenet unstable build 6349 is in CVS. The snapshots are being updated. Major changes: * Major bugfixes on recent work on NGRouting - pDNF was being initialized to 0.0 rather than 1.0 on estimators created without an initial specialization. Break compatibility with previous Estimator fields. - Send the maintenance requests out with HTL of the average HTL of started requests over the last hour, unless there are too few, in which case default to the maximum. Should reduce bias. * Try to manage RT flux - Accept out of band references if less than 40 nodes (in a typical 50 node RT) are both extant and not backed off. - Reinstate probabilistic referencing if more than 40 out of 50 nodes are extant and not backed off. Parameter is minRTNodesPRef. * Transient nodes should not accept requests, nor should they be added to the routing table. Occasionally nodes get added to the routing table other than by announcements and datasource reset, so this is possible. * Logging. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-CVS] freenet/src/freenet Version.java,1.545,1.546
Update of /cvsroot/freenet/freenet/src/freenet In directory sc8-pr-cvs1:/tmp/cvs-serv10591/src/freenet Modified Files: Version.java Log Message: 6350: Separate global estimators for search time and transfer rate. Therefore requestFailTime depends on the file size, as it should do. Fixes routing breakage w.r.t. large files. Index: Version.java === RCS file: /cvsroot/freenet/freenet/src/freenet/Version.java,v retrieving revision 1.545 retrieving revision 1.546 diff -u -w -r1.545 -r1.546 --- Version.java22 Nov 2003 17:56:01 - 1.545 +++ Version.java22 Nov 2003 19:03:56 - 1.546 @@ -20,7 +20,7 @@ public static String protocolVersion = 1.47; /** The build number of the current revision */ -public static final int buildNumber = 6349; +public static final int buildNumber = 6350; public static final int ignoreBuildsAfter = 6500; ___ cvs mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/rt NGRouting.java, 1.31, 1.32 NodeEstimatorFactory.java, 1.7, 1.8 NGRoutingTable.java, 1.48, 1.49 StandardNodeEstimator.java, 1.50, 1.51 StandardNodeEstimatorFac
Update of /cvsroot/freenet/freenet/src/freenet/node/rt
In directory sc8-pr-cvs1:/tmp/cvs-serv10591/src/freenet/node/rt
Modified Files:
NGRouting.java NodeEstimatorFactory.java NGRoutingTable.java
StandardNodeEstimator.java StandardNodeEstimatorFactory.java
ResponseTimeEstimator.java
Log Message:
6350:
Separate global estimators for search time and transfer rate.
Therefore requestFailTime depends on the file size, as it should do.
Fixes routing breakage w.r.t. large files.
Index: NGRouting.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/NGRouting.java,v
retrieving revision 1.31
retrieving revision 1.32
diff -u -w -r1.31 -r1.32
--- NGRouting.java 21 Nov 2003 21:12:43 - 1.31
+++ NGRouting.java 22 Nov 2003 19:03:55 - 1.32
@@ -193,7 +193,6 @@
else stdFileSize = 131072;
long normalizedTime = searchTime + (transferTime*stdFileSize/size);
if(lastEstimate.searchSuccessTime 0) {
- // searchSuccessTime is valid anyway
Core.diagnostics.occurrenceContinuous(normalizedSuccessTime,
normalizedTime);
Core.diagnostics.occurrenceContinuous(successSearchTime,
@@ -203,6 +202,10 @@
diffSearchTime);
Core.diagnostics.occurrenceContinuous(absDiffSearchSuccessTime,
Math.abs(diffSearchTime));
+
+ // we only want successes here - reporting everythings causes us to route to
+ // nodes that will just Query Reject. edt
+ ngrt.globalSearchTimeEstimator.reportTime(key, searchTime);
}
double rate;
if(size 16384 /* must be multi-segment */
@@ -215,6 +218,7 @@
Math.abs(diffRate));
}
Core.diagnostics.occurrenceContinuous(successTransferRate, rate);
+ngrt.globalTransferRateEstimator.reportTransferRate(k, rate/1000);
ngrt.reportRate(rate);
} else {
rate = -1.0;
@@ -240,11 +244,6 @@
count = maxSteps+1;
at = list.length;
long t = System.currentTimeMillis()-origStartTime;
- if(success routingRelated !isInsert) { // inserts include
AwaitingStoreData...
-// we only want successes here - reporting everythings causes us to route to
-// nodes that will just Query Reject. edt
-ngrt.globalEstimator.reportTime(key, t);
-}
if(routingRelated didNotQuicklyRNF !isInsert)
Core.diagnostics.occurrenceContinuous(searchFailedCount,
searchFailedCount);
if(isInsert (!noDiag)) {
Index: NodeEstimatorFactory.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/NodeEstimatorFactory.java,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -w -r1.7 -r1.8
--- NodeEstimatorFactory.java 22 Nov 2003 02:47:51 - 1.7
+++ NodeEstimatorFactory.java 22 Nov 2003 19:03:55 - 1.8
@@ -32,6 +32,7 @@
FieldSet estimator, Key k, boolean
needConnection,
NodeStats stats);
TimeEstimator createGlobalTimeEstimator();
+ TimeEstimator createGlobalRateEstimator();
/** Set the NGRoutingTable we are attached to */
void setNGRT(NGRoutingTable ngrt);
/**
Index: NGRoutingTable.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/NGRoutingTable.java,v
retrieving revision 1.48
retrieving revision 1.49
diff -u -w -r1.48 -r1.49
--- NGRoutingTable.java 22 Nov 2003 17:56:01 - 1.48
+++ NGRoutingTable.java 22 Nov 2003 19:03:55 - 1.49
@@ -45,7 +45,8 @@
final NodeEstimatorFactory factory;
final TimeEstimatorFactory timeFactory;
public final static String NGKEY = ngr1data;
-TimeEstimator globalEstimator;
+TimeEstimator globalSearchTimeEstimator;
+ TimeEstimator globalTransferRateEstimator;
public double fastestTransferSeen = 0;
public double defaultFastestTransferSeen;
@@ -118,16 +119,25 @@
public void loadEstimators() {
Core.logger.log(this, Loading estimators, Logger.NORMAL);
// First load the global estimator
-globalEstimator = null;
+globalSearchTimeEstimator = null;
+ globalTransferRateEstimator = null;
for(int i=0;iglobalFiles.length;i++) {
FileInputStream fi = null;
DataInputStream dis = null;
try {
fi = new FileInputStream(globalFiles[i]);
dis = new DataInputStream(fi);
-globalEstimator =
timeFactory.create(dis,ResponseTimeEstimator.convertTime(1000),ResponseTimeEstimator.convertTime(0));//0
ms to ~20 minutes
+globalSearchTimeEstimator =
+
[freenet-CVS] freenet/src/freenet Version.java,1.546,1.547
Update of /cvsroot/freenet/freenet/src/freenet In directory sc8-pr-cvs1:/tmp/cvs-serv11019/src/freenet Modified Files: Version.java Log Message: 6351: Fix NPE in FilterRoutingTable. Index: Version.java === RCS file: /cvsroot/freenet/freenet/src/freenet/Version.java,v retrieving revision 1.546 retrieving revision 1.547 diff -u -w -r1.546 -r1.547 --- Version.java22 Nov 2003 19:03:56 - 1.546 +++ Version.java22 Nov 2003 19:06:34 - 1.547 @@ -20,7 +20,7 @@ public static String protocolVersion = 1.47; /** The build number of the current revision */ -public static final int buildNumber = 6350; +public static final int buildNumber = 6351; public static final int ignoreBuildsAfter = 6500; ___ cvs mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/rt FilterRoutingTable.java, 1.12, 1.13
Update of /cvsroot/freenet/freenet/src/freenet/node/rt
In directory sc8-pr-cvs1:/tmp/cvs-serv11019/src/freenet/node/rt
Modified Files:
FilterRoutingTable.java
Log Message:
6351:
Fix NPE in FilterRoutingTable.
Index: FilterRoutingTable.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/FilterRoutingTable.java,v
retrieving revision 1.12
retrieving revision 1.13
diff -u -w -r1.12 -r1.13
--- FilterRoutingTable.java 22 Nov 2003 17:56:01 - 1.12
+++ FilterRoutingTable.java 22 Nov 2003 19:06:34 - 1.13
@@ -102,6 +102,7 @@
* @see
freenet.node.rt.RoutingTable#shouldReference(freenet.node.NodeReference,
freenet.message.StoreData)
*/
public boolean shouldReference(NodeReference nr, StoreData sd) {
+ if(nr == null) return false;
if(blocks(nr.getIdentity())) return false;
return rt.shouldReference(nr, sd);
}
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-dev] Unstable 6350 (and 6351)
Unstable build 6351 is in CVS. The snapshots are being updated. Changes: 6350: Separate global estimators for search time and transfer rate. This means that requestFailTime, the estimated cost of a retry, a dominating component in the NGRouting formula, now varies depending on the file size as well as the key. This should make routing more accurate for large files. 6351: Fixed a NullPointerException in FilterRoutingTable, thanks lostlogic. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-CVS] freenet/src/freenet/node/rt StandardNodeStats.java, 1.2, 1.3 StandardNodeEstimator.java, 1.51, 1.52
Update of /cvsroot/freenet/freenet/src/freenet/node/rt
In directory sc8-pr-cvs1:/tmp/cvs-serv17122/src/freenet/node/rt
Modified Files:
StandardNodeStats.java StandardNodeEstimator.java
Log Message:
6352:
Fix the bug that was causing stats.maxTransferFailedTime, and therefore the initial
transfer failed time on estimators, to be 0.
Up estimator fieldset version to 4.
Index: StandardNodeStats.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/StandardNodeStats.java,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -w -r1.2 -r1.3
--- StandardNodeStats.java 22 Nov 2003 17:56:01 - 1.2
+++ StandardNodeStats.java 22 Nov 2003 19:41:04 - 1.3
@@ -79,6 +79,9 @@
d = ne.tConnectSucceeded();
if(maxConnectSuccessTime d)
maxConnectSuccessTime = d;
+ d = ne.tTransferFailed();
+ if(maxTransferFailedTime d)
+ maxTransferFailedTime = d;
d = ne.tSearchFailed();
if(maxSearchFailedTime d)
maxSearchFailedTime = d;
Index: StandardNodeEstimator.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/StandardNodeEstimator.java,v
retrieving revision 1.51
retrieving revision 1.52
diff -u -w -r1.51 -r1.52
--- StandardNodeEstimator.java 22 Nov 2003 19:03:55 - 1.51
+++ StandardNodeEstimator.java 22 Nov 2003 19:41:04 - 1.52
@@ -103,7 +103,7 @@
public FieldSet toFieldSet() {
FieldSet fs = new FieldSet();
- fs.put(Version, 3);
+ fs.put(Version, 4);
fs.put(rpConnectFailed, rpConnectFailed.toFieldSet());
fs.put(rpTransferFailed, rpTransferFailed.toFieldSet());
fs.put(rpSearchFailed, rpSearchFailed.toFieldSet());
@@ -210,7 +210,7 @@
if(v == null) throw new EstimatorFormatException(no
Version);
try {
int version = Fields.hexToInt(v);
- if(version != 3)
+ if(version != 4)
throw new
EstimatorFormatException(Unsupported version +version, false);
} catch (NumberFormatException ex) {
EstimatorFormatException ee = new
EstimatorFormatException(Odd version: +v+ (+ex+));
@@ -1055,5 +1055,12 @@
*/
public long routeAccesses() {
return routeAccesses;
+ }
+
+ /**
+* @return
+*/
+ public double tTransferFailed() {
+ return rtTransferFailed.currentValue();
}
}
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-dev] Unstable 6352
Unstable 6352 is in CVS. This fixes the bug that caused the initial transfer failed time to be 0 on some estimators, and increments the (non-backwards-compatible) estimator passing version number, so that we don't need to worry about old builds with bad (optimistic) initial estimators (we can still talk to them, but we no longer accept their Estimators through StoreData or seednodes). -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-CVS] freenet/src/freenet/node/ds FSDataStore.java, 1.23, 1.24 DataStore.java, 1.5, 1.6
Update of /cvsroot/freenet/freenet/src/freenet/node/ds
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/ds
Modified Files:
FSDataStore.java DataStore.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: FSDataStore.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/ds/FSDataStore.java,v
retrieving revision 1.23
retrieving revision 1.24
diff -u -w -r1.23 -r1.24
--- FSDataStore.java18 Nov 2003 10:58:59 - 1.23
+++ FSDataStore.java22 Nov 2003 20:48:47 - 1.24
@@ -46,7 +46,8 @@
* @return a KeyOutputStream that transfers the written data
* into the store
*/
-public KeyOutputStream putData(Key k, long dataSize, FieldSet storables)
+public KeyOutputStream putData(Key k, long dataSize, FieldSet storables,
+ boolean ignoreDS)
throws IOException, KeyCollisionException {
// we need to know the byte-length of the Storables in advance
@@ -62,7 +63,7 @@
synchronized (dir.semaphore()) {
-if (dir.contains(fn))
+if (dir.contains(fn) !ignoreDS)
throw new KeyCollisionException();
if (maxDataSize 0 dataSize maxDataSize) {
Index: DataStore.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/ds/DataStore.java,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -w -r1.5 -r1.6
--- DataStore.java 16 Jun 2003 19:38:39 - 1.5
+++ DataStore.java 22 Nov 2003 20:48:48 - 1.6
@@ -35,8 +35,11 @@
* @param dataSize total length of data (including control bytes)
* @param storables Storable fields for this data
* @param indexNow whether to immediately index the key
+ * @param ignoreDS if true, don't check whether the data is in the store.
+ * Implementations are expected to buffer the data even though there is already
+ * a stored copy, and the commit will fail.
*/
-KeyOutputStream putData(Key k, long dataSize, FieldSet storables)
+KeyOutputStream putData(Key k, long dataSize, FieldSet storables, boolean
ignoreDS)
throws IOException, KeyCollisionException;
/**
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/states/FNP NewInsertRequest.java, 1.9, 1.10 NewDataRequest.java, 1.10, 1.11
Update of /cvsroot/freenet/freenet/src/freenet/node/states/FNP
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/states/FNP
Modified Files:
NewInsertRequest.java NewDataRequest.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: NewInsertRequest.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FNP/NewInsertRequest.java,v
retrieving revision 1.9
retrieving revision 1.10
diff -u -w -r1.9 -r1.10
--- NewInsertRequest.java 21 Nov 2003 21:12:44 - 1.9
+++ NewInsertRequest.java 22 Nov 2003 20:48:47 - 1.10
@@ -41,7 +41,7 @@
RequestInitiator ri = new RequestInitiator(id,
irmo.getReceivedTime());
Pending p = new InsertPending(id, (int) irmo.hopsToLive,
- irmo.searchKey, origRec, ft, ri, false);
+ irmo.searchKey, origRec, ft, ri, false,
false);
Core.diagnostics.occurrenceCounting(incomingInserts, 1);
return p.received(n, ri);
} catch (RequestAbortException rae) {
Index: NewDataRequest.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FNP/NewDataRequest.java,v
retrieving revision 1.10
retrieving revision 1.11
diff -u -w -r1.10 -r1.11
--- NewDataRequest.java 22 Nov 2003 17:56:01 - 1.10
+++ NewDataRequest.java 22 Nov 2003 20:48:47 - 1.11
@@ -45,7 +45,7 @@
Core.diagnostics.occurrenceContinuous(startedRequestHTL,
drmo.hopsToLive);
Pending p = new DataPending(id, (int) drmo.hopsToLive,
-drmo.searchKey, origRec, ft, ri, false);
+drmo.searchKey, origRec, ft, ri, false,
false);
Core.diagnostics.occurrenceCounting(incomingRequests, 1);
return p.received(n, ri);
} catch (RequestAbortException e) {
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/states/announcing NewInitialRequest.java, 1.7, 1.8
Update of /cvsroot/freenet/freenet/src/freenet/node/states/announcing
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/states/announcing
Modified Files:
NewInitialRequest.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: NewInitialRequest.java
===
RCS file:
/cvsroot/freenet/freenet/src/freenet/node/states/announcing/NewInitialRequest.java,v
retrieving revision 1.7
retrieving revision 1.8
diff -u -w -r1.7 -r1.8
--- NewInitialRequest.java 21 Nov 2003 21:12:44 - 1.7
+++ NewInitialRequest.java 22 Nov 2003 20:48:48 - 1.8
@@ -55,7 +55,7 @@
throw new BadStateException(Expecting MakeInitialRequest);
RequestInitiator ri = new RequestInitiator(id,
System.currentTimeMillis());
-return (new DataPending(id, htl, key, null, ft, ri, false)).received(n, ri);
+return (new DataPending(id, htl, key, null, ft, ri, false,
false)).received(n, ri);
}
public void lost(Node n) {
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/client InternalClient.java, 1.37, 1.38
Update of /cvsroot/freenet/freenet/src/freenet/client
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/client
Modified Files:
InternalClient.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: InternalClient.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/client/InternalClient.java,v
retrieving revision 1.37
retrieving revision 1.38
diff -u -w -r1.37 -r1.38
--- InternalClient.java 22 Nov 2003 17:56:01 - 1.37
+++ InternalClient.java 22 Nov 2003 20:48:48 - 1.38
@@ -498,15 +498,15 @@
throw new KeyException(got null Key);
Key k = ft.clientKey.getKey();
-if (ft.req.getNonLocal()) {
-// For now we implement non-local requests by
-// deleting the key from the datastore, just like
-// in FCP. Underwhelming :-( -gj
-if (node.ds.contains(k)) {
-node.ds.remove(k, false);
-}
-}
-
+//if (ft.req.getNonLocal()) {
+//// For now we implement non-local requests by
+//// deleting the key from the datastore, just like
+//// in FCP. Underwhelming :-( -gj
+//if (node.ds.contains(k)) {
+//node.ds.remove(k, false);
+//}
+//}
+//
Core.diagnostics.occurrenceContinuous(startedRequestHTL,
ft.req.htl);
@@ -539,7 +539,7 @@
RequestInitiator ri =
new RequestInitiator(id, System.currentTimeMillis());
Pending p = new DataPending(id, ft.req.htl, ft.clientKey.getKey(),
-null, ft, ri, false);
+null, ft, ri, false,
ft.req.getNonLocal());
// REDFLAG: do any calls within this scope
// cause events to be dispatched?
// Back to this.
@@ -634,7 +634,7 @@
new RequestInitiator(id, System.currentTimeMillis());
Pending p = new InsertPending(id, ft.req.htl, ft.clientKey.getKey(),
- null, ft, ri, false);
+ null, ft, ri, false,
ft.req.getNonLocal());
s = p.received(n, ri);
try {
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/message DataSend.java,1.13,1.14
Update of /cvsroot/freenet/freenet/src/freenet/message
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/message
Modified Files:
DataSend.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: DataSend.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/message/DataSend.java,v
retrieving revision 1.13
retrieving revision 1.14
diff -u -w -r1.13 -r1.14
--- DataSend.java 31 Oct 2003 19:21:20 - 1.13
+++ DataSend.java 22 Nov 2003 20:48:47 - 1.14
@@ -133,13 +133,14 @@
*
* @param nThe node to cache into.
* @param searchKeyThe key to cache as.
+ * @param ignoreDS If true, ignore the datastore rather than throwing
KeyCollisionException
* @return a DataState that will read the data into the store
* @exception StoreIOException If writing to the datastore fails.
* @exception IOException If creating the stream to CACHE the data fails.
* @exception DataNotValidIOException If the data does not validate
* for the provided key.
*/
-public ReceiveData cacheData(Node n, Key searchKey)
+public ReceiveData cacheData(Node n, Key searchKey, boolean ignoreDS)
throws IOException, DataNotValidIOException,
KeyCollisionException {
Storables storables = getStorables();
@@ -152,7 +153,7 @@
:+in, Logger.DEBUG);
KeyOutputStream out;
try {
- out = n.ds.putData(searchKey, length, storables);
+ out = n.ds.putData(searchKey, length, storables, ignoreDS);
} catch (IOException e) {
if(e instanceof StoreIOException) throw e;
else throw new StoreIOException(e);
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/rt NGRouting.java,1.32,1.33
Update of /cvsroot/freenet/freenet/src/freenet/node/rt
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/rt
Modified Files:
NGRouting.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: NGRouting.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/rt/NGRouting.java,v
retrieving revision 1.32
retrieving revision 1.33
diff -u -w -r1.32 -r1.33
--- NGRouting.java 22 Nov 2003 19:03:55 - 1.32
+++ NGRouting.java 22 Nov 2003 20:48:47 - 1.33
@@ -202,11 +202,10 @@
diffSearchTime);
Core.diagnostics.occurrenceContinuous(absDiffSearchSuccessTime,
Math.abs(diffSearchTime));
-
+}
// we only want successes here - reporting everythings causes us to route to
// nodes that will just Query Reject. edt
ngrt.globalSearchTimeEstimator.reportTime(key, searchTime);
-}
double rate;
if(size 16384 /* must be multi-segment */
transferTime 10 /* sanity check */) {
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node NewNodeContactor.java,1.2,1.3
Update of /cvsroot/freenet/freenet/src/freenet/node
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node
Modified Files:
NewNodeContactor.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: NewNodeContactor.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/NewNodeContactor.java,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -w -r1.2 -r1.3
--- NewNodeContactor.java 22 Nov 2003 17:56:00 - 1.2
+++ NewNodeContactor.java 22 Nov 2003 20:48:48 - 1.3
@@ -51,9 +51,9 @@
// Remove key from store if it exists
// FIXME: like in InternalClient, and FCP, maybe we want to
// keep it in the store and just pretend it's not for purposes of this
request?
- if (n.ds.contains(k)) {
- n.ds.remove(k, false);
- }
+// if (n.ds.contains(k)) {
+// n.ds.remove(k, false);
+// }
// Calculate HTL
int htl = Node.maxHopsToLive;
double occurrences = Core.diagnostics.getValue(startedRequestHTL,
@@ -67,7 +67,7 @@
}
RequestInitiator ri =
new RequestInitiator(id, System.currentTimeMillis());
- Pending p = new DataPending(id, htl, k, null, ct, ri, true);
+ Pending p = new DataPending(id, htl, k, null, ct, ri, true, true);
// Schedule a NodeContactorVector which returns the Pending
// And then schedules the RequestInitiator
NewNodeContactorRequest newReq =
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-CVS] freenet/src/freenet/node/states/request Pending.java, 1.84, 1.85 InsertPending.java, 1.29, 1.30 DataPending.java, 1.28, 1.29 AwaitingInsert.java, 1.10, 1.11
Update of /cvsroot/freenet/freenet/src/freenet/node/states/request
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/states/request
Modified Files:
Pending.java InsertPending.java DataPending.java
AwaitingInsert.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: Pending.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/request/Pending.java,v
retrieving revision 1.84
retrieving revision 1.85
diff -u -w -r1.84 -r1.85
--- Pending.java21 Nov 2003 21:12:43 - 1.84
+++ Pending.java22 Nov 2003 20:48:46 - 1.85
@@ -61,6 +61,8 @@
final boolean routeToNewestNodes;
+boolean ignoreDS = false;
+
/** Time marked when routing so that we can register time taken for NGrouting. DO
NOT RESET ON QUERYRESTARTED, as it is part of the node routing metric. */
long routedTime = -1;
@@ -102,9 +104,10 @@
* queries.
*/
Pending(long id, int htl, Key key, Peer orig, FeedbackToken ft, RequestInitiator
ri,
- boolean routeToNewestNodes) {
+ boolean routeToNewestNodes, boolean ignoreDS) {
super(id, htl, key, orig, ft, ri);
this.routeToNewestNodes = routeToNewestNodes;
+ this.ignoreDS = ignoreDS;
if(logDEBUG)
Core.logger.log(this, Created new Pending from scratch: +this,
@@ -126,8 +129,11 @@
outwardSender = p.outwardSender;
outwardIdentityLastUsed = p.outwardIdentityLastUsed;
routeToNewestNodes = p.routeToNewestNodes;
- } else
+ ignoreDS = p.ignoreDS;
+ } else {
routeToNewestNodes = false;
+ ignoreDS = false;
+ }
if(logDEBUG)
Core.logger.log(this, Created new Pending: +this+ from +
ancestor, new Exception(debug), Logger.DEBUG);
@@ -434,7 +440,7 @@
cancelRestart();
try {
-receivingData = dr.cacheData(n, searchKey);
+receivingData = dr.cacheData(n, searchKey, ignoreDS);
replyTime = System.currentTimeMillis();
// replyTime must be set AFTER verifying Storables
n.ft.remove(searchKey); // remove if in FT.
@@ -839,6 +845,7 @@
* unrecoverable error. Does nothing if the key is not found.
*/
void searchData(Node n) throws RequestAbortException {
+ if(ignoreDS) return;
if(logDEBUG)
Core.logger.log(this, searchData() on +this,
new Exception(debug), Logger.DEBUG);
Index: InsertPending.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/request/InsertPending.java,v
retrieving revision 1.29
retrieving revision 1.30
diff -u -w -r1.29 -r1.30
--- InsertPending.java 21 Nov 2003 21:12:43 - 1.29
+++ InsertPending.java 22 Nov 2003 20:48:47 - 1.30
@@ -56,8 +56,9 @@
*/
public InsertPending(long id, int htl, Key key, Peer orig,
FeedbackToken ft, RequestInitiator ri,
-boolean routeToNewestNodes) {
-super(id, htl, key, orig, ft, ri, routeToNewestNodes);
+boolean routeToNewestNodes,
+boolean ignoreDS) {
+super(id, htl, key, orig, ft, ri, routeToNewestNodes, ignoreDS);
}
/**
@@ -173,7 +174,7 @@
this.dim = dim;
try {
-receivingData = dim.cacheData(n, searchKey);
+receivingData = dim.cacheData(n, searchKey, ignoreDS);
} catch (KeyCollisionException e) {
// we've already sent the Accepted so we should try..
dim.eatData(n);
Index: DataPending.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/request/DataPending.java,v
retrieving revision 1.28
retrieving revision 1.29
diff -u -w -r1.28 -r1.29
--- DataPending.java21 Nov 2003 21:12:43 - 1.28
+++ DataPending.java22 Nov 2003 20:48:47 - 1.29
@@ -29,8 +29,9 @@
*/
public DataPending(long id, int htl, Key key, Peer orig,
FeedbackToken ft, RequestInitiator ri,
- boolean routeToNewestNodes) {
-super(id, htl, key, orig, ft, ri, routeToNewestNodes);
+ boolean routeToNewestNodes,
+ boolean ignoreDS) {
+super(id, htl,
[freenet-CVS] freenet/src/freenet/node/states/FCP NewClientPut.java, 1.17, 1.18 NewClientGet.java, 1.14, 1.15
Update of /cvsroot/freenet/freenet/src/freenet/node/states/FCP
In directory sc8-pr-cvs1:/tmp/cvs-serv28920/src/freenet/node/states/FCP
Modified Files:
NewClientPut.java NewClientGet.java
Log Message:
6353:
When we send out maintenance requests, when we get FCP or internal requests with skip
datastore enabled:
Don't actually delete the key from the datastore... just ignore the fact that it's
already there. The commit will fail, unless the original one didn't work.
Index: NewClientPut.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FCP/NewClientPut.java,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -w -r1.17 -r1.18
--- NewClientPut.java 21 Nov 2003 21:12:44 - 1.17
+++ NewClientPut.java 22 Nov 2003 20:48:47 - 1.18
@@ -97,12 +97,12 @@
ctBucket = n.bf.makeBucket(tmpLen);
ctStream = new FreeBucketInputStream(ckey.encode(data, mlen,
ctBucket),
n.bf, ctBucket);
-if (cpmo.getRemoveLocal()) {
- Key bkey = ckey.getKey();
- if (n.ds.contains(bkey)) {
- n.ds.remove(bkey, false);
- }
- }
+//if (cpmo.getRemoveLocal()) {
+// Key bkey = ckey.getKey();
+// if (n.ds.contains(bkey)) {
+// n.ds.remove(bkey, false);
+// }
+// }
// set up the FeedbackToken
FeedbackToken ft;
@@ -126,7 +126,8 @@
RequestInitiator ri =
new RequestInitiator(id, cpmo.getReceivedTime());
-s = new InsertPending(id, cpmo.getHTL(), ckey.getKey(), null, ft, ri,
false);
+s = new InsertPending(id, cpmo.getHTL(), ckey.getKey(), null, ft,
+ ri, false, cpmo.getRemoveLocal());
s = s.received(n, ri);
ctBucket = null;
Index: NewClientGet.java
===
RCS file: /cvsroot/freenet/freenet/src/freenet/node/states/FCP/NewClientGet.java,v
retrieving revision 1.14
retrieving revision 1.15
diff -u -w -r1.14 -r1.15
--- NewClientGet.java 22 Nov 2003 17:56:01 - 1.14
+++ NewClientGet.java 22 Nov 2003 20:48:47 - 1.15
@@ -59,17 +59,17 @@
sendMessage(new URIError(id, Key too big));
return null;
}
-if (cgmo.getRemoveLocal()) {
- if (n.ds.contains(k)) {
- n.ds.remove(k, false);
- }
-}
+//if (cgmo.getRemoveLocal()) {
+// if (n.ds.contains(k)) {
+// n.ds.remove(k, false);
+// }
+//}
FeedbackToken ft= new ClientGetToken(id, source, ckey, sendHint, uri,
timeSec,
n.bf);
RequestInitiator ri = new RequestInitiator(id,
cgmo.getReceivedTime());
return (new DataPending(id, cgmo.getHTL(),
-k, null, ft, ri, false)).received(n, ri);
+k, null, ft, ri, false,
cgmo.getRemoveLocal())).received(n, ri);
}
catch (KeyException e) {
if(Core.logger.shouldLog(Logger.DEBUG))
___
cvs mailing list
[EMAIL PROTECTED]
http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/cvs
[freenet-dev] Unstable 6353
Freenet unstable build 6353 is in CVS, snapshots are updating. Changes: When we refetch a recent key for maintenance of the routing table, when we get an FCP request with RemoveLocalKey, when we do an internal request with skip datastore enabled, don't actually delete the file from the store, instead ignore its presence for as long as possible. The request will (hopefully!) succeed despite the file already being in the store, and then it will try to commit, and get a collision, which will be handled normally. Unless something happened to the other item. This could be implemented slightly more efficiently w.r.t. disk usage. Testing would be appreciated, especially as this has an impact on common insertion tools. -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Progress report, and request for help
Hi All. Time for one of my regular progress reports: Next Generation Routing is in, and we are working feverishly to weed out the bugs, and coax it into working nicely. The process is challenging as often the effects of changes cannot be seen for several days, but we are making progress. We have made rapid progress on the issue of load-balancing, and are now focusing on how to ensure that node's specialize appropriately. Just yesterday we implemented a change that might address this problem although it will be a few days before we know how effective it is. One of the negative effects of our decision to migrate over to next-gen routing has been that we haven't put out a new Freenet release in quite a while, and this has left the project's finances in a less than ideal state, since most of our donations come in in the days after a release. The current status is that the project only has enough funding to keep Matthew employed until the middle of next week. As anyone who follows development closely will know, Matthew is central to providing momentum to the development process. As a result, I am appealing to supporters of the project to dig deep and make a contribution to keep things going. You can donate online via Paypal or E-Gold, and if you would prefer to contribute by mail, please email me directly to arrange it. If you would like to make an ongoing contribution to the project, you can even set up a monthly subscription of $5, $10, or $20. Either way, we really need your support to get us to the 0.6 release on which we are now focused. You can donate by visiting our donations page at: http://freenetproject.org/index.php?page=donate Many thanks, Ian. Freenet Project Coordinator ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Re: Negative trust
His analysis applies to any large-scale p2p network. There are at least two defenses: either create some sort of certification authority (perhaps a supervisory p2p network) or allow/encourage fragmentation of the target network. Come now, This is not impossible. GNUnet does it. And does it well. I posted a way to adapt this to Freenet's architecture a while back. It can be done. It just requires a big code over hall. You might be disagreeing with the conclusions of the paper on Sybil. If so, have you read the paper? If so, which conclusion are you disagreeing with? I am disagreeing with the paper. Not in it's conclusions but in it's premises. Or you might be saying that Freenet could create a CA. If so, can you be more specific? Not create a CA, but act as one. They state that a CA is necessary to prevent cancer nodes from making multiple identity's to insure privacy and prevent a group of nodes from attacking the network. I'll think you'll agree with me when I say that in terms of data storage Freenet does an excellent job insuring security despite not trusting the node with the data. If they think they could brute force a CHK I welcome them to try. In terms of responsibility for storing data, nobody is responsible in Freenet. They could pretend to be 10 nodes and then collect lots of data and delete it and nobody would care. The only way they got the data was to cache it in the first place. Or you might be saying that Freenet could allow or encourage network fragmentation. Are you? No I am not. There are still two other arias where cancer nodes can be a problem. First is flooding. This is what the GNUnet model solves. Here's the short version: If you give each node credit proportional to the amount of time they saved you by processing a request through them as opposed to someone else, and then allow them to use that credit towards your spending time processing their requests, then you don't need any outside authority. Both nodes know they are not being cheated. If they are then they don't process the requests. Simple as that. Now how does one build up credit in the first place? Simple. If CPU, network bandwidth or hardDrive space are not being used at any particular time, they go to waste. So even if a node has 0 credit you'll still process their request if you have idle resources. Thus you gain credit with them. This way no node can do more damage than The Amount of Benefit they have previously provided to the network + the slack resources in the network + the CPU required to check and then drop N requests. That's as good as it gets anywhere. The only problem this does not solve is if a node does a good job of processing requests over all, but always drops a single key. Freenet cannot truly solve this problem, because there is no way to know that they really should have had the data. BUT a central authority cannot solve this problem ether! The only way to do so would be for it to know where all the data on the network was stored. AND have all the requests routed and returned through it. Otherwise a node could claim it did not receive the data when it did. I don't think I need to explain why this is not a viable solution. ___ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
