[freenet-dev] Darknet vs opennet wording? was Re: Addressing the "Barlow" attack against opennet
On Wednesday 08 December 2010 01:08:51 David ?Bombe? Roden wrote: > On Wednesday 08 December 2010 00:44:51 Ed Tomlinson wrote: > > > But as you said previously no one uses darknet. How about a semi open net > > that uses a WOT attribute to decide what nodes to trust? > > That would allow a direct connection between a WoT identity and an IP address? > unless I understood you incorrectly. Sure. Thats why I said a WOT attribute. Some of your identies would be completely hidden but the one for the 'greynet' would have to eventually be resolveable to an IP. This could be made into a question for the node owner. eg. The following nodes want to connect via greynet do you trust them? Maybe the requesting node could send some info (a message with verifable info - I am xxx on yyy with WOT verifing) along witht the request. The issue with darknet has always been find enough nodes to trust. Using WOT we _might_ be able to automate this, at least to some extent. Question then becomes is using this idea better than opennet? Ed greynet = darknet with nodes found via a WOT.
Re: [freenet-dev] Darknet vs opennet wording? was Re: Addressing the "Barlow" attack against opennet
On Wednesday 08 December 2010 01:08:51 David ‘Bombe’ Roden wrote: > On Wednesday 08 December 2010 00:44:51 Ed Tomlinson wrote: > > > But as you said previously no one uses darknet. How about a semi open net > > that uses a WOT attribute to decide what nodes to trust? > > That would allow a direct connection between a WoT identity and an IP address— > unless I understood you incorrectly. Sure. Thats why I said a WOT attribute. Some of your identies would be completely hidden but the one for the 'greynet' would have to eventually be resolveable to an IP. This could be made into a question for the node owner. eg. The following nodes want to connect via greynet do you trust them? Maybe the requesting node could send some info (a message with verifable info - I am xxx on yyy with WOT verifing) along witht the request. The issue with darknet has always been find enough nodes to trust. Using WOT we _might_ be able to automate this, at least to some extent. Question then becomes is using this idea better than opennet? Ed greynet = darknet with nodes found via a WOT. ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Darknet vs opennet wording? was Re: Addressing the "Barlow" attack against opennet
On Tuesday 07 December 2010 12:21:07 Matthew Toseland wrote: > On Friday 03 December 2010 19:15:22 Klaus Koch wrote: > > > > It is a hard problem. But our traditional approach hasn't been terribly > > > > honest IMHO. > > > > We were talking on #freenet on how to explain new users in a few words > > (installer?) what freenet's security is all about and how to "warn" them of > > the shortcomings of opennet. I came up with the following text: > > > > "Freenet's security and anonymity is based on the idea that users connect to > > people they trust. Opennet mode (=LOW security level) is a convenience > > feature > > for new users who don't have trusted peers yet and it's security is not as > > strong as darknet (= MEDIUM/HIGH security level). Use this mode to befriend > > people you think you can trust. Get the highest security out of freenet by > > connection to your reallife friends!" > > > > somehow there's still missing that even connecting to a coworker is better > > than a random stranger, but I still struggle to put it into one of the > > sentences... > > IMHO that is precisely what people misunderstand most frequently. How about: > > Generally on Freenet you are only vulnerable to the users your node is > connected to. > Do you want Freenet to connect only to your friends? > > YES (DARKNET MODE): > If you have 5 or more friends who run Freenet, you should enable darknet > mode, and add them on the Friends page. Freenet will send your traffic > through them to their friends and the rest of the network. This greatly > improves your security, because you choose who you connect to. You should > only add people you know personally, online or offline. > > NO (OPENNET MODE): > Freenet can connect to other users automatically, if you don't know anyone on > Freenet. However, this is a convenience feature offering only minimal > security against a determined attacker. In opennet mode, the bad guys can > choose to connect to you, whereas in darknet mode, you choose who you connect > to. But as you said previously no one uses darknet. How about a semi open net that uses a WOT attribute to decide what nodes to trust? Ed
Re: [freenet-dev] Darknet vs opennet wording? was Re: Addressing the "Barlow" attack against opennet
On Tuesday 07 December 2010 12:21:07 Matthew Toseland wrote: > On Friday 03 December 2010 19:15:22 Klaus Koch wrote: > > > > It is a hard problem. But our traditional approach hasn't been terribly > > > > honest IMHO. > > > > We were talking on #freenet on how to explain new users in a few words > > (installer?) what freenet's security is all about and how to "warn" them of > > the shortcomings of opennet. I came up with the following text: > > > > "Freenet's security and anonymity is based on the idea that users connect to > > people they trust. Opennet mode (=LOW security level) is a convenience > > feature > > for new users who don't have trusted peers yet and it's security is not as > > strong as darknet (= MEDIUM/HIGH security level). Use this mode to befriend > > people you think you can trust. Get the highest security out of freenet by > > connection to your reallife friends!" > > > > somehow there's still missing that even connecting to a coworker is better > > than a random stranger, but I still struggle to put it into one of the > > sentences... > > IMHO that is precisely what people misunderstand most frequently. How about: > > Generally on Freenet you are only vulnerable to the users your node is > connected to. > Do you want Freenet to connect only to your friends? > > YES (DARKNET MODE): > If you have 5 or more friends who run Freenet, you should enable darknet > mode, and add them on the Friends page. Freenet will send your traffic > through them to their friends and the rest of the network. This greatly > improves your security, because you choose who you connect to. You should > only add people you know personally, online or offline. > > NO (OPENNET MODE): > Freenet can connect to other users automatically, if you don't know anyone on > Freenet. However, this is a convenience feature offering only minimal > security against a determined attacker. In opennet mode, the bad guys can > choose to connect to you, whereas in darknet mode, you choose who you connect > to. But as you said previously no one uses darknet. How about a semi open net that uses a WOT attribute to decide what nodes to trust? Ed ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Addressing the "Barlow" attack against opennet
On Friday 03 December 2010 14:15:22 Klaus Koch wrote: > > > It is a hard problem. But our traditional approach hasn't been terribly > > > honest IMHO. > > We were talking on #freenet on how to explain new users in a few words > (installer?) what freenet's security is all about and how to "warn" them of > the shortcomings of opennet. I came up with the following text: > > "Freenet's security and anonymity is based on the idea that users connect to > people they trust. Opennet mode (=LOW security level) is a convenience feature > for new users who don't have trusted peers yet and it's security is not as > strong as darknet (= MEDIUM/HIGH security level). Use this mode to befriend > people you think you can trust. Get the highest security out of freenet by > connection to your reallife friends!" > > somehow there's still missing that even connecting to a coworker is better > than a random stranger, but I still struggle to put it into one of the > sentences... There is also a lack of relative security. Is opennet more or less secure than tor? Is opennet more or less secure than using https? Is opennet more or less secure than bittorrent? Is darknet more secure than tor? etc. Answers to these question would help peope decide how to use freenet. Ed
Re: [freenet-dev] Addressing the "Barlow" attack against opennet
On Friday 03 December 2010 14:15:22 Klaus Koch wrote: > > > It is a hard problem. But our traditional approach hasn't been terribly > > > honest IMHO. > > We were talking on #freenet on how to explain new users in a few words > (installer?) what freenet's security is all about and how to "warn" them of > the shortcomings of opennet. I came up with the following text: > > "Freenet's security and anonymity is based on the idea that users connect to > people they trust. Opennet mode (=LOW security level) is a convenience feature > for new users who don't have trusted peers yet and it's security is not as > strong as darknet (= MEDIUM/HIGH security level). Use this mode to befriend > people you think you can trust. Get the highest security out of freenet by > connection to your reallife friends!" > > somehow there's still missing that even connecting to a coworker is better > than a random stranger, but I still struggle to put it into one of the > sentences... There is also a lack of relative security. Is opennet more or less secure than tor? Is opennet more or less secure than using https? Is opennet more or less secure than bittorrent? Is darknet more secure than tor? etc. Answers to these question would help peope decide how to use freenet. Ed ___ Devl mailing list Devl@freenetproject.org http://freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Splitfiles
On Saturday 13 February 2010 19:00:16 xor wrote: > First of all, I like your calculations very much and I wonder why nobody > calculated this before FEC was implemented. If I understood this correctly > then a 700mib file with block success rate p=0.58 will have a 48% total > success > chance. This sucks... > > On Saturday 13 February 2010 20:09:46 Evan Daniel wrote: > > For files of 20 segments (80 MiB) or more, we move to the > > double-layered interleaved scheme. I'm working on the interleaving > > code still (it isn't optimal for all numbers of data blocks yet). The > > simple segmenting scheme is better for smaller files, and the > > interleaved scheme for large ones. At 18 segments, the segmentation > > does better. By 20 segments, the interleaved code is slightly better. > > By 25 segments, the difference is approaching a 1.5x reduction in > > failure rates. (Details depend on block success rate. I'll post them > > on the bug report shortly.) > > > > I wonder why you do not want the interleaved scheme for all multi-segment > files? Why the arbitrary choice of 80 MiB files? Think this is NOT arbirary. It the place where it starts to make a difference in retrieval rates. Less that 80 should work just as well with the simpiler scheme... > It would suck if then people started to artificially bloat 50MiB files up to > 80MiB to improve their success rates... Ed
Re: [freenet-dev] Splitfiles
On Saturday 13 February 2010 19:00:16 xor wrote: > First of all, I like your calculations very much and I wonder why nobody > calculated this before FEC was implemented. If I understood this correctly > then a 700mib file with block success rate p=0.58 will have a 48% total > success > chance. This sucks... > > On Saturday 13 February 2010 20:09:46 Evan Daniel wrote: > > For files of 20 segments (80 MiB) or more, we move to the > > double-layered interleaved scheme. I'm working on the interleaving > > code still (it isn't optimal for all numbers of data blocks yet). The > > simple segmenting scheme is better for smaller files, and the > > interleaved scheme for large ones. At 18 segments, the segmentation > > does better. By 20 segments, the interleaved code is slightly better. > > By 25 segments, the difference is approaching a 1.5x reduction in > > failure rates. (Details depend on block success rate. I'll post them > > on the bug report shortly.) > > > > I wonder why you do not want the interleaved scheme for all multi-segment > files? Why the arbitrary choice of 80 MiB files? Think this is NOT arbirary. It the place where it starts to make a difference in retrieval rates. Less that 80 should work just as well with the simpiler scheme... > It would suck if then people started to artificially bloat 50MiB files up to > 80MiB to improve their success rates... Ed ___ Devl mailing list Devl@freenetproject.org http://osprey.vm.bytemark.co.uk/cgi-bin/mailman/listinfo/devl
[freenet-dev] selecting browser for Launch Freenet from systray
On Saturday 14 November 2009 21:19:43 Zero3 wrote: > Ed Tomlinson wrote: > > On Saturday 14 November 2009 16:02:11 Matthew Toseland wrote: > >> On Saturday 14 November 2009 17:19:15 Ed Tomlinson wrote: > >>> Hi, > >>> > >>> For a test I installed freenet on a windows 7 box. All went well. The > >>> new installation process is pretty simple. One small bone to pick. The > >>> first use wizard tells you NOT to use the same browser for freenet as you > >>> use normally - fine. However there does not seem to be a way to > >>> configure 'Launch Freenet' from the systray to use your prefered browser. > >>> I want to set it to use chrome... > >> It *does* use Chrome if available, in incognito mode. Ask Zero3 why it > >> isn't working. > >>> This should be easy but it does not seem to be... > > > > Think the problem is that I installed chrome after freenet and it _is_ what > > I want freenet to use. I see no option to tell freenet to switch to it. > > Thanks for the feedback :) > > Here is the deal: At the moment, there is a great difference in security > between the major browsers. Especially their incognito mode support. For > that reason, the launcher will dictate the choice of browser for the > user by trying to find the most secure one available on every launch. > > If you install Chrome, the launcher should pick it up at next launch. > More specifically, it will look for the registry string > "InstallLocation" under "HKEY_CURRENT_USER, > Software\Microsoft\Windows\CurrentVersion\Uninstall\Google Chrome". Can > you check if you have that key? Does not exist. I installed the 64 bit version. The uninstall entry is at: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432\Microsoft\Windows\CurrentVersion\Unistall\Google Chrome > The fproxy message is a bit old. It was made at a time where the plan > was to install a secondary browser/browser profile for the user, and use > that one solely for Freenet. As browsers in the meantime have started to > incorporate incognito modes, we are moving towards using those instead > (and IMHO, it is a much better solution. Last time we tried the profile > stuff - with FireFox - we failed miserably). > > I'm surprised you were able to install on Win7. I haven't tested the > installer on Win7 myself yet, and I've seen reports of installations > failures on Win7. It just worked out of the box? Aside from the chrome issue it worked as expected. Ed Tomlinson
Re: [freenet-dev] selecting browser for Launch Freenet from systray
On Saturday 14 November 2009 21:19:43 Zero3 wrote: > Ed Tomlinson wrote: > > On Saturday 14 November 2009 16:02:11 Matthew Toseland wrote: > >> On Saturday 14 November 2009 17:19:15 Ed Tomlinson wrote: > >>> Hi, > >>> > >>> For a test I installed freenet on a windows 7 box. All went well. The > >>> new installation process is pretty simple. One small bone to pick. The > >>> first use wizard tells you NOT to use the same browser for freenet as you > >>> use normally - fine. However there does not seem to be a way to > >>> configure 'Launch Freenet' from the systray to use your prefered browser. > >>> I want to set it to use chrome... > >> It *does* use Chrome if available, in incognito mode. Ask Zero3 why it > >> isn't working. > >>> This should be easy but it does not seem to be... > > > > Think the problem is that I installed chrome after freenet and it _is_ what > > I want freenet to use. I see no option to tell freenet to switch to it. > > Thanks for the feedback :) > > Here is the deal: At the moment, there is a great difference in security > between the major browsers. Especially their incognito mode support. For > that reason, the launcher will dictate the choice of browser for the > user by trying to find the most secure one available on every launch. > > If you install Chrome, the launcher should pick it up at next launch. > More specifically, it will look for the registry string > "InstallLocation" under "HKEY_CURRENT_USER, > Software\Microsoft\Windows\CurrentVersion\Uninstall\Google Chrome". Can > you check if you have that key? Does not exist. I installed the 64 bit version. The uninstall entry is at: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432\Microsoft\Windows\CurrentVersion\Unistall\Google Chrome > The fproxy message is a bit old. It was made at a time where the plan > was to install a secondary browser/browser profile for the user, and use > that one solely for Freenet. As browsers in the meantime have started to > incorporate incognito modes, we are moving towards using those instead > (and IMHO, it is a much better solution. Last time we tried the profile > stuff - with FireFox - we failed miserably). > > I'm surprised you were able to install on Win7. I haven't tested the > installer on Win7 myself yet, and I've seen reports of installations > failures on Win7. It just worked out of the box? Aside from the chrome issue it worked as expected. Ed Tomlinson ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] selecting browser for Launch Freenet from systray
On Saturday 14 November 2009 16:02:11 Matthew Toseland wrote: > On Saturday 14 November 2009 17:19:15 Ed Tomlinson wrote: > > Hi, > > > > For a test I installed freenet on a windows 7 box. All went well. The new > > installation process is pretty simple. One small bone to pick. The first > > use wizard tells you NOT to use the same browser for freenet as you use > > normally - fine. However there does not seem to be a way to configure > > 'Launch Freenet' from the systray to use your prefered browser. I want to > > set it to use chrome... > > It *does* use Chrome if available, in incognito mode. Ask Zero3 why it isn't > working. > > > > This should be easy but it does not seem to be... Think the problem is that I installed chrome after freenet and it _is_ what I want freenet to use. I see no option to tell freenet to switch to it. Thanks Ed
Re: [freenet-dev] selecting browser for Launch Freenet from systray
On Saturday 14 November 2009 16:02:11 Matthew Toseland wrote: > On Saturday 14 November 2009 17:19:15 Ed Tomlinson wrote: > > Hi, > > > > For a test I installed freenet on a windows 7 box. All went well. The new > > installation process is pretty simple. One small bone to pick. The first > > use wizard tells you NOT to use the same browser for freenet as you use > > normally - fine. However there does not seem to be a way to configure > > 'Launch Freenet' from the systray to use your prefered browser. I want to > > set it to use chrome... > > It *does* use Chrome if available, in incognito mode. Ask Zero3 why it isn't > working. > > > > This should be easy but it does not seem to be... Think the problem is that I installed chrome after freenet and it _is_ what I want freenet to use. I see no option to tell freenet to switch to it. Thanks Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] selecting browser for Launch Freenet from systray
Hi, For a test I installed freenet on a windows 7 box. All went well. The new installation process is pretty simple. One small bone to pick. The first use wizard tells you NOT to use the same browser for freenet as you use normally - fine. However there does not seem to be a way to configure 'Launch Freenet' from the systray to use your prefered browser. I want to set it to use chrome... This should be easy but it does not seem to be... Thanks Ed
[freenet-dev] selecting browser for Launch Freenet from systray
Hi, For a test I installed freenet on a windows 7 box. All went well. The new installation process is pretty simple. One small bone to pick. The first use wizard tells you NOT to use the same browser for freenet as you use normally - fine. However there does not seem to be a way to configure 'Launch Freenet' from the systray to use your prefered browser. I want to set it to use chrome... This should be easy but it does not seem to be... Thanks Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Current security measures may be harming performance; better measures may help
On Saturday 31 October 2009 11:13:35 Ed Tomlinson wrote: > Some data for you. Months ago I enlarged by store from 16G to 32G. My CHK > Store is now using 8.35G, the CHK cache uses 14.9G (its full). This implies > that we add to the cache at a very slow rate. I would be templed to simulate > random routing AND decreasing the HTL such that it starts caching eariler (eg > decrease the threshold by more than the number of hops randomly routed). > In the few days since 1239 went live my store has reached 9.12G and the cache has remained constant. One interesting thing to note. The write rates seem a bit strange. Writes 12,466 1,182 Access Rate 2.67 /s 2.36 /s Write Rate 0.12 /s 0.01 /s The store write rate has jumped and the cache rate fallen. This seems a little strange? Should not we always be caching more than storing? Thanks Ed
[freenet-dev] Current security measures may be harming performance; better measures may help
On Friday 30 October 2009 21:19:18 Matthew Toseland wrote: > Currently, requests are always routed the same way, but at high HTL we do not > cache either replies to requests or incoming inserts. > > Specifically, at HTL 18 and 17 we do not cache returned data from requests > (though we do check the datastore), and at HTL 18, 17 and 16 we do not cache > data from inserts. On average we spend 2 hops at HTL 18, including the > originator, so on average for an insert it is 4 hops before we cache, with a > minimum of 3 (or is it a minimum of 2? afaics we start at htl 18 and then we > may decrement it when sending to the next hop, so a minimum of 3). > > Decrement at HTL 18 is probabilistic, with a 50% probability. > > Simulations suggest that the "ideal" node is likely found around HTL 14 to > 15. So a significant proportion of requests and inserts will go past it while > still in the no caching phase. This may partly explain poor data retention, > which appears to affect some proportion of keys much more than the others. > > Hence we might get better data retention if we e.g. random routed while in > the no-cache phase. > > But here is another reason for random routing while in the no-cache phase: > > Lets assume that we only care about remote attackers. Generally they are much > more scary. So we are talking about the mobile attacker source tracing > attack. This means that a bad guy is a long way away, and he gets a few > requests by chance which were part of the same splitfile insert or request > originated by you. He is able to determine that they are part of the same, > interesting, splitfile. For each request, he knows 1) that it was routed to > him, and 2) its target location. He can thus determine where on the keyspace > the request could have come from. This is a big vague due to backoff etc, but > he can nonetheless identify an area where the originator is most likely > present, starting at his location and extending in one direction or the > other. In fact, he can identify the opposite end of it as the most likely > location of the originator. So he then tries to get peers closer to this > location, by announcement, path folding, changing his own location etc. If he > is right, he will then get requests from this source much more quickly. And > so he can keep on moving until he reaches the originator. It has been > suggested that we could mark requests so that they will not be routed to new > connections - the problem is this doesn't work for long-lived requests e.g. > big inserts. > > The number of samples the attacker gets is proportional to the number of hops > from the originator to the "ideal" node, on average, since samples after the > "ideal" are much less informative. It is also proportional to the number of > requests sent, and inversely to the size of the network. > > Random routing while the HTL is high, not to any specific location but to a > random peer at each hop (subject to e.g. backoff), would make the pre-ideal > samples much less useful, because they will each have effectively started at > a random node - not a truly random node, especially if we route randomly at > each hop, we won't have had enough hops for it to be a random node across the > whole keyspace, but it will still mean the picture is much more vague, and > the attacker will need a lot more samples. The post-ideal sample remains > useless. If the request reaches the attacker while it is still in the random > routing phase, this provides a useful sample to the attacker, but likely much > less useful than in the routed stage. > > So, just maybe, we could improve data persistence (if not necessarily overall > performance), and maintain the current no-cache-at-high-htl, and improve > security, by random routing as well as not caching while HTL is high. Worth > simulating perhaps? > > The next obvious solution is some form of bundling: Even if the bundle is not > encrypted, routing a large bunch of requests together for some distance gives > one sample instead of many. Short-lived bundles have the disadvantage that > there are many of them so the attacker gets more samples if they happen to > cross his path. However, we could do the don't-route-to-newbies trick with > short-lived bundles, using a fixed path for the bundle's lifetime. 10 bundles > each renewed once an hour beats hundreds of requests per hour! Long-lived > bundles would probably have to automatically move to new nodes, and therefore > could perhaps be traced back to source eventually - if the attacker managed > to hook one, or more likely trace a stream of requests back to one. > > Bundling is a lot more work, a lot more tuning, but of course more secure. It > would replace the current no cache for a few hops, and would still check the > local datastore. > > Encrypted tunnels are a further evolution of bundling: We send out various > randomly routed "anchors", which rendezvous to create a tunn
Re: [freenet-dev] Current security measures may be harming performance; better measures may help
On Friday 30 October 2009 21:19:18 Matthew Toseland wrote: > Currently, requests are always routed the same way, but at high HTL we do not > cache either replies to requests or incoming inserts. > > Specifically, at HTL 18 and 17 we do not cache returned data from requests > (though we do check the datastore), and at HTL 18, 17 and 16 we do not cache > data from inserts. On average we spend 2 hops at HTL 18, including the > originator, so on average for an insert it is 4 hops before we cache, with a > minimum of 3 (or is it a minimum of 2? afaics we start at htl 18 and then we > may decrement it when sending to the next hop, so a minimum of 3). > > Decrement at HTL 18 is probabilistic, with a 50% probability. > > Simulations suggest that the "ideal" node is likely found around HTL 14 to > 15. So a significant proportion of requests and inserts will go past it while > still in the no caching phase. This may partly explain poor data retention, > which appears to affect some proportion of keys much more than the others. > > Hence we might get better data retention if we e.g. random routed while in > the no-cache phase. > > But here is another reason for random routing while in the no-cache phase: > > Lets assume that we only care about remote attackers. Generally they are much > more scary. So we are talking about the mobile attacker source tracing > attack. This means that a bad guy is a long way away, and he gets a few > requests by chance which were part of the same splitfile insert or request > originated by you. He is able to determine that they are part of the same, > interesting, splitfile. For each request, he knows 1) that it was routed to > him, and 2) its target location. He can thus determine where on the keyspace > the request could have come from. This is a big vague due to backoff etc, but > he can nonetheless identify an area where the originator is most likely > present, starting at his location and extending in one direction or the > other. In fact, he can identify the opposite end of it as the most likely > location of the originator. So he then tries to get peers closer to this > location, by announcement, path folding, changing his own location etc. If he > is right, he will then get requests from this source much more quickly. And > so he can keep on moving until he reaches the originator. It has been > suggested that we could mark requests so that they will not be routed to new > connections - the problem is this doesn't work for long-lived requests e.g. > big inserts. > > The number of samples the attacker gets is proportional to the number of hops > from the originator to the "ideal" node, on average, since samples after the > "ideal" are much less informative. It is also proportional to the number of > requests sent, and inversely to the size of the network. > > Random routing while the HTL is high, not to any specific location but to a > random peer at each hop (subject to e.g. backoff), would make the pre-ideal > samples much less useful, because they will each have effectively started at > a random node - not a truly random node, especially if we route randomly at > each hop, we won't have had enough hops for it to be a random node across the > whole keyspace, but it will still mean the picture is much more vague, and > the attacker will need a lot more samples. The post-ideal sample remains > useless. If the request reaches the attacker while it is still in the random > routing phase, this provides a useful sample to the attacker, but likely much > less useful than in the routed stage. > > So, just maybe, we could improve data persistence (if not necessarily overall > performance), and maintain the current no-cache-at-high-htl, and improve > security, by random routing as well as not caching while HTL is high. Worth > simulating perhaps? > > The next obvious solution is some form of bundling: Even if the bundle is not > encrypted, routing a large bunch of requests together for some distance gives > one sample instead of many. Short-lived bundles have the disadvantage that > there are many of them so the attacker gets more samples if they happen to > cross his path. However, we could do the don't-route-to-newbies trick with > short-lived bundles, using a fixed path for the bundle's lifetime. 10 bundles > each renewed once an hour beats hundreds of requests per hour! Long-lived > bundles would probably have to automatically move to new nodes, and therefore > could perhaps be traced back to source eventually - if the attacker managed > to hook one, or more likely trace a stream of requests back to one. > > Bundling is a lot more work, a lot more tuning, but of course more secure. It > would replace the current no cache for a few hops, and would still check the > local datastore. > > Encrypted tunnels are a further evolution of bundling: We send out various > randomly routed "anchors", which rendezvous to create a tunn
[freenet-dev] Looking for a working Eclipse git plugin
On Tuesday 28 April 2009 01:24:16 bbackde at googlemail.com wrote: > On Tue, Apr 28, 2009 at 03:08, Daniel Cheng > wrote: > > Okay, after reading the irc log, i guess i know what's happending here. > > > > You have to generate a ssh-key using `ssh-keygen` and upload your > > *.pub file to your github account. > > It seems as I will run into some trouble because I use Windows and all > those nifty tools and howtos are for Linux > It will take some time until I can continue to contribute code :( > Even the egit eclipse plugin often hangs during checkout on my box. > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl If you want a good command line ssh client for windows look at putty. Be warned that if you use the puttygen program to create your keypair you probably need to reformat the .pub file before uploading. Ed
Re: [freenet-dev] Looking for a working Eclipse git plugin
On Tuesday 28 April 2009 01:24:16 bbac...@googlemail.com wrote: > On Tue, Apr 28, 2009 at 03:08, Daniel Cheng wrote: > > Okay, after reading the irc log, i guess i know what's happending here. > > > > You have to generate a ssh-key using `ssh-keygen` and upload your > > *.pub file to your github account. > > It seems as I will run into some trouble because I use Windows and all > those nifty tools and howtos are for Linux > It will take some time until I can continue to contribute code :( > Even the egit eclipse plugin often hangs during checkout on my box. > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl If you want a good command line ssh client for windows look at putty. Be warned that if you use the puttygen program to create your keypair you probably need to reformat the .pub file before uploading. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Git migration
On Friday 24 April 2009 11:09:17 Matthew Toseland wrote: > I am migrating the source to github. Apologies for not providing more notice. > SVN is now READ ONLY. What is the url to use to clone/pull this repository TIA Ed
Re: [freenet-dev] Git migration
On Friday 24 April 2009 11:09:17 Matthew Toseland wrote: > I am migrating the source to github. Apologies for not providing more notice. > SVN is now READ ONLY. What is the url to use to clone/pull this repository TIA Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Running testing - lost my peers
On Wednesday 08 April 2009 08:33:12 Matthew Toseland wrote: > On Wednesday 08 April 2009 03:21:26 Ed Tomlinson wrote: > > Hi, > > > > I've had two long standing 'friends'. Just noticed after I updated to the > latest testing version > > they were gone. Nothing shows on the 'friends' page nor does my peers > > files > have anything > > in it. If I copy and old peers files from a backup it seems to be ignored. > > This is impossible. Please verify that the problem actually exists: > 1. Shut down the node and then copy the old peers file in, and then start up > the node. > 2. Check wrapper.log, if they have signature failure exceptions or something > it should be logged. > 3. Is there anything special about these specific peers? Before posting I tried restoring the peers file. That time it ended up empty again. I tried again today. Now its working Guess we can ignore this for now. Thanks, Ed
[freenet-dev] Running testing - lost my peers
On Wednesday 08 April 2009 08:33:12 Matthew Toseland wrote: > On Wednesday 08 April 2009 03:21:26 Ed Tomlinson wrote: > > Hi, > > > > I've had two long standing 'friends'. Just noticed after I updated to the > latest testing version > > they were gone. Nothing shows on the 'friends' page nor does my peers > > files > have anything > > in it. If I copy and old peers files from a backup it seems to be ignored. > > This is impossible. Please verify that the problem actually exists: > 1. Shut down the node and then copy the old peers file in, and then start up > the node. > 2. Check wrapper.log, if they have signature failure exceptions or something > it should be logged. > 3. Is there anything special about these specific peers?
Re: [freenet-dev] Running testing - lost my peers
On Wednesday 08 April 2009 08:33:12 Matthew Toseland wrote: > On Wednesday 08 April 2009 03:21:26 Ed Tomlinson wrote: > > Hi, > > > > I've had two long standing 'friends'. Just noticed after I updated to the > latest testing version > > they were gone. Nothing shows on the 'friends' page nor does my peers > > files > have anything > > in it. If I copy and old peers files from a backup it seems to be ignored. > > This is impossible. Please verify that the problem actually exists: > 1. Shut down the node and then copy the old peers file in, and then start up > the node. > 2. Check wrapper.log, if they have signature failure exceptions or something > it should be logged. > 3. Is there anything special about these specific peers? Before posting I tried restoring the peers file. That time it ended up empty again. I tried again today. Now its working Guess we can ignore this for now. Thanks, Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Running testing - lost my peers
On Wednesday 08 April 2009 08:33:12 Matthew Toseland wrote: > On Wednesday 08 April 2009 03:21:26 Ed Tomlinson wrote: > > Hi, > > > > I've had two long standing 'friends'. Just noticed after I updated to the > latest testing version > > they were gone. Nothing shows on the 'friends' page nor does my peers > > files > have anything > > in it. If I copy and old peers files from a backup it seems to be ignored. > > This is impossible. Please verify that the problem actually exists: > 1. Shut down the node and then copy the old peers file in, and then start up > the node. > 2. Check wrapper.log, if they have signature failure exceptions or something > it should be logged. > 3. Is there anything special about these specific peers? ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Running testing - lost my peers
Hi, I've had two long standing 'friends'. Just noticed after I updated to the latest testing version they were gone. Nothing shows on the 'friends' page nor does my peers files have anything in it. If I copy and old peers files from a backup it seems to be ignored. Freenet 0.7 Build #1206 r26632 Freenet-ext Build #26 r23771 on amd64 Thanks, Ed
[freenet-dev] Running testing - lost my peers
Hi, I've had two long standing 'friends'. Just noticed after I updated to the latest testing version they were gone. Nothing shows on the 'friends' page nor does my peers files have anything in it. If I copy and old peers files from a backup it seems to be ignored. Freenet 0.7 Build #1206 r26632 Freenet-ext Build #26 r23771 on amd64 Thanks, Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Git vs hg: we must decide now!
On Wednesday 01 April 2009 11:05:43 Daniel Cheng wrote: > On Wed, Apr 1, 2009 at 8:36 AM, Daniel Cheng > wrote: > [...] > > > > I vote for git. > > May I change my vote? > I change to "abstain". > > * I am biased, because my work in jgit / git. > > * This should make some developer feel better. Actually this makes me feel worst. With you working on jgit/git it means that if we have a bug or need a feature added to git it will be easier to work thru the process since we have someone that understands it. From my pov this is a big plus for git... Ed > > * I am maintaining a jgit (java port of git) over freenet transport. > >And the upstream agreed (in principle) to merge it when ready. > > > > * Egit (git+eclipse integration) is preparing a proposal > > to become a eclipse project. > > > > * Eclipse is having a debate on which SCM to use. > > Everybody is talking about git, nobody care about hg. > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=249745. > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=197459 > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=257706 > > > > In the community, you can see: > > - Sun / Netbeans / OpenJDK / Java / Solaris camp is switching to hg. > > - Eclipse / Apache / Ruby / Linux camp is going for git > > - Ubuntu is going bzr (and nobody else care about them) > > > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >
Re: [freenet-dev] Git vs hg: we must decide now!
On Wednesday 01 April 2009 11:05:43 Daniel Cheng wrote: > On Wed, Apr 1, 2009 at 8:36 AM, Daniel Cheng > wrote: > [...] > > > > I vote for git. > > May I change my vote? > I change to "abstain". > > * I am biased, because my work in jgit / git. > > * This should make some developer feel better. Actually this makes me feel worst. With you working on jgit/git it means that if we have a bug or need a feature added to git it will be easier to work thru the process since we have someone that understands it. From my pov this is a big plus for git... Ed > > * I am maintaining a jgit (java port of git) over freenet transport. > >And the upstream agreed (in principle) to merge it when ready. > > > > * Egit (git+eclipse integration) is preparing a proposal > > to become a eclipse project. > > > > * Eclipse is having a debate on which SCM to use. > > Everybody is talking about git, nobody care about hg. > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=249745. > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=197459 > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=257706 > > > > In the community, you can see: > > - Sun / Netbeans / OpenJDK / Java / Solaris camp is switching to hg. > > - Eclipse / Apache / Ruby / Linux camp is going for git > > - Ubuntu is going bzr (and nobody else care about them) > > > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Bye folks
On December 27, 2008, Julien Cornuwel wrote: > I'm sorry to tell you that for some legal reasons (I live in France), I > have to stop working on Freenet. I've learned a lot with you guys and > I'm sad to have to say goodbye, but that's how it is when you live in a > so-called democracy... Julien, IF you can be more specific it probably would be of interest to all of us on this list. TIA Ed Tomlinson
Re: [freenet-dev] Bye folks
On December 27, 2008, Julien Cornuwel wrote: > I'm sorry to tell you that for some legal reasons (I live in France), I > have to stop working on Freenet. I've learned a lot with you guys and > I'm sad to have to say goodbye, but that's how it is when you live in a > so-called democracy... Julien, IF you can be more specific it probably would be of interest to all of us on this list. TIA Ed Tomlinson ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Naming of plugins in svn
On November 4, 2008, xor wrote: > > I just came up with the idea of renaming it to "Freetalk". > If nobody is against that I will do it today (wednesday). Oh great an IM system for freenet. Thats what freeTALK implies at least to me. Maybe FreeMessage or freemess might be better? Thanks Ed
Re: [freenet-dev] Naming of plugins in svn
On November 4, 2008, xor wrote: > > I just came up with the idea of renaming it to "Freetalk". > If nobody is against that I will do it today (wednesday). Oh great an IM system for freenet. Thats what freeTALK implies at least to me. Maybe FreeMessage or freemess might be better? Thanks Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] financial status
On October 30, 2008, Matthew Toseland wrote: > - FOAF routing and no-swap-on-opennet : likely improve overall performance > and > data persistence significantly, although hard to measure (anyone listening, > has freenet improved performance in the last few months??) All requests 6.328% 336,226 - this was about 4% when opennet was swapping CHKs 17.248% 100,189 SSKs 1.693% 236,037 Suspect an increaced hit ratio is leading to fewer hops.
Re: [freenet-dev] financial status
On October 30, 2008, Matthew Toseland wrote: > - FOAF routing and no-swap-on-opennet : likely improve overall performance > and > data persistence significantly, although hard to measure (anyone listening, > has freenet improved performance in the last few months??) All requests 6.328% 336,226 - this was about 4% when opennet was swapping CHKs 17.248% 100,189 SSKs 1.693% 236,037 Suspect an increaced hit ratio is leading to fewer hops. >From my perspective freenet hasbecome more responsive over the last month or >two. Thanks Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Freenet 0.7 build 1164
On September 18, 2008, Matthew Toseland wrote: > Freenet 0.7 build 1164 is out. Please let me know if auto-update doesn't work. > > The main change in 1164 is no-swap-on-opennet. I agree this is going to help with long term data retension. I do wonder what its effect on node 'security' will be?
Re: [freenet-dev] Freenet 0.7 build 1164
On September 18, 2008, Matthew Toseland wrote: > Freenet 0.7 build 1164 is out. Please let me know if auto-update doesn't work. > > The main change in 1164 is no-swap-on-opennet. I agree this is going to help with long term data retension. I do wonder what its effect on node 'security' will be? >From now any opennet node has a fixed location. This means that it can be >easily probed to see if it has specific pieces of data. This was a harder prospect in the swappable opennet... It also implies that opennet is going to become the repository of most of freenet's data - even for darknet nodes. What effect will this have on their security? Comments? Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 6, 2008, Daniel Cheng wrote: > On Wed, Aug 6, 2008 at 7:14 AM, Matthew Toseland > wrote: > > On Tuesday 05 August 2008 23:11, Ed Tomlinson wrote: > >> On August 5, 2008, Matthew Toseland wrote: > >> > On Sunday 03 August 2008 00:58, you wrote: > >> > > On August 2, 2008, Matthew Toseland wrote: > >> > > > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > >> > > > > On August 1, 2008, Michael Rogers wrote: > >> > > > > > Daniel Cheng wrote: > >> > > > > > > in a circular space, we can get infinite number of "average" by > >> > changing > >> > > > > > > point of reference. --- choose the point of reference with the > >> > smallest > >> > > > > > > standard deviation. > >> > > > > > > >> > > > > > From an infinite number of alternatives? Sounds like it might > >> > > > > > take > > a > >> > > > > > while. ;-) I think we can get away with just trying each location > > as > >> > the > >> > > > > > reference point, but that's still O(n^2) running time. > >> > > > > > > >> > > > > > How about this: the average of two locations is the location > > midway > >> > > > > > along the shortest line between them. So to estimate the average > > of a > >> > > > > > set of locations, choose two locations at random from the set and > >> > > > > > replace them with their average, and repeat until there's only > >> > > > > > one > >> > > > > > location in the set. > >> > > > > > > >> > > > > > It's alchemy but at least it runs in linear time. :-) > >> > > > > > >> > > > > Another idea that should run in linear time. Consider each key a > > point > >> > on > >> > > > the edge > >> > > > > of a circle (with a radius of 1). Convert each key (0=0 degress, > > 1=360) > >> > to > >> > > > an x, y cord and > >> > > > > average these numbers. Once all keys are averaged convert the (x, > > y) > >> > back > >> > > > into a key to > >> > > > > get the average. > >> > > > > > >> > > > > egx = sin (key * 360), y = cos(key * 360) assuming the angle > >> > > > > is > > in > >> > > > degrees not radians. > >> > > > > where a key is a number between 0 and 1 > >> > > > > >> > > > You miss the point. We already have what is effectively an angle, > >> > > > it's > >> > just in > >> > > > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the > > circular > >> > > > keyspace. > >> > > > >> > > No I have _not_ missed the point. If you map each key onto the rim of > >> > > a > >> > circle and > >> > > average resulting the x and y coords of all the keys you get an average > > in a > >> > circular > >> > > keyspace. Try it. > >> > > > >> > > If fact radius of the averaged x, y will also be a measure of just how > >> > specialized your > >> > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > >> > > >> > Cool! So the principle is that the closer the mid-point is to being > > actually > >> > on the circle, the more specialised the store? > >> > >> Yes. 'r' should be between 0 and 1. The closer to 0 the less specialized > > the store is. > >> It is not a perfect measure of specialization though. If the store is > > specialized in > >> two areas 180 degrees apart or 3 areas 120 degrees apart and so on, r could > > be small > >> with the store fairly specialized... > >> > >> > Somebody should implement this... We don't need to keep the actual > > samples, we > >> > can just keep a running average of x and y, right? What about > >> > sensitivity, > >> > can we reuse the bootstrapping-decaying-running-average code? Aka klein > >> > filter with sensitivity reducing over time so that for the first N > >>
Re: [freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 6, 2008, Daniel Cheng wrote: > On Wed, Aug 6, 2008 at 7:14 AM, Matthew Toseland > <[EMAIL PROTECTED]> wrote: > > On Tuesday 05 August 2008 23:11, Ed Tomlinson wrote: > >> On August 5, 2008, Matthew Toseland wrote: > >> > On Sunday 03 August 2008 00:58, you wrote: > >> > > On August 2, 2008, Matthew Toseland wrote: > >> > > > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > >> > > > > On August 1, 2008, Michael Rogers wrote: > >> > > > > > Daniel Cheng wrote: > >> > > > > > > in a circular space, we can get infinite number of "average" by > >> > changing > >> > > > > > > point of reference. --- choose the point of reference with the > >> > smallest > >> > > > > > > standard deviation. > >> > > > > > > >> > > > > > From an infinite number of alternatives? Sounds like it might > >> > > > > > take > > a > >> > > > > > while. ;-) I think we can get away with just trying each location > > as > >> > the > >> > > > > > reference point, but that's still O(n^2) running time. > >> > > > > > > >> > > > > > How about this: the average of two locations is the location > > midway > >> > > > > > along the shortest line between them. So to estimate the average > > of a > >> > > > > > set of locations, choose two locations at random from the set and > >> > > > > > replace them with their average, and repeat until there's only > >> > > > > > one > >> > > > > > location in the set. > >> > > > > > > >> > > > > > It's alchemy but at least it runs in linear time. :-) > >> > > > > > >> > > > > Another idea that should run in linear time. Consider each key a > > point > >> > on > >> > > > the edge > >> > > > > of a circle (with a radius of 1). Convert each key (0=0 degress, > > 1=360) > >> > to > >> > > > an x, y cord and > >> > > > > average these numbers. Once all keys are averaged convert the (x, > > y) > >> > back > >> > > > into a key to > >> > > > > get the average. > >> > > > > > >> > > > > egx = sin (key * 360), y = cos(key * 360) assuming the angle > >> > > > > is > > in > >> > > > degrees not radians. > >> > > > > where a key is a number between 0 and 1 > >> > > > > >> > > > You miss the point. We already have what is effectively an angle, > >> > > > it's > >> > just in > >> > > > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the > > circular > >> > > > keyspace. > >> > > > >> > > No I have _not_ missed the point. If you map each key onto the rim of > >> > > a > >> > circle and > >> > > average resulting the x and y coords of all the keys you get an average > > in a > >> > circular > >> > > keyspace. Try it. > >> > > > >> > > If fact radius of the averaged x, y will also be a measure of just how > >> > specialized your > >> > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > >> > > >> > Cool! So the principle is that the closer the mid-point is to being > > actually > >> > on the circle, the more specialised the store? > >> > >> Yes. 'r' should be between 0 and 1. The closer to 0 the less specialized > > the store is. > >> It is not a perfect measure of specialization though. If the store is > > specialized in > >> two areas 180 degrees apart or 3 areas 120 degrees apart and so on, r could > > be small > >> with the store fairly specialized... > >> > >> > Somebody should implement this... We don't need to keep the actual > > samples, we > >> > can just keep a running average of x and y, right? What about > >> > sensitivity, > >> > can we reuse the bootstrapping-decaying-running-average code? Aka klein > >> > filter with sensitivity reducing over time so that for
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 5, 2008, Matthew Toseland wrote: > On Sunday 03 August 2008 00:58, you wrote: > > On August 2, 2008, Matthew Toseland wrote: > > > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > > > > On August 1, 2008, Michael Rogers wrote: > > > > > Daniel Cheng wrote: > > > > > > in a circular space, we can get infinite number of "average" by > changing > > > > > > point of reference. --- choose the point of reference with the > smallest > > > > > > standard deviation. > > > > > > > > > > From an infinite number of alternatives? Sounds like it might take a > > > > > while. ;-) I think we can get away with just trying each location as > the > > > > > reference point, but that's still O(n^2) running time. > > > > > > > > > > How about this: the average of two locations is the location midway > > > > > along the shortest line between them. So to estimate the average of a > > > > > set of locations, choose two locations at random from the set and > > > > > replace them with their average, and repeat until there's only one > > > > > location in the set. > > > > > > > > > > It's alchemy but at least it runs in linear time. :-) > > > > > > > > Another idea that should run in linear time. Consider each key a point > on > > > the edge > > > > of a circle (with a radius of 1). Convert each key (0=0 degress, > > > > 1=360) > to > > > an x, y cord and > > > > average these numbers. Once all keys are averaged convert the (x, y) > back > > > into a key to > > > > get the average. > > > > > > > > eg x = sin (key * 360), y = cos(key * 360) assuming the angle is > > > > in > > > degrees not radians. > > > > where a key is a number between 0 and 1 > > > > > > You miss the point. We already have what is effectively an angle, it's > just in > > > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the circular > > > keyspace. > > > > No I have _not_ missed the point. If you map each key onto the rim of a > circle and > > average resulting the x and y coords of all the keys you get an average in > > a > circular > > keyspace. Try it. > > > > If fact radius of the averaged x, y will also be a measure of just how > specialized your > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > > Cool! So the principle is that the closer the mid-point is to being actually > on the circle, the more specialised the store? Yes. 'r' should be between 0 and 1. The closer to 0 the less specialized the store is. It is not a perfect measure of specialization though. If the store is specialized in two areas 180 degrees apart or 3 areas 120 degrees apart and so on, r could be small with the store fairly specialized... > Somebody should implement this... We don't need to keep the actual samples, > we > can just keep a running average of x and y, right? What about sensitivity, > can we reuse the bootstrapping-decaying-running-average code? Aka klein > filter with sensitivity reducing over time so that for the first N samples > it's effectively a simple running average and after that it's a klein filter > with sensitivity equal to that at the end of the first phase? Or would we > need to use a running average and reset it periodically? you just need to keep the totals for x and y along with the number of keys (call this n) . When a key is removed from the store subtract its x, y coord and reduce n by 1. reverse this when adding a key... Ed
Re: [freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 5, 2008, Matthew Toseland wrote: > On Sunday 03 August 2008 00:58, you wrote: > > On August 2, 2008, Matthew Toseland wrote: > > > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > > > > On August 1, 2008, Michael Rogers wrote: > > > > > Daniel Cheng wrote: > > > > > > in a circular space, we can get infinite number of "average" by > changing > > > > > > point of reference. --- choose the point of reference with the > smallest > > > > > > standard deviation. > > > > > > > > > > From an infinite number of alternatives? Sounds like it might take a > > > > > while. ;-) I think we can get away with just trying each location as > the > > > > > reference point, but that's still O(n^2) running time. > > > > > > > > > > How about this: the average of two locations is the location midway > > > > > along the shortest line between them. So to estimate the average of a > > > > > set of locations, choose two locations at random from the set and > > > > > replace them with their average, and repeat until there's only one > > > > > location in the set. > > > > > > > > > > It's alchemy but at least it runs in linear time. :-) > > > > > > > > Another idea that should run in linear time. Consider each key a point > on > > > the edge > > > > of a circle (with a radius of 1). Convert each key (0=0 degress, > > > > 1=360) > to > > > an x, y cord and > > > > average these numbers. Once all keys are averaged convert the (x, y) > back > > > into a key to > > > > get the average. > > > > > > > > eg x = sin (key * 360), y = cos(key * 360) assuming the angle is > > > > in > > > degrees not radians. > > > > where a key is a number between 0 and 1 > > > > > > You miss the point. We already have what is effectively an angle, it's > just in > > > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the circular > > > keyspace. > > > > No I have _not_ missed the point. If you map each key onto the rim of a > circle and > > average resulting the x and y coords of all the keys you get an average in > > a > circular > > keyspace. Try it. > > > > If fact radius of the averaged x, y will also be a measure of just how > specialized your > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > > Cool! So the principle is that the closer the mid-point is to being actually > on the circle, the more specialised the store? Yes. 'r' should be between 0 and 1. The closer to 0 the less specialized the store is. It is not a perfect measure of specialization though. If the store is specialized in two areas 180 degrees apart or 3 areas 120 degrees apart and so on, r could be small with the store fairly specialized... > Somebody should implement this... We don't need to keep the actual samples, > we > can just keep a running average of x and y, right? What about sensitivity, > can we reuse the bootstrapping-decaying-running-average code? Aka klein > filter with sensitivity reducing over time so that for the first N samples > it's effectively a simple running average and after that it's a klein filter > with sensitivity equal to that at the end of the first phase? Or would we > need to use a running average and reset it periodically? you just need to keep the totals for x and y along with the number of keys (call this n) . When a key is removed from the store subtract its x, y coord and reduce n by 1. reverse this when adding a key... Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 3, 2008, Ian Clarke wrote: > On Sat, Aug 2, 2008 at 6:58 PM, Ed Tomlinson wrote: > > No I have _not_ missed the point. If you map each key onto the rim of a > > circle and > > average resulting the x and y coords of all the keys you get an average in > > a circular > > keyspace. Try it. > > > > If fact radius of the averaged x, y will also be a measure of just how > > specialized your > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > > That's a neat approach, did you just made it up or is it an > established technique? Its an Ed solution. I did lots of plotting of strange functions when in school. It tends to give one a perspective on mapping and using different coordinate systems. Ed
Re: [freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 3, 2008, Ian Clarke wrote: > On Sat, Aug 2, 2008 at 6:58 PM, Ed Tomlinson <[EMAIL PROTECTED]> wrote: > > No I have _not_ missed the point. If you map each key onto the rim of a > > circle and > > average resulting the x and y coords of all the keys you get an average in > > a circular > > keyspace. Try it. > > > > If fact radius of the averaged x, y will also be a measure of just how > > specialized your > > store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) > > That's a neat approach, did you just made it up or is it an > established technique? Its an Ed solution. I did lots of plotting of strange functions when in school. It tends to give one a perspective on mapping and using different coordinate systems. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 2, 2008, Matthew Toseland wrote: > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > > On August 1, 2008, Michael Rogers wrote: > > > Daniel Cheng wrote: > > > > in a circular space, we can get infinite number of "average" by changing > > > > point of reference. --- choose the point of reference with the smallest > > > > standard deviation. > > > > > > From an infinite number of alternatives? Sounds like it might take a > > > while. ;-) I think we can get away with just trying each location as the > > > reference point, but that's still O(n^2) running time. > > > > > > How about this: the average of two locations is the location midway > > > along the shortest line between them. So to estimate the average of a > > > set of locations, choose two locations at random from the set and > > > replace them with their average, and repeat until there's only one > > > location in the set. > > > > > > It's alchemy but at least it runs in linear time. :-) > > > > Another idea that should run in linear time. Consider each key a point on > the edge > > of a circle (with a radius of 1). Convert each key (0=0 degress, 1=360) to > an x, y cord and > > average these numbers. Once all keys are averaged convert the (x, y) back > into a key to > > get the average. > > > > eg x = sin (key * 360), y = cos(key * 360) assuming the angle is in > degrees not radians. > > where a key is a number between 0 and 1 > > You miss the point. We already have what is effectively an angle, it's just > in > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the circular > keyspace. No I have _not_ missed the point. If you map each key onto the rim of a circle and average resulting the x and y coords of all the keys you get an average in a circular keyspace. Try it. If fact radius of the averaged x, y will also be a measure of just how specialized your store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) Ed
Re: [freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 2, 2008, Matthew Toseland wrote: > On Saturday 02 August 2008 02:41, Ed Tomlinson wrote: > > On August 1, 2008, Michael Rogers wrote: > > > Daniel Cheng wrote: > > > > in a circular space, we can get infinite number of "average" by changing > > > > point of reference. --- choose the point of reference with the smallest > > > > standard deviation. > > > > > > From an infinite number of alternatives? Sounds like it might take a > > > while. ;-) I think we can get away with just trying each location as the > > > reference point, but that's still O(n^2) running time. > > > > > > How about this: the average of two locations is the location midway > > > along the shortest line between them. So to estimate the average of a > > > set of locations, choose two locations at random from the set and > > > replace them with their average, and repeat until there's only one > > > location in the set. > > > > > > It's alchemy but at least it runs in linear time. :-) > > > > Another idea that should run in linear time. Consider each key a point on > the edge > > of a circle (with a radius of 1). Convert each key (0=0 degress, 1=360) to > an x, y cord and > > average these numbers. Once all keys are averaged convert the (x, y) back > into a key to > > get the average. > > > > eg x = sin (key * 360), y = cos(key * 360) assuming the angle is in > degrees not radians. > > where a key is a number between 0 and 1 > > You miss the point. We already have what is effectively an angle, it's just > in > 0 to 1 instead of 0 to 360 deg / 2*pi rads. The problem is the circular > keyspace. No I have _not_ missed the point. If you map each key onto the rim of a circle and average resulting the x and y coords of all the keys you get an average in a circular keyspace. Try it. If fact radius of the averaged x, y will also be a measure of just how specialized your store is... (eg r = sqrt(average(x cords)^2+average(y cords)^2) Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 1, 2008, Michael Rogers wrote: > Daniel Cheng wrote: > > in a circular space, we can get infinite number of "average" by changing > > point of reference. --- choose the point of reference with the smallest > > standard deviation. > > From an infinite number of alternatives? Sounds like it might take a > while. ;-) I think we can get away with just trying each location as the > reference point, but that's still O(n^2) running time. > > How about this: the average of two locations is the location midway > along the shortest line between them. So to estimate the average of a > set of locations, choose two locations at random from the set and > replace them with their average, and repeat until there's only one > location in the set. > > It's alchemy but at least it runs in linear time. :-) Another idea that should run in linear time. Consider each key a point on the edge of a circle (with a radius of 1). Convert each key (0=0 degress, 1=360) to an x, y cord and average these numbers. Once all keys are averaged convert the (x, y) back into a key to get the average. eg x = sin (key * 360), y = cos(key * 360) assuming the angle is in degrees not radians. where a key is a number between 0 and 1 Ed
Re: [freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On August 1, 2008, Michael Rogers wrote: > Daniel Cheng wrote: > > in a circular space, we can get infinite number of "average" by changing > > point of reference. --- choose the point of reference with the smallest > > standard deviation. > > From an infinite number of alternatives? Sounds like it might take a > while. ;-) I think we can get away with just trying each location as the > reference point, but that's still O(n^2) running time. > > How about this: the average of two locations is the location midway > along the shortest line between them. So to estimate the average of a > set of locations, choose two locations at random from the set and > replace them with their average, and repeat until there's only one > location in the set. > > It's alchemy but at least it runs in linear time. :-) Another idea that should run in linear time. Consider each key a point on the edge of a circle (with a radius of 1). Convert each key (0=0 degress, 1=360) to an x, y cord and average these numbers. Once all keys are averaged convert the (x, y) back into a key to get the average. eg x = sin (key * 360), y = cos(key * 360) assuming the angle is in degrees not radians. where a key is a number between 0 and 1 Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Proposed solutions so far was Re:?Friend-of-a-friend routing
On July 31, 2008, Florent Daigni?re wrote: > * Matthew Toseland [2008-07-31 17:37:41]: > > > On Tuesday 22 July 2008 23:33, Florent Daigni?re wrote: > > > * Robert Mead [2008-07-22 18:21:14]: > > > > > > > So it sounds like it will be about as secure as it currently is, but a > > > > lot more efficient. 5 to 3 is a big difference. > > > > > > ... On a small, ideal network. > > > > On a larger, messier network, it should make *more* difference. That is, if > > routing is the problem. > > Am I the only one thinking that routing isn't the problem but churn is? > Not at all. Think that churn is making data much harder to find. While I think FOF routing will help shorten path lenghts, I do not think its going to help find older data all that much faster. I would love to see some simulations where the average location of the store influences swapping. It would be very interesting to see the effect on finding keys, especially rarely referenced keys, when this sort of swapping is in effect. Ed
Re: [freenet-dev] Proposed solutions so far was Re:?F riend-of-a-friend routing
On July 31, 2008, Florent Daignière wrote: > * Matthew Toseland <[EMAIL PROTECTED]> [2008-07-31 17:37:41]: > > > On Tuesday 22 July 2008 23:33, Florent Daignière wrote: > > > * Robert Mead <[EMAIL PROTECTED]> [2008-07-22 18:21:14]: > > > > > > > So it sounds like it will be about as secure as it currently is, but a > > > > lot more efficient. 5 to 3 is a big difference. > > > > > > ... On a small, ideal network. > > > > On a larger, messier network, it should make *more* difference. That is, if > > routing is the problem. > > Am I the only one thinking that routing isn't the problem but churn is? > Not at all. Think that churn is making data much harder to find. While I think FOF routing will help shorten path lenghts, I do not think its going to help find older data all that much faster. I would love to see some simulations where the average location of the store influences swapping. It would be very interesting to see the effect on finding keys, especially rarely referenced keys, when this sort of swapping is in effect. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] git-over-fproxy
On June 5, 2008, David ?Bombe? Roden wrote: > On Thursday 05 June 2008 12:17:05 bbackde at googlemail.com wrote: > > > Why git, why not mercurial (better multi-platform support than git, > > same or more features)? > > Why mercurial, why not git? > > This kind of question is pointless. And annoying. Especially annoying. David, This is the best reply I have seen to this sort of 'supid' message in along time. Ed
Re: [freenet-dev] git-over-fproxy
On June 5, 2008, David ‘Bombe’ Roden wrote: > On Thursday 05 June 2008 12:17:05 [EMAIL PROTECTED] wrote: > > > Why git, why not mercurial (better multi-platform support than git, > > same or more features)? > > Why mercurial, why not git? > > This kind of question is pointless. And annoying. Especially annoying. David, This is the best reply I have seen to this sort of 'supid' message in along time. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Usability issues
On May 6, 2008, Matthew Toseland wrote: > On Tuesday 06 May 2008 15:50, Robert Hailey wrote: > > > > On May 5, 2008, at 5:49 PM, Matthew Toseland wrote: > > > [...] > > > - Freenet is slow, especially in this case where it had an > > > untweaked > > > browser. After a long delay, two sites loaded, after a longer delay, > > > several > > > sites loaded at once; this strongly suggests imho that the problem > > > is caused > > > by not finding the browser. User reports significant improvement in > > > speed > > > after a few minutes. > > > - Connection limit is definitely a problem: many images which should > > > be in the > > > same container not loading; same for the Potentially Dangerous Content > > > warning. > > > > Surely the browser isn't the one which understands freenet containers. > > Are we coalescing requests for the same container internally? > > Not at the client level. There is room for further improvement here. But we > do > do some coalescing: we won't have multiple local requests in flight for the > same key, for example. > > Most of the above issues were caused by the browser not being found, this has > been fixed in 90% of cases by the changes I made to browse.cmd. > > However the fact remains that a newbie node is still rather disappointingly > slow, even with a tweaked browser. If the user is in the habit of stoping and starting the node and has a queue with a large number of blocks, browsing will be very slow (or impossible) until the rebuild if the queue completes. See bug 2334. Ed
Re: [freenet-dev] Usability issues
On May 6, 2008, Matthew Toseland wrote: > On Tuesday 06 May 2008 15:50, Robert Hailey wrote: > > > > On May 5, 2008, at 5:49 PM, Matthew Toseland wrote: > > > [...] > > > - Freenet is slow, especially in this case where it had an > > > untweaked > > > browser. After a long delay, two sites loaded, after a longer delay, > > > several > > > sites loaded at once; this strongly suggests imho that the problem > > > is caused > > > by not finding the browser. User reports significant improvement in > > > speed > > > after a few minutes. > > > - Connection limit is definitely a problem: many images which should > > > be in the > > > same container not loading; same for the Potentially Dangerous Content > > > warning. > > > > Surely the browser isn't the one which understands freenet containers. > > Are we coalescing requests for the same container internally? > > Not at the client level. There is room for further improvement here. But we > do > do some coalescing: we won't have multiple local requests in flight for the > same key, for example. > > Most of the above issues were caused by the browser not being found, this has > been fixed in 90% of cases by the changes I made to browse.cmd. > > However the fact remains that a newbie node is still rather disappointingly > slow, even with a tweaked browser. If the user is in the habit of stoping and starting the node and has a queue with a large number of blocks, browsing will be very slow (or impossible) until the rebuild if the queue completes. See bug 2334. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Why Freenet is so SLOW! / Finding data
On December 12, 2007, Robert Hailey wrote: > > On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > > Ok that could be interesting. Although ideally we'd have a > > circular-keyspace-aware averager. > >> ... > > Ok. I suggest you commit, I will review post-commit. > > Committed it r16508, though it has changed slightly according to my > experiments with swap-biasing (e.g. shows number of cache/store writes > as well). Were should this appear? Does it take a while to show up? I have r16509 running and am not seeing anything new (yet). > From what I have discovered (or theorized) thus far, using the > average location of the entire store to bias against is way too much > of an anchor. This is good, because it takes up way to much memory to > remember such a running average anyway. If we end up implementing such > a bias, in the end, it will likely just take into account the last few > inserts or successes (a small constant amount). In this way, the > pressure of your peers can still pull you into a new location > (dragging the weight of the last few inserts with you; which will be > updated). Keeping the average key location and standard deviation just requires a few counters along hooks to update them when adding or deleting the store... It also requires a single pass to get the initial numbers. Should not be expensive except for that first pass. I agree that using the average would be very restrictive. Average plus or minus standard deviation would give a much better 'limit'. I suspect that It would take a fair length of time for it to start limiting at all (e.g. 2 x standard deviation would be greater than 1). > Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), > after running with a recently-stored-location bias, I must have pulled > my peers closer to me as well, as now even with the patch off, I do > not see the storeDistance go nearly so high (staying around 0.004). > Or, since the network has no anchor, maybe I've pressured the whole > wheel to stop turning. For at least my node; this (valuing the store > as one peer) has not shown any increase in store hit rates, biasing > against the succeeding requests, however, has in the past raised it to > about 1%; but I would be even more concerning with implementing that. > > With these stats I have noticed what may be an odd constant-location- > shift between what the node is requested to insert and retrieve, like > the network does not look for data in the exact same place it stores > it?!?!?! Suspect, but still can't confirm anything yet. Ed
Re: [freenet-dev] Why Freenet is so SLOW! / Finding data
On December 12, 2007, Robert Hailey wrote: > > On Dec 11, 2007, at 5:59 PM, Matthew Toseland wrote: > > Ok that could be interesting. Although ideally we'd have a > > circular-keyspace-aware averager. > >> ... > > Ok. I suggest you commit, I will review post-commit. > > Committed it r16508, though it has changed slightly according to my > experiments with swap-biasing (e.g. shows number of cache/store writes > as well). Were should this appear? Does it take a while to show up? I have r16509 running and am not seeing anything new (yet). > From what I have discovered (or theorized) thus far, using the > average location of the entire store to bias against is way too much > of an anchor. This is good, because it takes up way to much memory to > remember such a running average anyway. If we end up implementing such > a bias, in the end, it will likely just take into account the last few > inserts or successes (a small constant amount). In this way, the > pressure of your peers can still pull you into a new location > (dragging the weight of the last few inserts with you; which will be > updated). Keeping the average key location and standard deviation just requires a few counters along hooks to update them when adding or deleting the store... It also requires a single pass to get the initial numbers. Should not be expensive except for that first pass. I agree that using the average would be very restrictive. Average plus or minus standard deviation would give a much better 'limit'. I suspect that It would take a fair length of time for it to start limiting at all (e.g. 2 x standard deviation would be greater than 1). > Whereas previously I have seen the 'storeDistance' to be 0.065 (IIRC), > after running with a recently-stored-location bias, I must have pulled > my peers closer to me as well, as now even with the patch off, I do > not see the storeDistance go nearly so high (staying around 0.004). > Or, since the network has no anchor, maybe I've pressured the whole > wheel to stop turning. For at least my node; this (valuing the store > as one peer) has not shown any increase in store hit rates, biasing > against the succeeding requests, however, has in the past raised it to > about 1%; but I would be even more concerning with implementing that. > > With these stats I have noticed what may be an odd constant-location- > shift between what the node is requested to insert and retrieve, like > the network does not look for data in the exact same place it stores > it?!?!?! Suspect, but still can't confirm anything yet. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Why Freenet is so SLOW! / r16372
Hi, Some feedback on this. Before updating to testing I was seeing about 10% of my Preemptive Rejection Reasons attributed to freeHeapPercentThreshold now its: 101639Output bandwidth liability 1216 > On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > > > On Friday 30 November 2007 23:03, Robert Hailey wrote: > >> > >> A much-more aggressive patch (which makes all RequestHandler sends > >> Async except the 'last') shows could be 12x; but am concerned over > >> the > >> byte-counter side effects with which I am not familiar. > > > > [...] > > - To get the correct byte counts. > > This is really important, because it underlies the major load limiting > > systems. However it is only important for terminal messages: we can > > reasonably assume the rest will be piggybacked if necessary. > > > > In RequestHandler, most of the sendSync()'s are terminal, and > > therefore must > > remain. The one that is the most interesting is the one you > > mentioned above: > > sending the Accepted message. > > > Well... I found the time to get the "more aggressive" patch working. > > The general idea is that (after the final request) because the ~only~ > reason to keep the thread waiting on the final transmission is the > byte count, to delegate that to the 'sent()' callback thread. > > On my machine (which was showing pre-emptive reject cause as > ">threadLimit", because of the high bandwidth ceiling I have set), I > saw a increase in node bandwidth (in & out), and saw a new pre-emptive > rejection cause "insufficient output bandwidth". Realizing then that > the bandwidth ceiling (being so high) was the cause for my node > behaving differently than others, I changed my config to less-than > that observed value, and "output bandwidth liability" became > predominate (as Matthew said), and re-ran the tests with the more- > expected results: > > For my machine, it appears to save 10 to 49 threads (the JVM-supplied > value), varying largely presumably due to variances in the length of > send queues. Also, I've seen quite high numbers for 'pooled threads > awaiting work'. > > After cursory inspection, I suspect that the other related handlers > (Insert/SSK) would not see the same benefit as RequestHandler; and are > not modified. > > It may not therefore show a significant change to the common user (as > the ack appears to have), but (at least for me, i.e. ">threadLimit" > reject) it appears to give the user the ability to use more bandwidth, > or perhaps to have longer send-queues (service slower peers). > > Question: Is it a bug that messages back to the source are not counted > as the liability for handling a request, or is this intentional? (e.g. > sendSync(dnf, null); ) > > Question: Should there be a note in the config page about bandwidth > limits effect on threads? It seems the more bandwidth you allocate, > the more requests you serve, the more threads freenet uses. > > Commited in r16372 > > -- > Robert Hailey > > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >
Re: [freenet-dev] Why Freenet is so SLOW! / r16372
Hi, Some feedback on this. Before updating to testing I was seeing about 10% of my Preemptive Rejection Reasons attributed to freeHeapPercentThreshold now its: 101639Output bandwidth liability 1216 > On Dec 1, 2007, at 6:02 PM, Matthew Toseland wrote: > > > On Friday 30 November 2007 23:03, Robert Hailey wrote: > >> > >> A much-more aggressive patch (which makes all RequestHandler sends > >> Async except the 'last') shows could be 12x; but am concerned over > >> the > >> byte-counter side effects with which I am not familiar. > > > > [...] > > - To get the correct byte counts. > > This is really important, because it underlies the major load limiting > > systems. However it is only important for terminal messages: we can > > reasonably assume the rest will be piggybacked if necessary. > > > > In RequestHandler, most of the sendSync()'s are terminal, and > > therefore must > > remain. The one that is the most interesting is the one you > > mentioned above: > > sending the Accepted message. > > > Well... I found the time to get the "more aggressive" patch working. > > The general idea is that (after the final request) because the ~only~ > reason to keep the thread waiting on the final transmission is the > byte count, to delegate that to the 'sent()' callback thread. > > On my machine (which was showing pre-emptive reject cause as > ">threadLimit", because of the high bandwidth ceiling I have set), I > saw a increase in node bandwidth (in & out), and saw a new pre-emptive > rejection cause "insufficient output bandwidth". Realizing then that > the bandwidth ceiling (being so high) was the cause for my node > behaving differently than others, I changed my config to less-than > that observed value, and "output bandwidth liability" became > predominate (as Matthew said), and re-ran the tests with the more- > expected results: > > For my machine, it appears to save 10 to 49 threads (the JVM-supplied > value), varying largely presumably due to variances in the length of > send queues. Also, I've seen quite high numbers for 'pooled threads > awaiting work'. > > After cursory inspection, I suspect that the other related handlers > (Insert/SSK) would not see the same benefit as RequestHandler; and are > not modified. > > It may not therefore show a significant change to the common user (as > the ack appears to have), but (at least for me, i.e. ">threadLimit" > reject) it appears to give the user the ability to use more bandwidth, > or perhaps to have longer send-queues (service slower peers). > > Question: Is it a bug that messages back to the source are not counted > as the liability for handling a request, or is this intentional? (e.g. > sendSync(dnf, null); ) > > Question: Should there be a note in the config page about bandwidth > limits effect on threads? It seems the more bandwidth you allocate, > the more requests you serve, the more threads freenet uses. > > Commited in r16372 > > -- > Robert Hailey > > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Why Freenet is so SLOW! / Finding data
On December 7, 2007, Robert Hailey wrote: > > One thing that may be happening that somewant offsets this problem ? > > could be > > clustering of locations. ?e.g. even though my nodes location changes ? > > often it could > > be that it favours some subsets of the total location space. ?It ? > > would be worthing > > adding a metric to track how long a node spends in a given area of ? > > the location > > space. > > Hmm... that may be implementable as a running average of (location ? > over time). i.e. every fixed interval (hour? / 5-minute) average one ? > 'location' into the running average. Which amounts to a location- > shifting 'velocity' measurement, or 'average-location'. > I was thinking more along the lines of a histogram like 0.5 used to use to show specialization. Ed
[freenet-dev] Why Freenet is so SLOW! / Finding data
On December 7, 2007, Robert Hailey wrote: > > On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > > > Hi, > > > > I think freenet is slow because its having problems finding data. > > The hit rates in 0.5 ranged around 5%. > > in 0.7 I see 2% on cache and 0% on store. We just are not finding > > data. My location shifts so fast that > > other nodes have no idea what is in my node's 8G store. I strongly > > suggest that we experiment with letting > > the store contents limit how far from the current location we can > > swap. This could easily be done with > > a tunable such that the effect could be turned off or could be made > > very strict. We could even use > > a feedback loop to find an optimal value with the store/cache hit > > rate being used to control just how strict > > we are. The idea would be to minimize the swapping limitation while > > maximizing the hit rate which > > would decrease the path length and hence the traffic. > > > > In my case the node usually uses a good chunk (80-90%) of the > > bandwidth I give to it. I think the > > bandwidth limits work. I would agree that backout is probably not > > needed and think it would be a > > good idea to try with it turn off - again this would improve routing > > which I think is the core of 0.7 > > performance problems. > > > > Thanks > > Ed > > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible). This is an interesting way to tackle the problem. Might I suggest that instead of doing this for data that we are about to discard we do it select a key from the DS at random and reinsert if its far from our current location. One thing that may be happening that somewant offsets this problem could be clustering of locations. e.g. even though my nodes location changes often it could be that it favours some subsets of the total location space. It would be worthing adding a metric to track how long a node spends in a given area of the location space. Ed
Re: [freenet-dev] Why Freenet is so SLOW! / Finding data
On December 7, 2007, Robert Hailey wrote: > > One thing that may be happening that somewant offsets this problem > > could be > > clustering of locations. e.g. even though my nodes location changes > > often it could > > be that it favours some subsets of the total location space. It > > would be worthing > > adding a metric to track how long a node spends in a given area of > > the location > > space. > > Hmm... that may be implementable as a running average of (location > over time). i.e. every fixed interval (hour? / 5-minute) average one > 'location' into the running average. Which amounts to a location- > shifting 'velocity' measurement, or 'average-location'. > I was thinking more along the lines of a histogram like 0.5 used to use to show specialization. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Why Freenet is so SLOW! / Finding data
On December 7, 2007, Robert Hailey wrote: > > On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > > > Hi, > > > > I think freenet is slow because its having problems finding data. > > The hit rates in 0.5 ranged around 5%. > > in 0.7 I see 2% on cache and 0% on store. We just are not finding > > data. My location shifts so fast that > > other nodes have no idea what is in my node's 8G store. I strongly > > suggest that we experiment with letting > > the store contents limit how far from the current location we can > > swap. This could easily be done with > > a tunable such that the effect could be turned off or could be made > > very strict. We could even use > > a feedback loop to find an optimal value with the store/cache hit > > rate being used to control just how strict > > we are. The idea would be to minimize the swapping limitation while > > maximizing the hit rate which > > would decrease the path length and hence the traffic. > > > > In my case the node usually uses a good chunk (80-90%) of the > > bandwidth I give to it. I think the > > bandwidth limits work. I would agree that backout is probably not > > needed and think it would be a > > good idea to try with it turn off - again this would improve routing > > which I think is the core of 0.7 > > performance problems. > > > > Thanks > > Ed > > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible). This is an interesting way to tackle the problem. Might I suggest that instead of doing this for data that we are about to discard we do it select a key from the DS at random and reinsert if its far from our current location. One thing that may be happening that somewant offsets this problem could be clustering of locations. e.g. even though my nodes location changes often it could be that it favours some subsets of the total location space. It would be worthing adding a metric to track how long a node spends in a given area of the location space. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Why Freenet is so SLOW! Proposal for new load management
Hi, I think freenet is slow because its having problems finding data. The hit rates in 0.5 ranged around 5%. in 0.7 I see 2% on cache and 0% on store. We just are not finding data. My location shifts so fast that other nodes have no idea what is in my node's 8G store. I strongly suggest that we experiment with letting the store contents limit how far from the current location we can swap. This could easily be done with a tunable such that the effect could be turned off or could be made very strict. We could even use a feedback loop to find an optimal value with the store/cache hit rate being used to control just how strict we are. The idea would be to minimize the swapping limitation while maximizing the hit rate which would decrease the path length and hence the traffic. In my case the node usually uses a good chunk (80-90%) of the bandwidth I give to it. I think the bandwidth limits work. I would agree that backout is probably not needed and think it would be a good idea to try with it turn off - again this would improve routing which I think is the core of 0.7 performance problems. Thanks Ed On November 30, 2007, Matthew Toseland wrote: > On Friday 30 November 2007 08:54, Florent Daigni?re wrote: > > > Obviously I expect the above will evolve over time. However, IMHO it > > > would > be > > > significantly better at matching demand to supply than the current > algorithm > > > while not sacrificing routing, and would therefore be a major improvement > in > > > performance, which is one of the most important problems Freenet needs to > > > deal with. > > > > You proposal is far away from tit-for-tat ... How do you prevent > > leachers from forgetting to send a "I want to process requests" flag ? > > Tit-for-tat can be layered on top of any sensible load management scheme (not > easily on the current system though). It's not something we need to deal with > right now. >
Re: [freenet-dev] Why Freenet is so SLOW! Proposal for new load management
Hi, I think freenet is slow because its having problems finding data. The hit rates in 0.5 ranged around 5%. in 0.7 I see 2% on cache and 0% on store. We just are not finding data. My location shifts so fast that other nodes have no idea what is in my node's 8G store. I strongly suggest that we experiment with letting the store contents limit how far from the current location we can swap. This could easily be done with a tunable such that the effect could be turned off or could be made very strict. We could even use a feedback loop to find an optimal value with the store/cache hit rate being used to control just how strict we are. The idea would be to minimize the swapping limitation while maximizing the hit rate which would decrease the path length and hence the traffic. In my case the node usually uses a good chunk (80-90%) of the bandwidth I give to it. I think the bandwidth limits work. I would agree that backout is probably not needed and think it would be a good idea to try with it turn off - again this would improve routing which I think is the core of 0.7 performance problems. Thanks Ed On November 30, 2007, Matthew Toseland wrote: > On Friday 30 November 2007 08:54, Florent Daignière wrote: > > > Obviously I expect the above will evolve over time. However, IMHO it > > > would > be > > > significantly better at matching demand to supply than the current > algorithm > > > while not sacrificing routing, and would therefore be a major improvement > in > > > performance, which is one of the most important problems Freenet needs to > > > deal with. > > > > You proposal is far away from tit-for-tat ... How do you prevent > > leachers from forgetting to send a "I want to process requests" flag ? > > Tit-for-tat can be layered on top of any sensible load management scheme (not > easily on the current system though). It's not something we need to deal with > right now. > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] [freenet-cvs] r15647 - in trunk/freenet/src/freenet: clients/http l10n pluginmanager
On October 30, 2007, Florent Daigni?re wrote: > * David ???Bombe??? Roden [2007-10-30 10:39:55]: > > > On Tue, 2007-10-30 at 10:21 +0100, Florent Daigni?re wrote: > > > > > I'm not convinced by that patch... I mean, the original idea was that > > > "auto-reload-from-emu" was a feature to help debugging, not one to be > > > used by everyone. > > > > Okay, so do we still need it at all? > > I'm not sure :) > > > I have no problem removing the > > feature altogether, although we should have some way of reloading a > > plugin to update to a newer version. A button in the web interface to > > trigger the reload manually would do, though. > > Yeah, having an "update" button is sensible. Is there some reason we cannot have the plugins on freenet itself. IMHO this would be an optimal solution. Ed
Re: [freenet-dev] [freenet-cvs] r15647 - in trunk/fre enet/src/freenet: clients/http l10n pluginmanager
On October 30, 2007, Florent Daignière wrote: > * David ???Bombe??? Roden <[EMAIL PROTECTED]> [2007-10-30 10:39:55]: > > > On Tue, 2007-10-30 at 10:21 +0100, Florent Daignière wrote: > > > > > I'm not convinced by that patch... I mean, the original idea was that > > > "auto-reload-from-emu" was a feature to help debugging, not one to be > > > used by everyone. > > > > Okay, so do we still need it at all? > > I'm not sure :) > > > I have no problem removing the > > feature altogether, although we should have some way of reloading a > > plugin to update to a newer version. A button in the web interface to > > trigger the reload manually would do, though. > > Yeah, having an "update" button is sensible. Is there some reason we cannot have the plugins on freenet itself. IMHO this would be an optimal solution. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] High Speed Links
On Monday 21 May 2007 06:25, Florent Daigni?re wrote: > * Volodya [2007-05-21 11:16:13]: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > jarvil at gmail.com wrote: > > > Hello, > > > > > > I proposed an advanced option on the peers (friends) page for high speed > > > links. > > > > > > In the present code, if you increase the outgoing bandwith the peers > > > inevitably end up in backed off mode as the ones who cannot deal with > > > the higher throughput backoff the incoming connection. I dont know the > > > method used for Backed Off values but some basic math tells me that > > > mode nodes can only receive 10K/s for each connection. Let me explain. > > > > > > If you have one node on default setting of 15K/s outgoing bandwith and > > > 60K/s incoming (4xoutgoing). If a user has 6 connections you have a > > > max of 10K incoming for each connection. 10K/s is double the speed of > > > a modem connection and hardly broadband speed. IMHO This severely > > > limits the speed of freenet. > > > > > > What I would like to see is the ability to set individual bandwith on > > > peers OR designate a peer as high speed which excludes it from the > > > bandwith management on the normal peers. This would send a message to > > > the other peer requesting a high speed link which would appear on > > > their peer listing as request for high speed and the speed requested. > > > If they agree then the link operates at the new speed sending data at > > > the maximum speed specified until there is no more data to send. > > > > > > Regards > > > > > > Jarvil > > > > This will also help those of us who are running more than one node and they > > are on the > > same LAN. > > > > I have tried to explain it on irc: it doesn't and won't help... and > probably will f*ck up the load-balancing. > > Even if we have "faster than other" links, the node doesn't take that > into account when it routes requests : the "fast link" isn't more likely > to be used than any other one because of how routing works. We route to > what we think to be the shortest path not the local-fastest one (to > avoid missrouting) This is not strictly true. We try to use the best route. If it happens to be backed off we skip it unless all nodes are backed off. If a link is faster it should be backed off less... If the nodes have agreed on a faster link, load balancing should be able to take it into account. > What would help is a way to "share" datastores amongst different trusted > nodes. > > NextGen$ >
Re: [freenet-dev] High Speed Links
On Monday 21 May 2007 06:25, Florent Daignière wrote: > * Volodya <[EMAIL PROTECTED]> [2007-05-21 11:16:13]: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > [EMAIL PROTECTED] wrote: > > > Hello, > > > > > > I proposed an advanced option on the peers (friends) page for high speed > > > links. > > > > > > In the present code, if you increase the outgoing bandwith the peers > > > inevitably end up in backed off mode as the ones who cannot deal with > > > the higher throughput backoff the incoming connection. I dont know the > > > method used for Backed Off values but some basic math tells me that > > > mode nodes can only receive 10K/s for each connection. Let me explain. > > > > > > If you have one node on default setting of 15K/s outgoing bandwith and > > > 60K/s incoming (4xoutgoing). If a user has 6 connections you have a > > > max of 10K incoming for each connection. 10K/s is double the speed of > > > a modem connection and hardly broadband speed. IMHO This severely > > > limits the speed of freenet. > > > > > > What I would like to see is the ability to set individual bandwith on > > > peers OR designate a peer as high speed which excludes it from the > > > bandwith management on the normal peers. This would send a message to > > > the other peer requesting a high speed link which would appear on > > > their peer listing as request for high speed and the speed requested. > > > If they agree then the link operates at the new speed sending data at > > > the maximum speed specified until there is no more data to send. > > > > > > Regards > > > > > > Jarvil > > > > This will also help those of us who are running more than one node and they > > are on the > > same LAN. > > > > I have tried to explain it on irc: it doesn't and won't help... and > probably will f*ck up the load-balancing. > > Even if we have "faster than other" links, the node doesn't take that > into account when it routes requests : the "fast link" isn't more likely > to be used than any other one because of how routing works. We route to > what we think to be the shortest path not the local-fastest one (to > avoid missrouting) This is not strictly true. We try to use the best route. If it happens to be backed off we skip it unless all nodes are backed off. If a link is faster it should be backed off less... If the nodes have agreed on a faster link, load balancing should be able to take it into account. > What would help is a way to "share" datastores amongst different trusted > nodes. > > NextGen$ > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Easier reference swapping
Hi, Why not take the simple way? The last time I had to reset my node I ran a swapbot that used irc to get some refs. If I remember correctly there is a project that creates an anonymous irc network. Why not have nodes create or connect to this network, at least some of the time, and use a standardized swapbot to get references. If a person running a node is willing to be less anonymous, we could even use freenode... Comments? Ed On Tuesday 06 March 2007 05:37, Florent Daigni?re wrote: > * Volodya [2007-03-06 05:44:35]: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Matthew Toseland wrote: > > > I don't understand why a password and IP address is easier than a > > > one-time reference. I suppose it has the advantage of being able to > > > write it down - but for it to be secure it would need to be a one-time > > > password; you'd need to generate a new one every time ... > > > > > > Hmmm. Maybe we should provide both mechanisms? > > > > One thing that might be done is not having an increadibly secure password > > protection (just > > secure enough), but when somebody adds themselves via password they get > > added in the > > disabled mode, then the person tells you "It asks me to tell you to enable > > me" and you do > > so. If somebody intersepts the password in between and uses it, the second > > person will get > > a request to inform you that password has been used already, so you just go > > and delete the > > bugger who used it. > > > > In other words: Bring security away from the machine and to the person. > > > > - Volodya > > > > So far a node is *passive* and won't react upon reception of any unknown data. > If we want to tell the user that the password has already been used, we > would need to change that behaviour :/ > > I'm not sure it's a good idea. > > NextGen$ > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >
Re: [freenet-dev] Easier reference swapping
Hi, Why not take the simple way? The last time I had to reset my node I ran a swapbot that used irc to get some refs. If I remember correctly there is a project that creates an anonymous irc network. Why not have nodes create or connect to this network, at least some of the time, and use a standardized swapbot to get references. If a person running a node is willing to be less anonymous, we could even use freenode... Comments? Ed On Tuesday 06 March 2007 05:37, Florent Daignière wrote: > * Volodya <[EMAIL PROTECTED]> [2007-03-06 05:44:35]: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > Matthew Toseland wrote: > > > I don't understand why a password and IP address is easier than a > > > one-time reference. I suppose it has the advantage of being able to > > > write it down - but for it to be secure it would need to be a one-time > > > password; you'd need to generate a new one every time ... > > > > > > Hmmm. Maybe we should provide both mechanisms? > > > > One thing that might be done is not having an increadibly secure password > > protection (just > > secure enough), but when somebody adds themselves via password they get > > added in the > > disabled mode, then the person tells you "It asks me to tell you to enable > > me" and you do > > so. If somebody intersepts the password in between and uses it, the second > > person will get > > a request to inform you that password has been used already, so you just go > > and delete the > > bugger who used it. > > > > In other words: Bring security away from the machine and to the person. > > > > - Volodya > > > > So far a node is *passive* and won't react upon reception of any unknown data. > If we want to tell the user that the password has already been used, we > would need to change that behaviour :/ > > I'm not sure it's a good idea. > > NextGen$ > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] ip v6 when no ip v6 is present
On Tuesday 23 January 2007 07:39, Ed Tomlinson wrote: If I compile in IPv6 the error changes to: Jan 23, 2007 13:44:36:838 (freenet.node.Node, PacketSender thread for 0, NORMAL): Connected: 0 Routing Backed Off: 0 Too New: 0 Too Old: 0 Disconnected: 10 Never Connected: 0 Disabled: 0 Bursting: 0 Listening: 0 Listen Only: 0 Jan 23, 2007 13:44:39:458 (freenet.io.comm.UdpSocketManager, PacketSender thread for 0, NORMAL): Error while sending packet to IP v6 address: 2002:53e9:61d2:0:0:0:0:1%2:3478: java.io.IOException: Network is unreachable java.io.IOException: Network is unreachable at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) at java.net.DatagramSocket.send(DatagramSocket.java:624) at freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) at freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) at freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) at freenet.node.PacketSender.realRun(PacketSender.java:293) at freenet.node.PacketSender.run(PacketSender.java:123) at java.lang.Thread.run(Thread.java:797) Jan 23, 2007 13:44:41:878 (freenet.node.Node, PacketSender thread for 0, NORMAL): Connected: 0 Routing Backed Off: 0 Too New: 0 Too Old: 0 Disconnected: 10 Never Connected: 0 Disabled: 0 Bursting: 0 Listening: 0 Listen Only: 0 which repeats for ever and does not connect to any peers (1011 worked as of 12 hours ago with 6 long standing peers) Thanks Ed > Hi, > > For some reason my node thinks it should be attempting connections using ip > v6. There is not v6 built into my kernel. > What happens is: > > an 23, 2007 12:37:19:587 (freenet.io.comm.UdpSocketManager, PacketSender > thread for 0, NORMAL): Error while sending packet to IP > v6 address: 2002::61d2:0:0:0:0:1%2:: java.net.SocketException: > Protocol family unavailable > java.net.SocketException: Protocol family unavailable > at java.net.PlainDatagramSocketImpl.send(Native Method) > at > java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) > at java.net.DatagramSocket.send(DatagramSocket.java:624) > at > freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) > at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) > at > freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) > at > freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) > at > freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) > at > freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) > at freenet.node.PacketSender.realRun(PacketSender.java:293) > at freenet.node.PacketSender.run(PacketSender.java:123) > at java.lang.Thread.run(Thread.java:797) > > is repeated every couple of seconds. This is the first node in my peers file. > > This makes my node rather useless... > > How does freenet decide to try v6? How does not tell it not to use v6 ever? > In any case it should not loop. > > Thanks > Ed > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >
[freenet-dev] ip v6 when no ip v6 is present
Hi, For some reason my node thinks it should be attempting connections using ip v6. There is not v6 built into my kernel. What happens is: an 23, 2007 12:37:19:587 (freenet.io.comm.UdpSocketManager, PacketSender thread for 0, NORMAL): Error while sending packet to IP v6 address: 2002::61d2:0:0:0:0:1%2:: java.net.SocketException: Protocol family unavailable java.net.SocketException: Protocol family unavailable at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) at java.net.DatagramSocket.send(DatagramSocket.java:624) at freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) at freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) at freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) at freenet.node.PacketSender.realRun(PacketSender.java:293) at freenet.node.PacketSender.run(PacketSender.java:123) at java.lang.Thread.run(Thread.java:797) is repeated every couple of seconds. This is the first node in my peers file. This makes my node rather useless... How does freenet decide to try v6? How does not tell it not to use v6 ever? In any case it should not loop. Thanks Ed
Re: [freenet-dev] ip v6 when no ip v6 is present
On Tuesday 23 January 2007 07:39, Ed Tomlinson wrote: If I compile in IPv6 the error changes to: Jan 23, 2007 13:44:36:838 (freenet.node.Node, PacketSender thread for 0, NORMAL): Connected: 0 Routing Backed Off: 0 Too New: 0 Too Old: 0 Disconnected: 10 Never Connected: 0 Disabled: 0 Bursting: 0 Listening: 0 Listen Only: 0 Jan 23, 2007 13:44:39:458 (freenet.io.comm.UdpSocketManager, PacketSender thread for 0, NORMAL): Error while sending packet to IP v6 address: 2002:53e9:61d2:0:0:0:0:1%2:3478: java.io.IOException: Network is unreachable java.io.IOException: Network is unreachable at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) at java.net.DatagramSocket.send(DatagramSocket.java:624) at freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) at freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) at freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) at freenet.node.PacketSender.realRun(PacketSender.java:293) at freenet.node.PacketSender.run(PacketSender.java:123) at java.lang.Thread.run(Thread.java:797) Jan 23, 2007 13:44:41:878 (freenet.node.Node, PacketSender thread for 0, NORMAL): Connected: 0 Routing Backed Off: 0 Too New: 0 Too Old: 0 Disconnected: 10 Never Connected: 0 Disabled: 0 Bursting: 0 Listening: 0 Listen Only: 0 which repeats for ever and does not connect to any peers (1011 worked as of 12 hours ago with 6 long standing peers) Thanks Ed > Hi, > > For some reason my node thinks it should be attempting connections using ip > v6. There is not v6 built into my kernel. > What happens is: > > an 23, 2007 12:37:19:587 (freenet.io.comm.UdpSocketManager, PacketSender > thread for 0, NORMAL): Error while sending packet to IP > v6 address: 2002::61d2:0:0:0:0:1%2:: java.net.SocketException: > Protocol family unavailable > java.net.SocketException: Protocol family unavailable > at java.net.PlainDatagramSocketImpl.send(Native Method) > at > java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) > at java.net.DatagramSocket.send(DatagramSocket.java:624) > at > freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) > at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) > at > freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) > at > freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) > at > freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) > at > freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) > at freenet.node.PacketSender.realRun(PacketSender.java:293) > at freenet.node.PacketSender.run(PacketSender.java:123) > at java.lang.Thread.run(Thread.java:797) > > is repeated every couple of seconds. This is the first node in my peers file. > > This makes my node rather useless... > > How does freenet decide to try v6? How does not tell it not to use v6 ever? > In any case it should not loop. > > Thanks > Ed > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] ip v6 when no ip v6 is present
Hi, For some reason my node thinks it should be attempting connections using ip v6. There is not v6 built into my kernel. What happens is: an 23, 2007 12:37:19:587 (freenet.io.comm.UdpSocketManager, PacketSender thread for 0, NORMAL): Error while sending packet to IP v6 address: 2002::61d2:0:0:0:0:1%2:: java.net.SocketException: Protocol family unavailable java.net.SocketException: Protocol family unavailable at java.net.PlainDatagramSocketImpl.send(Native Method) at java.net.PlainDatagramSocketImpl.send(PlainDatagramSocketImpl.java:134) at java.net.DatagramSocket.send(DatagramSocket.java:624) at freenet.io.comm.UdpSocketManager.sendPacket(UdpSocketManager.java:613) at freenet.node.FNPPacketMangler.sendPacket(FNPPacketMangler.java:508) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:496) at freenet.node.FNPPacketMangler.sendAuthPacket(FNPPacketMangler.java:461) at freenet.node.FNPPacketMangler.sendFirstHalfDHPacket(FNPPacketMangler.java:439) at freenet.node.FNPPacketMangler.sendHandshake(FNPPacketMangler.java:1506) at freenet.node.PacketSender.realRun(PacketSender.java:293) at freenet.node.PacketSender.run(PacketSender.java:123) at java.lang.Thread.run(Thread.java:797) is repeated every couple of seconds. This is the first node in my peers file. This makes my node rather useless... How does freenet decide to try v6? How does not tell it not to use v6 ever? In any case it should not loop. Thanks Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Re: [freenet-cvs] r11047 - trunk/freenet/src/freenet/node/updater
Hi,
As prevously discussed, can we please make this a config option?
Thanks
Ed
On Friday 24 November 2006 19:17, nextgens at emu.freenetproject.org wrote:
> Author: nextgens
> Date: 2006-11-25 00:17:11 + (Sat, 25 Nov 2006)
> New Revision: 11047
>
> Modified:
>trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> Log:
> Don't try to start the updater if we aren't running under the wrapper.
>
> Modified: trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> ===
> --- trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> 2006-11-24 23:12:44 UTC (rev 11046)
> +++ trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> 2006-11-25 00:17:11 UTC (rev 11047)
> @@ -169,6 +169,10 @@
>*/
> void enable(boolean enable) throws InvalidConfigValueException {
> logMINOR = Logger.shouldLog(Logger.MINOR, this);
> + if(!node.isUsingWrapper()){
> + Logger.normal(this, "Don't try to start the updater as
> we are not running under the wrapper.");
> + return;
> + }
> NodeUpdater main = null, ext = null;
> synchronized(this) {
> boolean enabled = (mainUpdater != null &&
> mainUpdater.isRunning());
>
> ___
> cvs mailing list
> cvs at freenetproject.org
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs
>
>
[freenet-dev] Re: [freenet-cvs] r11047 - trunk/freenet/src/freenet/node/updater
Hi,
As prevously discussed, can we please make this a config option?
Thanks
Ed
On Friday 24 November 2006 19:17, [EMAIL PROTECTED] wrote:
> Author: nextgens
> Date: 2006-11-25 00:17:11 + (Sat, 25 Nov 2006)
> New Revision: 11047
>
> Modified:
>trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> Log:
> Don't try to start the updater if we aren't running under the wrapper.
>
> Modified: trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> ===
> --- trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> 2006-11-24 23:12:44 UTC (rev 11046)
> +++ trunk/freenet/src/freenet/node/updater/NodeUpdaterManager.java
> 2006-11-25 00:17:11 UTC (rev 11047)
> @@ -169,6 +169,10 @@
>*/
> void enable(boolean enable) throws InvalidConfigValueException {
> logMINOR = Logger.shouldLog(Logger.MINOR, this);
> + if(!node.isUsingWrapper()){
> + Logger.normal(this, "Don't try to start the updater as
> we are not running under the wrapper.");
> + return;
> + }
> NodeUpdater main = null, ext = null;
> synchronized(this) {
> boolean enabled = (mainUpdater != null &&
> mainUpdater.isRunning());
>
> ___
> cvs mailing list
> cvs@freenetproject.org
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/cvs
>
>
___
Devl mailing list
Devl@freenetproject.org
http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Part Time Node
On Wednesday 01 November 2006 20:39, toad wrote: > In theory yes. In practice no. The reason is this: > - You will probably get your peers from #freenet-refs > - Most people there seem to drop nodes when they are offline for a few > hours. Actually I think it can work just fine. When you exchange your refs tell your friend you are up 16hrs per day. Alternately put 16hrDay in your node name. If you communicate what you are doing is strongly suspect that people will not drop you too quickly. I know that to get deleted here you need to be down for over 4 weeks. Not everyone drops after a down of an hour or two... Ed > If your peers are understanding - perhaps because they are true darknet > peers (i.e. nodes run by people you actually know, for example), then > yes, it can work, modulo timezone issues etc. > > :< > > On Thu, Nov 02, 2006 at 12:25:31AM +0100, G.Roethlin wrote: > > In the old 0.3 and 0.5 days I used to have a part time node, which > > was active about 16 hours a day. It worked reasonably well (my node > > was bussy scratching at it's output bandwith limit). Could this work > > in 0.7 freenet too? > > > > cb >
Re: [freenet-dev] Part Time Node
On Wednesday 01 November 2006 20:39, toad wrote: > In theory yes. In practice no. The reason is this: > - You will probably get your peers from #freenet-refs > - Most people there seem to drop nodes when they are offline for a few > hours. Actually I think it can work just fine. When you exchange your refs tell your friend you are up 16hrs per day. Alternately put 16hrDay in your node name. If you communicate what you are doing is strongly suspect that people will not drop you too quickly. I know that to get deleted here you need to be down for over 4 weeks. Not everyone drops after a down of an hour or two... Ed > If your peers are understanding - perhaps because they are true darknet > peers (i.e. nodes run by people you actually know, for example), then > yes, it can work, modulo timezone issues etc. > > :< > > On Thu, Nov 02, 2006 at 12:25:31AM +0100, G.Roethlin wrote: > > In the old 0.3 and 0.5 days I used to have a part time node, which > > was active about 16 hours a day. It worked reasonably well (my node > > was bussy scratching at it's output bandwith limit). Could this work > > in 0.7 freenet too? > > > > cb > ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Bandwidth limits...
On Monday 16 October 2006 08:04, Florent Daigni?re wrote: > * Ed Tomlinson [2006-10-16 08:01:51]: > > > Hi, > > > > At toad's prompting I tried an experiment. The results were interesting. > > I changed by output bandwidth limit to 100k/s. The result. NO change > > in the output rates. It still averages about 5k/s with peaks of about 20. > > > > Think there is something fishy with our bandwidth limitations. > > > > Ed > > Or our load-balancing scheme is too conservative... btw, increasing your > bandwidth limit doesn't mean that your peers will be able to cope with > the new amount of data nor that they will be willing to send your more. Problem, as I see it, is that we base output rates on the rates that we recieve data from nodes. Most do not have symetric bandwidth (I have 20-30x more input bandwidth). In other words, even though I limit my output bandwidth, I can handle much more input 20-30x more input... Freenet does not seem to understand this. Ed
Re: [freenet-dev] Bandwidth limits...
On Monday 16 October 2006 08:04, Florent Daignière wrote: > * Ed Tomlinson <[EMAIL PROTECTED]> [2006-10-16 08:01:51]: > > > Hi, > > > > At toad's prompting I tried an experiment. The results were interesting. > > I changed by output bandwidth limit to 100k/s. The result. NO change > > in the output rates. It still averages about 5k/s with peaks of about 20. > > > > Think there is something fishy with our bandwidth limitations. > > > > Ed > > Or our load-balancing scheme is too conservative... btw, increasing your > bandwidth limit doesn't mean that your peers will be able to cope with > the new amount of data nor that they will be willing to send your more. Problem, as I see it, is that we base output rates on the rates that we recieve data from nodes. Most do not have symetric bandwidth (I have 20-30x more input bandwidth). In other words, even though I limit my output bandwidth, I can handle much more input 20-30x more input... Freenet does not seem to understand this. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Bandwidth limits...
Hi, At toad's prompting I tried an experiment. The results were interesting. I changed by output bandwidth limit to 100k/s. The result. NO change in the output rates. It still averages about 5k/s with peaks of about 20. Think there is something fishy with our bandwidth limitations. Ed
[freenet-dev] Bandwidth limits...
Hi, At toad's prompting I tried an experiment. The results were interesting. I changed by output bandwidth limit to 100k/s. The result. NO change in the output rates. It still averages about 5k/s with peaks of about 20. Think there is something fishy with our bandwidth limitations. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Re: [freenet-cvs] r10661 - trunk/freenet/src/freenet/clients/http
Hi, What happens if one just happens to find out the node that cannot be removed is obsolete, compromised or untrustworthy? A warning might be a better idea here? Ed On Saturday 14 October 2006 07:57, nextgens at freenetproject.org wrote: > Author: nextgens > Date: 2006-10-14 11:57:08 + (Sat, 14 Oct 2006) > New Revision: 10661 > > Modified: >trunk/freenet/src/freenet/clients/http/DarknetConnectionsToadlet.java > Log: > Small hack on fproxy to deny node removal if there isn't one week of > inactivity. > Maybe it should be done in Node.removeDarknetConnection insteed.
[freenet-dev] Re: [freenet-cvs] r10661 - trunk/freenet/src/freenet/clients/http
Hi, What happens if one just happens to find out the node that cannot be removed is obsolete, compromised or untrustworthy? A warning might be a better idea here? Ed On Saturday 14 October 2006 07:57, [EMAIL PROTECTED] wrote: > Author: nextgens > Date: 2006-10-14 11:57:08 + (Sat, 14 Oct 2006) > New Revision: 10661 > > Modified: >trunk/freenet/src/freenet/clients/http/DarknetConnectionsToadlet.java > Log: > Small hack on fproxy to deny node removal if there isn't one week of > inactivity. > Maybe it should be done in Node.removeDarknetConnection insteed. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Routing not working? Please post your store stats
On Friday 06 October 2006 19:01, toad wrote: > 1. THE STORE IS *LESS* EFFECTIVE THAN THE CACHE! What is the expected ratio of store to cache hits? Given that nodes seem to cluster (if the old testnet graphs still apply) then this clustering should result in the cache having lots of entries that are close to the node location. This implies that the cache will ofter have the key. If might be interesting to get the average key location and standard deviation of all the keys in the cache and store. If the my thoughs are correct the store should have a average keyvalue near the nodes location with a smaller deviation. Second question raised by mrogers, why are there more cache queries than store queries? Thanks Ed > > > Please could people post their store statistics? Cache hits, store hits, > cached keys, stored keys. > > So far: > [23:11] # Cached keys: 6,389 (199 MiB) > [23:11] # Stored keys: 24,550 (767 MiB) > [23:09] # Cache hits: 217 / 12,738 (1%) > [23:09] # Store hits: 14 / 10,818 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 59.56 > > [23:12] # Cached keys: 17,930 (560 MiB) > [23:12] # Stored keys: 24,895 (777 MiB) > [23:14] # Cache hits: 178 / 3,767 (4%) > [23:14] # Store hits: 11 / 2,970 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 22.47 > > [23:14] # Cached keys: 45,148 (1.37 GiB) > [23:14] # Stored keys: 16,238 (507 MiB) > [23:11] # Cache hits: 41 / 861 (4%) > [23:11] # Store hits: 5 / 677 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 2.95 > > Thus, in practice, the cache is far more efficient than the store. > > The cache caches every key fetched or inserted through this node. > > The store stores only keys inserted, and of those, only those for which > there is no closer node to the key amongst our peers. > > > The cache being more effective than the store (and note that the above > is for CHKs only) implies either: > 1. Routing is broken. > 2. There is more location churn than the store can cope with. > 3. There is more data churn than the store can cope with. > > > 2. SUSPICIONS OF EXCESSIVE LOCATION CHURN > - > > ljn1981 said that his node would often do a swap and then reverse it. > However several people say their location is more or less what it was. > It is necessary to make a log of a node's location changes over time... > > > 3. PROBE REQUESTS NOT WORKING > - > > "Probe requests" are a new class of requests which simply take a > location, and try to find the next location - the lowest location > greater than the one they started with. Here's a recent trace (these can > be triggered by telneting to 2323 and typing PROBEALL:, then watching > wrapper.log): > > LOCATION 1: 0.00917056526893234 > LOCATION 2: 0.009450590423585203 > LOCATION 3: 0.009507800765948482 > LOCATION 4: 0.03378227720218496 > [ delays ] > LOCATION 5: 0.033884263580090224 > [ delays ] > LOCATION 6: 0.03557139211207139 > LOCATION 7: 0.04136594238104219 > LOCATION 8: 0.06804731119243879 > LOCATION 9: 0.06938071503433951 > LOCATION 10: 0.11468659860500963 > [ big delays ] > LOCATION 11: 0.11498938134581993 > LOCATION 12: 0.11800179518614218 > LOCATION 13: 0.1180104005154885 > LOCATION 14: 0.11907112718505641 > LOCATION 15: 0.3332896508938398 > [ biggish delays ] > LOCATION 16: 0.6963082287578662 > LOCATION 17: 0.7003642648424434 > LOCATION 18: 0.7516363167204175 > LOCATION 19: 0.7840227104081505 > LOCATION 20: 0.8238921670991454 > LOCATION 21: 0.8551853934902863 > LOCATION 22: 0.8636946791670825 > LOCATION 23: 0.8755575572906827 > LOCATION 24: 0.883042607673485 > LOCATION 25: 0.8910451777595195 > LOCATION 26: 0.8930966991557874 > LOCATION 27: 0.8939968594038799 > LOCATION 28: 0.894079854085 > LOCATION 29: 0.8941104802690825 > LOCATION 30: 0.9103443172876444 > LOCATION 31: 0.9103717579924239 > LOCATION 32: 0.9107237145701387 > LOCATION 33: 0.9108357699627044 > LOCATION 34: 0.9130496893125409 > LOCATION 35: 0.9153056056305631 > [ delays ] > LOCATION 36: 0.9180229911856111 > LOCATION 37: 0.9184676396364483 > LOCATION 38: 0.9198162081803294 > LOCATION 39: 0.9232383399833453 > [ big delays ] > LOCATION 40: 0.9232484869765467 > LOCATION 41: 0.9398827726484242 > LOCATION 42: 0.9420672052844097 > LOCATION 43: 0.9442367949642505 > LOCATION 44: 0.952129695833 > [ big delays ] > LOCATION 45: 0.9521866483104723 > LOCATION 46: 0.9562645053030697 > LOCATION 47: 0.9715290823566148 > LOCATION 48: 0.9722492845296398 > LOCATION 49: 0.974283274258849 > [ big delays ... ] > > Clearly there are more than around 50 nodes on freenet at any given > time, and the above includes some really big jumps, as well as some > really small ones. This may be a problem with probe requests, but it > is suspicious... >
Re: [freenet-dev] Routing not working? Please post your store stats
On Friday 06 October 2006 19:01, toad wrote: > 1. THE STORE IS *LESS* EFFECTIVE THAN THE CACHE! What is the expected ratio of store to cache hits? Given that nodes seem to cluster (if the old testnet graphs still apply) then this clustering should result in the cache having lots of entries that are close to the node location. This implies that the cache will ofter have the key. If might be interesting to get the average key location and standard deviation of all the keys in the cache and store. If the my thoughs are correct the store should have a average keyvalue near the nodes location with a smaller deviation. Second question raised by mrogers, why are there more cache queries than store queries? Thanks Ed > > > Please could people post their store statistics? Cache hits, store hits, > cached keys, stored keys. > > So far: > [23:11] # Cached keys: 6,389 (199 MiB) > [23:11] # Stored keys: 24,550 (767 MiB) > [23:09] # Cache hits: 217 / 12,738 (1%) > [23:09] # Store hits: 14 / 10,818 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 59.56 > > [23:12] # Cached keys: 17,930 (560 MiB) > [23:12] # Stored keys: 24,895 (777 MiB) > [23:14] # Cache hits: 178 / 3,767 (4%) > [23:14] # Store hits: 11 / 2,970 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 22.47 > > [23:14] # Cached keys: 45,148 (1.37 GiB) > [23:14] # Stored keys: 16,238 (507 MiB) > [23:11] # Cache hits: 41 / 861 (4%) > [23:11] # Store hits: 5 / 677 (0%) > > (Cached hits / cached keys) / (Stored hits / stored keys) = 2.95 > > Thus, in practice, the cache is far more efficient than the store. > > The cache caches every key fetched or inserted through this node. > > The store stores only keys inserted, and of those, only those for which > there is no closer node to the key amongst our peers. > > > The cache being more effective than the store (and note that the above > is for CHKs only) implies either: > 1. Routing is broken. > 2. There is more location churn than the store can cope with. > 3. There is more data churn than the store can cope with. > > > 2. SUSPICIONS OF EXCESSIVE LOCATION CHURN > - > > ljn1981 said that his node would often do a swap and then reverse it. > However several people say their location is more or less what it was. > It is necessary to make a log of a node's location changes over time... > > > 3. PROBE REQUESTS NOT WORKING > - > > "Probe requests" are a new class of requests which simply take a > location, and try to find the next location - the lowest location > greater than the one they started with. Here's a recent trace (these can > be triggered by telneting to 2323 and typing PROBEALL:, then watching > wrapper.log): > > LOCATION 1: 0.00917056526893234 > LOCATION 2: 0.009450590423585203 > LOCATION 3: 0.009507800765948482 > LOCATION 4: 0.03378227720218496 > [ delays ] > LOCATION 5: 0.033884263580090224 > [ delays ] > LOCATION 6: 0.03557139211207139 > LOCATION 7: 0.04136594238104219 > LOCATION 8: 0.06804731119243879 > LOCATION 9: 0.06938071503433951 > LOCATION 10: 0.11468659860500963 > [ big delays ] > LOCATION 11: 0.11498938134581993 > LOCATION 12: 0.11800179518614218 > LOCATION 13: 0.1180104005154885 > LOCATION 14: 0.11907112718505641 > LOCATION 15: 0.3332896508938398 > [ biggish delays ] > LOCATION 16: 0.6963082287578662 > LOCATION 17: 0.7003642648424434 > LOCATION 18: 0.7516363167204175 > LOCATION 19: 0.7840227104081505 > LOCATION 20: 0.8238921670991454 > LOCATION 21: 0.8551853934902863 > LOCATION 22: 0.8636946791670825 > LOCATION 23: 0.8755575572906827 > LOCATION 24: 0.883042607673485 > LOCATION 25: 0.8910451777595195 > LOCATION 26: 0.8930966991557874 > LOCATION 27: 0.8939968594038799 > LOCATION 28: 0.894079854085 > LOCATION 29: 0.8941104802690825 > LOCATION 30: 0.9103443172876444 > LOCATION 31: 0.9103717579924239 > LOCATION 32: 0.9107237145701387 > LOCATION 33: 0.9108357699627044 > LOCATION 34: 0.9130496893125409 > LOCATION 35: 0.9153056056305631 > [ delays ] > LOCATION 36: 0.9180229911856111 > LOCATION 37: 0.9184676396364483 > LOCATION 38: 0.9198162081803294 > LOCATION 39: 0.9232383399833453 > [ big delays ] > LOCATION 40: 0.9232484869765467 > LOCATION 41: 0.9398827726484242 > LOCATION 42: 0.9420672052844097 > LOCATION 43: 0.9442367949642505 > LOCATION 44: 0.952129695833 > [ big delays ] > LOCATION 45: 0.9521866483104723 > LOCATION 46: 0.9562645053030697 > LOCATION 47: 0.9715290823566148 > LOCATION 48: 0.9722492845296398 > LOCATION 49: 0.974283274258849 > [ big delays ... ] > > Clearly there are more than around 50 nodes on freenet at any given > time, and the above includes some really big jumps, as well as some > really small ones. This may be a problem with probe requests, but it > is suspicious... > ___ Devl mailing list Devl@freenetproject.org http://emu.freen
[freenet-dev] Routing not working? Please post your store stats
And some more numbers Cached keys: 95,542 (~2.91 GiB) Cache hits: 1,250 / 16,010 (7%) Stored keys: 27,863 (~870 MiB) Store hits: 46 / 12,697 (0%) on Friday 06 October 2006 19:59, Florian Holzinger wrote: > > # Cached keys: 102,152 (~3.11 GiB) > # Stored keys: 20,967 (~655 MiB) > # Cache hits: 1,162 / 17,611 (6%) > # Store hits: 88 / 15,298 (0%) > > > toad wrote: > > 1. THE STORE IS *LESS* EFFECTIVE THAN THE CACHE! > > > > > > Please could people post their store statistics? Cache hits, store hits, > > cached keys, stored keys. > > > > So far: > > [23:11] # Cached keys: 6,389 (199 MiB) > > [23:11] # Stored keys: 24,550 (767 MiB) > > [23:09] # Cache hits: 217 / 12,738 (1%) > > [23:09] # Store hits: 14 / 10,818 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 59.56 > > > > [23:12] # Cached keys: 17,930 (560 MiB) > > [23:12] # Stored keys: 24,895 (777 MiB) > > [23:14] # Cache hits: 178 / 3,767 (4%) > > [23:14] # Store hits: 11 / 2,970 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 22.47 > > > > [23:14] # Cached keys: 45,148 (1.37 GiB) > > [23:14] # Stored keys: 16,238 (507 MiB) > > [23:11] # Cache hits: 41 / 861 (4%) > > [23:11] # Store hits: 5 / 677 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 2.95 > > > > Thus, in practice, the cache is far more efficient than the store. > > > > The cache caches every key fetched or inserted through this node. > > > > The store stores only keys inserted, and of those, only those for which > > there is no closer node to the key amongst our peers. > > > > > > The cache being more effective than the store (and note that the above > > is for CHKs only) implies either: > > 1. Routing is broken. > > 2. There is more location churn than the store can cope with. > > 3. There is more data churn than the store can cope with. > > > > > > 2. SUSPICIONS OF EXCESSIVE LOCATION CHURN > > - > > > > ljn1981 said that his node would often do a swap and then reverse it. > > However several people say their location is more or less what it was. > > It is necessary to make a log of a node's location changes over time... > > > > > > 3. PROBE REQUESTS NOT WORKING > > - > > > > "Probe requests" are a new class of requests which simply take a > > location, and try to find the next location - the lowest location > > greater than the one they started with. Here's a recent trace (these can > > be triggered by telneting to 2323 and typing PROBEALL:, then watching > > wrapper.log): > > > > LOCATION 1: 0.00917056526893234 > > LOCATION 2: 0.009450590423585203 > > LOCATION 3: 0.009507800765948482 > > LOCATION 4: 0.03378227720218496 > > [ delays ] > > LOCATION 5: 0.033884263580090224 > > [ delays ] > > LOCATION 6: 0.03557139211207139 > > LOCATION 7: 0.04136594238104219 > > LOCATION 8: 0.06804731119243879 > > LOCATION 9: 0.06938071503433951 > > LOCATION 10: 0.11468659860500963 > > [ big delays ] > > LOCATION 11: 0.11498938134581993 > > LOCATION 12: 0.11800179518614218 > > LOCATION 13: 0.1180104005154885 > > LOCATION 14: 0.11907112718505641 > > LOCATION 15: 0.3332896508938398 > > [ biggish delays ] > > LOCATION 16: 0.6963082287578662 > > LOCATION 17: 0.7003642648424434 > > LOCATION 18: 0.7516363167204175 > > LOCATION 19: 0.7840227104081505 > > LOCATION 20: 0.8238921670991454 > > LOCATION 21: 0.8551853934902863 > > LOCATION 22: 0.8636946791670825 > > LOCATION 23: 0.8755575572906827 > > LOCATION 24: 0.883042607673485 > > LOCATION 25: 0.8910451777595195 > > LOCATION 26: 0.8930966991557874 > > LOCATION 27: 0.8939968594038799 > > LOCATION 28: 0.894079854085 > > LOCATION 29: 0.8941104802690825 > > LOCATION 30: 0.9103443172876444 > > LOCATION 31: 0.9103717579924239 > > LOCATION 32: 0.9107237145701387 > > LOCATION 33: 0.9108357699627044 > > LOCATION 34: 0.9130496893125409 > > LOCATION 35: 0.9153056056305631 > > [ delays ] > > LOCATION 36: 0.9180229911856111 > > LOCATION 37: 0.9184676396364483 > > LOCATION 38: 0.9198162081803294 > > LOCATION 39: 0.9232383399833453 > > [ big delays ] > > LOCATION 40: 0.9232484869765467 > > LOCATION 41: 0.9398827726484242 > > LOCATION 42: 0.9420672052844097 > > LOCATION 43: 0.9442367949642505 > > LOCATION 44: 0.952129695833 > > [ big delays ] > > LOCATION 45: 0.9521866483104723 > > LOCATION 46: 0.9562645053030697 > > LOCATION 47: 0.9715290823566148 > > LOCATION 48: 0.9722492845296398 > > LOCATION 49: 0.974283274258849 > > [ big delays ... ] > > > > Clearly there are more than around 50 nodes on freenet at any given > > time, and the above includes some really big jumps, as well as some > > really small ones. This may be a problem with probe requests, but it > > is suspicious... > > > > > > > > > > ___ > > Devl mailing list > > Devl at
Re: [freenet-dev] Routing not working? Please post your store stats
And some more numbers Cached keys: 95,542 (~2.91 GiB) Cache hits: 1,250 / 16,010 (7%) Stored keys: 27,863 (~870 MiB) Store hits: 46 / 12,697 (0%) on Friday 06 October 2006 19:59, Florian Holzinger wrote: > > # Cached keys: 102,152 (~3.11 GiB) > # Stored keys: 20,967 (~655 MiB) > # Cache hits: 1,162 / 17,611 (6%) > # Store hits: 88 / 15,298 (0%) > > > toad wrote: > > 1. THE STORE IS *LESS* EFFECTIVE THAN THE CACHE! > > > > > > Please could people post their store statistics? Cache hits, store hits, > > cached keys, stored keys. > > > > So far: > > [23:11] # Cached keys: 6,389 (199 MiB) > > [23:11] # Stored keys: 24,550 (767 MiB) > > [23:09] # Cache hits: 217 / 12,738 (1%) > > [23:09] # Store hits: 14 / 10,818 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 59.56 > > > > [23:12] # Cached keys: 17,930 (560 MiB) > > [23:12] # Stored keys: 24,895 (777 MiB) > > [23:14] # Cache hits: 178 / 3,767 (4%) > > [23:14] # Store hits: 11 / 2,970 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 22.47 > > > > [23:14] # Cached keys: 45,148 (1.37 GiB) > > [23:14] # Stored keys: 16,238 (507 MiB) > > [23:11] # Cache hits: 41 / 861 (4%) > > [23:11] # Store hits: 5 / 677 (0%) > > > > (Cached hits / cached keys) / (Stored hits / stored keys) = 2.95 > > > > Thus, in practice, the cache is far more efficient than the store. > > > > The cache caches every key fetched or inserted through this node. > > > > The store stores only keys inserted, and of those, only those for which > > there is no closer node to the key amongst our peers. > > > > > > The cache being more effective than the store (and note that the above > > is for CHKs only) implies either: > > 1. Routing is broken. > > 2. There is more location churn than the store can cope with. > > 3. There is more data churn than the store can cope with. > > > > > > 2. SUSPICIONS OF EXCESSIVE LOCATION CHURN > > - > > > > ljn1981 said that his node would often do a swap and then reverse it. > > However several people say their location is more or less what it was. > > It is necessary to make a log of a node's location changes over time... > > > > > > 3. PROBE REQUESTS NOT WORKING > > - > > > > "Probe requests" are a new class of requests which simply take a > > location, and try to find the next location - the lowest location > > greater than the one they started with. Here's a recent trace (these can > > be triggered by telneting to 2323 and typing PROBEALL:, then watching > > wrapper.log): > > > > LOCATION 1: 0.00917056526893234 > > LOCATION 2: 0.009450590423585203 > > LOCATION 3: 0.009507800765948482 > > LOCATION 4: 0.03378227720218496 > > [ delays ] > > LOCATION 5: 0.033884263580090224 > > [ delays ] > > LOCATION 6: 0.03557139211207139 > > LOCATION 7: 0.04136594238104219 > > LOCATION 8: 0.06804731119243879 > > LOCATION 9: 0.06938071503433951 > > LOCATION 10: 0.11468659860500963 > > [ big delays ] > > LOCATION 11: 0.11498938134581993 > > LOCATION 12: 0.11800179518614218 > > LOCATION 13: 0.1180104005154885 > > LOCATION 14: 0.11907112718505641 > > LOCATION 15: 0.3332896508938398 > > [ biggish delays ] > > LOCATION 16: 0.6963082287578662 > > LOCATION 17: 0.7003642648424434 > > LOCATION 18: 0.7516363167204175 > > LOCATION 19: 0.7840227104081505 > > LOCATION 20: 0.8238921670991454 > > LOCATION 21: 0.8551853934902863 > > LOCATION 22: 0.8636946791670825 > > LOCATION 23: 0.8755575572906827 > > LOCATION 24: 0.883042607673485 > > LOCATION 25: 0.8910451777595195 > > LOCATION 26: 0.8930966991557874 > > LOCATION 27: 0.8939968594038799 > > LOCATION 28: 0.894079854085 > > LOCATION 29: 0.8941104802690825 > > LOCATION 30: 0.9103443172876444 > > LOCATION 31: 0.9103717579924239 > > LOCATION 32: 0.9107237145701387 > > LOCATION 33: 0.9108357699627044 > > LOCATION 34: 0.9130496893125409 > > LOCATION 35: 0.9153056056305631 > > [ delays ] > > LOCATION 36: 0.9180229911856111 > > LOCATION 37: 0.9184676396364483 > > LOCATION 38: 0.9198162081803294 > > LOCATION 39: 0.9232383399833453 > > [ big delays ] > > LOCATION 40: 0.9232484869765467 > > LOCATION 41: 0.9398827726484242 > > LOCATION 42: 0.9420672052844097 > > LOCATION 43: 0.9442367949642505 > > LOCATION 44: 0.952129695833 > > [ big delays ] > > LOCATION 45: 0.9521866483104723 > > LOCATION 46: 0.9562645053030697 > > LOCATION 47: 0.9715290823566148 > > LOCATION 48: 0.9722492845296398 > > LOCATION 49: 0.974283274258849 > > [ big delays ... ] > > > > Clearly there are more than around 50 nodes on freenet at any given > > time, and the above includes some really big jumps, as well as some > > really small ones. This may be a problem with probe requests, but it > > is suspicious... > > > > > > > > > > ___ > > Devl mailing list > > Devl@fre
[freenet-dev] Re: Freenet 0.7 licensing
tly because the concept is so universal, and so fundamental, and so basic. And that is why the GPLv2 is a great license. I can't stress that enough. ?Sure, other licenses can say the same thing, but what the GPLv2 did was to be the first open-source license that made that "tit-for-tat" a legal license that was widely deployed. That's something that the FSF and rms should be proud of, rather than trying to ruin by adding all these totally unnecessary things that are ephemeral, and depend on some random worry of the day. That's also why I ended up changing the kernel license to the GPLv2. The original Linux source license said basically: "Give all source back, and never charge any money". ?It took me a few months, but I realized that the "never charge any money" part was just asinine. ?It wasn't the point. ? The point was always "give back in kind". Btw, on a personal note, I can even tell you where that "never charge any money" requirement came from. ?It came from my own frustrations with Minix as a poor student, where the cost of getting the system ($169 USD back then) was just absolutely prohibitive. ?I really disliked having to spend a huge amount of money (to me) for something that I just needed to make my machine useful. In other words, my original license very much had a "fear and loathing" component to it. ?It was exactly that "never charge any money" part. But I realized that in the end, it was never really about the money, and that what I really looked for in a license was the "fairness" thing. And that's what the GPLv2 is. ?It's "fair". ?It asks everybody - regardless of circumstance - for the same thing. ?It asks for the effort that was put into improving the software to be given back to the common good. ?You can use the end result any way you want (and if you want to use it for "bad" things, be my guest), but we ask the same exact thing of everybody - give your modifications back. That's true grace. ?Realizing that the petty concerns don't matter, whether they are money or DRM, or patents, or anything else. And that's why I chose the GPLv2. ?I did it back when the $169 I paid for Minix still stung me, because I just decided that that wasn't what it was all about. And I look at the additions to the GPLv3, and I still say: "That's not what it's all about". My original license was petty and into details. ?I don't need to go back to those days. ?I found a better license. ?And it's the GPLv2. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo at vger.kernel.org More majordomo info at ?http://vger.kernel.org/majordomo-info.html Please read the FAQ at ?http://www.tux.org/lkml/ - On Saturday 23 September 2006 11:53, toad wrote: > Linus's insistence that if it is not specified, then the default is GPL > 2 only, is part of the reason why we are doing this. One of the > advantages of GPL 3 is that it solves compatibility problems with > various licenses, many of which are widely used for java related code, > for example ASL2 (we would like to use some ASL2 code in Freenet, the > Apache Commons Compress library). > > We may want to upgrade to GPL3 only in future, for compatibility > reasons, but for the time being the proposal is that we make it > explicitly "GPL 2 or later". We should have this discussion on the > mailing list, so I have CC'ed it; where did you get the below PDF from? > Nobody has responded to my original mailing list post. > > In terms of specifics... The FSF has always been political. It has > sharply defined political goals. "DRM abuse", as they call it, is a > direct threat to the FSF's political goals as expressed in the GPL2, > and so they have reacted to it. Software patents likewise: IBM is trying > to have its cake and eat it too: Funding linux on the one hand, and > campaigning for ever stronger and wider software patents on the other > hand in order to suborn Linux and make it *impossible* to develop it > without corporate patronage; this could reasonably be termed (legal) > theft. Freenet is also political... > > On Sat, Sep 23, 2006 at 09:58:11AM -0400, Ed Tomlinson wrote: > > Hi > > > > Have you seen this? > > > > Ed > > > > On Friday 22 September 2006 14:57, you wrote: > > > Hi, I am trying to clarify a minor licensing issue with Freenet 0.7. > > > Since you contributed to it, I must ask: At the time of your commits, it > > > was not clear whether Freenet was GPL 2 or later, or just GPL 2. We > > > would like it to be GPL 2 or later, so we can transparently upgrade to > > > GPL 3 if necessary (it has various advantages, the most practical of > > > which being that it is compatible with various other free licenses such > > > as the Apache Software License). The code will remain GPL 2 for the time > > > being (GPL 3 isn't even out yet), but we want it to be forward > > > compatible if possible. Could you please either: > > > a) Tell me that you support the code being "GPL 2 or later" > > > b) Tell me that you don't (Ideally with reasons!) > > > > > > Thanks. >
Re: [freenet-dev] Re: Freenet 0.7 licensing
ld. And that's exactly because the concept is so universal, and so fundamental, and so basic. And that is why the GPLv2 is a great license. I can't stress that enough. Sure, other licenses can say the same thing, but what the GPLv2 did was to be the first open-source license that made that "tit-for-tat" a legal license that was widely deployed. That's something that the FSF and rms should be proud of, rather than trying to ruin by adding all these totally unnecessary things that are ephemeral, and depend on some random worry of the day. That's also why I ended up changing the kernel license to the GPLv2. The original Linux source license said basically: "Give all source back, and never charge any money". It took me a few months, but I realized that the "never charge any money" part was just asinine. It wasn't the point. The point was always "give back in kind". Btw, on a personal note, I can even tell you where that "never charge any money" requirement came from. It came from my own frustrations with Minix as a poor student, where the cost of getting the system ($169 USD back then) was just absolutely prohibitive. I really disliked having to spend a huge amount of money (to me) for something that I just needed to make my machine useful. In other words, my original license very much had a "fear and loathing" component to it. It was exactly that "never charge any money" part. But I realized that in the end, it was never really about the money, and that what I really looked for in a license was the "fairness" thing. And that's what the GPLv2 is. It's "fair". It asks everybody - regardless of circumstance - for the same thing. It asks for the effort that was put into improving the software to be given back to the common good. You can use the end result any way you want (and if you want to use it for "bad" things, be my guest), but we ask the same exact thing of everybody - give your modifications back. That's true grace. Realizing that the petty concerns don't matter, whether they are money or DRM, or patents, or anything else. And that's why I chose the GPLv2. I did it back when the $169 I paid for Minix still stung me, because I just decided that that wasn't what it was all about. And I look at the additions to the GPLv3, and I still say: "That's not what it's all about". My original license was petty and into details. I don't need to go back to those days. I found a better license. And it's the GPLv2. Linus - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ - On Saturday 23 September 2006 11:53, toad wrote: > Linus's insistence that if it is not specified, then the default is GPL > 2 only, is part of the reason why we are doing this. One of the > advantages of GPL 3 is that it solves compatibility problems with > various licenses, many of which are widely used for java related code, > for example ASL2 (we would like to use some ASL2 code in Freenet, the > Apache Commons Compress library). > > We may want to upgrade to GPL3 only in future, for compatibility > reasons, but for the time being the proposal is that we make it > explicitly "GPL 2 or later". We should have this discussion on the > mailing list, so I have CC'ed it; where did you get the below PDF from? > Nobody has responded to my original mailing list post. > > In terms of specifics... The FSF has always been political. It has > sharply defined political goals. "DRM abuse", as they call it, is a > direct threat to the FSF's political goals as expressed in the GPL2, > and so they have reacted to it. Software patents likewise: IBM is trying > to have its cake and eat it too: Funding linux on the one hand, and > campaigning for ever stronger and wider software patents on the other > hand in order to suborn Linux and make it *impossible* to develop it > without corporate patronage; this could reasonably be termed (legal) > theft. Freenet is also political... > > On Sat, Sep 23, 2006 at 09:58:11AM -0400, Ed Tomlinson wrote: > > Hi > > > > Have you seen this? > > > > Ed > > > > On Friday 22 September 2006 14:57, you wrote: > > > Hi, I am trying to clarify a minor licensing issue with Freenet 0.7. > > > Since you contributed to it, I must ask: At the time of your commits, it > > > was not clear whether Freenet was GPL 2 or later, or just GPL 2. We > > > would like it to be GPL 2 or later, s
[Tech] Re: [freenet-dev] What is backoff for?
On Wednesday 09 August 2006 10:49, Michael Rogers wrote: > Matthew Toseland wrote: > > Because we have load propagation, it is entirely legitimate to > > send requests ANYWAY, and let upstream nodes deal with the slowness - > > I'm inclined to agree with this argument, but I'm not sure it follows > that we should send fewer requests to slow nodes. This will necessarily > lead to fast nodes getting a larger proportion of requests, even if they > don't get a larger absolute number of requests. Maybe that's acceptable, > but I think we should distinguish between two questions: > 1) should we rely on senders to slow down instead of backing off? > 2) should we send a smaller proportion of requests to slow nodes? This is ineffect what the code I wrote in back in May does. It used a know alg to do slow down the rate of requests to node rejecting requests. Ed
Re: [Tech] Re: [freenet-dev] What is backoff for?
On Wednesday 09 August 2006 10:49, Michael Rogers wrote: > Matthew Toseland wrote: > > Because we have load propagation, it is entirely legitimate to > > send requests ANYWAY, and let upstream nodes deal with the slowness - > > I'm inclined to agree with this argument, but I'm not sure it follows > that we should send fewer requests to slow nodes. This will necessarily > lead to fast nodes getting a larger proportion of requests, even if they > don't get a larger absolute number of requests. Maybe that's acceptable, > but I think we should distinguish between two questions: > 1) should we rely on senders to slow down instead of backing off? > 2) should we send a smaller proportion of requests to slow nodes? This is ineffect what the code I wrote in back in May does. It used a know alg to do slow down the rate of requests to node rejecting requests. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] freenet-ext.jar cleanup
On Sunday 30 July 2006 18:57, Florent Daigni?re wrote: > * Juiceman [2006-07-30 17:19:04]: > > > On 7/30/06, Florent Daigni?re (NextGen$) > > wrote: > > >Hi, > > > > > >Would someone mind if I remove "pentium" optimized native big > > >integer libraries ? what about pentiummmx too ? > > >The asset beeing a space gain. > > > > > >Atm, we have : none, pentium, pentiummmx, pentium2, pentium3, > > >pentium4, k6, k62, k63, athlon, x86_64 > > > > > >IMHO, freenet 0.7 can't run "well enough" on a pentium. I'm > > >running a node on a pentium2 and I'm already short of resources. > > > > > >I've got one other idea: what about distributing > > >YetAnotherJarfile with only optimized libraries ? > > > > > >one for windows, one for linux (half space gain) > > >or one per processor type (including both linux and win32 libs : > > >big space gain) > > > > > >any thought ? > > > > > > > Honestly, I think this is more work than it is worth. Freenet.jar is > > half the size of the freenet-ext.jar but is downloaded dozen's of > > times more often. In the big picture this is a small percentage of > > the bandwidth. Our target audiance is broadband users to which 1 > > extra megabyte once in a great while is not an issue. Also, I think > > this will lead to possible support and configuration issues if users > > get the wrong .jar's or mess around trying for better performance from > > different files, > > > > I would much rather see the energy spent making the nodeupdater detect > > new versions amd update the freenet-ext.jar file in-Freenet. My 2 > > cents. > > The problem is that many people aren't using update-over-freenet but the > mirrors. Even then they download the .ext very seldomly. Think that there is nothing to gain (except complexitity) by splitting this jar. Thanks Ed Tomlinson
Re: [freenet-dev] freenet-ext.jar cleanup
On Sunday 30 July 2006 18:57, Florent Daignière wrote: > * Juiceman <[EMAIL PROTECTED]> [2006-07-30 17:19:04]: > > > On 7/30/06, Florent Daignière (NextGen$) <[EMAIL PROTECTED]> > > wrote: > > >Hi, > > > > > >Would someone mind if I remove "pentium" optimized native big > > >integer libraries ? what about pentiummmx too ? > > >The asset beeing a space gain. > > > > > >Atm, we have : none, pentium, pentiummmx, pentium2, pentium3, > > >pentium4, k6, k62, k63, athlon, x86_64 > > > > > >IMHO, freenet 0.7 can't run "well enough" on a pentium. I'm > > >running a node on a pentium2 and I'm already short of resources. > > > > > >I've got one other idea: what about distributing > > >YetAnotherJarfile with only optimized libraries ? > > > > > >one for windows, one for linux (half space gain) > > >or one per processor type (including both linux and win32 libs : > > >big space gain) > > > > > >any thought ? > > > > > > > Honestly, I think this is more work than it is worth. Freenet.jar is > > half the size of the freenet-ext.jar but is downloaded dozen's of > > times more often. In the big picture this is a small percentage of > > the bandwidth. Our target audiance is broadband users to which 1 > > extra megabyte once in a great while is not an issue. Also, I think > > this will lead to possible support and configuration issues if users > > get the wrong .jar's or mess around trying for better performance from > > different files, > > > > I would much rather see the energy spent making the nodeupdater detect > > new versions amd update the freenet-ext.jar file in-Freenet. My 2 > > cents. > > The problem is that many people aren't using update-over-freenet but the > mirrors. Even then they download the .ext very seldomly. Think that there is nothing to gain (except complexitity) by splitting this jar. Thanks Ed Tomlinson ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Darknet Connections Raw Mode
On Sunday 02 July 2006 15:04, Florent Daigni?re wrote: > * Ian Clarke [2006-07-02 11:52:19]: > > > This is exactly the type of thing I was worried would happen. > > Automated access to information about the node is exactly what FCP is > > intended for, and exactly what fproxy isn't intended for. We > > definitely should not be encouraging people to write scripts to > > scrape fproxy pages by trying to make them easier to parse. Fproxy > > is not, and will never be intended to provide a stable API for this > > kind of thing. > > > > Please revert this, and instead, work on bug #200 (https:// > > bugs.freenetproject.org/view.php?id=200) > > > > Ian. > > I do agree ; it's not a good idea. > > Reverted in r9429 > > NextGen$ Thank You. Ed
Re: [freenet-dev] Darknet Connections Raw Mode
On Sunday 02 July 2006 15:04, Florent Daignière wrote: > * Ian Clarke <[EMAIL PROTECTED]> [2006-07-02 11:52:19]: > > > This is exactly the type of thing I was worried would happen. > > Automated access to information about the node is exactly what FCP is > > intended for, and exactly what fproxy isn't intended for. We > > definitely should not be encouraging people to write scripts to > > scrape fproxy pages by trying to make them easier to parse. Fproxy > > is not, and will never be intended to provide a stable API for this > > kind of thing. > > > > Please revert this, and instead, work on bug #200 (https:// > > bugs.freenetproject.org/view.php?id=200) > > > > Ian. > > I do agree ; it's not a good idea. > > Reverted in r9429 > > NextGen$ Thank You. Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Do we actually need VPN support to forward stuff? was Re: [freenet-dev] More stable node locations
On Saturday 01 July 2006 04:00, Florent Daigni?re wrote: > * Matthew Toseland [2006-07-01 02:33:37]: > > > Well, is it entirely clear that we do need VPN support? > > No. :) No here too. If anything why not SSL tunnels? No need for any other drivers then. I know that Corps are turning away from VPN toward SSL tunnels. VPN drivers tend to cause unexpected problems. SSL tunnels tend to be cleaner... Ed > Freenet is about a DHT ; VPN support seems to be "ahead" of that. It > could be done using plugins ... but should definitly not be part of the > main distribution imho. > > And FYI, accessing tun/tap|virtual interfaces from java is almost NOT > possible : > 1) java is missing raw sockets > 2) you need r00t rights to create the virtual interface, > configure the routing table, the firewall, ... > > A plugin using JNI calls could do that though... If we add a dependency > on "pppd" it would be easier (we create ppp over freenet :p) > > > > > Do games/Network Neighbourhood/etc send packets to a specific interface > > address to discover servers on the LAN, or do they just broadcast > > (255.255.255.255)? I > > They send either broadcasts or join a multicast group > > > suppose they would broadcast. In which case, does > > loopback receive these broadcasts? How do they receive responses? Do > > they bind to a specific interface? If so, how can Hamatchi work when > > there is both a physical LAN and a virtual one? I doubt that most games > > would look for every interface with an address that looks like a LAN > > address, and bind to each...? > > They do. > > > If we can receive the broadcasts on > > loopback, and if we can receive the responses to them on loopback, then > > we may indeed be able to do this without any nasty platform specific > > VPN code. > > We WILL need platform specific code > > > However, if the VPN code is freely available under a suitable > > license for all interesting platforms, then it's not necessarily a big > > deal if we can't. > > > > Apart from network discovery, it is entirely possible to bind to > > addresses in the loopback range, assign permanent IPs to peers, and > > edit the hosts.txt file on windows or /etc/hosts on linux to include > > permanent name entries. Without any nasty platform specific code. The > > only worry is that tunneling TCP may not be entirely straightforward as > > we don't have any similar streaming code at present. Tunneling UDP is > > fairly trivial. > > Agreed, but does it worth the additionnal complexity ? > > > > > On Fri, Jun 30, 2006 at 04:47:04PM -0400, Colin Davis wrote: > > > > > > > >I'm not entirely sure why this would need to be done outside of > > > >Java... > > > >Can't we already bind to an infinate number of IP addresses? If we > > > >pick > > > >IPaddress to bind to that aren't taken, such as 172.100.100.* we > > > >should > > > >be able to bind to those without an issue. > > > > > > I agree with the rest of my post, and think that Hamachi-style > > > functionality would be THE killer app, but I don't know what I was > > > smoking when I wrote that part. > > > Of course we would need some sort of .lib per-system to bind to IP > > > addresses. It's trivial to do, and we're already using .libs for > > > native acceleration. I think it's acceptable, considering if it's not > > > there, or can't load, freenet still works, the Hamachi sharing just > > > doesn't. > > > > > > > > > Please consider it. I know implementing a VPN-style connection is a > > > pain, but since we ALREADY have connections to those people that span > > > NATs, it's not as hard as it otherwise would be. > > > Doing so will ENSURE darknet is VERY popular- There are lots of > > > Filesharing/IM apps- There aren't any OSS which do this (easily). > > > > > > -Colin > > > > > > > > > > > > > >> > > > >> > > > >>There are OSS apps that do this, it's just that it's difficult to > > > >>set up > > > >>as what you are doing is creating a VPN. That would be extremely > > > >>difficult to do over Java. > > > >> > > > >>However, the idea of sharing services out to your darknet peers is > > > >>possible, if it is sufficiently useful. Certainly exposing samba > > > >>shares > > > >>or other TCP-based services is possible (if they are allowed to > > > >>localhost or LAN already). > > > >> > > > >>As far as UDP-based games go, isn't it always going to perform > > > >>better to > > > >>connect directly to the IP address of your friend? Admittedly you > > > >>have > > > >>to password the server, and find their IP address... I wonder if > > > >>there's > > > >>something in the idea of dyndns over freenet (as opposed to ARKs; > > > >>make > > > >>toad.freenet resolve via a local lookup of the ARK or the > > > >>connection to > > > >>toad's current IP address)... we could have the node insert (and > > > >>keep up > > > >>to date) lines for your darknet neighbours in hosts.txt. :) > > > >> > > > >>It
Re: Do we actually need VPN support to forward stuff? was Re: [freenet-dev] More stable node locations
On Saturday 01 July 2006 04:00, Florent Daignière wrote: > * Matthew Toseland <[EMAIL PROTECTED]> [2006-07-01 02:33:37]: > > > Well, is it entirely clear that we do need VPN support? > > No. :) No here too. If anything why not SSL tunnels? No need for any other drivers then. I know that Corps are turning away from VPN toward SSL tunnels. VPN drivers tend to cause unexpected problems. SSL tunnels tend to be cleaner... Ed > Freenet is about a DHT ; VPN support seems to be "ahead" of that. It > could be done using plugins ... but should definitly not be part of the > main distribution imho. > > And FYI, accessing tun/tap|virtual interfaces from java is almost NOT > possible : > 1) java is missing raw sockets > 2) you need r00t rights to create the virtual interface, > configure the routing table, the firewall, ... > > A plugin using JNI calls could do that though... If we add a dependency > on "pppd" it would be easier (we create ppp over freenet :p) > > > > > Do games/Network Neighbourhood/etc send packets to a specific interface > > address to discover servers on the LAN, or do they just broadcast > > (255.255.255.255)? I > > They send either broadcasts or join a multicast group > > > suppose they would broadcast. In which case, does > > loopback receive these broadcasts? How do they receive responses? Do > > they bind to a specific interface? If so, how can Hamatchi work when > > there is both a physical LAN and a virtual one? I doubt that most games > > would look for every interface with an address that looks like a LAN > > address, and bind to each...? > > They do. > > > If we can receive the broadcasts on > > loopback, and if we can receive the responses to them on loopback, then > > we may indeed be able to do this without any nasty platform specific > > VPN code. > > We WILL need platform specific code > > > However, if the VPN code is freely available under a suitable > > license for all interesting platforms, then it's not necessarily a big > > deal if we can't. > > > > Apart from network discovery, it is entirely possible to bind to > > addresses in the loopback range, assign permanent IPs to peers, and > > edit the hosts.txt file on windows or /etc/hosts on linux to include > > permanent name entries. Without any nasty platform specific code. The > > only worry is that tunneling TCP may not be entirely straightforward as > > we don't have any similar streaming code at present. Tunneling UDP is > > fairly trivial. > > Agreed, but does it worth the additionnal complexity ? > > > > > On Fri, Jun 30, 2006 at 04:47:04PM -0400, Colin Davis wrote: > > > > > > > >I'm not entirely sure why this would need to be done outside of > > > >Java... > > > >Can't we already bind to an infinate number of IP addresses? If we > > > >pick > > > >IPaddress to bind to that aren't taken, such as 172.100.100.* we > > > >should > > > >be able to bind to those without an issue. > > > > > > I agree with the rest of my post, and think that Hamachi-style > > > functionality would be THE killer app, but I don't know what I was > > > smoking when I wrote that part. > > > Of course we would need some sort of .lib per-system to bind to IP > > > addresses. It's trivial to do, and we're already using .libs for > > > native acceleration. I think it's acceptable, considering if it's not > > > there, or can't load, freenet still works, the Hamachi sharing just > > > doesn't. > > > > > > > > > Please consider it. I know implementing a VPN-style connection is a > > > pain, but since we ALREADY have connections to those people that span > > > NATs, it's not as hard as it otherwise would be. > > > Doing so will ENSURE darknet is VERY popular- There are lots of > > > Filesharing/IM apps- There aren't any OSS which do this (easily). > > > > > > -Colin > > > > > > > > > > > > > >> > > > >> > > > >>There are OSS apps that do this, it's just that it's difficult to > > > >>set up > > > >>as what you are doing is creating a VPN. That would be extremely > > > >>difficult to do over Java. > > > >> > > > >>However, the idea of sharing services out to your darknet peers is > > > >>possible, if it is sufficiently useful. Certainly exposing samba > > > >>shares > > > >>or other TCP-based services is possible (if they are allowed to > > > >>localhost or LAN already). > > > >> > > > >>As far as UDP-based games go, isn't it always going to perform > > > >>better to > > > >>connect directly to the IP address of your friend? Admittedly you > > > >>have > > > >>to password the server, and find their IP address... I wonder if > > > >>there's > > > >>something in the idea of dyndns over freenet (as opposed to ARKs; > > > >>make > > > >>toad.freenet resolve via a local lookup of the ARK or the > > > >>connection to > > > >>toad's current IP address)... we could have the node insert (and > > > >>keep up > > > >>to date) lines for your darknet neighbours in hosts.txt. :)
[freenet-dev] Re: [freenet-cvs] r9379 - in trunk/freenet/src/freenet: clients/http node
On Monday 26 June 2006 17:06, Matthew Toseland wrote: > Nice, but not entirely meaningful as we don't know what the normal range > of requests' target locations is ... does SNMP work at the moment? Could > we use it to plot a frequency distribution graph in MRTG? All this metric measures is now much better could we have routed requests if there was no backoff. It makes no attempt to gauge how close to the actual target we were. Are you suggesting we want to measure this too? I am not entirely sure what you are suggesting here... > Oh and misrouting has one S. :) And variables should start with a lower > case letter. I'll correct this (spell checkers are nice. What they do to code is not). Ed
[freenet-dev] Re: [freenet-cvs] r9379 - in trunk/freenet/src/freenet: clients/http node
On Monday 26 June 2006 17:06, Matthew Toseland wrote: > Nice, but not entirely meaningful as we don't know what the normal range > of requests' target locations is ... does SNMP work at the moment? Could > we use it to plot a frequency distribution graph in MRTG? All this metric measures is now much better could we have routed requests if there was no backoff. It makes no attempt to gauge how close to the actual target we were. Are you suggesting we want to measure this too? I am not entirely sure what you are suggesting here... > Oh and misrouting has one S. :) And variables should start with a lower > case letter. I'll correct this (spell checkers are nice. What they do to code is not). Ed ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] More stable node location
With a node up 7x24 I am currently seeing .35 variation in location on a daily basis. I have 8-10 connected peers. Ed On Wednesday 21 June 2006 12:01, Ruud Javi wrote: > >Is there a serious problem with node location stability? Oskar's > >simulations suggest not. Anything which impacts location swapping will > >need to be simulated, of course. > > Well... I can only talk for myself, but my own node's location is changing > often to total different values. Yesterday I was at about 0.4 and at this > moment I am around 0.1 . Note that I did not add or remove peers in between. > With the size of the current network, I think that a change of about 0.3 is > extremely big and unwanted. > > Anyway, I am unsure how serious this problem is. So far, I am able to > retrieve all Freesites inside Freenet .7 , also old ones. Maybe when the > network grows, it's harder to find keys and it does seems to be a serious > problem. > > > > >My main concern with treating offline nodes as online for purposes of > >swapping is that swaps cannot involve those offline nodes; they are > >static for the period while they are offline, this may not be good for > >location swapping. > > Agreed on. Most optimistic view against that is that when the node comes > back on he will have the same location. If a swap occurs, no matter how you > treat off-line nodes, the effect for the offline node is none untill it gets > back on. > > As long as we will not have much DNF's, as told above, this is probably not > an issue. > > > > > >On Wed, Jun 21, 2006 at 04:25:40PM +0200, Ruud Javi wrote: > > > The following text is describing a way to have a more stable node > >location, > > > by treating temporary offline nodes as online nodes. > > > > > > The location of your node is depending on your neighbors. If your > > > neighbor?s locations are all around 0.5, then your node will also try to > > > get a location close to 0.5 > > > > > > When somebody is inserting content into Freenet, specific keys will go > >to > > > specific locations. Others are able to retrieve this content as long as > > > your node is at that location (or close). For that reason it?s a good > >thing > > > if a node would stay at a specific location. > > > > > > If the network is stable, no location-swaps would occur. The network > >would > > > not be stable if nodes join the network or leave the network. This can > >be > > > as well temporary (non 24/7 nodes) or permanent (nodes joining/leaving). > > > > > > Against permanent changes is not that much possible; when new nodes > >arrive > > > it is necessary that this has an effect on node locations. > > > > > > Against temporary changes we can do something. If a neighbor of you > >would > > > go offline (bedtime), your node would choose another location, as most > > > optimal. Instead of this your node could just treat the offline node as > >an > > > online node for some time (perhaps 24 hours). Of course your node could > >not > > > change the location with an offline node, but it could decide not to > >change > > > location with an online node. The idea is that once the offline node > >would > > > come back online, you would want your old location back. > > > > > > In this way your node?s location would most probably be more stable as > >the > > > current situation. > > > > > > Last questions: > > > - Is a more stable node location a big advantage? > > > - Will routing be worse if a lot of your neighbors are temporary > > > offline and you would not change node location? > >-- > >Matthew J Toseland - toad at amphibian.dyndns.org > >Freenet Project Official Codemonkey - http://freenetproject.org/ > >ICTHUS - Nothing is impossible. Our Boss says so. > > > ><< signature.asc >> > > > > > >___ > >Devl mailing list > >Devl at freenetproject.org > >http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > _ > MSN Search, for accurate results! http://search.msn.nl > > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > >
