[freenet-dev] Thoughts about Freenet deployment on Windows platforms.
Le 23/07/2013 16:07, Juiceman a ?crit : > Sent from my wireless phone. > On Jul 23, 2013 6:20 AM, "Matthew Toseland" > wrote: >> >> On Tuesday 23 Jul 2013 11:01:16 Matthew Toseland wrote: >>> On Monday 22 Jul 2013 11:47:17 Rom. wrote: Hello, I would like to expose some thoughts about the current move for the deployment of Freenet on windows platforms. Currently as far as i understand, toad and operhiem1 are the only to maintain the windows installer of Freenet. This Freenet installer and all Freenet EXE (freenet.exe, freenetlauncher.exe ) are written in AutoHotKey. Due to its limitations (Unicode support) and recurrent false positives from anti viruses, it has been considered to move away from > AutoHotKey. Toad summarize thoughts here: https://bugs.freenetproject.org/view.php?id=5456#c9883. After reading this, i have started (with curiosity in mind) to work on an installer written with InnoSetup (https://github.com/freenet/wininstaller-staging/issues/12), which is > a true and dedicated software for creating installer for Windows. >>> >>> Right. This looks promising. But during this process and with some previous experiences related to Freenet on windows (https://bitbucket.org/romnbb/freenetfortraveler) a question came to my mind: Does Freenet needs an installer on Windows ? What this installer currently does : - Check port availability and writing settings in freenet.ini file before its first launch. - Determine how much ram to allocate to java (wrapper.java.maxmemory) - Extract all Freenet related files What if Freenet package is provided as a Zip file (Toad post: https://bugs.freenetproject.org/view.php?id=5842) and all pre-required settings handled, at a "First launch step", by the main program on windows : Freenet.exe (aka Freenet Tray) ? Upsides: - A simple zip file can easily fits the maintainability requirement. Files that change the most are related to the core of freenet (freenet.jar, plugins and maybe others) not those related to windows. - There is no doubts or problems creating zip files natively from > Linux. >>> >>> How will antiviruses deal with it? Will the file insight type thingies > still complain about it because it's not a file they've seen before? Or > will they only look at the files inside it? If a zip file bypasses the need > to deal with this, it'd be sensible; if not, it probably makes more sense > to have a discrete installer. >>> >>> One thing we've been considering is uploading to the website a few days > after we've uploaded to auto-update. All nodes download the installer, so > the file will at least be "visible", although it won't have been run. We > could even run it, with some no-op flag, but that would probably be more > trouble than it's worth. Downsides: - Users need to know what to do with this Zip file. That, most > probably, why Installers are commonly used on Windows : user will click, click , click , click ... (without reading most of the time...) >>> >>> Right. It may make it a bit harder. Which is bad, and can only be > tolerated if it's offset by fewer problems with AV's. >>> - Handle of pre-settings must be ported to the main program > (freenet.exe). And with that come an other related question. If so, what to do about Freenet.exe (freenet tray), which is the > desktop interface between Users and Freenet main interface (freenet proxy). Keep using AutoHotKey ? Re-write it in another language (Java ? FreePascal ?). >>> >>> As far as AV's are concerned, AHK isn't ideal, but if they've seen the > file before they're less likely to complain about it; if these files are > updated infrequently, we can probably get away with it? Maybe i have missed some important things about this deployment on windows that can changed my thoughts if i am aware of. So, feedbacks are welcome. Whatever the direction taken, i will gladly help to maintain the > windows side of Freenet :) . >>> >>> Awesome! Best regards, Romain. PS: sorry if my english appears to be a bit "crude". It's not my > native language and long text doesn't help. >>> >>> Your english is fine above. >>> >> One other thing: For invites, we'd probably end up distributing a zip, > with some extra files, which the installer would recognise and deal with. >> >> So the big question is, will an AV always flag a foreign zip file, or > will it only concern itself with the executables inside that zip file? >> >> > > I 'think' most AV look at executable files. Exe, dll, pdf. Etc... > > Also some look at heuristics such as trying to install in the autorun > section of the Windows registry. If the installer platform is somewhat > mainstream it is easier. I have seen InnoSetup
[freenet-dev] MAST Attack
On 2013/07/23 (Jul), at 11:54 AM, Matthew Toseland wrote: > We haven't quantified this. Maybe we should, but it doesn't seem a good use > of scarce resources to implement a toolkit for an attack that we're not sure > how to beat! At least not if I do it. I don't think we'd need a full toollkit, just some stats on: (1) the porportionality of long links [and how long they are] (I think we actually have a target value for this), (2) the probability of being able to insert your node "between" two others (routing-wise), (3) the average "width" of the network (in number of connections), ... and maybe some reasonable assumptions concerning how often an insert takes place (e.g. once a day?). As I see the attack, it would go much like a binary search: (1) effectively divide the network in half (i.e. most request going from one half to the other cross one of your nodes), (2) wait for a predicted insert to cross the tripwire (statistically you'll catch 1-in-4, so +4 days?) (3) divide the originating half-address-space in half again (works best with 50% more nodes [for three partitions]) (4) wait for another insert (now you'll catch 3/4 of "most"/50% requests, so that is 3-in-8 requests, so +2 days?) (5) move the stray partition to subdivide the wedge the request came from (6) wait for another insert (now at 7/8 address space, so another +2 days?) So under ideal attack conditions (where the target has but one unchanging peer connection, and the attacker has an as-of-yet-unmeasured number of nodes at his disposal), with an arbitrarily guessed network size of 10k... it would take 16 days to zero in on the target; and 18d for 20k, 20d for 40k, etc. What is the current network size, anyway? :-) The problem is, we focus on how easy steps 2/4/6 are, when I'm not even sure how feasible the rest is. Suppose that in the above case I have *more* than one peer connection, and even just the first hop of an insert is randomized (no random routing code required; just a random routable peer). Then that 16 day estimate is effectively multiplied by my number of peers, and becomes 1 year (and the attacker has to account for the fact that he might be chasing my peers rather than me, but that wouldn't be too hard... hmmm, maybe if they change). But in terms of the attack cost, we would need to know how many nodes it would take to "divide the network in half". I agree that this would be *significantly* cheaper than connecting to every node, but IMO the payoff for the attacker is very slow and unsure; what's more, one could only target *one* source, so it would have to be a high-value target (IMO). I would be happy to just ensure that the current code starts inserts at a random peer. -- Robert Hailey
[freenet-dev] Summary of recent opennet discussions
> One problem with the yubikey thing is it takes time to deliver them. > Hence the need to be able to just buy an invite e.g. with BTC. I agree that yubikey can't be the only way to introduce core nodes. There should be a (near) instant method as well. > On the level of tunnels - what I would consider real security - it's > a major unsolved problem in academia. (Tunnel setup on DHTs) I'm probably speaking out of my ass now (I promise to read more on it), but isn't the whole problem here that on DHTs, nodes don't have access to the complete node pool, so attacker can influence a set of nodes chosen for the tunnel, making compromised nodes progressively more likely to be chosen? If this is correct, can't we just side-step this problem by keeping full node tables in all bootstrap nodes? For a million nodes and 1K structure per node (which is probably an upper estimate), it's just 1G total. I think it can easily be fully distributed (i.e., without any partitioning) over a set of bootstrap nodes; when new core nodes contact bootstrap nodes, bootstrap nodes collectively produce whatever bootstrap information is required (set of node references for tunneling, location, etc), and update node information in their tables (like IP address, last seen time, etc). >>> If you trust your friends less than you trust the jack-boots, >>> then you probably don't have much to fear from the jack-boots. >> >> As I'd mentioned in a previous email, it's not a question of trust >> as is, it's a question of damage my friend can do to me if he >> decides to betray my trust. > > No. If they can prove that a given IP address at a given time > uploaded a given illegal file, game over. Or downloaded it, but > uploaders are worth more. That's the only safe assumption. Then neither darknet nor (current) opennet can work - at all. Basically, I'm trusting my friends to not disclose that I'm running Freenet - but no more than that. >> I still don't get why it happens. Are torrent clients run by >> significantly different slice of users? Again: I can reliably >> demonstrate speeds of *at least* 500 KB/s for any torrent with 10+ >> peers; with 20 peers, 90% of torrents, taken from all over the >> world, will max out my internet connection. This indicates average >> upload speeds of at least 50 KB/s, and probably closer to 100 KB/s >> (and with most connections being asymmetric in one way or another, >> we can safely assume that upload bandwidth is the limiting >> factor). > > Because torrent traffic is more bursty than Freenet? I.e. most of the > time they are idle? It's one possible explanation. I wonder if we can get a distribution of traffic on a node over the applications accessing it? Can I find out somehow how much physical traffic is associated with, i.e., FMS? This would be a very interesting stat, IMO. VD.
[freenet-dev] MAST Attack
On 2013/07/23 (Jul), at 5:08 AM, Matthew Toseland wrote: > if I can't beat MAST I'll go mad. Is this "Mobile Attacker Static Target"? I was not able to quickly find a past thread with such a title. > MAST: Listen for a predictable request/insert, triangulate roughly where > the originator is on the network based on the requests you receive, > announce to that location, repeat until you have the target. Cheap. Really, > really cheap. You've probably already done this, but for purposes of discussion (or my understanding), maybe we can break down the possibilities. > MAST: Listen for a predictable request/insert, Can we narrow the field to inserts? or at least judge them as separate problems? AFAICS, triangulating predictable GETs (a noisy operation, like watching for an SSK update) is far different from trying to catch a PUT (a quiet single-shot event), trying to solve them both with the same mechanism *will* make you go mad. IMO the best way to avoid predictability with GET ops is top-notch caching & cache invalidation, and I seem to recall a promising discussion of a 2-step "insert-then-reveal" mechanism for the top insert block (the only one that is predictable. Is that enough to kill the predictability, or did we find that you have to protect every insert (not just the top)? > triangulate roughly where the originator is on the network And by that, I suppose you mean "which peer you received the request from" tends to point towards the originator? To attack this point, you will probably either have to (1) route the request randomly, (2) initially route the request to a random location [and then to the target], or (3) make a "temporary and only for inserts" peer connection (which I guess converts the static target to a mobile target). > based on the requests you receive, Can we make it unlikely that an attacker receives a request? In a healthy network that would already be the case, so I presume that the efforts to secure opennet address this part. > announce to that location, I don't think we can avoid this, as that is a basic operation. > repeat until you have the target. I get the impression that there is more theory in that fragment than is initially obvious. You are presuming (perhaps correctly) that the attacker has a semi-continuous stream of requests that he can trace back, and that at each location in the network (that he chooses) he has a high probability of being somewhere in the path of these requests. Really? -- Robert Hailey
[freenet-dev] Thoughts about Freenet deployment on Windows platforms.
Sent from my wireless phone. On Jul 23, 2013 6:20 AM, "Matthew Toseland" wrote: > > On Tuesday 23 Jul 2013 11:01:16 Matthew Toseland wrote: > > On Monday 22 Jul 2013 11:47:17 Rom. wrote: > > > Hello, > > > > > > I would like to expose some thoughts about the current move for the > > > deployment of Freenet on windows platforms. > > > > > > Currently as far as i understand, toad and operhiem1 are the only to > > > maintain the windows installer of Freenet. > > > This Freenet installer and all Freenet EXE (freenet.exe, > > > freenetlauncher.exe ) are written in AutoHotKey. > > > > > > Due to its limitations (Unicode support) and recurrent false positives > > > from anti viruses, it has been considered to move away from AutoHotKey. > > > Toad summarize thoughts here: > > > https://bugs.freenetproject.org/view.php?id=5456#c9883. > > > > > > After reading this, i have started (with curiosity in mind) to work on > > > an installer written with InnoSetup > > > (https://github.com/freenet/wininstaller-staging/issues/12), which is a > > > true and dedicated software for creating installer for Windows. > > > > Right. This looks promising. > > > > > > But during this process and with some previous experiences related to > > > Freenet on windows (https://bitbucket.org/romnbb/freenetfortraveler) a > > > question came to my mind: > > > > > > Does Freenet needs an installer on Windows ? > > > > > > What this installer currently does : > > > - Check port availability and writing settings in freenet.ini file > > > before its first launch. > > > - Determine how much ram to allocate to java (wrapper.java.maxmemory) > > > - Extract all Freenet related files > > > > > > What if Freenet package is provided as a Zip file > > > (Toad post: https://bugs.freenetproject.org/view.php?id=5842) and all > > > pre-required settings handled, at a "First launch step", by the main > > > program on windows : Freenet.exe (aka Freenet Tray) ? > > > > > > Upsides: > > > - A simple zip file can easily fits the maintainability requirement. > > > Files that change the most are related to the core of freenet > > > (freenet.jar, plugins and maybe others) not those related to windows. > > > - There is no doubts or problems creating zip files natively from Linux. > > > > How will antiviruses deal with it? Will the file insight type thingies still complain about it because it's not a file they've seen before? Or will they only look at the files inside it? If a zip file bypasses the need to deal with this, it'd be sensible; if not, it probably makes more sense to have a discrete installer. > > > > One thing we've been considering is uploading to the website a few days after we've uploaded to auto-update. All nodes download the installer, so the file will at least be "visible", although it won't have been run. We could even run it, with some no-op flag, but that would probably be more trouble than it's worth. > > > > > > Downsides: > > > - Users need to know what to do with this Zip file. That, most probably, > > > why Installers are commonly used on Windows : user will click, click , > > > click , click ... (without reading most of the time...) > > > > Right. It may make it a bit harder. Which is bad, and can only be tolerated if it's offset by fewer problems with AV's. > > > > > - Handle of pre-settings must be ported to the main program (freenet.exe). > > > > > > And with that come an other related question. > > > If so, what to do about Freenet.exe (freenet tray), which is the desktop > > > interface between Users and Freenet main interface (freenet proxy). > > > Keep using AutoHotKey ? Re-write it in another language (Java ? > > > FreePascal ?). > > > > As far as AV's are concerned, AHK isn't ideal, but if they've seen the file before they're less likely to complain about it; if these files are updated infrequently, we can probably get away with it? > > > > > > Maybe i have missed some important things about this deployment on > > > windows that can changed my thoughts if i am aware of. > > > > > > So, feedbacks are welcome. > > > > > > > > > Whatever the direction taken, i will gladly help to maintain the windows > > > side of Freenet :) . > > > > Awesome! > > > > > > Best regards, > > > Romain. > > > > > > > > > PS: sorry if my english appears to be a bit "crude". It's not my native > > > language and long text doesn't help. > > > > Your english is fine above. > > > One other thing: For invites, we'd probably end up distributing a zip, with some extra files, which the installer would recognise and deal with. > > So the big question is, will an AV always flag a foreign zip file, or will it only concern itself with the executables inside that zip file? > > I 'think' most AV look at executable files. Exe, dll, pdf. Etc... Also some look at heuristics such as trying to install in the autorun section of the Windows registry. If the installer platform is somewhat mainstream it is easier. I have seen InnoSetup before where as I had not seen ahk bef
Re: [freenet-dev] Thoughts about Freenet deployment on Windows platforms.
On Tuesday 23 Jul 2013 16:56:03 Rom. wrote: > > > AV's are capable of reading inside zip (and probably other common > archives). Hence, if freenet.exe is flagged, the archive will probably > be flagged too. I have some doubts about the fact that if AV's have seen > the file before they will be less susceptible to complain. It's more a > deal with file patterns (and heuristics, like juiceman said). As > AutoHotKey Exes contain the script interpreter, which is a common > pattern (and this pattern appears in true malwares written in AHK). But > AV's behavior seems to be erratic for this point. Both are true. Most modern AV's will complain on files that they haven't seen before. For example, a recent user gave up when he got this: http://www.symantec.com/security_response/writeup.jsp?docid=2010-051308-1854-99 It's a good idea in principle - especially if the file has been substituted, it may even has a bogus signature, which is very hard to detect as there are so many SSL providers. It's an even better idea if like Freenet it's not signed at all - we can fix that though. > > Some reading if you are interested (a bit old, but still regularly true) : > AutoIt : > http://www.autoitscript.com/forum/topic/34658-are-my-autoit-exes-really-infected/ > AutoHotKey: http://www.donationcoder.com/forum/index.php?topic=15210.0 > AutoHotKey: > http://www.autohotkey.com/board/topic/29203-an-open-letter-for-antiviral-software-companies/ > That looks like what we've been seeing so far. :( signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] MAST Attack
On Tuesday 23 Jul 2013 16:30:57 Robert Hailey wrote: > > On 2013/07/23 (Jul), at 5:08 AM, Matthew Toseland wrote: > > > if I can't beat MAST I'll go mad. > > Is this "Mobile Attacker Static Target"? I was not able to quickly find a > past thread with such a title. Mobile attacker source tracing. Documented on the mailing lists sometimes, FMS etc, and there is a page on the wiki that is reasonably complete IIRC. > > > MAST: Listen for a predictable request/insert, triangulate roughly where > > the originator is on the network based on the requests you receive, > > announce to that location, repeat until you have the target. Cheap. Really, > > really cheap. > > You've probably already done this, but for purposes of discussion (or my > understanding), maybe we can break down the possibilities. > > > MAST: Listen for a predictable request/insert, > > Can we narrow the field to inserts? or at least judge them as separate > problems? AFAICS, triangulating predictable GETs (a noisy operation, like > watching for an SSK update) is far different from trying to catch a PUT (a > quiet single-shot event), trying to solve them both with the same mechanism > *will* make you go mad. Inserts are much more valuable most of the time. This is an assumption we make, which is sometimes wrong, but it's reasonable given that Freenet is a storage network. Fetches are harder because there might be multiple people fetching, but they're easier because there's no way to prevent the keys being predicted in advance. Whereas for inserts, the best defence is to insert as SSK, i.e. with random keys. Unfortunately you need to insert some predictable keys sometimes (SSK top blocks, chat posts, and reinserts). > > IMO the best way to avoid predictability with GET ops is top-notch caching & > cache invalidation, and I seem to recall a promising discussion of a 2-step > "insert-then-reveal" mechanism for the top insert block (the only one that is > predictable. Is that enough to kill the predictability, or did we find that > you have to protect every insert (not just the top)? It's not a bad idea, but it's not published, and its effect is unclear / hasn't been simulated or modelled thoroughly. Whereas proper tunnels there is lots of stuff on. Also, it's quite expensive. > > > triangulate roughly where the originator is on the network > > And by that, I suppose you mean "which peer you received the request from" > tends to point towards the originator? Yes. The attack reached you, and it's for a target location. You can deduce that it's most likely to come from the part of the keyspace where it would have been routed through you. This is relatively simple (the circular keyspace is a problem), and you combine it between lots of requests to get an increasingly accurate picture of where the inserter is likely to be. > > To attack this point, you will probably either have to (1) route the request > randomly, This is the best option. > (2) initially route the request to a random location [and then to the > target], or This is easier to trace than random routing. > (3) make a "temporary and only for inserts" peer connection (which I guess > converts the static target to a mobile target). Not sure what you are talking about here. You mean instead of bundles as tunnels, you want to random route to set up a connection, and then transfer the data, avoiding the performance penalty of relaying it through 5 random nodes? Not a huge difference given that a normal insert visits 20 nodes - and the recipient knows (passively) exactly who is requesting/inserting the data, and what they are inserting/requesting (even better than if they're a peer in the current network). Which may be bad as you'll probably need multiple such connections for performance reasons. > > > based on the requests you receive, > > Can we make it unlikely that an attacker receives a request? In a healthy > network that would already be the case, so I presume that the efforts to > secure opennet address this part. The problem is if you're doing a big reinsert there's a good chance he'll receive it. He can increase this linearly by getting more connections. > > > announce to that location, > > I don't think we can avoid this, as that is a basic operation. I'm not sure. If we can impose some sort of cost on identity creation, and then create the location randomly, we can make this more expensive. > > > repeat until you have the target. > > I get the impression that there is more theory in that fragment than is > initially obvious. > > You are presuming (perhaps correctly) that the attacker has a semi-continuous > stream of requests that he can trace back, and that at each location in the > network (that he chooses) he has a high probability of being somewhere in the > path of these requests. Really? Right, he needs enough requests initially to get a target location that is likely to be closer to the originator than he
[freenet-dev] GSoC mid-term review in one week!
Mid-terms can be submitted from the 29th of July. Students should have some idea where they are going and some code by now ... signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Securing opennet *might* be possible ... but it will be radical
It might be possible to do a mostly-distributed revalidation protocol, i.e. avoid having to contact the bootstrap/seednodes regularly, but still limit nodes and connections per IP address to make Sybil more expensive... Re: Would you pay to join opennet? From: toad-notrust@h2RzPS4fEzP0zU43GAfEgxqK2Y55~kEUNR01cWvYApI Date: Monday 22 Jul 2013 14:32:04 Groups: freenet Followup-To: freenet Message was signed by Matthew John Toseland (2010-2015 key) (Key ID: 0x75941D88). The signature is valid, but the key's validity is unknown. Public@yfAQXNfkUI3g5ghjRoRTMm2s2J8DBM9WuKJYqtkfTXc wrote: > toad-notrust@h2RzPS4fEzP0zU43GAfEgxqK2Y55~kEUNR01cWvYApI wrote : >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >> >> sonny@L3O94oWz6k~8A0dVG0jpJc55ohubTtp4RHGIVdyGBqw wrote: >> >>> toad-notrust@h2RzPS4fEzP0zU43GAfEgxqK2Y55~kEUNR01cWvYApI wrote : -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Would you pay to join opennet? Bear in mind that a paid system could have an interesting level of security - not quite up to that of a real darknet, but most attacks would be far more expensive. When I say pay, I mean pay once, to create a node. You can then stay on opennet for as long as you like. And there'd be a fallback: Transient mode. Less anonymity, but it could still use tunnels. Significantly less performance. What paying to join opennet gets you (along with having some specified amount of bandwidth per peer), is the ability to be part of the *backbone*, i.e. the storage network. Hopefully the network would be somewhat faster because of the bandwidth requirements. And it would have proper tunnels. Straw poll, but thoughts welcome. Some of the technical details spelled out on the 1.0 proposal (http://piratepad.net/TQYpA3SDu7). No idea re legal issues. Existing users would likely be grandfathered in, for a limited period and subject to the same bandwidth requirements (and IP address requirements). >>> >>> Nope! I can't afford it, second it kills my anonymity! >> >> How so? The fact that you are running Freenet (assuming it's opennet and >> you use the seednodes to bootstrap, or download it from the website), >> usually isn't anonymous. >> >> Also I didn't specify a price; if people are willing to pay $1 rather >> than $100 that would be useful information. >> - -- >> This FMS identity is not secure, it is run on my behalf by a friend. I >> will sign anything important. >> -BEGIN PGP SIGNATURE- >> Version: GnuPG v1.4.12 (GNU/Linux) >> >> iEYEAREIAAYFAlHsV6kACgkQYUNbc3WUHYjn2ACfUcxHdMrusjAyk/UgkMSSCzgC >> 6O4AoK09NyccAi0q+N/mwD7WkVPaz9vf >> =xxsP >> -END PGP SIGNATURE- >> > There could be lots of ways to pay. > > If the recent DefCon estimate that freenet has only about 32T storage > that's abysmal for a distributed storage network of 1+ nodes. I have a > hard time believing it's that low but with the move to laptops, pad and > phones it could be. People running a node 24/7 with 500G+ storage and good > bandwidth or even low bandwidth these days might be a rarity. Maybe look > at storage, bandwidth and uptime to promote good transient to core. I > think that's more valuable that a couple of bucks. > > Right now take out the seed nodes and opennet dies slowly over days and > maybe weeks. With the proposal take out meganodes and all the seednodes > and core nodes have a 24 hour death sentence and opennet is no more, a > quicker collapse. It leaves things where you always wanted it darknet > only, or scattered disconnected darknets. DoS'ing the seednodes prevents nodes from joining opennet if they are not currently connected. Lets ignore the question of payment for now and talk about IP address scarcity: Currently: - Right now, the seednodes individually limit announcements per IP address per 24 hour period. Relatively easy: - Seednodes should connect to each other and broadcast who is announcing, and thus collectively limit announcements per IP address per day. - Seednodes should include the location as well, and limit the number of locations that can be used by an IP address in one day. (E.g. to 2, so we allow one reinstall). Harder: - To announce through seednodes, you need a certificate. This is created by the bootstrap nodes (seednodes), and includes a random location chosen by them. - The certificate also includes your IP address, so you need to revalidate it centrally every time it changes. When your IP address changes you will usually need to reannounce anyway, so this isn't really any different to the current situation. - We can ensure that no one node can connect to more than N other nodes by asking it to give us a collision-detecting SSK to insert (based on its node key) listing its peers at a specific point in time, like PISCES does. - Preventing an attacker from manufacturing Sybil nodes via path folding: We require a certificate to be an opennet connection. -
Re: [freenet-dev] Bundles
On Tuesday 23 Jul 2013 01:45:26 Matthew Toseland wrote: > What about bundles? > - Inserts are mainly slow because of the downstream nodes, right? Inserts go > to 20+ nodes atm. > - So the number of parallel bundles necessary, even to maintain full speed, > may be relatively low? > - Lets say we go 5 hops random routed as a bundle (i.e. without diverging). > This should get us a significant distance from the originator. Then we random > route individual requests for another 2 hops (line + starburst). Then we do > normal inserts (but without the don't-cache-at-high-htl hack, since we don't > cache up to starting the regular inserts). This would slow down inserts by > roughly 1/3rd. If we can limit it to 4 or 5 such bundles, we can probably > achieve performance close to not bundling at all, but going the same number > of hops. > - Short term: Accept the requests one at a time. No storage. Possibly check > uptime before accepting, maybe option to accept on previous node if it goes > down etc. > - Medium term: Store the data (on disk!). Once sent committed, we will send > the data even if we restart. Then insert it (as a local insert). Limit the > number pending per peer etc. Fire and forget. Completion: Top block is a > FEC'ed SSK/PSK splitfile (fetchable from a single URI, not too big, but > divided up among the various bundle inserters), with a highish threshold, so > it is completed collaboratively. Security/reliability tradeoff. Possibility > of recovering data and doing a regular insert. > - Long term we'd like to start a large number of inserts all at once at the > destination, as this further reduces the possibilities for MAST. But it has > load management implications. > - 20 hops is probably too high anyway - but it may be problematic / > significant testing etc to establish the correct HTL. > > Implement basic bundle protocol (no storage, relatively small bundles, > probably need an uptime check before accepting), and tests to compare speed > of different numbers of bundles vs regular inserts going the same distance. > E.g. by a hack to use a lower HTL on starting so they both go the same number > of hops. How many bundles do we need to get 80% of the performance of just > sending the inserts? > > Even if we send a bundle to every single peer, we'd still have a significant > security gain relative to the current situation... > > Various people have contributed to this idea's evolution over the years... > One possible complication nextgens pointed out: We may need to implement the bundles API, and group related requests (from the same splitfile insert, or even FCP connection), relatively early, i.e. before we formally deploy this. Otherwise we have passive attackers picking up correlations between different pseudonymous identities. But testing it to get the basic code and the performance parameters should be easy enough. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Thoughts about Freenet deployment on Windows platforms.
On Tuesday 23 Jul 2013 11:01:16 Matthew Toseland wrote: > On Monday 22 Jul 2013 11:47:17 Rom. wrote: > > Hello, > > > > I would like to expose some thoughts about the current move for the > > deployment of Freenet on windows platforms. > > > > Currently as far as i understand, toad and operhiem1 are the only to > > maintain the windows installer of Freenet. > > This Freenet installer and all Freenet EXE (freenet.exe, > > freenetlauncher.exe ) are written in AutoHotKey. > > > > Due to its limitations (Unicode support) and recurrent false positives > > from anti viruses, it has been considered to move away from AutoHotKey. > > Toad summarize thoughts here: > > https://bugs.freenetproject.org/view.php?id=5456#c9883. > > > > After reading this, i have started (with curiosity in mind) to work on > > an installer written with InnoSetup > > (https://github.com/freenet/wininstaller-staging/issues/12), which is a > > true and dedicated software for creating installer for Windows. > > Right. This looks promising. > > > > But during this process and with some previous experiences related to > > Freenet on windows (https://bitbucket.org/romnbb/freenetfortraveler) a > > question came to my mind: > > > > Does Freenet needs an installer on Windows ? > > > > What this installer currently does : > > - Check port availability and writing settings in freenet.ini file > > before its first launch. > > - Determine how much ram to allocate to java (wrapper.java.maxmemory) > > - Extract all Freenet related files > > > > What if Freenet package is provided as a Zip file > > (Toad post: https://bugs.freenetproject.org/view.php?id=5842) and all > > pre-required settings handled, at a "First launch step", by the main > > program on windows : Freenet.exe (aka Freenet Tray) ? > > > > Upsides: > > - A simple zip file can easily fits the maintainability requirement. > > Files that change the most are related to the core of freenet > > (freenet.jar, plugins and maybe others) not those related to windows. > > - There is no doubts or problems creating zip files natively from Linux. > > How will antiviruses deal with it? Will the file insight type thingies still > complain about it because it's not a file they've seen before? Or will they > only look at the files inside it? If a zip file bypasses the need to deal > with this, it'd be sensible; if not, it probably makes more sense to have a > discrete installer. > > One thing we've been considering is uploading to the website a few days after > we've uploaded to auto-update. All nodes download the installer, so the file > will at least be "visible", although it won't have been run. We could even > run it, with some no-op flag, but that would probably be more trouble than > it's worth. > > > > Downsides: > > - Users need to know what to do with this Zip file. That, most probably, > > why Installers are commonly used on Windows : user will click, click , > > click , click ... (without reading most of the time...) > > Right. It may make it a bit harder. Which is bad, and can only be tolerated > if it's offset by fewer problems with AV's. > > > - Handle of pre-settings must be ported to the main program (freenet.exe). > > > > And with that come an other related question. > > If so, what to do about Freenet.exe (freenet tray), which is the desktop > > interface between Users and Freenet main interface (freenet proxy). > > Keep using AutoHotKey ? Re-write it in another language (Java ? > > FreePascal ?). > > As far as AV's are concerned, AHK isn't ideal, but if they've seen the file > before they're less likely to complain about it; if these files are updated > infrequently, we can probably get away with it? > > > > Maybe i have missed some important things about this deployment on > > windows that can changed my thoughts if i am aware of. > > > > So, feedbacks are welcome. > > > > > > Whatever the direction taken, i will gladly help to maintain the windows > > side of Freenet :) . > > Awesome! > > > > Best regards, > > Romain. > > > > > > PS: sorry if my english appears to be a bit "crude". It's not my native > > language and long text doesn't help. > > Your english is fine above. > One other thing: For invites, we'd probably end up distributing a zip, with some extra files, which the installer would recognise and deal with. So the big question is, will an AV always flag a foreign zip file, or will it only concern itself with the executables inside that zip file? signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Fwd: Bundles
One last gasp on security. There are lots of things we can do in the longer term, notably darknet enhancements and PISCES tunnels. But if I can't beat MAST I'll go mad. ROUGH PRIORITY LIST: (For me; there are others working on WoT, UI enhancements etc) - Smallish usability fixes etc. (Including e.g. wrapper fix, Spider fixes) - First stage of bundles. (Fix MAST!) - Rip db4o out. (Major usability improvement IMHO) If I'm lucky I'll be here by the time I leave - Darknet enhancements. - New SSKs. (Important for lots of things) - ... - PISCES. - ... - Opennet: IP/bandwidth scarcity. - ... --- Begin Message --- What about bundles? - Inserts are mainly slow because of the downstream nodes, right? Inserts go to 20+ nodes atm. - So the number of parallel bundles necessary, even to maintain full speed, may be relatively low? - Lets say we go 5 hops random routed as a bundle (i.e. without diverging). This should get us a significant distance from the originator. Then we random route individual requests for another 2 hops (line + starburst). Then we do normal inserts (but without the don't-cache-at-high-htl hack, since we don't cache up to starting the regular inserts). This would slow down inserts by roughly 1/3rd. If we can limit it to 4 or 5 such bundles, we can probably achieve performance close to not bundling at all, but going the same number of hops. - Short term: Accept the requests one at a time. No storage. Possibly check uptime before accepting, maybe option to accept on previous node if it goes down etc. - Medium term: Store the data (on disk!). Once sent committed, we will send the data even if we restart. Then insert it (as a local insert). Limit the number pending per peer etc. Fire and forget. Completion: Top block is a FEC'ed SSK/PSK splitfile (fetchable from a single URI, not too big, but divided up among the various bundle inserters), with a highish threshold, so it is completed collaboratively. Security/reliability tradeoff. Possibility of recovering data and doing a regular insert. - Long term we'd like to start a large number of inserts all at once at the destination, as this further reduces the possibilities for MAST. But it has load management implications. - 20 hops is probably too high anyway - but it may be problematic / significant testing etc to establish the correct HTL. Implement basic bundle protocol (no storage, relatively small bundles, probably need an uptime check before accepting), and tests to compare speed of different numbers of bundles vs regular inserts going the same distance. E.g. by a hack to use a lower HTL on starting so they both go the same number of hops. How many bundles do we need to get 80% of the performance of just sending the inserts? Even if we send a bundle to every single peer, we'd still have a significant security gain relative to the current situation... Various people have contributed to this idea's evolution over the years... signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl--- End Message --- signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Summary of recent opennet discussions
On Tuesday 23 Jul 2013 09:32:31 Victor Denisov wrote: > > One problem with the yubikey thing is it takes time to deliver them. > > Hence the need to be able to just buy an invite e.g. with BTC. > > I agree that yubikey can't be the only way to introduce core nodes. > There should be a (near) instant method as well. > > > On the level of tunnels - what I would consider real security - it's > > a major unsolved problem in academia. (Tunnel setup on DHTs) > > I'm probably speaking out of my ass now (I promise to read more on it), > but isn't the whole problem here that on DHTs, nodes don't have access > to the complete node pool, so attacker can influence a set of nodes > chosen for the tunnel, making compromised nodes progressively more > likely to be chosen? If this is correct, can't we just side-step this > problem by keeping full node tables in all bootstrap nodes? > > For a million nodes and 1K structure per node (which is probably an > upper estimate), it's just 1G total. I think it can easily be fully > distributed (i.e., without any partitioning) over a set of bootstrap > nodes; when new core nodes contact bootstrap nodes, bootstrap nodes > collectively produce whatever bootstrap information is required (set of > node references for tunneling, location, etc), and update node > information in their tables (like IP address, last seen time, etc). How does this help with tunnel setup? Also, it's only practical if core nodes have very high uptime; update traffic is the problem for such schemes in practice. > > >>> If you trust your friends less than you trust the jack-boots, > >>> then you probably don't have much to fear from the jack-boots. > >> > >> As I'd mentioned in a previous email, it's not a question of trust > >> as is, it's a question of damage my friend can do to me if he > >> decides to betray my trust. > > > > No. If they can prove that a given IP address at a given time > > uploaded a given illegal file, game over. Or downloaded it, but > > uploaders are worth more. That's the only safe assumption. > > Then neither darknet nor (current) opennet can work - at all. Basically, > I'm trusting my friends to not disclose that I'm running Freenet - but > no more than that. No, anything with proper tunnels can work fine. > > >> I still don't get why it happens. Are torrent clients run by > >> significantly different slice of users? Again: I can reliably > >> demonstrate speeds of *at least* 500 KB/s for any torrent with 10+ > >> peers; with 20 peers, 90% of torrents, taken from all over the > >> world, will max out my internet connection. This indicates average > >> upload speeds of at least 50 KB/s, and probably closer to 100 KB/s > >> (and with most connections being asymmetric in one way or another, > >> we can safely assume that upload bandwidth is the limiting > >> factor). > > > > Because torrent traffic is more bursty than Freenet? I.e. most of the > > time they are idle? > > It's one possible explanation. I wonder if we can get a distribution of > traffic on a node over the applications accessing it? Can I find out > somehow how much physical traffic is associated with, i.e., FMS? This > would be a very interesting stat, IMO. We don't collect that data at present. We could, if we can identify requests associated with FMS. > > VD. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Thoughts about Freenet deployment on Windows platforms.
On Monday 22 Jul 2013 11:47:17 Rom. wrote: > Hello, > > I would like to expose some thoughts about the current move for the > deployment of Freenet on windows platforms. > > Currently as far as i understand, toad and operhiem1 are the only to > maintain the windows installer of Freenet. > This Freenet installer and all Freenet EXE (freenet.exe, > freenetlauncher.exe ) are written in AutoHotKey. > > Due to its limitations (Unicode support) and recurrent false positives > from anti viruses, it has been considered to move away from AutoHotKey. > Toad summarize thoughts here: > https://bugs.freenetproject.org/view.php?id=5456#c9883. > > After reading this, i have started (with curiosity in mind) to work on > an installer written with InnoSetup > (https://github.com/freenet/wininstaller-staging/issues/12), which is a > true and dedicated software for creating installer for Windows. Right. This looks promising. > > But during this process and with some previous experiences related to > Freenet on windows (https://bitbucket.org/romnbb/freenetfortraveler) a > question came to my mind: > > Does Freenet needs an installer on Windows ? > > What this installer currently does : > - Check port availability and writing settings in freenet.ini file > before its first launch. > - Determine how much ram to allocate to java (wrapper.java.maxmemory) > - Extract all Freenet related files > > What if Freenet package is provided as a Zip file > (Toad post: https://bugs.freenetproject.org/view.php?id=5842) and all > pre-required settings handled, at a "First launch step", by the main > program on windows : Freenet.exe (aka Freenet Tray) ? > > Upsides: > - A simple zip file can easily fits the maintainability requirement. > Files that change the most are related to the core of freenet > (freenet.jar, plugins and maybe others) not those related to windows. > - There is no doubts or problems creating zip files natively from Linux. How will antiviruses deal with it? Will the file insight type thingies still complain about it because it's not a file they've seen before? Or will they only look at the files inside it? If a zip file bypasses the need to deal with this, it'd be sensible; if not, it probably makes more sense to have a discrete installer. One thing we've been considering is uploading to the website a few days after we've uploaded to auto-update. All nodes download the installer, so the file will at least be "visible", although it won't have been run. We could even run it, with some no-op flag, but that would probably be more trouble than it's worth. > > Downsides: > - Users need to know what to do with this Zip file. That, most probably, > why Installers are commonly used on Windows : user will click, click , > click , click ... (without reading most of the time...) Right. It may make it a bit harder. Which is bad, and can only be tolerated if it's offset by fewer problems with AV's. > - Handle of pre-settings must be ported to the main program (freenet.exe). > > And with that come an other related question. > If so, what to do about Freenet.exe (freenet tray), which is the desktop > interface between Users and Freenet main interface (freenet proxy). > Keep using AutoHotKey ? Re-write it in another language (Java ? > FreePascal ?). As far as AV's are concerned, AHK isn't ideal, but if they've seen the file before they're less likely to complain about it; if these files are updated infrequently, we can probably get away with it? > > Maybe i have missed some important things about this deployment on > windows that can changed my thoughts if i am aware of. > > So, feedbacks are welcome. > > > Whatever the direction taken, i will gladly help to maintain the windows > side of Freenet :) . Awesome! > > Best regards, > Romain. > > > PS: sorry if my english appears to be a bit "crude". It's not my native > language and long text doesn't help. Your english is fine above. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
