Re: [freenet-chat] [Forwarded from FMS] 0.8.0, the two big security issues
I guess I should reply to this... On Monday 12 July 2010 22:09:12 3BUIb3S50i wrote: > o...@lkxpu0~cdv6dh0idyw4mbwkusgn~h~bs3qqvxyoxsay wrote : > > The Seeker wrote: > > > >> On 7/11/2010 6:21 AM, o...@lkxpu0~cdv6dh0idyw4mbwkusgn~h~bs3qqvxyoxsay > wrote: > >>> joh...@6kzjmqcftzffej0wthb29r63t5jkjg2xy5hzsvitg1a wrote: > >>> > Matthew Toseland > > IMHO we should attempt to fix, or at least realistically work around, > the two > big known security issues for 0.8.0, and get a paper published at the > same time > as the release. These are: > 1. The Pitch Black attack. Oskar has a good idea how to fix it but has > not yet > simulated a fix. This blocks publishing a paper, and it also prevents > use of > darknet anywhere where there may be internal attackers. As I understand > it > implementation should not be particularly difficult - the main work > needed here > is to implement it in a simple simulator and tweak it until it works, > right? > 2. The mobile attacker source tracing attack. What this means is an > attacker > knows what is to be inserted (or requested), and he is initially > distant from > the inserter. He recognises the blocks, and uses the keys' locations > (and path > folding, and possibly announcement) to move towards the originator, > gaining more > and more of the stream as he moves closer. This is primarily a problem > on > opennet, but it is also feasible on darknet - it's just massively more > expensive. It can be worked around for inserts by: > i) Inserting with a random splitfile key. THIS IS IMPLEMENTED AS OF > 1255, > provided you insert to SSK@, AND > ii) Providing an easy to use selective reinsert mechanism, AND > iii) Putting a timestamp on the inserts on any small reinsert, and only > routing > to nodes that were connected prior to that timestamp. > IMHO the second and third items are relatively easy. > > At the same time, we can substantially improve data persistence (1255 > already > does that for big files, but the insert tweaks that are going to be > tested real > soon now would probably gain us a lot more), ship Freetalk, WoT and > FlogHelper > for improved end-user functionality, a fixed wininstaller, lots of bug > fixes and > minor usability tweaks, and everything else we've done since 0.7.5. > > And having a paper published at the same time would surely help with > publicity > amongst certain kinds of folk. > >>> > >>> *lol* > >>> Is this the same Toad who managed to break all nodes since 1250+? > >>> Must have been fun for latest users, he will have to publish a lot of > >>> papers to attract more users than are currently leaving. As I understand it: - We had a few relatively simple problems around 1250. - A few builds later we introduced even segment splitting. This was disruptive in that it changed the CHKs resulting from inserts, and it did not introduce proper back compatibility code. - I therefore attempted to make all planned metadata changes at once in 1255, resulting in a great many changes at once, including some bad bugs. However it did include much improved back compatiblity support. - I did try to test thoroughly but it is difficult when there are very few testers. - Anyhow, each build since 1255 fixes a bunch of bugs, most but not all of which were introduced in 1255. > >>> > >>> New, promised features are worthless if the node is broken and resets > your > >>> datastore or up- and downloads. Fortunately we haven't had a datastore resetting bug for a *long* time. Arguably the salted hash store is incapable of such a problem short of corruption of the metadata files... As regards resetting uploads and downloads: - 1255 included a small change that might have resulted in corruption being detected when it hadn't before. - 1255 changed the internal data structures quite a bit, and this combined with a long-standing bug related to defragmentation to cause catastrophe for some people who had always defrag on startup enabled. - Fundamentally, *all* databases regularly corrupt themselves when exposed to the real world: End users with finite disk space, power cuts, unclean shutdowns, overclocked or overheating CPUs, and so on. Hence we need auto-backups. - One of the reasons that we have not yet released 0.8.0 is that there is not yet any auto-backup for the downloads database. It has been planned for some time, sorry I haven't got around to it. > >>> > >>> What is he smoking to call this *improved persistence*? The biggest change in 1255 was a couple of changes to FEC encoding, to split segments more evenly and to use extra cross-segment redundancy for files of 80MB+. All simulation work (yes we actually simulated this in advance rather than just doing things at random, thanks to evanbd) shows that this should dramatically increase the retrievability of larger files. There are also good
[freenet-chat] [Forwarded from FMS] 0.8.0, the two big security issues
o...@lkxpu0~cdv6dh0idyw4mbwkusgn~h~bs3qqvxyoxsay wrote : > The Seeker wrote: > >> On 7/11/2010 6:21 AM, o...@lkxpu0~cdv6dh0idyw4mbwkusgn~h~bs3qqvxyoxsay wrote: >>> joh...@6kzjmqcftzffej0wthb29r63t5jkjg2xy5hzsvitg1a wrote: >>> Matthew Toseland IMHO we should attempt to fix, or at least realistically work around, the two big known security issues for 0.8.0, and get a paper published at the same time as the release. These are: 1. The Pitch Black attack. Oskar has a good idea how to fix it but has not yet simulated a fix. This blocks publishing a paper, and it also prevents use of darknet anywhere where there may be internal attackers. As I understand it implementation should not be particularly difficult - the main work needed here is to implement it in a simple simulator and tweak it until it works, right? 2. The mobile attacker source tracing attack. What this means is an attacker knows what is to be inserted (or requested), and he is initially distant from the inserter. He recognises the blocks, and uses the keys' locations (and path folding, and possibly announcement) to move towards the originator, gaining more and more of the stream as he moves closer. This is primarily a problem on opennet, but it is also feasible on darknet - it's just massively more expensive. It can be worked around for inserts by: i) Inserting with a random splitfile key. THIS IS IMPLEMENTED AS OF 1255, provided you insert to SSK@, AND ii) Providing an easy to use selective reinsert mechanism, AND iii) Putting a timestamp on the inserts on any small reinsert, and only routing to nodes that were connected prior to that timestamp. IMHO the second and third items are relatively easy. At the same time, we can substantially improve data persistence (1255 already does that for big files, but the insert tweaks that are going to be tested real soon now would probably gain us a lot more), ship Freetalk, WoT and FlogHelper for improved end-user functionality, a fixed wininstaller, lots of bug fixes and minor usability tweaks, and everything else we've done since 0.7.5. And having a paper published at the same time would surely help with publicity amongst certain kinds of folk. >>> >>> *lol* >>> Is this the same Toad who managed to break all nodes since 1250+? >>> Must have been fun for latest users, he will have to publish a lot of >>> papers to attract more users than are currently leaving. >>> >>> New, promised features are worthless if the node is broken and resets your >>> datastore or up- and downloads. >>> >>> What is he smoking to call this *improved persistence*? >> >> Thanks for all your hard work testing pre-release builds, it's thanks to the >> input of people like you during testing that we get the quality of product that >> we do. > > If you tried being sarcastic you failed. > > We all are running pre-release builds, no matter what Toad calls them and > no matter whether he declares a build to be 0.80. > > Is there any developer reading and writing here? > Or maybe in Frost? > Can you explain to me why Frost was secure enough for Toad on 0.5 but not > on 0.7? > If he is panicked by the bots (which bots btw.), shouldn't it at least be > possible to announce a build in a keyed board if he still rejects to > communicate? > > Do you think adding hashes to metadata was an improvement? > > As expected the real bug hasn't been fixed, files still get corrupted when > being inserted, did I write already that this seems to be a bug in FEC data > (some downloaders are affected, some not, the older the file the lower the > chance to get it uncorrupted)? > > True, the downloading node now detects this corruption and throws away all > blocks, just saying hash mismatch. > With previous builds one was able to repair corrupted files, read about > Quickpar. > Now your node says "sorry, this file is toad, I can't pass it on to you". > > Great improvement, isn't it? > Can you please ask him to remove hash check as long as he doesn't fix the > real bug? > We know ourselves when a file is corrupted. > We don't need the node to detect it when downloading the file, we need a > node not corrupting the file when it is being inserted. > > I could continue my rant with p0's comment about NNTP not being of primary > interest for Freetalk, Webinterface to be sufficient. > > Over and out. ___ chat mailing list chat@freenetproject.org Archived: http://news.gmane.org/gmane.network.freenet.general Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/chat Or mailto:chat-requ...@freenetproject.org?subject=unsubscribe
