On Wed, Jan 29, 2003 at 03:05:19PM +0100, Michael Schierl wrote: > [I did not get that message via the mailing list, so I copied it from > gmane.network.freenet.devel. Sorry if the References are broken] > > Matthew Toseland schrieb: > > > GJ recently added a doc on FEC. Unfortunately the Wiki is down at the > > moment. Maybe it's in CVS somewhere. > > if you mean FCP-FEC.txt - that's what i have (from the devl archive). > And it is not better than the FreenetClientProtocol and the MetadataSpec > on freenetproject.org
Hmmm. It is out of date? Maybe something should be done about that. > > > > i prepended fec split blocks with Version\nRevision=1\nEnd\n as all > > > other files. as there is no diffenent description this SHOULD work > > > for fec splitfiles as well. > > > The reason that metadata in FEC splitfile chunks is deprecated is to > > allow a new feature called "automatic healing". > > Good idea. If you see someone who is ill, try to heal him. But what to > do if you cannot heal him, e.g. if he's HIV positive? > > a) Shoot him (that's what Freenet 549/550 does). This will solve the > problem with HIV fastest. > > b) Let him exist, but don't try to heal him (e.g. let user download it, > but disable healing when you see the first chunk that has metadata.) This is clearly desirable if the cost is not too high. > > c) Try to heal him with Voodoo (assume all metadata for all chunks is > the same). This is not worth the effort. If we were making the change post-1.0, it would be, but right now, it isn't. > > > The idea is that when > > you download a splitfile, after you have enough blocks and the original > > file has been reconstructed, some of the blocks that failed to fetch are > > regenerated and reinserted. > > Does that mean if all data blocks succeeded (no blocks failed), there > will be inserted some check blocks again even if thy are there or does > that mean no healing is done? No. If it got some bad blocks, and still managed to put together the file, it will regenerate some of the bad blocks and insert them. > > > This does not work if we do not know what > > metadata you have prepended to the files, and since the metadata will > > never be used anyway it doesn't make much sense. > > Is that Version/REvision=1/End no longer mandatory for files (that are > referenced from a manifest file) as well? It is not necessary for FEC chunks. FEC chunks do not need any other metadata, so it is superfluous. The docs probably need updating. > > If yes, what would you prefer: using it or not using it. Using it has > the advantage that your don't have to reinsert your old files, not using > it has the advantage that I don't have to distinguish between splitfile > chunks and files (as I do it now). I'm not sure how the round-up-to-next-power-of-2 size code works. Does it only apply to data or does it apply to (metadata + data) ? If the latter, your inserts will have ended up as twice their intended size... > > Another use for the metadata is for "test inserts" (different metadata, > so there are no key collisions), but as RemoveLocalKey seems to work > quite well now, i might remove that from FIW as well. > > > Also, there is code to > > automatically detect corrupt files, which is currently turned on, and > > which chokes on files with metadata. I have made it turn off on non-FEC > > splitfiles, but for FEC splitfiles I will need some help from GJ. > > I haven't looked into the freenet code recently, but checking if a chunk > has metadata and use that for switching off both that check and the > healing can't be too difficult. GJ has some doubts about it being all that easy. > > > > b) do *not* break data that is already on Freenet. Set a version flag > > > for newer files and don't make old files irretrievable. > > > > We did not know it was being used. > > Okay, here is an honest question that has been asked here many times: > are you active on freenet or not? Yup. I run a freesite. I can't tell you which one without shooting you afterwards. I also download files for testing purposes, I browse freenet for various reasons but mostly testing. I do not run any of the board apps though. > > Sometimes when I read devl, it seems to me that those devlopers don't > think that there are people, that > > a) use it or I do use it. > b) depend on it I am skeptical about "depend on it".. > c) don't know an alternative to it (I don't know one either.) Interesting. An alternative to it for what? Certainly there is nothing with quite the same goals and abilities, but most things freenet is used for can be and are done with other things. > > I thought releasing 0.5 was thought to help those people that use > freenet, not to break one feature at each new release. (Perhaps a TLA > forces you to do so, as the DSB has disappeared now and they don't like > you to hold data longer than half a year in Freenet. ;-)) Heh. But yeah, backwards compatibility, in a stable release series, is a good thing, as long as it does not occupy enormous resources. Hopefully we can find a quick hack to support these files. > > And for inserting a site you have 2 choices: fishtools or fiw. And it > seems that most of the new sites are with fiw. There are other alternatives if you don't need big files. And it is possible to only use freenet.client.cli.Main. > > Here some sites that use both FIW (seen from the banner) and splitfiles. > > SSK at M7yZgrl8gwtAe1xEcR5Xyv4tFsoPAgM/fectest/2// (yes, that's mine) > SSK@~tof3Yh258dhoAvC5h3oDvRZUIcPAgM/fascism// > SSK at ueuIs1YFvMzA6OedkN9XOgfWxskPAgM/SFE/4// > SSK at kDMuL~spMPkdQY6JOxqwfek2nhMPAgM/pepper/5// > i'm not sure about SSK at -w495UL3mfSlWC2c~nRAuG2fAWwPAgM/TFEE// > > and some more I don't want to post here. (no, that's no child porn) > > > As I have explained above, it is better to have no metadata on these > > chunks. However it makes sense to have back compatibility on the > > unstable branch. A mistake, well, what do we have CVS for? > > To fix it? (Staying at the old build is also possible w/o CVS.) > > > This was not > > a full release. For your information, GJ wanted to completely remove > > support for non-redundant splitfiles, because they are unreliable. But > > we do still support non-redundant splitfiles. > > Good decision. I'd hate to know having megs or gigs of data in my > datastore no one can retrieve... > > > > i haven't tested that, but if it works, be happy. If not, blame > > > Matthew. > > It would be more convincing if you did it. The freesite link on TFE > > will point to your site no matter what we do. > > Sure. But I 'know' that lotsa FIW users are lurking on devl (one day > after that request for freesites within jar files I got two mails asking > me when FIW will support that jar freesites.) > > So they can fix it themselves (if it works depends on the question above > - need of metadata in normal files - as this will send all files without > metadata). > > > > e) don't pay any more money to Matthew. He shall starve to death. > > Firstly, it was technically GJ's decision/code. > > Okay, then shoot GJ ;-) GJ knows the splitfile code better than anyone. He wrote it. He continues to work on it fairly heavily despite no financial recompense whatsoever, and little of any other kind of recompense. He's a vital part of the team, but he makes mistakes occasionally just like everyone else. > > > Secondly, we're pre-1.0. > > Hey, thanks, that's a good argument. FIW is even pre-0.10. Have to > remember that one. :) > > > Thirdly, it wasn't even an official release. So get a fucking sense of > > proportion. > > So it's not encouraged to dl newer builds via the update snapshot > function? Then I suggest to remove that link from the start menu items > of the Windows Installer. The file may stay there, but no function in > start menu. > > BTW, renaming it to "Get latest snapshot" would be a good idea as well. > Firstly, it always downloads something, even if it's the same version, > and secondly, it breaks the "u" shortcut in the start menu (both for > uninstall and for update). > > > Set mainport.params.servlet.7.params.sfDoParanoidChecks=false to disable > > paranoid CHK checking. This will be the default on the next stable > > build. This should make FEC files with metadata work. > > thank you. seems to work as exspected - i can download again :-))))) > > mihi -- Matthew Toseland toad at amphibian.dyndns.org/amphibian at users.sourceforge.net Full time freenet hacker. http://freenetproject.org/ Freenet Distribution Node (temporary) at ICTHUS. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20030129/0d0301a6/attachment.pgp>
