Re: [freenet-dev] Opera Unite versus Freenet
On Saturday 16 January 2010 15:19:32 David ‘Bombe’ Roden wrote: > On Thursday 14 January 2010 17:18:31 Matthew Toseland wrote: > > > Content of Evil was uploaded from a dial-up modem for several years! > > A-HA! So you admit to being CofE? :) LOL, if I told you that I'd have to kill you. And no, I'm not CofE. Really. As you'll easily conclude by going to read it (it's mirrored on 0.7). He said he used a modem to upload it. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Opera Unite versus Freenet
Matthew Toseland skrev: > On Saturday 16 January 2010 15:19:32 David ‘Bombe’ Roden wrote: >> On Thursday 14 January 2010 17:18:31 Matthew Toseland wrote: >> >>> Content of Evil was uploaded from a dial-up modem for several years! >> A-HA! So you admit to being CofE? :) > > LOL, if I told you that I'd have to kill you. And no, I'm not CofE. Really. > As you'll easily conclude by going to read it (it's mirrored on 0.7). He said > he used a modem to upload it. /me looks suspiciously at Matthew :P - Zero3 ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Moving the french wiki to sourceforge?
On Friday 15 January 2010 23:38:16 Ian Clarke wrote: > On Fri, Jan 8, 2010 at 6:09 PM, Matthew Toseland > wrote: > > On Tuesday 05 January 2010 01:35:19 Ian Clarke wrote: > >> 2010/1/4 Clément Vollet > >> > >> > Since the official english wiki is being moved to sourceforge, can we > >> > also > >> > put > >> > the french wiki there too? > >> > It's already on mediawiki, so it should be pretty straightforward (?). > >> > > >> > >> You may want to wait a bit, as its possible we'll be migrating elsewhere, > >> perhaps to emu (our own server). If so, you'll be welcome to migrate the > >> french version there too. > > > > We do not want to host it on emu! There are many free wiki hosting > > services, and we do not want to have to > > admin anything that isn't absolutely vital. Moving the wiki off emu is an > > important step towards getting rid of emu. > > Another important step will be moving the bug tracker. > > So what is the alternative? Sourceforge seems to be a usability > nightmare (as its always been). You are the only person to have had any serious usability issue with it. However it is slow. It should probably be hosted elsewhere but it does need to be an externally hosted (= *not* on a php host, no manual upgrades) MediaWiki. > > We should move stuff off Emu if we can, but we shouldn't infinitely > delay something the project needs if Emu is the only viable option. No, I have neither the time nor the knowledge to admin emu competently, so it is important that as little as possible depend on it. And we have made much progress to that goal. Anyway there are lots of other free wiki providers. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] JetBrains IDEA license key for Freenet development
Once again, JetBrains has kindly provided an open source license to the project for Freenet development. If you would like access to this, please email me off-list (but remember its only for Freenet work). Ian. -- Ian Clarke CEO, SenseArray Email: i...@sensearray.com Ph: +1 512 422 3588 ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
What happened here? I can't see this note! --- Begin Message --- A NOTE has been added to this issue. == https://bugs.freenetproject.org/view.php?id=3693 == Reported By:Matthew John Toseland (Toad) Assigned To: == Project:Freenet Issue ID: 3693 Category: spider/librarian Reproducibility:have not tried Severity: minor Priority: normal Status: new Target Version: 0.8.0 == Date Submitted: 2009-11-10 13:00 UTC Last Modified: 2010-01-11 23:54 UTC == Summary:Put before in XML indexes Description: This would allow us to only parse once, saving *a lot* of disk reads/writes/encrypts/decrypts. We will need to be back compatible in parsing for a while, which can be best accomplished by a version number change in the main document (index.xml). == Relationships ID Summary -- child of0003694 Decompress and parse in one action == -- (0006463) Mike Bush (reporter) - 2010-01-11 23:54 https://bugs.freenetproject.org/view.php?id=3693#c6463 -- Doing it this way round would prevent the ability to produce partial results, I'm in favour of scrapping them in the XMLIndex though since it's not available in this version of the Library interface and the speed/memory usage improvement is obviously much more important. The parser can be much simpler without that provision too. If no one has started it, you can assign it to me. Issue History Date ModifiedUsername FieldChange == 2009-11-10 13:00 Matthew John Toseland (Toad)New Issue 2009-11-10 13:00 Matthew John Toseland (Toad)milestone => 0.8 2009-11-10 13:00 Matthew John Toseland (Toad)svn-revision => 2009-11-10 13:33 Matthew John Toseland (Toad)Relationship added child of 0003694 2009-11-19 08:18 Juiceman Issue Monitored: Juiceman 2010-01-11 23:54 Mike Bush Note Added: 0006463 == --- End Message --- signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
He deleted it again: 2010-01-11 23:54 Mike Bush Note Added: 0006463 2010-01-12 00:03 Mike Bush Note Deleted: 0006463 ;) - Zero3 Matthew Toseland skrev: > What happened here? I can't see this note! > > > > > Emne: > [Freenet 0003693]: Put before in XML indexes > Fra: > Mantis Bug Tracker > Dato: > Mon, 11 Jan 2010 23:54:11 + > Til: > t...@amphibian.dyndns.org > > Til: > t...@amphibian.dyndns.org > > > A NOTE has been added to this issue. > == > https://bugs.freenetproject.org/view.php?id=3693 > == > Reported By:Matthew John Toseland (Toad) > Assigned To: > == > Project:Freenet > Issue ID: 3693 > Category: spider/librarian > Reproducibility:have not tried > Severity: minor > Priority: normal > Status: new > Target Version: 0.8.0 > == > Date Submitted: 2009-11-10 13:00 UTC > Last Modified: 2010-01-11 23:54 UTC > == > Summary:Put before in XML indexes > Description: > This would allow us to only parse once, saving *a lot* of disk > reads/writes/encrypts/decrypts. We will need to be back compatible in > parsing for a while, which can be best accomplished by a version number > change in the main document (index.xml). > == > Relationships ID Summary > -- > child of0003694 Decompress and parse in one action > == > > -- > (0006463) Mike Bush (reporter) - 2010-01-11 23:54 > https://bugs.freenetproject.org/view.php?id=3693#c6463 > -- > Doing it this way round would prevent the ability to produce partial > results, I'm in favour of scrapping them in the XMLIndex though since it's > not available in this version of the Library interface and the speed/memory > usage improvement is obviously much more important. The parser can be much > simpler without that provision too. > > If no one has started it, you can assign it to me. > > Issue History > Date ModifiedUsername FieldChange > == > 2009-11-10 13:00 Matthew John Toseland (Toad)New Issue > > > 2009-11-10 13:00 Matthew John Toseland (Toad)milestone => 0.8 > > > 2009-11-10 13:00 Matthew John Toseland (Toad)svn-revision => > > > 2009-11-10 13:33 Matthew John Toseland (Toad)Relationship added child of > 0003694 > 2009-11-19 08:18 Juiceman Issue Monitored: Juiceman > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > == > > > > > > ___ > Devl mailing list > Devl@freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
On Tuesday 19 January 2010 16:14:11 Christian Funder Sommerlund (Zero3) wrote: > He deleted it again: > > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > 2010-01-12 00:03 Mike Bush Note Deleted: 0006463 > > ;) > > - Zero3 > > Matthew Toseland skrev: > > What happened here? I can't see this note! > > > > > > > > > > Emne: > > [Freenet 0003693]: Put before in XML indexes > > Fra: > > Mantis Bug Tracker > > Dato: > > Mon, 11 Jan 2010 23:54:11 + > > Til: > > t...@amphibian.dyndns.org > > > > Til: > > t...@amphibian.dyndns.org > > > > > > A NOTE has been added to this issue. > > == > > https://bugs.freenetproject.org/view.php?id=3693 > > == > > Reported By:Matthew John Toseland (Toad) > > Assigned To: > > == > > Project:Freenet > > Issue ID: 3693 > > Category: spider/librarian > > Reproducibility:have not tried > > Severity: minor > > Priority: normal > > Status: new > > Target Version: 0.8.0 > > == > > Date Submitted: 2009-11-10 13:00 UTC > > Last Modified: 2010-01-11 23:54 UTC > > == > > Summary:Put before in XML indexes > > Description: > > This would allow us to only parse once, saving *a lot* of disk > > reads/writes/encrypts/decrypts. We will need to be back compatible in > > parsing for a while, which can be best accomplished by a version number > > change in the main document (index.xml). > > == > > Relationships ID Summary > > -- > > child of0003694 Decompress and parse in one action > > == > > > > -- > > (0006463) Mike Bush (reporter) - 2010-01-11 23:54 > > https://bugs.freenetproject.org/view.php?id=3693#c6463 > > -- > > Doing it this way round would prevent the ability to produce partial > > results, I'm in favour of scrapping them in the XMLIndex though since it's > > not available in this version of the Library interface and the speed/memory > > usage improvement is obviously much more important. The parser can be much > > simpler without that provision too. > > > > If no one has started it, you can assign it to me. > > > > Issue History > > Date ModifiedUsername FieldChange > > > > == > > 2009-11-10 13:00 Matthew John Toseland (Toad)New Issue > > > > > > 2009-11-10 13:00 Matthew John Toseland (Toad)milestone => > > 0.8 > > > > 2009-11-10 13:00 Matthew John Toseland (Toad)svn-revision => > > > > > > 2009-11-10 13:33 Matthew John Toseland (Toad)Relationship added child > > of > > 0003694 > > 2009-11-19 08:18 Juiceman Issue Monitored: Juiceman > > > > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > > > > == Doh, was I wrong to assign it to him then? signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] If anyone would like to do some math for WoT please read
On Wednesday 13 January 2010 00:33:21 xor wrote: > > Currenty, the WoT IdentityFetcher permanently subscribes to the trust list > USK > of EVERY known identity, resulting in a total load of n*n USK subscriptions > on > the network - each identity subscribes to the USK of every other identity. > > (Even though it does O(n²), we will soon release the current code as WoT 0.4 > on the plugins page labeled as testing because it is guaranteed to fetch all > identities, i.e. guraanteed to "just work" and has proven to be very stable > in > all other means. And WoT 0.4 in general is optimized for new features & > stability, not for performance. 0.5 is planned to fix usability and > performance issues and bugs of course. - First implement all functions to > work > properly, then optimize and see if it still works the same as the previous > version, that's my plan) IMHO O(n) per node is not as big a problem as you might think on Freenet, because much of the work ends up being shared between different nodes. It is definitely not the same as O(n^2). However, we do need to deal with it, as n will eventually get rather large, and we have to track multiple keys for each of these identities. > > I propose the following change for WoT 0.5 to get rid of the permanent O(N*N): > > - Permanent USK subscriptions should only be done to rank1 identities: The > identities to which an OwnIdentity has assigned a (positive of course) trust > value (= the users trustees). This should give us "small world > network"-effects > already. And if I'm right it changes everything from n² to some logarithmic > function of n. That is to be proved :) > > - The trust lists which are fetched of the rank1 identities will give us new > edition hints for new trust lists of other identities. Its called "hints" > because we cannot tell the USK manager that those editions REALLY exist, they > might be spoofed, i.e. too high. The USK manager supports specifying an > edition as hint-only. > > - So for rank 2 identities (Y is rank 2 <=> an own identity trusts X and X > trusts Y): If we receive a new edition hint for a rank2 identity, immediately > start a fetch for the hinted edition. Give the fetch a certain amount of time > to succeed, abort it if it doesn't. I propose 1 hour as the time limit. Tell > the USK manager NOT to attempt to search for any newer editions than the > hinted one: We want to use the information gathered from our rank1 identities > instead. But do not disable the fetching attempts of older editions than the > hinted one - to detect bogus hints. All you need to do is fetch that one slot. Give it say 3 attempts. That won't take an hour and you don't need a time limit, it's a simple request. > > - For rank3 and above identities, do the same as for rank2 but only allow a > hint-fetch-attempt every N days. N is to be carefully tweaked, I propose we > start with 3. > > - If any identity has not had a fetch attempt for >= M days, try to fetch > them > once. In other words, each identity is fetched at max every M days so that > identities which are not "hit" by the random heuristic above still get > fetched > every now and then. Reasonable. > DO allow the USK manager to search for editions higher > than the current hinted one. Do this to catch bugs in the other heuristics. > Log if an edition was fetched which was higher than the latest received hint. > I propose M to be 7 days. Fetching many slots ahead by doing a full USK retrieval uses much more resources, is it worth it? > > Now the question is: Who can calculate the maximal, minimal and average > complexity of my algorithm? :) > > I'm aware of the fact that the last point (fetch all identities every 7 days) > introduces some O(n²) again but IMHO its necessary for preventing bugs which > could cause some identities to be NEVER fetched. We can disable it after > we're > sure that it's not needed. Is it? Won't hints suffice to quickly deal with any such problem? > > And if anyone has suggestions for improvements, feel free to give them. I > consider my proposed rank1 and rank2 behavior as quite natural, but for rank3 > and above we might for example use an exponential function of rank, score and > whatever for the minimal delay between fetches. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Library - request for testers
On Monday 11 January 2010 16:49:30 Ximin Luo wrote: > On 11/01/10 16:30, Matthew Toseland wrote: > > On Sunday 10 January 2010 11:35:47 Ximin Luo wrote: > >> http://github.com/freenet/plugin-Library-staging > >> > >> If anyone has some free time, please check out the above repo and run the > >> test: > >> > >> $ git clone git://github.com/freenet/plugin-Library-staging.git $ cd > >> plugin-Library-staging $ ant junit > >> > >> Open up ./test/plugins/Library/index/BIndexTest.java and fiddle about with > >> the numbers at the top, and run the test with different settings. > >> BIndexTest is the first test to run; you can just hit ctrl-C after it's > >> done to skip the rest. > >> > >> What you're testing is an algorithm to write to a on-freenet b-tree. Ie. > >> most of the data is on freenet, and you're just pulling/pushing in > >> (approximately) the minimum nodes needed to update it, plus a new root > >> node. > >> > >> This would be useful for search indexes (which are huge), and could also > >> be useful for other things that need to store a huge on-freenet data > >> structure. Obviously, it needs a shit load of testing first, so any help > >> would be appreciated. > > > > IMHO any such on-freenet data structure should be single owner, to avoid > > concurrent updates / finding the latest version problems, and the data > > should all be duplicated locally so it can be easily reinserted. > > The only thing that is "owned" is the USK root pointer to the rest of the data > structure. The data nodes are currently stored as CHK files, so they are > effectively immutable, which means concurrency isn't a problem. Multiple root > USKs from different owners could point to the same CHK nodes, making fork > operations trivial. Right. But each root has an owner. And a forked index will have both a different root and some set of different sub-paths. The point is we can't have a single linear USK space with private key known by many people, because this will be chaotic. The correct model is to fork and merge a la DSCM's or wiki-over-DSCM's. > > > However, this is very important stuff, because It will allow us to generate > > indexes gradually during spidering and much more efficiently (no waiting > > several days while doing hideous numbers of seeks while it writes the > > index). There may be other uses too in the medium term, as infinity0 hints > > at, e.g. published databases etc. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] [REQ]Pushing the french translation on website-official
On Wednesday 13 January 2010 10:24:57 Clément Vollet wrote: > Hello, > > I think that we can push the french translation of the website, since almost > all important pages are translated (that is index, download, whatis, people, > donate, sponsors). What's left is the faq (about 50% done), but I don't think > it's sufficient to block the release of the translated site. > Moreover, all pages which aren't translated are displayed in english. > > For now, we can push it without doing any langage recognition, and do that > latter (there is links to other language on top right of the site). > > I'd like to add that I made some changes to the css to fix the menu > displaying > (it wasn't scaling well with font size), especially when hovering item. > However, the items will now be centered, since I can't find a way to fix this > without center them. > > So, please push (except if you have something more important to do, of course > :p ), or if not, tell me why. Ok, will try to push this soon. signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Uservoice update
Top 5 items on Uservoice: 1) One GUI for all (263 votes) IMHO integrating Freetalk and FlogHelper will go a long way towards satisfying people who voted for this, although the original poster wants a dedicated browser thingy. 2) Write a killer file-sharing application (232 votes) IMHO there are a number of issues here: - Current data persistence sucks. We are working on it and need to do more work on it. - It is relatively hard to search for files. We need a good WoT-based spamproof fast file search system. - General demand from many folks for insert on demand. Maybe it worked on Frost, maybe it didn't, but Frost is dead. The main advantages of insert on demand are 1) that you can "share" a directory immediately, 2) you don't need to wait for the insert, 3) the set of available data, or available data at a given speed, is larger. I am skeptical, there are several problems with it (trust, security, different views of "network pollution" ...), but in any case improving data persistence and making it easy to search for files are both important, and once we have these we can consider insert on demand. And there *are* options for relatively safe reinsert on demand, although they are messy... Or am I being unduly dogmatic and long-termist here? Maybe a simple WoT-based file search and share and reinsert on demand system would gain us so many more users that we should do it anyway, even if insert on demand is risky and not necessary long term? This does rely on distributed searching, and it could be a while ... 3) Add a "pause" feature (188 votes) This might actually be partly due to the frequently fixed and re-broken "Freenet doesn't restart after I change IP address / my laptop sleeps / etc" bug. We need to fix that ASAP. It is also closely related to the general fact that it takes quite some time for a node to get up to speed on opennet after it's been down for a while. I have several ideas about how to improve that. We need to make Freenet more low-uptime-friendly, even if that means using unreasonable levels of data redundancy. It is also closely related to the system tray icon. We have one on Windows, we will soon have one on Mac, once we have both I will write one for Linux. But the specific demand is that we have some means for the node to "pause" its connections, so that it can get them back quickly. There has been much debate on this on IRC, lists and the bug tracker, it is probably possible and is related to some others ... Other options include just cutting the bandwidth limit or turning off the client layer etc. 4) use the port 80,443,53,1863 for comunication (171 votes) Remains remarkably popular! There are some simple options such as using popular UDP ports, but any serious progress is going to require a TCP transport plugin, and probably an HTTP one as well, maybe with multiplexing support for smaller web servers. SSL would be nice too. Of course these will only work when port forwarding works - but it does work a lot of the time. In any case, this requires transport plugins, which are quite feasible but may well be a largish amount of work. 5) Implement reinsert on demand (169 votes) See the item on filesharing above! Since Frost has been largely abandoned, there is a major gap for filesharing it seems... 6) Chat (108 votes) Included because #5 and #2 duplicate each other. The originator sees this as real-time. digger3 is working on a prototype of real time chat over Freenet, which may be extended to microblogging. He says lag is 10-30 sec and can be lowered considerably by relatively small network changes... signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Opera Unite versus Freenet
On Saturday 16 January 2010 15:19:32 David ?Bombe? Roden wrote: > On Thursday 14 January 2010 17:18:31 Matthew Toseland wrote: > > > Content of Evil was uploaded from a dial-up modem for several years! > > A-HA! So you admit to being CofE? :) LOL, if I told you that I'd have to kill you. And no, I'm not CofE. Really. As you'll easily conclude by going to read it (it's mirrored on 0.7). He said he used a modem to upload it. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/60bfe825/attachment.pgp>
[freenet-dev] Opera Unite versus Freenet
Matthew Toseland skrev: > On Saturday 16 January 2010 15:19:32 David ?Bombe? Roden wrote: >> On Thursday 14 January 2010 17:18:31 Matthew Toseland wrote: >> >>> Content of Evil was uploaded from a dial-up modem for several years! >> A-HA! So you admit to being CofE? :) > > LOL, if I told you that I'd have to kill you. And no, I'm not CofE. Really. > As you'll easily conclude by going to read it (it's mirrored on 0.7). He said > he used a modem to upload it. /me looks suspiciously at Matthew :P - Zero3
[freenet-dev] Moving the french wiki to sourceforge?
On Friday 15 January 2010 23:38:16 Ian Clarke wrote: > On Fri, Jan 8, 2010 at 6:09 PM, Matthew Toseland > wrote: > > On Tuesday 05 January 2010 01:35:19 Ian Clarke wrote: > >> 2010/1/4 Cl?ment Vollet > >> > >> > Since the official english wiki is being moved to sourceforge, can we > >> > also > >> > put > >> > the french wiki there too? > >> > It's already on mediawiki, so it should be pretty straightforward (?). > >> > > >> > >> You may want to wait a bit, as its possible we'll be migrating elsewhere, > >> perhaps to emu (our own server). ?If so, you'll be welcome to migrate the > >> french version there too. > > > > We do not want to host it on emu! There are many free wiki hosting > > services, and we do not want to have to > > admin anything that isn't absolutely vital. Moving the wiki off emu is an > > important step towards getting rid of emu. > > Another important step will be moving the bug tracker. > > So what is the alternative? Sourceforge seems to be a usability > nightmare (as its always been). You are the only person to have had any serious usability issue with it. However it is slow. It should probably be hosted elsewhere but it does need to be an externally hosted (= *not* on a php host, no manual upgrades) MediaWiki. > > We should move stuff off Emu if we can, but we shouldn't infinitely > delay something the project needs if Emu is the only viable option. No, I have neither the time nor the knowledge to admin emu competently, so it is important that as little as possible depend on it. And we have made much progress to that goal. Anyway there are lots of other free wiki providers. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/f8bad37a/attachment.pgp>
[freenet-dev] JetBrains IDEA license key for Freenet development
Once again, JetBrains has kindly provided an open source license to the project for Freenet development. If you would like access to this, please email me off-list (but remember its only for Freenet work). Ian. -- Ian Clarke CEO, SenseArray Email: ian at sensearray.com Ph: +1 512 422 3588 -- next part -- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/0e3536b3/attachment.html>
[freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
What happened here? I can't see this note! -- next part -- An embedded message was scrubbed... From: Mantis Bug Tracker Subject: [Freenet 0003693]: Put before in XML indexes Date: Mon, 11 Jan 2010 23:54:11 + Size: 4405 URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/1bd6f210/attachment.mht> -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/1bd6f210/attachment.pgp>
[freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
He deleted it again: 2010-01-11 23:54 Mike Bush Note Added: 0006463 2010-01-12 00:03 Mike Bush Note Deleted: 0006463 ;) - Zero3 Matthew Toseland skrev: > What happened here? I can't see this note! > > > > > Emne: > [Freenet 0003693]: Put before in XML indexes > Fra: > Mantis Bug Tracker > Dato: > Mon, 11 Jan 2010 23:54:11 + > Til: > toad at amphibian.dyndns.org > > Til: > toad at amphibian.dyndns.org > > > A NOTE has been added to this issue. > == > https://bugs.freenetproject.org/view.php?id=3693 > == > Reported By:Matthew John Toseland (Toad) > Assigned To: > == > Project:Freenet > Issue ID: 3693 > Category: spider/librarian > Reproducibility:have not tried > Severity: minor > Priority: normal > Status: new > Target Version: 0.8.0 > == > Date Submitted: 2009-11-10 13:00 UTC > Last Modified: 2010-01-11 23:54 UTC > == > Summary:Put before in XML indexes > Description: > This would allow us to only parse once, saving *a lot* of disk > reads/writes/encrypts/decrypts. We will need to be back compatible in > parsing for a while, which can be best accomplished by a version number > change in the main document (index.xml). > == > Relationships ID Summary > -- > child of0003694 Decompress and parse in one action > == > > -- > (0006463) Mike Bush (reporter) - 2010-01-11 23:54 > https://bugs.freenetproject.org/view.php?id=3693#c6463 > -- > Doing it this way round would prevent the ability to produce partial > results, I'm in favour of scrapping them in the XMLIndex though since it's > not available in this version of the Library interface and the speed/memory > usage improvement is obviously much more important. The parser can be much > simpler without that provision too. > > If no one has started it, you can assign it to me. > > Issue History > Date ModifiedUsername FieldChange > == > 2009-11-10 13:00 Matthew John Toseland (Toad)New Issue > > > 2009-11-10 13:00 Matthew John Toseland (Toad)milestone => 0.8 > > > 2009-11-10 13:00 Matthew John Toseland (Toad)svn-revision => > > > 2009-11-10 13:33 Matthew John Toseland (Toad)Relationship added child of > 0003694 > 2009-11-19 08:18 Juiceman Issue Monitored: Juiceman > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > == > > > > > > ___ > Devl mailing list > Devl at freenetproject.org > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
[freenet-dev] Fwd: [Freenet 0003693]: Put before in XML indexes
On Tuesday 19 January 2010 16:14:11 Christian Funder Sommerlund (Zero3) wrote: > He deleted it again: > > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > 2010-01-12 00:03 Mike Bush Note Deleted: 0006463 > > ;) > > - Zero3 > > Matthew Toseland skrev: > > What happened here? I can't see this note! > > > > > > > > > > Emne: > > [Freenet 0003693]: Put before in XML indexes > > Fra: > > Mantis Bug Tracker > > Dato: > > Mon, 11 Jan 2010 23:54:11 + > > Til: > > toad at amphibian.dyndns.org > > > > Til: > > toad at amphibian.dyndns.org > > > > > > A NOTE has been added to this issue. > > == > > https://bugs.freenetproject.org/view.php?id=3693 > > == > > Reported By:Matthew John Toseland (Toad) > > Assigned To: > > == > > Project:Freenet > > Issue ID: 3693 > > Category: spider/librarian > > Reproducibility:have not tried > > Severity: minor > > Priority: normal > > Status: new > > Target Version: 0.8.0 > > == > > Date Submitted: 2009-11-10 13:00 UTC > > Last Modified: 2010-01-11 23:54 UTC > > == > > Summary:Put before in XML indexes > > Description: > > This would allow us to only parse once, saving *a lot* of disk > > reads/writes/encrypts/decrypts. We will need to be back compatible in > > parsing for a while, which can be best accomplished by a version number > > change in the main document (index.xml). > > == > > Relationships ID Summary > > -- > > child of0003694 Decompress and parse in one action > > == > > > > -- > > (0006463) Mike Bush (reporter) - 2010-01-11 23:54 > > https://bugs.freenetproject.org/view.php?id=3693#c6463 > > -- > > Doing it this way round would prevent the ability to produce partial > > results, I'm in favour of scrapping them in the XMLIndex though since it's > > not available in this version of the Library interface and the speed/memory > > usage improvement is obviously much more important. The parser can be much > > simpler without that provision too. > > > > If no one has started it, you can assign it to me. > > > > Issue History > > Date ModifiedUsername FieldChange > > > > == > > 2009-11-10 13:00 Matthew John Toseland (Toad)New Issue > > > > > > 2009-11-10 13:00 Matthew John Toseland (Toad)milestone => > > 0.8 > > > > 2009-11-10 13:00 Matthew John Toseland (Toad)svn-revision => > > > > > > 2009-11-10 13:33 Matthew John Toseland (Toad)Relationship added child > > of > > 0003694 > > 2009-11-19 08:18 Juiceman Issue Monitored: Juiceman > > > > 2010-01-11 23:54 Mike Bush Note Added: 0006463 > > > > == Doh, was I wrong to assign it to him then? -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/1bc03dc2/attachment.pgp>
[freenet-dev] If anyone would like to do some math for WoT please read
On Wednesday 13 January 2010 00:33:21 xor wrote: > > Currenty, the WoT IdentityFetcher permanently subscribes to the trust list > USK > of EVERY known identity, resulting in a total load of n*n USK subscriptions > on > the network - each identity subscribes to the USK of every other identity. > > (Even though it does O(n?), we will soon release the current code as WoT 0.4 > on the plugins page labeled as testing because it is guaranteed to fetch all > identities, i.e. guraanteed to "just work" and has proven to be very stable > in > all other means. And WoT 0.4 in general is optimized for new features & > stability, not for performance. 0.5 is planned to fix usability and > performance issues and bugs of course. - First implement all functions to > work > properly, then optimize and see if it still works the same as the previous > version, that's my plan) IMHO O(n) per node is not as big a problem as you might think on Freenet, because much of the work ends up being shared between different nodes. It is definitely not the same as O(n^2). However, we do need to deal with it, as n will eventually get rather large, and we have to track multiple keys for each of these identities. > > I propose the following change for WoT 0.5 to get rid of the permanent O(N*N): > > - Permanent USK subscriptions should only be done to rank1 identities: The > identities to which an OwnIdentity has assigned a (positive of course) trust > value (= the users trustees). This should give us "small world > network"-effects > already. And if I'm right it changes everything from n? to some logarithmic > function of n. That is to be proved :) > > - The trust lists which are fetched of the rank1 identities will give us new > edition hints for new trust lists of other identities. Its called "hints" > because we cannot tell the USK manager that those editions REALLY exist, they > might be spoofed, i.e. too high. The USK manager supports specifying an > edition as hint-only. > > - So for rank 2 identities (Y is rank 2 <=> an own identity trusts X and X > trusts Y): If we receive a new edition hint for a rank2 identity, immediately > start a fetch for the hinted edition. Give the fetch a certain amount of time > to succeed, abort it if it doesn't. I propose 1 hour as the time limit. Tell > the USK manager NOT to attempt to search for any newer editions than the > hinted one: We want to use the information gathered from our rank1 identities > instead. But do not disable the fetching attempts of older editions than the > hinted one - to detect bogus hints. All you need to do is fetch that one slot. Give it say 3 attempts. That won't take an hour and you don't need a time limit, it's a simple request. > > - For rank3 and above identities, do the same as for rank2 but only allow a > hint-fetch-attempt every N days. N is to be carefully tweaked, I propose we > start with 3. > > - If any identity has not had a fetch attempt for >= M days, try to fetch > them > once. In other words, each identity is fetched at max every M days so that > identities which are not "hit" by the random heuristic above still get > fetched > every now and then. Reasonable. > DO allow the USK manager to search for editions higher > than the current hinted one. Do this to catch bugs in the other heuristics. > Log if an edition was fetched which was higher than the latest received hint. > I propose M to be 7 days. Fetching many slots ahead by doing a full USK retrieval uses much more resources, is it worth it? > > Now the question is: Who can calculate the maximal, minimal and average > complexity of my algorithm? :) > > I'm aware of the fact that the last point (fetch all identities every 7 days) > introduces some O(n?) again but IMHO its necessary for preventing bugs which > could cause some identities to be NEVER fetched. We can disable it after > we're > sure that it's not needed. Is it? Won't hints suffice to quickly deal with any such problem? > > And if anyone has suggestions for improvements, feel free to give them. I > consider my proposed rank1 and rank2 behavior as quite natural, but for rank3 > and above we might for example use an exponential function of rank, score and > whatever for the minimal delay between fetches. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/4bdc2c6a/attachment.pgp>
[freenet-dev] Library - request for testers
On Monday 11 January 2010 16:49:30 Ximin Luo wrote: > On 11/01/10 16:30, Matthew Toseland wrote: > > On Sunday 10 January 2010 11:35:47 Ximin Luo wrote: > >> http://github.com/freenet/plugin-Library-staging > >> > >> If anyone has some free time, please check out the above repo and run the > >> test: > >> > >> $ git clone git://github.com/freenet/plugin-Library-staging.git $ cd > >> plugin-Library-staging $ ant junit > >> > >> Open up ./test/plugins/Library/index/BIndexTest.java and fiddle about with > >> the numbers at the top, and run the test with different settings. > >> BIndexTest is the first test to run; you can just hit ctrl-C after it's > >> done to skip the rest. > >> > >> What you're testing is an algorithm to write to a on-freenet b-tree. Ie. > >> most of the data is on freenet, and you're just pulling/pushing in > >> (approximately) the minimum nodes needed to update it, plus a new root > >> node. > >> > >> This would be useful for search indexes (which are huge), and could also > >> be useful for other things that need to store a huge on-freenet data > >> structure. Obviously, it needs a shit load of testing first, so any help > >> would be appreciated. > > > > IMHO any such on-freenet data structure should be single owner, to avoid > > concurrent updates / finding the latest version problems, and the data > > should all be duplicated locally so it can be easily reinserted. > > The only thing that is "owned" is the USK root pointer to the rest of the data > structure. The data nodes are currently stored as CHK files, so they are > effectively immutable, which means concurrency isn't a problem. Multiple root > USKs from different owners could point to the same CHK nodes, making fork > operations trivial. Right. But each root has an owner. And a forked index will have both a different root and some set of different sub-paths. The point is we can't have a single linear USK space with private key known by many people, because this will be chaotic. The correct model is to fork and merge a la DSCM's or wiki-over-DSCM's. > > > However, this is very important stuff, because It will allow us to generate > > indexes gradually during spidering and much more efficiently (no waiting > > several days while doing hideous numbers of seeks while it writes the > > index). There may be other uses too in the medium term, as infinity0 hints > > at, e.g. published databases etc. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/b201f983/attachment.pgp>
[freenet-dev] [REQ]Pushing the french translation on website-official
On Wednesday 13 January 2010 10:24:57 Cl?ment Vollet wrote: > Hello, > > I think that we can push the french translation of the website, since almost > all important pages are translated (that is index, download, whatis, people, > donate, sponsors). What's left is the faq (about 50% done), but I don't think > it's sufficient to block the release of the translated site. > Moreover, all pages which aren't translated are displayed in english. > > For now, we can push it without doing any langage recognition, and do that > latter (there is links to other language on top right of the site). > > I'd like to add that I made some changes to the css to fix the menu > displaying > (it wasn't scaling well with font size), especially when hovering item. > However, the items will now be centered, since I can't find a way to fix this > without center them. > > So, please push (except if you have something more important to do, of course > :p ), or if not, tell me why. Ok, will try to push this soon. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/8703b6fe/attachment.pgp>
[freenet-dev] Uservoice update
Top 5 items on Uservoice: 1) One GUI for all (263 votes) IMHO integrating Freetalk and FlogHelper will go a long way towards satisfying people who voted for this, although the original poster wants a dedicated browser thingy. 2) Write a killer file-sharing application (232 votes) IMHO there are a number of issues here: - Current data persistence sucks. We are working on it and need to do more work on it. - It is relatively hard to search for files. We need a good WoT-based spamproof fast file search system. - General demand from many folks for insert on demand. Maybe it worked on Frost, maybe it didn't, but Frost is dead. The main advantages of insert on demand are 1) that you can "share" a directory immediately, 2) you don't need to wait for the insert, 3) the set of available data, or available data at a given speed, is larger. I am skeptical, there are several problems with it (trust, security, different views of "network pollution" ...), but in any case improving data persistence and making it easy to search for files are both important, and once we have these we can consider insert on demand. And there *are* options for relatively safe reinsert on demand, although they are messy... Or am I being unduly dogmatic and long-termist here? Maybe a simple WoT-based file search and share and reinsert on demand system would gain us so many more users that we should do it anyway, even if insert on demand is risky and not necessary long term? This does rely on distributed searching, and it could be a while ... 3) Add a "pause" feature (188 votes) This might actually be partly due to the frequently fixed and re-broken "Freenet doesn't restart after I change IP address / my laptop sleeps / etc" bug. We need to fix that ASAP. It is also closely related to the general fact that it takes quite some time for a node to get up to speed on opennet after it's been down for a while. I have several ideas about how to improve that. We need to make Freenet more low-uptime-friendly, even if that means using unreasonable levels of data redundancy. It is also closely related to the system tray icon. We have one on Windows, we will soon have one on Mac, once we have both I will write one for Linux. But the specific demand is that we have some means for the node to "pause" its connections, so that it can get them back quickly. There has been much debate on this on IRC, lists and the bug tracker, it is probably possible and is related to some others ... Other options include just cutting the bandwidth limit or turning off the client layer etc. 4) use the port 80,443,53,1863 for comunication (171 votes) Remains remarkably popular! There are some simple options such as using popular UDP ports, but any serious progress is going to require a TCP transport plugin, and probably an HTTP one as well, maybe with multiplexing support for smaller web servers. SSL would be nice too. Of course these will only work when port forwarding works - but it does work a lot of the time. In any case, this requires transport plugins, which are quite feasible but may well be a largish amount of work. 5) Implement reinsert on demand (169 votes) See the item on filesharing above! Since Frost has been largely abandoned, there is a major gap for filesharing it seems... 6) Chat (108 votes) Included because #5 and #2 duplicate each other. The originator sees this as real-time. digger3 is working on a prototype of real time chat over Freenet, which may be extended to microblogging. He says lag is 10-30 sec and can be lowered considerably by relatively small network changes... -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100119/e135101f/attachment.pgp>
