[freenet-dev] Plugin autoupdating question
On Tuesday 01 December 2009 18:49:21 Evan Daniel wrote: > On Tue, Dec 1, 2009 at 12:30 PM, xor wrote: > > - The autoupdater MUST call plugin.terminate() before unloading the > > plugin because plugins use databases and those must be properly closed. > > > > - Deadlock bugs however might prevent plugins from terminating properly, > > terminate() might not return. > > > > So the question is: If a bug is introduced which permanently breaks > > terminate(), even after restarting the node, will the autoupdater handle > > that? > > > > IMHO if it is told to update a plugin and the node is restarted in > > between then it should update the plugin before even starting it after > > the restart. That would prevent this issue > > > > Further, toad please remember that you should add logic to the node which > > calls terminate() on all plugins before node shutdown, otherwise the > > WoT/Freetalk databases might get corrupted, resulting in un-debuggle > > weird issues. > > Surely the on-disk database is always in a valid (or at least > recoverable) state? That is one of the points of using a database, > right? That it can recover to a valid state after a software or > hardware crash, such as abrupt termination of the program or loss of > power? > It's quite irrelevant actually, the point is that there is no point in NOT calling terminate() =) -- next part -- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091204/11a561bb/attachment.html> -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091204/11a561bb/attachment.pgp>
[freenet-dev] Freenet tray icon
My wife made the following discovery about the Windows Freenet tray icon: -- Forwarded message -- From: Janie Mehew <...> Date: Fri, Dec 4, 2009 at 11:33 AM Subject: Freenet tray icon - forward to mailing list To: Ian Clarke Hi, I have noticed that the freenet tray icon seems to be using more resources than it needs to. If I open process explorer, and sort the entries by CSwitch Delta, the freenettray.exe program is always in the top 5-10. On my computer it seems to be around 200 per update interval. This indicates that the tray icon is using more CPU than it probably needs to. For an explanation, see here: http://blogs.technet.com/sysinternals/archive/2004/04/27/452830.aspx By comparison, my product's tray icon polls our service over RPC every few seconds, and never seems to have a CSwitch delta higher than 5. I've attached a screenshot of process explorer. Janie -- Ian Clarke CEO, Uprizer Labs Email: ian at uprizer.com Ph: +1 512 422 3588 -- next part -- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091204/4ceb4ec6/attachment.html> -- next part -- A non-text attachment was scrubbed... Name: freenettray.PNG Type: image/png Size: 50595 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091204/4ceb4ec6/attachment.png>
[freenet-dev] Plugin autoupdating question
On Fri, Dec 4, 2009 at 6:16 AM, xor wrote: > On Tuesday 01 December 2009 18:49:21 Evan Daniel wrote: >> On Tue, Dec 1, 2009 at 12:30 PM, xor wrote: >> > - The autoupdater MUST call plugin.terminate() before unloading the >> > plugin because plugins use databases and those must be properly closed. >> > >> > - Deadlock bugs however might prevent plugins from terminating properly, >> > terminate() might not return. >> > >> > So the question is: If a bug is introduced which permanently breaks >> > terminate(), even after restarting the node, will the autoupdater handle >> > that? >> > >> > IMHO if it is told to update a plugin and the node is restarted in >> > between then it should update the plugin before even starting it after >> > the restart. That would prevent this issue >> > >> > Further, toad please remember that you should add logic to the node >> > which >> > calls terminate() on all plugins before node shutdown, otherwise the >> > WoT/Freetalk databases might get corrupted, resulting in un-debuggle >> > weird issues. >> >> Surely the on-disk database is always in a valid (or at least >> recoverable) state? That is one of the points of using a database, >> right? That it can recover to a valid state after a software or >> hardware crash, such as abrupt termination of the program or loss of >> power? >> > > It's quite irrelevant actually, the point is that there is no point in NOT > calling terminate() =) Well, the node doesn't always shut down cleanly, in which case it can't guarantee to call terminate(). Having the database get corrupted in such a case would be bad. (I agree with you that the node should call terminate() whenever possible.) Evan Daniel
[freenet-dev] Why does announcement take so long?
tly > reduced, and should apply separately to each category e.g. a new announcement > connection can only kick out an existing connection every N minutes. We have > a similar limit based on the number of successful requests since we last > dropped a connected peer. We might want this to apply just to path folding, > or to apply for each category, maybe with different limits. I dont think one should make several changes at once - better to implement and evaluate a new method defensively. But maybe the idea to limit on the number of requests is also good: when an announcement connection has seen a number of successful request - it should have used those to fold with the rest of the network, and can be immediately dropped. > It looks like improving announcement is probably going to have to include > alchemy, but well, evanbd and vivee started that, and they're theoreticians, > they can tell us what to set the parameters to :) I still only see limited way for defensive experimentation, alchemy or improved simulations here - rather than theory. The result can only be observed if the implementation is global. But maybe I am wrong :-) > Thoughts? I would just like to re-iterate one point: to have additional slots for announcing (and reconnecting) for a node X, is IMHO /not/ meant for connecting nodes to sit on some "waiting-list" to get upgraded to persistent connections to node X. Instead, it is meant to allow connecting nodes to start operate and observe traffic in the network as soon as possible! This is what gives nodes connections. The hypothesis is that by operating in the network as soon as possible, even though only for minutes - and protecting such slots from path folding, this allows several things. This includes nodes that are connecting in parallell to e.g. see each others announcements, start normal requests themselves, observe other requests and so on - but I wrote more about it in the original mail. The key is to not have the same kind of 10-minute limit to these extra slots - I think that is offering them too much. What about trying simply with a period of 2 or 3 minutes before such connections can be eaten up by other announcements or reconnects - and shift it upwards if it doesnt work? This will quickly give an announcing nodes at least a number of peers from those that had connections for more than 2 or 3 minutes - a good chance for nodes unless the pressure is very strong on the network. > The conversation between me and evanbd (and a little with vivee) on this > topic is on this bug: > https://bugs.freenetproject.org/view.php?id=3753 Two comments after having read this conversation 1. I think it is wrong to let nodes be moved from being temporary peers to normal persistent peers /unless/ there are suddently free slots in the persistent queue when the grace period is over (or if it happens during the grace period). The temporary slots should only be to allow nodes quickly start operating in the network - path folding should be made to where there are available persistent slots since that is the goal. The arguments are above (and in the first mail). 2. One point to make regarding to interpreting the nice data that evanbd has collected: even though a large fraction of nodes are still probably going to be online Y hours afterwards - it is a question of how relevant that is to reconnecting. And how many nodes we really want to offer the reconnection possibility. During the course of operating in the network nodes are likely to see many different peers over the hours. There are perhaps implementation reasons to limit the number of peers that can reconnect using some heuristic. Lets say that the nodes implement the previously discussed sharing of store/cache states. Then, keeping track of previous opennet peers and their previous states may become quite costy. Such things could either limit the fraction of nodes we really prefer Y hours later, or that it is simply costy implementation-wise. I have not thought much about it, but it does not seem clear to allow anyone we previously saw to reconnect.. Cheers, /vive -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091204/30cd2a8e/attachment.pgp>
Re: [freenet-dev] Plugin autoupdating question
On Tuesday 01 December 2009 18:49:21 Evan Daniel wrote: On Tue, Dec 1, 2009 at 12:30 PM, xor x...@gmx.li wrote: - The autoupdater MUST call plugin.terminate() before unloading the plugin because plugins use databases and those must be properly closed. - Deadlock bugs however might prevent plugins from terminating properly, terminate() might not return. So the question is: If a bug is introduced which permanently breaks terminate(), even after restarting the node, will the autoupdater handle that? IMHO if it is told to update a plugin and the node is restarted in between then it should update the plugin before even starting it after the restart. That would prevent this issue Further, toad please remember that you should add logic to the node which calls terminate() on all plugins before node shutdown, otherwise the WoT/Freetalk databases might get corrupted, resulting in un-debuggle weird issues. Surely the on-disk database is always in a valid (or at least recoverable) state? That is one of the points of using a database, right? That it can recover to a valid state after a software or hardware crash, such as abrupt termination of the program or loss of power? It's quite irrelevant actually, the point is that there is no point in NOT calling terminate() =) signature.asc Description: This is a digitally signed message part. ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Plugin autoupdating question
On Fri, Dec 4, 2009 at 6:16 AM, xor x...@gmx.li wrote: On Tuesday 01 December 2009 18:49:21 Evan Daniel wrote: On Tue, Dec 1, 2009 at 12:30 PM, xor x...@gmx.li wrote: - The autoupdater MUST call plugin.terminate() before unloading the plugin because plugins use databases and those must be properly closed. - Deadlock bugs however might prevent plugins from terminating properly, terminate() might not return. So the question is: If a bug is introduced which permanently breaks terminate(), even after restarting the node, will the autoupdater handle that? IMHO if it is told to update a plugin and the node is restarted in between then it should update the plugin before even starting it after the restart. That would prevent this issue Further, toad please remember that you should add logic to the node which calls terminate() on all plugins before node shutdown, otherwise the WoT/Freetalk databases might get corrupted, resulting in un-debuggle weird issues. Surely the on-disk database is always in a valid (or at least recoverable) state? That is one of the points of using a database, right? That it can recover to a valid state after a software or hardware crash, such as abrupt termination of the program or loss of power? It's quite irrelevant actually, the point is that there is no point in NOT calling terminate() =) Well, the node doesn't always shut down cleanly, in which case it can't guarantee to call terminate(). Having the database get corrupted in such a case would be bad. (I agree with you that the node should call terminate() whenever possible.) Evan Daniel ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
Re: [freenet-dev] Freenet tray icon
Hey Thanks for the feedback :). ~200 context switches per second (or whatever interval we are talking about here) does sound like a lot in light of how little work the tray icon has when idling. This is not a problem at all for modern CPUs though. Note that Chrome - one of the lightest browsers available - in that screenshot triggers 10x this number (not meant as an excuse, but rather to illustrate how small a number we are talking about). What the tray icon actually does when idle is to check the service status and look for a killfile every 5 seconds. I suspect that the cause of these context switches are the timer for these check. I'm not entirely sure how the timers are implemented in Autohotkey (the language the tray manager is coded in), but chances are that the implementation is less-than-ideal for our purpose. Nevertheless, we are (IMHO) talking very fine details here. In the perfect world, with unlimited time, it might be worth looking into. But in the real world, there are so many more important things to work at than worrying about a couple of hundred context switches :). - Zero3 Ian Clarke skrev: My wife made the following discovery about the Windows Freenet tray icon: -- Forwarded message -- From: *Janie Mehew* ... Date: Fri, Dec 4, 2009 at 11:33 AM Subject: Freenet tray icon - forward to mailing list To: Ian Clarke ian.cla...@gmail.com mailto:ian.cla...@gmail.com Hi, I have noticed that the freenet tray icon seems to be using more resources than it needs to. If I open process explorer, and sort the entries by CSwitch Delta, the freenettray.exe program is always in the top 5-10. On my computer it seems to be around 200 per update interval. This indicates that the tray icon is using more CPU than it probably needs to. For an explanation, see here: http://blogs.technet.com/sysinternals/archive/2004/04/27/452830.aspx By comparison, my product's tray icon polls our service over RPC every few seconds, and never seems to have a CSwitch delta higher than 5. I've attached a screenshot of process explorer. Janie -- Ian Clarke CEO, Uprizer Labs Email: i...@uprizer.com mailto:i...@uprizer.com Ph: +1 512 422 3588 ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl ___ Devl mailing list Devl@freenetproject.org http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
