Re: [freenet-support] routing table
On Sun, Feb 15, 2004 at 09:00:36PM +, Jim Dixon wrote: On Fri, 13 Feb 2004, Paul Derbyshire wrote: Freeing up RAM is not related to routing table at all. Unfortunately, Freenet code contains a bug (a so called memory leak) which takes memory from your OS, but then forgets about it, not using it and not returning it - so the amount of memory used by your node grows constantly, until you start getting Out Of Memory errors (or OOMs for short). How the hell is that even *possible*? It's written in a language with garbage collecting memory management for chrissake, and the Java GC *is* smart enough to collect circular object graphs that have become unreachable by running threads. What makes you think that? Is it a VM bug or is it just creating objects it theoretically could reach (thus they don't get GC'd), but ignores forever? This would be the case if in fact Freenet had a memory leak of the type described. However, whenever someone tells you authoritatively that such a memory leak exists, you have to wonder why they don't fix it, if they are so certain about it. space leaks probably do exist in the code somewhere. I suspect that Freeing up RAM is not related to routing table at all is simply wrong. It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. As I have said on numerous occasions: RAM is cheap. Freedom or working software is not cheap, it's bloody expensive. 79MB is way under what most nodes use. We may be able to get perm node usage down to 80MB at some point, but it's not an immediate priority. Don't you think it would be nice to have a working network? My single inactive node doesn't transmit any messages. The only thing that could account for the 79 MB of memory used would seem to be routing information relating to the 98 nodes it knows about: Uhmm, no. That's not the only thing that uses memory in fred. RTFS :) -- Routing Table status: Feb 15, 2004 12:58:00 PM Number of node references 98 Attempted to contact node references 98 Contacted node references 24 Connections with Successful Transfers 2 Backed off nodes 0 -- -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
On Sun, Feb 15, 2004 at 10:37:16PM +, Jim Dixon wrote: On Sun, 15 Feb 2004, Niklas Bergh wrote: It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. My single inactive node doesn't transmit any messages. The only thing that could account for the 79 MB of memory used would seem to be routing information relating to the 98 nodes it knows about: Please help out. Fire up a memory profiler of your choice at your machine and tell me what it is that occupies all that memory. When I do the same on my machine the node wont use more than 10-15 megs of memory. I don't understand. Are you saying that if you run top it shows only 10-15 MB of RAM in use by Freenet? No, he is saying that the memory profiler he uses slows the node down by so much that the node doesn't use much RAM. Looking at three different systems I see Redhat 8.0Freenet build 506590 MB Redhat 7.1.2 Freenet build 5068 127 MB Linux ?? Freenet build 506578 MB On the third system dmesg doesn't return anything; it's probably Redhat 8.0. These are the SIZE figures; RSS is a couple of MB smaller and SHARE runs around 8 MB. Suggest how I might get better definition and I will run it tomorrow. -- Jim Dixon [EMAIL PROTECTED] tel +44 117 982 0786 mobile +44 797 373 7881 Be liberal in what you accept,Jon Postel and conservative in what you send. RFC 793 http://jxcl.sourceforge.net Java unit test coverage http://xlattice.sourceforge.net p2p communications infrastructure ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED] -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
[EMAIL PROTECTED] Mon Feb 16 14:05:35 2004 wrote in an attachment: It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. As I have said on numerous occasions: RAM is cheap. Freedom or working software is not cheap, it's bloody expensive. 79MB is way under what most nodes use. We may be able to get perm node usage down to 80MB at some point, but it's not an immediate priority. Don't you think it would be nice to have a working network? It would be. Instead we have rivers of meaningless messages, memory hogs, buggy code. And lots of flippant remarks, of course. -- Jim Dixon [EMAIL PROTECTED] tel +44 117 982 0786 mobile +44 797 373 7881 Be liberal in what you accept,Jon Postel and conservative in what you send. RFC 793 http://jxcl.sourceforge.net Java unit test coverage http://xlattice.sourceforge.net p2p communications infrastructure ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
On Fri, 13 Feb 2004, Paul Derbyshire wrote: Freeing up RAM is not related to routing table at all. Unfortunately, Freenet code contains a bug (a so called memory leak) which takes memory from your OS, but then forgets about it, not using it and not returning it - so the amount of memory used by your node grows constantly, until you start getting Out Of Memory errors (or OOMs for short). How the hell is that even *possible*? It's written in a language with garbage collecting memory management for chrissake, and the Java GC *is* smart enough to collect circular object graphs that have become unreachable by running threads. What makes you think that? Is it a VM bug or is it just creating objects it theoretically could reach (thus they don't get GC'd), but ignores forever? This would be the case if in fact Freenet had a memory leak of the type described. However, whenever someone tells you authoritatively that such a memory leak exists, you have to wonder why they don't fix it, if they are so certain about it. I suspect that Freeing up RAM is not related to routing table at all is simply wrong. It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. My single inactive node doesn't transmit any messages. The only thing that could account for the 79 MB of memory used would seem to be routing information relating to the 98 nodes it knows about: -- Routing Table status: Feb 15, 2004 12:58:00 PM Number of node references 98 Attempted to contact node references 98 Contacted node references 24 Connections with Successful Transfers 2 Backed off nodes 0 -- -- Jim Dixon [EMAIL PROTECTED] tel +44 117 982 0786 mobile +44 797 373 7881 Be liberal in what you accept,Jon Postel and conservative in what you send. RFC 793 http://jxcl.sourceforge.net Java unit test coverage http://xlattice.sourceforge.net p2p communications infrastructure ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. My single inactive node doesn't transmit any messages. The only thing that could account for the 79 MB of memory used would seem to be routing information relating to the 98 nodes it knows about: Please help out. Fire up a memory profiler of your choice at your machine and tell me what it is that occupies all that memory. When I do the same on my machine the node wont use more than 10-15 megs of memory. /N ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
On Sun, 15 Feb 2004, Niklas Bergh wrote: It is noticeable that Freenet uses a ridiculous amount of RAM. If I run top on a node that connects to only one other node, and sporadically at that, I see that it is using 79 MB of memory. The number doesn't appear to grow -- there is no evidence of a memory leak -- but it starts out and remains huge. My single inactive node doesn't transmit any messages. The only thing that could account for the 79 MB of memory used would seem to be routing information relating to the 98 nodes it knows about: Please help out. Fire up a memory profiler of your choice at your machine and tell me what it is that occupies all that memory. When I do the same on my machine the node wont use more than 10-15 megs of memory. I don't understand. Are you saying that if you run top it shows only 10-15 MB of RAM in use by Freenet? Looking at three different systems I see Redhat 8.0Freenet build 506590 MB Redhat 7.1.2 Freenet build 5068 127 MB Linux ?? Freenet build 506578 MB On the third system dmesg doesn't return anything; it's probably Redhat 8.0. These are the SIZE figures; RSS is a couple of MB smaller and SHARE runs around 8 MB. Suggest how I might get better definition and I will run it tomorrow. -- Jim Dixon [EMAIL PROTECTED] tel +44 117 982 0786 mobile +44 797 373 7881 Be liberal in what you accept,Jon Postel and conservative in what you send. RFC 793 http://jxcl.sourceforge.net Java unit test coverage http://xlattice.sourceforge.net p2p communications infrastructure ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
Is it a VM bug or is it just creating objects it theoretically could reach (thus they don't get GC'd), but ignores forever? The second it what is defined as a 'memory leak' in GC'd environments. /N ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
On Sat, Feb 14, 2004 at 09:34:06AM +0100, Niklas Bergh wrote: Is it a VM bug or is it just creating objects it theoretically could reach (thus they don't get GC'd), but ignores forever? The second it what is defined as a 'memory leak' in GC'd environments. Or a space leak. /N -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. signature.asc Description: Digital signature ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
RE: [freenet-support] routing table
Title: Message The routingtable filesare thertnodes_* and rtprops_* files in your choosenfreenet install folder. However.. these filesoccupies onlya few kilobytes of your harddrive. Is it HD space or RAM memory you want to free up? /N -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert GreenageSent: den 13 februari 2004 04:32To: [EMAIL PROTECTED]; [EMAIL PROTECTED]Subject: [freenet-support] routing table which folder in windows contains the routing table that needs to be deleted in order to free up memory? --- Robert Greenage --- [EMAIL PROTECTED] --- EarthLink: It's your Internet. ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
RE: [freenet-support] routing table
Title: Message I want to free up RAM memory. I only have 192 mbs. I seem to run out of it very fast. --- Robert Greenage --- [EMAIL PROTECTED] --- EarthLink: It's your Internet. - Original Message - From: Niklas Bergh To: [EMAIL PROTECTED];[EMAIL PROTECTED] Sent: 2/13/04 5:09:02 AM Subject: RE: [freenet-support] routing table The routingtable filesare thertnodes_* and rtprops_* files in your choosenfreenet install folder. However.. these filesoccupies onlya few kilobytes of your harddrive. Is it HD space or RAM memory you want to free up? /N -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert GreenageSent: den 13 februari 2004 04:32To: [EMAIL PROTECTED]; [EMAIL PROTECTED]Subject: [freenet-support] routing table which folder in windows contains the routing table that needs to be deleted in order to free up memory? --- Robert Greenage --- [EMAIL PROTECTED] --- EarthLink: It's your Internet. ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] routing table
Title: Message Freeing up RAM is not related to routing table at all. Unfortunately, Freenet code contains a bug (a so called "memory leak") which takes memory from your OS, but then "forgets" about it, not using it and not returning it - so the amount of memory used by your node grows constantly, until you start getting Out Of Memory errors (or OOMs for short). So far, attempts by developers to catchthis particular bugweren't successful. The only advice I can give is to restart a node once in a while (every 6 to 12 hours seems to be a good choice). Regards, Victor Denisov,CEO, Jera Systems, Moscow, Russia - Original Message - From: Robert Greenage To: [EMAIL PROTECTED] Sent: Friday, February 13, 2004 4:15 PM Subject: RE: [freenet-support] routing table I want to free up RAM memory. I only have 192 mbs. I seem to run out of it very fast. --- Robert Greenage --- [EMAIL PROTECTED] --- EarthLink: It's your Internet. - Original Message - From: Niklas Bergh To: [EMAIL PROTECTED];[EMAIL PROTECTED] Sent: 2/13/04 5:09:02 AM Subject: RE: [freenet-support] routing table The routingtable filesare thertnodes_* and rtprops_* files in your choosenfreenet install folder. However.. these filesoccupies onlya few kilobytes of your harddrive. Is it HD space or RAM memory you want to free up? /N -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert GreenageSent: den 13 februari 2004 04:32To: [EMAIL PROTECTED]; [EMAIL PROTECTED]Subject: [freenet-support] routing table which folder in windows contains the routing table that needs to be deleted in order to free up memory? --- Robert Greenage --- [EMAIL PROTECTED] --- EarthLink: It's your Internet. ___Support mailing list[EMAIL PROTECTED]http://news.gmane.org/gmane.network.freenet.supportUnsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/supportOr mailto:[EMAIL PROTECTED] ___ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]
Re: [freenet-support] Routing table corrupt! Trying to reseed. throws EOFs
On Sat, Nov 30, 2002 at 12:19:22AM +0100, [EMAIL PROTECTED] wrote: Build Number 624 29.11.2002 23:37:58 (freenet.fs.dir.NativeFSDirectory$NWalk, main): Exhausted supply of files29.11.2002 23:37:58 (freenet.fs.dir.NativeFSDirectory, main): starting with D:\_freenet\store (8589934592) 29.11.2002 23:37:58 (freenet.fs.dir.NativeFSDirectory, main): get LRU keys 29.11.2002 23:37:58 (freenet.fs.dir.NativeFSDirectory, main): walk 29.11.2002 23:37:58 (freenet.node.Main, main): loading data store 29.11.2002 23:37:58 (freenet.fs.dir.NativeFSDirectory, main): get dirty 29.11.2002 23:37:58 (freenet.node.Main, main): loading routing table 29.11.2002 23:37:58 (freenet.config.Params, main): getOption(listenPort) 29.11.2002 23:37:58 (freenet.config.Params, main): getOption(listenPort) 29.11.2002 23:37:59 (freenet.node.Main, main): One of the routing table files was corrupt. java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:397) at java.io.DataInputStream.readLong(DataInputStream.java:418) at freenet.support.SimpleDataObjectStore.preload(SimpleDataObjectStore.java:62) at freenet.support.SimpleDataObjectStore.init(SimpleDataObjectStore.java:50) at freenet.node.Main.main(Main.java:553) 29.11.2002 23:37:59 (freenet.node.Main, main): One of the routing table files was corrupt. java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:397) at java.io.DataInputStream.readLong(DataInputStream.java:418) at freenet.support.SimpleDataObjectStore.preload(SimpleDataObjectStore.java:62) at freenet.support.SimpleDataObjectStore.init(SimpleDataObjectStore.java:50) at freenet.node.Main.main(Main.java:553) It would seem that both your routing table files were chopped off. One possibility is that the partition ran out of space..? How big are the files in question? 29.11.2002 23:37:59 (freenet.node.Main, main): Routing table corrupt! Trying to reseed. 29.11.2002 23:37:59 (freenet.node.Main, main): Unexpected Exception: java.io.EOFException java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:397) at java.io.DataInputStream.readLong(DataInputStream.java:418) at freenet.support.SimpleDataObjectStore.preload(SimpleDataObjectStore.java:62) at freenet.support.SimpleDataObjectStore.init(SimpleDataObjectStore.java:50) at freenet.node.Main.main(Main.java:571) java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:397) at java.io.DataInputStream.readLong(DataInputStream.java:418) at freenet.support.SimpleDataObjectStore.preload(SimpleDataObjectStore.java:62) at freenet.support.SimpleDataObjectStore.init(SimpleDataObjectStore.java:50) at freenet.node.Main.main(Main.java:571) [end of log] -- Matthew Toseland [EMAIL PROTECTED] [EMAIL PROTECTED] Freenet/Coldstore open source hacker. Employed full time by Freenet Project Inc. from 11/9/02 to 11/1/03 http://freenetproject.org/ msg02399/pgp0.pgp Description: PGP signature
Re: [freenet-support] Routing table status and transient nodes.
On Sat, Nov 30, 2002 at 04:33:17PM +, Roger Hayter wrote: Can anyone explain to me the difference in success rates in contacting a given set of nodes for a transient node from that achieved by a permanent node? I have a permanent node which has managed to contact a group of seed nodes about 500 times after 13 trials. The transient node (running for a shorter time) with the same set of seed nodes has managed to contact them about 400 times out of 600 trials. Is there a difference in the type of contact attempted, or the time the node waits for a reply: or something? The main difference is that a permanent node gets much more traffic, and so is much more dynamic. The more permanent nodes we have, the less the strain on the seednodes. -- Matthew Toseland [EMAIL PROTECTED] [EMAIL PROTECTED] Freenet/Coldstore open source hacker. Employed full time by Freenet Project Inc. from 11/9/02 to 11/1/03 http://freenetproject.org/ msg02400/pgp0.pgp Description: PGP signature
Re: [freenet-support] Routing table status and transient nodes.
In message [EMAIL PROTECTED], Greg Wooledge [EMAIL PROTECTED] writes Roger Hayter ([EMAIL PROTECTED]) wrote: I have a permanent node which has managed to contact a group of seed nodes about 500 times after 13 trials. I would imagine something is wrong with your computer, firewall, network, Java VM, freenet.conf, etc. Here's what I've got on a non-transient build-535 node with Sun Java 1.4.1_01 on Linux 2.4.18 after two weeks: Number of node references: 49 Contacted node references: 46 Backed off node references: 3 Total Trials: 642128 Total Successes: 391024 Implementation: freenet.node.rt.CPAlgoRoutingTable This is not to say that it's totally impossible you've found a Freenet bug. But I'd suspect a configuration error is more likely. I am not thinking of it as a bug, more a result of prioritisation of activity. While all the things you mention could be at fault, this behaviour has varied widely with build number (currently 627), I have changed nothing (well, apart from ...). The only distinctive feature is that it is a very slow computer (P70). I just wondered if anyone had any clues as to how I could give it more time to hear answers, by throttling some other activity for instance. -- Roger Hayter ___ support mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org/cgi-bin/mailman/listinfo/support
