On Thursday 18 February 2010 08:26:55 Daniel Cheng wrote: > Hi all, > > Do we still concern memory usage? > I am seeing the memory usage of freenet climb higher and higher in > recently release. > It is no longer possible to run a node in an 128m memory box. > > With 3c02c397bfea7418d5d311ba481c3b3c7df96e2e, may I say the promise/envision > of low-memory/embeded usage dead/failed ?
We do care about memory usage. At least I do. Several recent builds have fixed memory leaks and similar issues. However, it is also true that more memory can substantially improve performance, and some features need more than others: - Disk-based temp files are encrypted unless physical security is set to low. A small amount of such data can be cached in RAM and therefore not encrypted. This makes a noticeable difference in practice. - Library searches can use a lot of memory if searching for popular words. - Persistent downloads and anything using the client layer database are speeded up considerably if they are able to use the cached objects rather than having to read and reconstruct them from disk. - WoT and Freetalk can use quite a bit of RAM, and cause the node to use significant resources. - Compression and decompression, and FEC encoding, can use significant amounts of RAM; the number that can run in parallel can be determined (if seeking is not an issue which sometimes it isn't) based on memory limits. Thus, we intend to provide auto-detection of Freenet's java memory limit based on the total system RAM in the near future. Then the RAM temp files limit, and other things, would be auto-calibrated based on the overall limit. This is largely implemented for Mac/Linux/etc, but is not yet enabled because of problems on Mac. Also, eventually we will implement bloom filter sharing, and this will use a significant (but configurable) amount of memory also, although there are ways to do such checks periodically if we have long-term requests. And to answer the original point, yes we do need to do more work on reducing memory usage. But maybe not as the most immediate priority before 0.8.0. At the moment I am dealing with a bug relating to a lot of node resources being used for fetching USKs when Freetalk is loaded. After that I'll probably be doing feature work e.g. the new FEC arrangements. Do you have any direct evidence (however anecdotal) that build X uses a lot more memory to process the same workload (same store size, same queued downloads etc) than earlier build Y? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20100218/48557f6d/attachment.pgp>
