On Tuesday 10 November 2009 15:22:37 Daniel Cheng wrote: > On Mon, Nov 9, 2009 at 10:39 PM, Evan Daniel <evanbd at gmail.com> wrote: > > On Mon, Nov 9, 2009 at 8:35 AM, Matthew Toseland > > <toad at amphibian.dyndns.org> wrote: > >> Some Chinese feedback: Main point is there are a lot of old PCs with very > >> limited RAM, although new ones have plenty of RAM. Broadband is however > >> reasonable in general, making it a better candidate than Iran. Suggested > >> making startup on startup configurable, substantially reducing memory > >> footprint etc. My worry with the former is that opennet won't last > >> forever, and low-uptime darknet is very problematic. > >> > >> IMHO we could reasonably cut our default memory limit to 128M. The main > >> issues are: > >> - Sorting out Library memory issues (bug #3685) > >> - Sorting out Freetalk memory issues (I haven't seen OOMs from Freetalk > >> recently, maybe this really is fixed?) > >> - Sorting out - or ignoring - the startup memory spike on big inserts (4G > >> needs 192MB). > >> > >> On fast systems, a higher memory limit means less CPU spent in garbage > >> collection, so maybe there is an argument for reinstating the panel in the > >> wizard that asks the user how much memory to allocate... > >> > >> Our friend has also localised the wininstaller (this is subject to > >> technical issues Zero3 hopefully will be able to resolve), and jSite (I > >> will deal with this soon). > > > > It is my understanding (haven't personally tested) that we could > > reduce the memory footprint by reducing the per-thread stack size > > (-Xss option). ?Similarly, reducing the max thread count might help > > (we don't usually use all the threads, but worst-case behavior is what > > causes OOMs). > > > > Should we base the memory for RAM buckets on the configured maximum, > > instead of having a constant default? ?These are used for caching > > decoded data, right? ?So more buckets means that Library gets far more > > responsive on repeat searches. ?(Just tested this -- increasing RAM > > buckets to 80 MiB does seem to mean that Library skips directly to the > > "Parsing subindex" step on repeat or partially repeat searches. ?Of > > course, it's still glacially slow after that.) > > On an idle node, most of the memory are used by the FailureTable. > Is there any ways to cut down this memory without hurting routing?
Here's the more complex explanation, having dealt with the low-hanging fruit. :) requestorNodes and requestedNodes use WeakReference's, but we don't create a new WeakReference, we use the one that the PeerNode has created, so this is not a problem for either GC overhead or memory usage. Conceivably we could make requestedTimeouts relative to requestedTimes, more code though. As far as I can see we use all the fields... We do keep multiple entries (within the same FailureTableEntry) for separate requests, but we need this for HTL-sensitive per-node failure table routing. Also, the key itself is quite large. I have planned to reduce disk i/o by eliminating the on-disk cooldown queue, and using the in-RAM failure table plus some extra data structures: - A data structure tracking persistent requests (SendableGet's, e.g. a splitfile segment), and when we last tried them. This should be tiny. - If we haven't tried that specific request recently, we check the failure table. This deals with several cases: 1) Some other node tried that key recently (*not* covered by current code), 2) There are two copies of the same segment queued, e.g. for different files on the same freesite, or just multiple copies of the same download, 3) Something wierd is happening. - Arguably case 2) ought to be covered by an explicit mechanism for detecting such things. And arguably we should ignore case 3). But that does still leave case 1) for the failure table. I have in the past considered storing the failure table on disk, but this would slow down routing considerably: Even if we have a bloom filter (which would have to be rebuilt occasionally), when we get a hit we'll have to look it up on disk. Our disk i/o is already quite high, although I have a couple of ways to reduce it significantly planned (RAM-only cooldown queue and occupied/wrong store bitmaps for the stores). Given a bloom filter the latency cost wouldn't be huge though. This would have to be a separate database from the client layer database, because the client layer database can stall for minutes on end sometimes - we should fix that but we can't afford to wait for it. We would also want a separate database jobs thread for it, but this would be more broadly useful as we could store other node info in it; we can't access it on the incoming data threads because we don't want it to delay forwarding other messages. I am not sure whether it would interfere with the plans for the cooldown queue, given if we had a bloom filter? One advantage is it could be bigger. Size is an issue: - Currently there is a limit of 20k entries. This is fairly arbitrary. - My node has 5000 after 16 hours uptime with a 30KB/sec output limit. But faster nodes could well go past the 20k entry limit. - We keep FTEs for an hour after the last input. - When we get a key, we offer it to all nodes who have requested it, or who we have requested it from, in the last hour. We also use the structure to decide whether we want the data, which depends not only on whether our client layer wants it, but also on whether other nodes have asked us for it recently (i.e. within the FailureTableEntry's lifetime). - After a failure from a specific node, we have a cooldown period of up to 10 minutes during which we will misroute - prefer not to route to that node. - In future, we may send RecentlyFailed to a node which asks for a key which has been requested many times recently. If it was on-disk, we could make it significantly bigger - but ULPRs are inherently unreliable, and the longer it is since the node requested the key the more likely it is that it is no longer of interest. In the future, (reliable, compared to ULPRs anyway) passive requests would certainly be disk-based, because there would be a lot of them. That does not necessarily mean it's the best option for now though. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20091110/06722286/attachment.pgp>
