On Wednesday 06 May 2009 14:43:59 Victor Denisov wrote: > Matthew Toseland wrote: > > This is the downside of db4o. If it is a widespread problem, we're gonna have > > to revert it. Which means throwing away more than 6 months work largely > > funded by Google's $18K. > > I think that using a database is a good idea (although I personally > would've opted for a relational database such as Derby). So I'd prefer > to try and understand and fix the issue rather than hiding from it :-). > > > My database queue is usually pretty empty, even with queued downloads, but I > > have 8G and fast mirrored disks... > > The problem's that Freenet *doesn't* even use the amount of memory I > provide it with (I'm yet to see it use more than 120 megs out of 320 I > allow for the heap). I'd be willing to dedicate as much memory as > required if only it'd help. > > My hard drives are nothing special - 250Gb 7200 RPM Seagate ones, 16 Mb > cache, SATA2, no NCQ - though definitely not the slowest out there. I > see ~35 Mb/s read speed and ~28 Mb/s write speed for medium-sized files > and ~5 Mb/s to 8 Mb/s for small files in the tests I'd done. I'll > probably have to test the same from inside Java to make absolutely sure > that it's not some weird JVM issue on my platform, though. > > > 2650 handles is strange, on unix we are generally limited to 1024 and > > generally we don't exceed that. Both of your problems may be caused by flaky > > hardware, but frankly we do need to run on flaky real world hardware. :| > > I don't have Freenet running right now, will check it later. But I2P is > using 2670 handles right now, and Azureus uses 1450 - so 2600 for > Freenet is definitely nothing out of the ordinary on Windows. Oh, and > the highest handle user on my machine is MySQL, which uses ~69000 > handles and works absolutely fine :-). > > >> Same here. Enormous disk queues. I've also compared i/o counts with i/o > >> bytes read/written - that's how I know that i/o operations are small. In > >> the statistics screen, I routinely see 100+ outstanding database jobs. > >> It can't be good. > > > > This just confirms that disk I/O is the problem ... and almost certainly > > caused by db4o as it goes away if nothing is queued. > > My thinking exactly. Would providing you with a snapshot of CPU/memory > performance under YourKit Profiler (I have academic licenses for both > 7.5 and 8.0, IIRC) or VisualVM (which is now a part of the JDK > distributive) on my machine help? Any logging I can turn on to help? > BTW, I have logging set to ERROR for now, as with NORMAL level it logs > ~2Mb per minute, adding noticeably to overall disk contention. > > Regards, > Victor Denisov.
One other thing, for both you and Juiceman: How's the CPU usage? Given how much RAM you have I would expect node.db4o to be cached in memory (how big is it?). But doing a read through the OS to the OS disk cache may cost a lot of CPU (context switch etc) ... Is there a lot of CPU usage for the freenet process? To the point that it might be the cause of the poor overall system performance? And how much CPU usage is system?
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:support-requ...@freenetproject.org?subject=unsubscribe
