On Wednesday 06 May 2009 14:43:59 Victor Denisov wrote: > Matthew Toseland wrote: > > This is the downside of db4o. If it is a widespread problem, we're gonna have > > to revert it. Which means throwing away more than 6 months work largely > > funded by Google's $18K. > > I think that using a database is a good idea (although I personally > would've opted for a relational database such as Derby). So I'd prefer > to try and understand and fix the issue rather than hiding from it :-). > > > My database queue is usually pretty empty, even with queued downloads, but I > > have 8G and fast mirrored disks... > > The problem's that Freenet *doesn't* even use the amount of memory I > provide it with (I'm yet to see it use more than 120 megs out of 320 I > allow for the heap). I'd be willing to dedicate as much memory as > required if only it'd help. > > My hard drives are nothing special - 250Gb 7200 RPM Seagate ones, 16 Mb > cache, SATA2, no NCQ - though definitely not the slowest out there. I > see ~35 Mb/s read speed and ~28 Mb/s write speed for medium-sized files > and ~5 Mb/s to 8 Mb/s for small files in the tests I'd done. I'll > probably have to test the same from inside Java to make absolutely sure > that it's not some weird JVM issue on my platform, though. > > > 2650 handles is strange, on unix we are generally limited to 1024 and > > generally we don't exceed that. Both of your problems may be caused by flaky > > hardware, but frankly we do need to run on flaky real world hardware. :| > > I don't have Freenet running right now, will check it later. But I2P is > using 2670 handles right now, and Azureus uses 1450 - so 2600 for > Freenet is definitely nothing out of the ordinary on Windows. Oh, and > the highest handle user on my machine is MySQL, which uses ~69000 > handles and works absolutely fine :-). > > >> Same here. Enormous disk queues. I've also compared i/o counts with i/o > >> bytes read/written - that's how I know that i/o operations are small. In > >> the statistics screen, I routinely see 100+ outstanding database jobs. > >> It can't be good. > > > > This just confirms that disk I/O is the problem ... and almost certainly > > caused by db4o as it goes away if nothing is queued. > > My thinking exactly. Would providing you with a snapshot of CPU/memory > performance under YourKit Profiler (I have academic licenses for both > 7.5 and 8.0, IIRC) or VisualVM (which is now a part of the JDK > distributive) on my machine help? Any logging I can turn on to help? > BTW, I have logging set to ERROR for now, as with NORMAL level it logs > ~2Mb per minute, adding noticeably to overall disk contention. > > Regards, > Victor Denisov.
One other thing, for both you and Juiceman: How's the CPU usage? Given how much RAM you have I would expect node.db4o to be cached in memory (how big is it?). But doing a read through the OS to the OS disk cache may cost a lot of CPU (context switch etc) ... Is there a lot of CPU usage for the freenet process? To the point that it might be the cause of the poor overall system performance? And how much CPU usage is system? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/support/attachments/20090506/f8c8b6fd/attachment.pgp>
