I want to re-do these just to be sure, but what I think I've observed is as follows...
I've done 2 tests using ram disks, one was an application fragment that I've referred to before - performs extensive traversal of the graph. Once the cache had become hot the performance was good for each iteration, but periodically (I assume when garbage collection kicked in) the impact of that garbage collection was significantly more with ram disk configuration than without it. The second test was to see how fast nodes could be looked up by an indexed 'id' value (not their node id, but a domain id integer value. I have one particular domain entity of which there are about 3.5m instances, all indexed by id. I created a test that read these id's from the origin SQL database into memory then iterated thru' each value and retrieved the corresponding node from neo4j via the index. All the neo4j graphdb directory structure had been copied to a ram disk area so I had expected the test to be cpu bound, but watching it with jconsole it had very low cpu utilisation and its performance wasn't much better (maybe 10%) than when the graphdb directory structure had been on a local disk. On a local disk the test retrieved about 12k nodes per second, with the ram disk it retrieved approximately 13k nodes per second. BTW, I'm starting to think that my own API's around neo4j might be adding significant overhead - not because they do much but that I wrap fly-weight wrapper objects around each node as I retrieve each node then often discard it. I notice on jconsole that the heap graph looks shows a very regular pattern of rapid heap growth followed by reduction - I suspect that reflecting an accumulation of these very short lived wrapper objects that then get garbage collected. This reflects the approach and is perhaps exacerbated for queries that need to traverse the object graph. Rather than use the neo4j traversal framework I'm tendiing to build simple accessors methos on each type for traversing to their neighbour types, then building queries atop of these that traverse the domain model in terms of the lower level accessors. This is attractive in terms of reducing coupling to neo4j, but I think might be the cause of lots of very short-lived fly-weight objects and collections of them being created and discarded.... any comment - is this an anti-pattern and a rationale to use the neo4j traversal framework mechanism....? On 3 Jul 2011, at 12:47, Michael Hunger wrote: > Paul, > > could you provide some details which kinds of perf-tests don't perform on a > RAM disk? > > Thanks Michael > > Sent from my iBrick4 > > > Am 02.07.2011 um 00:55 schrieb Paul Bandler <pband...@cseuk.co.uk>: > >> >> On 30 Jun 2011, at 23:42, Michael Hunger wrote: >> >>> I would love to look into the memory mapping issues on Solaris. >> >> There is no issue - just user mis-understanding of how it works - or doesn't >> if ones database is located on an NFS file-system. A little documentation >> on the necessary preconditions would be in order perhaps. >> >> Still have some other performance issues I don't fully understand - like why >> when locating the database on a ram disk certain performance tests don'g >> seem to run any quicker and exhibit low cpu utilisation as if they're still >> I/O bound, but that will have to wait for next week. >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
