I understood that on windows the memory mapped sizes needed to be included in the heap, since they are not allocated outside the heap as they are on linux/mac. So in this case he needs a larger heap (and make sure the memory mapped files are much smaller than the heap). The relevant part of the configuration settings doc says:
When running Neo4j on Windows the size of the memory-mapped nioneo configurations need to be added to the heap size parameter. On Linux and Unix-systems memory mapped IO is not included in the heap size. I still think that the solution to this case is to group the different relationship types into separate sub-graphs, so that the performance of traversing HAS_ONE is not affected by the number of relationships of CONTAINS. Of course traversing the CONTAINS will still be slow without increasing the cache, as you suggest. On Thu, Jun 16, 2011 at 12:07 AM, Michael Hunger < michael.hun...@neotechnology.com> wrote: > Agelos, > > sorry, didn't want to sound that way. > > 512M ram is not very much for larger graphs. Neo4j has to cache nodes, > relationships in the heap as well as you own datastructures. > > The memory mapped files for the datastores are kept outside the heap. > > Normally with your 4G I'd suggest using about 1.5G for heap and 1.5G for > the memory mapped files. > http://wiki.neo4j.org/content/Configuration_Settings > > Do you have a small test-case available that creates your graph and runs > your traversal? Then I could have a look at that and also do some > profiling to determine the issues for this slowdown. > > The indexing doesn't help as it also has to hit caches or disk. The graph > traversal is normally a very efficient operation that shouldn't experience > this bad performance. > > Cheers > > Michael > > > P.S. I just use my mail client for handling the mailing list and it works > fine for me. Imho Gmail groups threads automatically. > > > Am 15.06.2011 um 17:40 schrieb Agelos Pikoulas: > > > Re: [Neo4j] Slow Traversals on Nodes with too many > > Relationships > > > > I have to respectfully agree with Rick Bullotta. > > > > I was suspecting the big-O is not linear for this case. > > > > To verify I added x4 Container nodes (400.000) and their appropriate > > Relationships, and it is now *unbelievably* slow : > > It does not take x4 more, but it takes more than 30-40 seconds for each > > next() Remind you 100K nodes = ~2secs for each next() !!! > > > > And only to make matters worse, the subsequent runs weren't fast either - > > they actually took more time than the first > > (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) > > > > The whole setup is running on > > Eclipse 3.6, with -Xmx512m on JavaVM, > > Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ > > Vertex 2). The neo4J data resides on this SSD. > > The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. > > > > I wonder what would happen if the Container nodes were a few million > (which > > will be my case) - it will run forever. > > > > Could you please looking into my suggestion - i.e "Using a 'smart' behind > > the scenes Indexing on both *RelationshipType* and *Direction* that > > Traversals actually use to boost things up" ? > > > > To another topic, how does one use this mailing list - I use it through > > gmail and I am utterly lost - is there a better client/UI to actually > > post/reply into threads ? > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
