There are 322 million relationships, no 57 million relationships. The only propierty I save in the relationship is a String. In this 322 million relationships there are 3 diferents types of relations. Do you think that 13 GB is a strange number? Why?
Mike El mié, 24-02-2010 a las 08:36 -0500, Rick Bullotta escribió: > I guess the bigger question to me is why 57 million relationships would > require 13 GB for the propertystore file and 67 GB for the string > propertystore. > > Something doesn't seem right. Can you describe what types of properties > you're storing on each node/relationship? > > Rick > > -----Original Message----- > From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On > Behalf Of Miguel Ángel Águila Lorente > Sent: Wednesday, February 24, 2010 6:29 AM > To: Neo user discussions > Subject: Re: [Neo] Java outof 64 GB ram > > Around 57.000.000 nodes and 322.000.000 relationships. Now I'm trying > with the following configuration: > > neostore.nodestore.db.mapped_memory=913M > neostore.relationshipstore.db.mapped_memory=11G > neostore.propertystore.db.mapped_memory=50M > neostore.propertystore.db.strings.mapped_memory=100M > neostore.propertystore.db.arrays.mapped_memory=0M > > I think that I had understood wrong the neo.props file. Now I'm > expecting what happen. > > El mié, 24-02-2010 a las 11:19 +0000, Rick Bullotta escribió: > > Hello, Miguel. > > > > Approximately how many nodes and relationships are in your graph database? > > > > Rick > > > > > > > > > > -----Original Message----- > > From: Miguel Ángel Águila Lorente <magu...@ac.upc.edu> > > Date: Wed, 24 Feb 2010 11:06:35 > > To: Neo user discussions<user@lists.neo4j.org> > > Subject: Re: [Neo] Java outof 64 GB ram > > > > Hi, > > > > firstly I used the numbers of 9 bytes * number_of_nodes and 33 bytes * > > number_of_relations for the neo.props configure, but it doesn't work. > > > > After that I tried with my actual configuration: > > > > neostore.nodestore.db.mapped_memory=180G > > neostore.relationshipstore.db.mapped_memory=6000G > > neostore.propertystore.db.mapped_memory=50M > > neostore.propertystore.db.strings.mapped_memory=100M > > neostore.propertystore.db.arrays.mapped_memory=0M > > > > This are the numbers of my database: > > > > 488.2M neostore.nodestore.db > > 12.6G neostore.propertystore.db > > 67G neostore.propertystore.db.strings > > 9.9G neostore.relationshipstore.db > > > > In the page configuartion seetings the only thing (in my view) that > > could help me is the garbage collector but I've ever thought that > > garbage collector works when I need space and I have things to remove, > > therefore I'm not sure if this information can help me. > > > > El mié, 24-02-2010 a las 10:38 +0100, Johan Svensson escribió: > > > Hi, > > > > > > What is the error/problem you get when executing the program? > > > > > > It is not a good idea to use all available RAM for the Java heap. Have > > > a look at http://wiki.neo4j.org/content/Configuration_Settings for > > > more information on how to configure Neo4j to make good use of > > > available RAM. > > > > > > Regards, > > > -Johan > > > > > > On Tue, Feb 23, 2010 at 6:11 PM, Miguel Angel Aguila <magu...@ac.upc.edu> > > > wrote: > > > > Yes I give 64 GB and only 60 GB in diferent executions. > > > > > > > > 2010/2/23 Mattias Persson <matt...@neotechnology.com> > > > > > > > >> One more thing... how much heap have you given the JVM? You control > > > >> how much RAM your JVM gets with the -Xmx option to the java command. > > > >> F.ex: > > > >> > > > >> java -Xmx10G -cp ..... > > > >> > > > >> Please see more information regarding performance and tweaking at > > > >> http://wiki.neo4j.org/content/Neo4j_Performance_Guide > > > >> > > > >> 2010/2/23 Mattias Persson <matt...@neotechnology.com>: > > > >> > I see that you're using an older version of Neo4j. Have you tried > > > >> > using > > > >> 1.0? > > > >> > > > > >> > http://components.neo4j.org/neo4j-kernel > > > >> > http://neo4j.org/download > > > >> > > > > >> > Maven > > > >> > groupId: org.neo4j > > > >> > artifactId: neo4j-kernel > > > >> > version: 1.0 > > > >> > > > > >> > ...and at first glance I don't see any direct problem with your code. > > > >> > > > > >> > 2010/2/23 Miguel Ángel Águila <magu...@ac.upc.edu>: > > > >> >> Hello, > > > >> >> > > > >> >> I'm doing a code to walk around all the nodes and get the node with > > > >> >> the > > > >> >> maximum out degree. The main problem is that there are around 57 > > > >> >> million > > > >> >> of nodes and 322 million of edges in the neo database, and using a > > > >> >> 64 GB > > > >> >> RAM is insufficient to execute the program. It is the code: > > > >> >> Can anyone see what I'm doing wrong? I don't understand. > > > >> >> > > > >> >> Thanks. > > > >> >> Mike > > > >> >> > > > >> >> > > > >> >> > > > >> >> public static void getNodeMaxOutDegreeNeo(final NeoService neo, > > > >> >> final IndexService indexService) { > > > >> >> Node idNode; > > > >> >> long degree; > > > >> >> long maxDegree = 0; > > > >> >> IndexHits<Node> nodes; > > > >> >> long idDegree; > > > >> >> long idMaxDegree; > > > >> >> Transaction tx = neo.beginTx(); > > > >> >> try { > > > >> >> nodes = indexService.getNodes("TYPE", "titles"); > > > >> >> Iterator<Node> it = nodes.iterator(); > > > >> >> Node idNodeMaxDegree=it.next(); > > > >> >> Iterator<Relationship> relIterator = > > > >> >> idNodeMaxDegree.getRelationships( > > > >> >> NeoDataBase.MyRelationshipTypes.REF, > > > >> >> Direction.OUTGOING).iterator(); > > > >> >> > > > >> >> while (relIterator.hasNext()) { > > > >> >> relIterator.next(); > > > >> >> maxDegree++; > > > >> >> } > > > >> >> long counter = 0; > > > >> >> while (it.hasNext()) { > > > >> >> if ( ++counter % 50000 == 0 ) { > > > >> >> tx.success(); > > > >> >> tx.finish(); > > > >> >> tx = neo.beginTx(); > > > >> >> } > > > >> >> idNode = it.next(); > > > >> >> relIterator = > > > >> >> idNode.getRelationships(NeoDataBase.MyRelationshipTypes.REF, > > > >> >> Direction.OUTGOING).iterator(); > > > >> >> degree = 0; > > > >> >> while (relIterator.hasNext()) { > > > >> >> relIterator.next(); > > > >> >> degree++; > > > >> >> } > > > >> >> if (degree >= maxDegree) { > > > >> >> if(degree==maxDegree) { > > > >> >> idDegree= > > > >> >> Long.getLong((String)idNode.getProperty("ID_TITLE")); > > > >> >> idMaxDegree= > > > >> >> Long.getLong((String)idNodeMaxDegree.getProperty("ID_TITLE")); > > > >> >> if(idDegree>idMaxDegree) { > > > >> >> idNodeMaxDegree = idNode; > > > >> >> } > > > >> >> } > > > >> >> else { > > > >> >> maxDegree = degree; > > > >> >> idNodeMaxDegree = idNode; > > > >> >> } > > > >> >> } > > > >> >> } > > > >> >> > > > >> >> System.out.println("OId = > > > >> >> "+(String)idNodeMaxDegree.getProperty("ID_TITLE")); > > > >> >> System.out.println("Title = > > > >> >> "+(String)idNodeMaxDegree.getProperty("NAME")); > > > >> >> System.out.println("#refs = "+maxDegree); > > > >> >> tx.success(); > > > >> >> } > > > >> >> finally > > > >> >> { > > > >> >> tx.finish(); > > > >> >> } > > > >> >> } > > > _______________________________________________ > > > Neo mailing list > > > User@lists.neo4j.org > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > _______________________________________________ > > Neo mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > > Neo mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
