Hi Peter, The limits that you have specified is enough for me. Thanks you again.
Núria. 2009/12/26 Peter Neubauer <peter.neuba...@neotechnology.com> > Hi Núria, > the current ID-scheme of using Integers for IDs for both Nodes, > Relationships and Properties limits the possible node space size to 4 > Billion nodes, 4 Billion Relationships and 4 Billion properties. Of > course one could switch to Long as IDs, but that will increase the > reserved amount of bytes and cause possible performance penalties. > However, this is the current limit, after that you have to start > thinking about sharding along a suitable domain-specific criteria. > What size and domain are you imagining? > > However, when dealing with bigger nodespaces you probably want to > increase RAM of your server machine and think about SSD in order to > keep the often-used parts of your graph cached and minimize IO cost. > > HTH > > Cheers, > > /peter neubauer > > COO and Sales, Neo Technology > > GTalk: neubauer.peter > Skype peter.neubauer > Phone +46 704 106975 > LinkedIn http://www.linkedin.com/in/neubauer > Twitter http://twitter.com/peterneubauer > > http://www.neo4j.org - Relationships count. > http://gremlin.tinkerpop.com - PageRank in 2 lines of code. > http://www.linkedprocess.org - Computing at LinkedData scale. > > > > On Sat, Dec 26, 2009 at 4:10 PM, Núria Trench <nuriatre...@gmail.com> > wrote: > > Hi, > > > > I have just finished parsing and creating the database with the latest > > index-util-0.9-SNAPSHOT available in your repository. It has been > finished > > succesfully so I must thank you for your interest and useful help. > > And, finally, I have one last question. I have been created 180 million > of > > edges and 20 million of nodes. Is it possible to create a bigger amount > of > > edges and nodes with Neo4j? Do you have a limit? > > > > Thank your very much again. > > > > 2009/12/21 Núria Trench <nuriatre...@gmail.com> > > > >> Hi again Mattias, > >> > >> I'm still trying to parse all the data in order to create the graph. I > will > >> report the results as soon as possible. > >> Thank you very much for your interest. > >> > >> Núria. > >> > >> 2009/12/21 Mattias Persson <matt...@neotechnology.com> > >> > >> Hi again, > >>> > >>> any luck with this yet? > >>> > >>> 2009/12/11 Núria Trench <nuriatre...@gmail.com>: > >>> > Thank you very much Mattias. I will test it as soon as possible and > I'll > >>> > will tell you something. > >>> > > >>> > Núria. > >>> > > >>> > 2009/12/11 Mattias Persson <matt...@neotechnology.com> > >>> > > >>> >> I've tried this a couple of times now and first of all I see some > >>> >> problems in your code: > >>> >> > >>> >> 1) In the method createRelationsTitleImage you have an inverted > "head > >>> >> != -1" check where it should be "head == -1" > >>> >> > >>> >> 2) You index relationships in createRelationsBetweenTitles method, > >>> >> this isn't ok since the index can only manage nodes. > >>> >> > >>> >> And I recently committed a "fix" which removed the caching layer in > >>> >> the LuceneIndexBatchInserterImpl (and therefore also > >>> >> LuceneFulltextIndexBatchInserter). This probably fixes your > problems. > >>> >> I'm also working on a performance fix which makes consecutive > getNodes > >>> >> calls faster. > >>> >> > >>> >> So I think that with these fixes (1) and (2) and the latest > index-util > >>> >> 0.9-SNAPSHOT your sample will run fine. Also you could try without > >>> >> calling optimize. See more information at > >>> >> http://wiki.neo4j.org/content/Indexing_with_BatchInserter > >>> >> > >>> >> 2009/12/10 Mattias Persson <matt...@neotechnology.com>: > >>> >> > To continue this thread in the user list: > >>> >> > > >>> >> > Thanks Núria, I've gotten your samples code/files and I'm running > it > >>> >> > now to try to reproduce you problem. > >>> >> > > >>> >> > 2009/12/9 Núria Trench <nuriatre...@gmail.com>: > >>> >> >> I have finished uploading the 4 csv files. You'll see an e-mail > with > >>> the > >>> >> >> other 3 csv files packed in a rar file. > >>> >> >> Thanks, > >>> >> >> > >>> >> >> Núria. > >>> >> >> > >>> >> >> 2009/12/9 Núria Trench <nuriatre...@gmail.com> > >>> >> >>> > >>> >> >>> Yes, you are right. But there is one csv file that is too big to > be > >>> >> packed > >>> >> >>> with other files and I am reducing it. > >>> >> >>> I am sending the other files now. > >>> >> >>> > >>> >> >>> 2009/12/9 Mattias Persson <matt...@neotechnology.com> > >>> >> >>>> > >>> >> >>>> By the way, you might consider packing those files (with zip or > >>> tar.gz > >>> >> >>>> or something) cause they will shrink quite well > >>> >> >>>> > >>> >> >>>> 2009/12/9 Mattias Persson <matt...@neotechnology.com>: > >>> >> >>>> > Great, but I only got the images.csv file... I'm starting to > >>> test > >>> >> with > >>> >> >>>> > that at least > >>> >> >>>> > > >>> >> >>>> > 2009/12/9 Núria Trench <nuriatre...@gmail.com>: > >>> >> >>>> >> Hi again, > >>> >> >>>> >> > >>> >> >>>> >> The errors show up after being parsed 2 csv files to create > all > >>> the > >>> >> >>>> >> nodes, > >>> >> >>>> >> just in the moment of calling the method "getSingleNode" for > >>> >> looking > >>> >> >>>> >> up the > >>> >> >>>> >> tail and head node for creating all the edges by reading the > >>> other > >>> >> two > >>> >> >>>> >> csv > >>> >> >>>> >> files. > >>> >> >>>> >> > >>> >> >>>> >> I am sending with Sprend the four csv files that will help > you > >>> to > >>> >> >>>> >> trigger > >>> >> >>>> >> index behaviour. > >>> >> >>>> >> > >>> >> >>>> >> Thank you, > >>> >> >>>> >> > >>> >> >>>> >> Núria. > >>> >> >>>> >> > >>> >> >>>> >> 2009/12/9 Mattias Persson <matt...@neotechnology.com> > >>> >> >>>> >>> > >>> >> >>>> >>> Hmm, I've no idea... but does the errors show up early in > the > >>> >> process > >>> >> >>>> >>> or do you have to insert a LOT of data to trigger it? In > such > >>> case > >>> >> >>>> >>> you > >>> >> >>>> >>> could send me a part of them... maybe using > >>> http://www.sprend.se, > >>> >> >>>> >>> WDYT? > >>> >> >>>> >>> > >>> >> >>>> >>> 2009/12/9 Núria Trench <nuriatre...@gmail.com>: > >>> >> >>>> >>> > Hi Mattias, > >>> >> >>>> >>> > > >>> >> >>>> >>> > The data isn't confident but the files are very big (5,5 > >>> GB). > >>> >> >>>> >>> > How can I send you this data? > >>> >> >>>> >>> > > >>> >> >>>> >>> > 2009/12/9 Mattias Persson <matt...@neotechnology.com> > >>> >> >>>> >>> >> > >>> >> >>>> >>> >> Yep I got the java code, thanks. Yeah if the data is > >>> confident > >>> >> or > >>> >> >>>> >>> >> sensitive you can just send me the formatting, else > >>> consider > >>> >> >>>> >>> >> sending > >>> >> >>>> >>> >> the files as well (or a subset if they are big). > >>> >> >>>> >>> >> > >>> >> >>>> >>> >> 2009/12/9 Núria Trench <nuriatre...@gmail.com>: > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Mattias Persson, [matt...@neotechnology.com] > >>> >> Neo Technology, www.neotechnology.com > >>> >> _______________________________________________ > >>> >> Neo mailing list > >>> >> User@lists.neo4j.org > >>> >> https://lists.neo4j.org/mailman/listinfo/user > >>> >> > >>> > _______________________________________________ > >>> > Neo mailing list > >>> > User@lists.neo4j.org > >>> > https://lists.neo4j.org/mailman/listinfo/user > >>> > > >>> > >>> > >>> > >>> -- > >>> Mattias Persson, [matt...@neotechnology.com] > >>> Neo Technology, www.neotechnology.com > >>> _______________________________________________ > >>> Neo mailing list > >>> User@lists.neo4j.org > >>> https://lists.neo4j.org/mailman/listinfo/user > >>> > >> > >> > > _______________________________________________ > > Neo mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
