Hi Mattias, Thank you very much for fixing the problem so fast. I will try it as soon as the new changes will be available in the maven repository.
Núria. 2009/12/4 Mattias Persson <matt...@neotechnology.com> > I fixed the problem and also added a cache per key for faster > getNodes/getSingleNode lookup during the insert process. However the > cache assumes that there's nothing in the index when the process > starts (which almost always will be true) to speed things up even > further. > > You can control the cache size and if it should be used by overriding > the (this is also documented in the Javadoc): > > boolean useCache() > int getMaxCacheSizePerKey() > > methods in your LuceneIndexBatchInserterImpl instance. The new changes > should be available in the maven repository within an hour. > > 2009/12/4 Mattias Persson <matt...@neotechnology.com>: > > I think I found the problem... it's indexing as it should, but it > > isn't reflected in getNodes/getSingleNode properly until you > > flush/optimize/shutdown the index. I'll try to fix it today! > > > > 2009/12/3 Núria Trench <nuriatre...@gmail.com>: > >> Thank you very much for your response. > >> If you need more information, you only have to send an e-mail and I will > try > >> to explain it better. > >> > >> Núria. > >> > >> 2009/12/3 Mattias Persson <matt...@neotechnology.com> > >> > >>> This is something I'd like to reproduce and I'll do some testing on > >>> this tomorrow > >>> > >>> 2009/12/3 Núria Trench <nuriatre...@gmail.com>: > >>> > Hello, > >>> > > >>> > Last week, I decided to download your graph database core in order to > use > >>> > it. First, I created a new project to parse my CSV files and create a > new > >>> > graph database with Neo4j. This CSV files contain 150 milion edges > and 20 > >>> > milion nodes. > >>> > > >>> > When I finished to write the code which will create the graph > database, I > >>> > executed it and, after six hours of execution, the program crashes > >>> because > >>> > of a Lucene exception. The exception is related to the index merging > and > >>> it > >>> > has the following message: > >>> > "mergeFields produced an invalid result: docCount is 385282378 but > fdx > >>> file > >>> > size is 3082259028; now aborting this merge to prevent index > corruption" > >>> > > >>> > I have searched on the net and I found that it is a lucene bug. The > >>> > libraries used for executing my project were: > >>> > neo-1.0-b10 > >>> > index-util-0.7 > >>> > lucene-core-2.4.0 > >>> > > >>> > So, I decided to use a newer Lucene version. I found that you have a > >>> newer > >>> > index-util version so I updated the libraries: > >>> > neo-1.0-b10 > >>> > index-util-0.9 > >>> > lucene-core-2.9.1 > >>> > > >>> > When I had updated those libraries, I tried to execute my project > again > >>> and > >>> > I found that, in many occassions, it was not indexing properly. So, I > >>> tried > >>> > to optimize the index after every time I indexed something. This was > a > >>> > solution because, after that, it was indexing properly but the time > >>> > execution increased a lot. > >>> > > >>> > I am not using transactions, instead of this, I am using the Batch > >>> Inserter > >>> > with the LuceneIndexBatchInserter. > >>> > > >>> > So, my question is: What can I do to solve this problem? If use > >>> > index-util-0.7 I cannot finish the execution of creating the graph > >>> database > >>> > and I use index-util-0.9 I have to optimize the index in every > insertion > >>> and > >>> > the execution never ever ends. > >>> > > >>> > Thank you very much in advance, > >>> > > >>> > Núria. > >>> > _______________________________________________ > >>> > Neo mailing list > >>> > User@lists.neo4j.org > >>> > https://lists.neo4j.org/mailman/listinfo/user > >>> > > >>> > >>> > >>> > >>> -- > >>> Mattias Persson, [matt...@neotechnology.com] > >>> Neo Technology, www.neotechnology.com > >>> _______________________________________________ > >>> Neo mailing list > >>> User@lists.neo4j.org > >>> https://lists.neo4j.org/mailman/listinfo/user > >>> > >> _______________________________________________ > >> Neo mailing list > >> User@lists.neo4j.org > >> https://lists.neo4j.org/mailman/listinfo/user > >> > > > > > > > > -- > > Mattias Persson, [matt...@neotechnology.com] > > Neo Technology, www.neotechnology.com > > > > > > -- > Mattias Persson, [matt...@neotechnology.com] > Neo Technology, www.neotechnology.com > _______________________________________________ > Neo mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
