Hi, it is hard to make predictions on how long it will take to migrate a store to 1.5. What the migration time actually depends on is node and relationship population, number of properties and number of dynamic records in use. This correlates to file size but one cannot determine the time based on just that, since not in use records still take up space but do not contribute to the migration time. Also indices are not touched, so they have nothing to do with upgrade time.
Based on our runs, a store of the size you mention should take in the order of 1-2 hours to upgrade, depending of course on OS, filesystem, hardware etc. As far as the safety of your data goes: It is highly recommended to do a backup of your data in a safe location before starting the upgrade process and after shutting down cleanly the neo4j database. However, we have taken all the measures we could to make sure that the process is as safe as possible and we also keep a copy of the existing database files under <database_dir>/upgrade_backup just to be extra cautious. In the unlikely case anything goes south, you can always fix the problem, restore the old database and try the upgrade again. hope that helps, CG On Wed, Oct 12, 2011 at 9:30 AM, st.pa <st...@web.de> wrote: > > Hi, this is great, I was still holding my breath and at first didnt even > notice, then I got the snapshot and changed classpaths in my project to > use 1.5.m, when I noticed the following error > > org.neo4j.graphdb.TransactionFailureException: Could not create data > source [nioneodb], see nested exception for cause of error > org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:153) > org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:112) > at > org.neo4j.kernel.EmbeddedGraphDbImpl.<init>(EmbeddedGraphDbImpl.java:190) > at > org.neo4j.kernel.EmbeddedGraphDatabase.<init>(EmbeddedGraphDatabase.java:80) > ... > Caused by: > org.neo4j.kernel.impl.storemigration.UpgradeNotAllowedByConfigurationException: > To enable automatic upgrade, please set allow_store_upgrade in > configuration properties > at > org.neo4j.kernel.impl.storemigration.ConfigMapUpgradeConfiguration.checkConfigurationAllowsAutomaticUpgrade(ConfigMapUpgradeConfiguration.java:41) > at > org.neo4j.kernel.impl.storemigration.StoreUpgrader.attemptUpgrade(StoreUpgrader.java:50) > at > org.neo4j.kernel.impl.nioneo.store.NeoStore.tryToUpgradeStores(NeoStore.java:127) > at > org.neo4j.kernel.impl.nioneo.store.NeoStore.verifyCorrectTypeDescriptorAndVersion(NeoStore.java:98) > at > org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.loadStorage(CommonAbstractStore.java:196) > at > org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.<init>(CommonAbstractStore.java:104) > at > org.neo4j.kernel.impl.nioneo.store.AbstractStore.<init>(AbstractStore.java:120) > at org.neo4j.kernel.impl.nioneo.store.NeoStore.<init>(NeoStore.java:72) > at > org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.<init>(NeoStoreXaDataSource.java:137) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.neo4j.kernel.impl.transaction.XaDataSourceManager.create(XaDataSourceManager.java:75) > at > org.neo4j.kernel.impl.transaction.TxModule.registerDataSource(TxModule.java:147) > ... 24 more > > > which tells me to allow the neo-store to be updated. That spooked me so > much that I didnt yet try it in earnest, focusing instead on rewriting > my app to work without thread-interrupts because that was apparently > messing with lucene, and trying to find the reason why the > worker-threads' transactions always keep deadlocking... as much as I > desired that cypher-feature with the RelationshipIndexLookup, there are > work-arounds, but I'll get around to testing it sometime soon. > > Before I do upgrade the store, > > 39 MB neostore.nodestore.db > 412 MB neostore.propertystore.db > 4315 MB neostore.propertystore.db.arrays > 1857 MB neostore.propertystore.db.strings > 1090 MB neostore.relationshipstore.db > plus 1.6GB of indices, > > any idea how long such a neostore-upgrade might take from 1.4.1. to that > 1.5.snapshot? like seconds, minutes, hours, or worse? > > > Am 02.10.2011 19:25, schrieb Andres Taylor: >> And now it's ready. >> >> If you are using SNAPSHOT, please update your dependencies, and check it >> out. >> http://docs.neo4j.org/chunked/snapshot/query-start.html#start-relationship-by-index-lookup >> >> Thanks again for providing feedback. It's much more fun coding when I know >> someone is using it. >> >> Andrés >> >> On Fri, Sep 30, 2011 at 10:37 AM, Andres Taylor< >> andres.tay...@neotechnology.com> wrote: >> >>> Hey! >>> >>> This is something that has been planned since the very first iteration of >>> Cypher, but no-one seemed to need it, so I have pushed it in front of me. >>> >>> Now we seem to have an user that wants it, so I'll add it. >>> >>> Don't hold your breath though - it'll be ready when it's ready. >>> >>> Andrés >>> >>> >>> On Thu, Sep 29, 2011 at 1:10 PM, st.pa<st...@web.de> wrote: >>> >>>> hi. >>>> when i use a cypher query to get the queued nodes associated via >>>> queue-relationships to a repository-node which has several million other >>>> relationships then it takes about six minutes to fill that result list: >>>> >>>> <pre> >>>> List<Node> list = new ArrayList<Node>(); >>>> StringBuffer s = new StringBuffer(); >>>> s.append("START repository = ("); >>>> s.append((Long) controller.getRepositoryNode().getId()); >>>> s.append(") MATCH (repository)-[:"); >>>> s.append(Rels.REL_REPOSITORY_QUEUE); >>>> s.append("]->(node) RETURN node ORDER BY node."); >>>> s.append(Keys.KEY_TIMESTAMP); >>>> s.append(" SKIP "); >>>> s.append(Integer.toString(offset)); >>>> s.append(" LIMIT "); >>>> s.append(Integer.toString(buffersize)); >>>> Log.log(s.toString()); >>>> Query query = this.parser.parse(s.toString()); >>>> ExecutionResult result = this.engine.execute(query); >>>> Iterator<Node> iterator = result.columnAs("node"); >>>> while (iterator.hasNext()) { >>>> list.add(iterator.next()); >>>> } >>>> </pre> >>>> >>>> but when i directly query the relationshipIndex containing all >>>> queue-relationships and assuming that sorting by timestamp is equivalent >>>> to sorting by node-ids then it completes in just half a minute, which is >>>> a dozen times faster: >>>> >>>> <pre> >>>> List<Node> list = new ArrayList<Node>(); >>>> IndexHits<Relationship> hits = >>>> controller.getRelationIndexRepositoryUrlQueue().query( >>>> Keys.KEY_TYPE,Rels.REL_REPOSITORY_QUEUE, >>>> controller.getRepositoryNode(),null >>>> ); >>>> List<Long> ids = new ArrayList<Long>(); >>>> try { >>>> for (Relationship hit : hits) { >>>> ids.add( >>>> hit.getEndNode().getId() >>>> ); >>>> } >>>> } finally { >>>> hits.close(); >>>> } >>>> Collections.sort(ids); >>>> for (int i = offset; i< Math.min(offset + buffersize,ids.size()); i++) { >>>> list.add( >>>> this.graph.getNodeById( >>>> ids.get(i) >>>> ) >>>> ); >>>> } >>>> </pre> >>>> >>>> now my question. how can i efficiently get such a result list sorted for >>>> some node property other than id. and question number two: how would a >>>> cypher statement look like if it supported relationship-indices. maybe >>>> >>>> <pre> >>>> START x = (1) >>>> MATCH (x)-[r:(MyRelationIndex,type,MyRelationType)]->(y) >>>> RETURN r,y >>>> ORDER BY y.somePropertyOtherThanId >>>> </pre> >>>> >>>> similar to nodeIndexLookups, where i give the name of the relationship >>>> index to use, key and value for the relationship, start and end nodes >>>> are given by (x) and (y) in this case. i'd like that feature very much >>>> and if i knew how i'd probably even help to get on with it but anyway i >>>> can be patient. >>>> >>>> >>>> kindly, st.p. >>>> _______________________________________________ >>>> Neo4j mailing list >>>> User@lists.neo4j.org >>>> https://lists.neo4j.org/mailman/listinfo/user >>>> >>> >>> >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
