The best solution currently may be to skip the IndexProvider part and just implement a "SpatialIndex implements Index<Node>" or something which can be instantiated manually in the code.
2010/11/26 Mattias Persson <matt...@neotechnology.com> > > > 2010/11/26 Craig Taverner <cr...@amanzi.com> > > This is interesting to me too, not just for the spatial index, but for the >> composite index which I'm working on. That too is a tree based index >> directly in the graph with the leaf nodes connected directly to the data >> nodes (indexed nodes). This is obviously very convenience because the >> index >> query in internally just a traversal. So there is great value in having >> the >> index in the same database as the data. It certainly comes as a surprise >> to >> me that this will be problematic. >> >> In my own applications, the index code is just like application code, >> using >> the same transactions and everything. That is clean and simple. Is it >> really >> necessary to have indexing somehow separated in a way that causes these >> issues? >> > > Each index can be associated with different implementations (f.ex. lucene, > spatial, whatever), so somewhere needs to be stored a mapping between index > name and its configuration. Everything stored in neo4j should go via a > logical log to be transactional and safe for recovery and also work well in > an HA environment where logical logs are the main means of transferring > data. > > Currently the commands for creating an index (storing a mapping > indexName->config) is stored in the lucene logical log (the main > implementation of an IndexProvider) because we don't want to mess with those > internals of neo4j right before a major release. Since the spatial index > uses the nioneo kernel data source there are no such commands to store in > that logical log, so doing it "the right way" is impossible a.t.m. Sure we > can store index configuration right there when an index is created, skipping > the logical log, but it won't work in an HA environment or be transactional > safe for recovery or any of that. The actual indexing/querying (which is > really operations on the graph) will work just as good as anything. > > Did that make it any more clear to you? > > >> >> >> On Fri, Nov 26, 2010 at 12:03 PM, Mattias Persson < >> matt...@neotechnology.com >> > wrote: >> >> > 2010/11/26 Peter Neubauer <peter.neuba...@neotechnology.com> >> > >> > > Mhh, >> > > it was the public constructors. >> > > >> > > Also, the Spatial Index will be operating on the same Neo4j graph >> > > instance and build an internal index. How am I going to server the XA >> > > Datasource etc in that case? Could I get hold of the one for the >> > > Kernel and return it maybe, or is there another mechaism signalling >> > > that this index is actually belonging to the same XA datasource as the >> > > kernel? >> > > >> > >> > That's what I meant in my previous post. I'll try to fix that, but I >> just >> > realized that there's internal problems which makes the possible >> solution >> > not work in an HA environment and feel quite "hackish". It's because of >> > "create index" and "delete index" commands lives in the lucene-index >> > component/logical log. And to move them to kernel could be a mismatch >> and >> > also it's too close to the next release to touch any of those things. >> > >> > >> > > >> > > Cheers, >> > > >> > > /peter neubauer >> > > >> > > GTalk: neubauer.peter >> > > Skype peter.neubauer >> > > Phone +46 704 106975 >> > > LinkedIn http://www.linkedin.com/in/neubauer >> > > Twitter http://twitter.com/peterneubauer >> > > >> > > http://www.neo4j.org - Your high performance graph >> > database. >> > > http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing >> party. >> > > >> > > >> > > >> > > On Thu, Nov 25, 2010 at 9:38 PM, Mattias Persson >> > > <matt...@neotechnology.com> wrote: >> > > > 2010/11/25 Peter Neubauer <peter.neuba...@neotechnology.com> >> > > > >> > > >> Hi all, >> > > >> Craig and me thought of registering Neo4j-Spatial as a new >> > > >> IndexProvider, so you can set up a NodeIndex with all the >> geographic >> > > >> hints and setup you need, and then just index geometry nodes and >> ask >> > > >> spatial queries over it in order to make the component better >> usable, >> > > >> hopefully as easy as Lucene. >> > > >> >> > > >> Now, upon implementing >> > > >> >> > > >> >> > > >> > >> https://github.com/neo4j/neo4j-spatial/blob/master/src/main/java/org/neo4j/gis/spatial/indexprovider/SpatialIndexProvider.java >> > > >> and registering it like >> > > >> >> > > >> >> > > >> > >> https://github.com/neo4j/neo4j-spatial/blob/master/src/main/resources/META-INF/services/org.neo4j.kernel.KernelExtension >> > > >> , >> > > >> the test >> > > >> >> > > >> > >> https://github.com/neo4j/neo4j-spatial/blob/master/src/test/java/org/neo4j/gis/spatial/IndexProviderTest.java >> > > >> is failing, saying that no index provider "spatial" was found. >> > > >> Anything wrong in the process? >> > > >> >> > > > >> > > > Sounds great, and be sure to express any difficulties/shortcomings >> with >> > > the >> > > > API so that it can improved for these kinds of scenarios. >> > > > >> > > > It may be that you have to make the constructor public (I don't know >> in >> > > what >> > > > way the service API uses reflection and all that). Also there's this >> > > > assumption that an IndexProvider provides its own data source. The >> > > spatial >> > > > index provider uses the graph itself (right?) so it could be a >> problem. >> > > Let >> > > > me look at and fix that, and then document how that's done for such >> a >> > > > scenario (hopefully tomorrow). >> > > > >> > > > >> > > >> >> > > >> Cheers, >> > > >> >> > > >> /peter neubauer >> > > >> >> > > >> GTalk: neubauer.peter >> > > >> Skype peter.neubauer >> > > >> Phone +46 704 106975 >> > > >> LinkedIn http://www.linkedin.com/in/neubauer >> > > >> Twitter http://twitter.com/peterneubauer >> > > >> >> > > >> http://www.neo4j.org - Your high performance graph >> > > database. >> > > >> http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing >> > party. >> > > >> _______________________________________________ >> > > >> Neo4j mailing list >> > > >> User@lists.neo4j.org >> > > >> https://lists.neo4j.org/mailman/listinfo/user >> > > >> >> > > > >> > > > >> > > > >> > > > -- >> > > > Mattias Persson, [matt...@neotechnology.com] >> > > > Hacker, Neo Technology >> > > > www.neotechnology.com >> > > > _______________________________________________ >> > > > Neo4j mailing list >> > > > User@lists.neo4j.org >> > > > https://lists.neo4j.org/mailman/listinfo/user >> > > > >> > > _______________________________________________ >> > > Neo4j mailing list >> > > User@lists.neo4j.org >> > > https://lists.neo4j.org/mailman/listinfo/user >> > > >> > >> > >> > >> > -- >> > Mattias Persson, [matt...@neotechnology.com] >> > Hacker, Neo Technology >> > www.neotechnology.com >> > _______________________________________________ >> > Neo4j mailing list >> > User@lists.neo4j.org >> > https://lists.neo4j.org/mailman/listinfo/user >> > >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user >> > > > > -- > Mattias Persson, [matt...@neotechnology.com] > Hacker, Neo Technology > www.neotechnology.com > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
