Let me first look into the details of the current Btree implementation and see how it can be expanded to other datatypes, then look into your typing support in Amanzi and figure out if they match. If so we can port those parts to graph-collections.
> Date: Thu, 30 Jun 2011 01:15:31 +0200 > From: cr...@amanzi.com > To: user@lists.neo4j.org > Subject: Re: [Neo4j] neo4j-graph-collections > > It is technically possible, but it is a somewhat specialized index, not a > normal BTree, so I think you would want both (mine and a classic btree). My > index performs better for certain data patterns, is best with semi-ordered > data and moderately even distributions (since it has no rebalancing), and > requires the developer to pick a good starting 'resolution' which means they > should know something about their data. Perhaps we just port some of the > typing support into a btree in the collections project? > > On Wed, Jun 29, 2011 at 4:19 PM, Niels Hoogeveen > <pd_aficion...@hotmail.com>wrote: > > > > > Craig, > > Would it be possible to merge your work on Amanzi with the work the Neo > > team has done on the Btree component that is now in neo4j-graph-collections, > > so we can eventually have one implementation that meets a broad variety of > > needs? > > Niels > > > > > Date: Wed, 29 Jun 2011 15:34:47 +0200 > > > From: cr...@amanzi.com > > > To: user@lists.neo4j.org > > > Subject: Re: [Neo4j] neo4j-graph-collections > > > > > > I have previously used two solutions to deal with multiple types in > > btrees: > > > > > > - My first index in 2009 was a btree-like n-dim index using generics > > to > > > support int[], long[], float[] and double[] (no strings). I used this > > for > > > TimeLine (long[1]) and Location (double[2]). The knowledge about what > > type > > > was used was in the code for constructing the index (whether a new > > index or > > > accessing an existing index in the graph). > > > - In December I started my amanzi-index (on > > > github<https://github.com/craigtaverner/amanzi-index>) > > > that is also btree-like, n-dimensional. But this time it can index > > multiple > > > types in the same tree (so a float, int and string in the same tree, > > instead > > > of being forced to have all properties of the same type). It is a > > re-write > > > of the previous index to support Strings, and mixed types. This time > > it does > > > save the type information in meta-data at the tree root. > > > > > > The idea of using a 'comparator' class for the types is similar, but > > simpler > > > than the idea I implemented for amanzi-index, where I have mapper classes > > > that describe not only how to compare types, but also how to map from > > values > > > to index keys and back. This includes (to some extent) the concept of the > > > lucene analyser, since the mapper can decide on custom distribution of, > > for > > > example, strings and category indexes. > > > > > > For both of these indexes, you configure the index up front, and then > > only > > > call index.add(node) to index a node. This will fit in well with the new > > > auto-indexing ideas in neo4j. > > > > > > On Wed, Jun 29, 2011 at 2:25 PM, Niels Hoogeveen > > > <pd_aficion...@hotmail.com>wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > At this moment Btree only supports the primitive datatype long, while > > Rtree > > > > only supports the datatype double. For Btree it makes sense to at least > > > > support strings, floats, doubles and ints too. Use cases for these data > > > > types are pretty obvious and are Btree backed in (almost) every RDBMS > > > > product around.I think the best solution would be to create Comparator > > > > objects wrapping these primitive data types and store the class name of > > the > > > > comparator in root of the index tree. This allows users to create their > > own > > > > comparators for datatypes not covered yet. It would make sense people > > would > > > > want to store BigInt and BigDecimal objects in a Btree too, others may > > want > > > > to store dates (instead of datetime), fractions, complex numbers or > > even > > > > more exotic data types. > > > > Niels > > > > > From: sxk1...@hotmail.com > > > > > To: user@lists.neo4j.org > > > > > Date: Tue, 28 Jun 2011 22:43:24 -0700 > > > > > Subject: Re: [Neo4j] neo4j-graph-collections > > > > > > > > > > > > > > > I've read through this thread in more detail and have a few thoughts, > > > > when you talk about type I am assuming that you are referring to an > > > > interface that both (Btree,Rtree) can implement, for the data types I'd > > like > > > > to understand the use cases first before implementing the different > > data > > > > types, maybe we could store types of Object instead of Long or Double > > and > > > > implement comparators in a more meaningful fashion. Also I was > > wondering > > > > if unit tests would need to be extracted out of the spatial component > > and > > > > embedded inside the graph-collections component as well or whether we'd > > > > potentially need to write brand new unit tests as well. > > > > > Craig as I mentioned I'd love to help, let me know if it would be > > > > possible to fork a repo or to talk in more detail this week. > > > > > Regards > > > > > > > > > > > From: pd_aficion...@hotmail.com > > > > > > To: user@lists.neo4j.org > > > > > > Date: Wed, 29 Jun 2011 01:35:43 +0200 > > > > > > Subject: Re: [Neo4j] neo4j-graph-collections > > > > > > > > > > > > > > > > > > As to the issue of n-dim doubles, it would be interesting to > > consider > > > > creating a set of classes of type Orderable (supporting <, <=, >, >= > > > > operations), this we can use in both Rtree and Btree. Right now Btree > > only > > > > supports datatype Long. This should also become more generic. A first > > step > > > > we can take is at least wrap the common datatypes in Orderable classes. > > > > > > Niels > > > > > > > > > > > > > Date: Wed, 29 Jun 2011 00:32:15 +0200 > > > > > > > From: cr...@amanzi.com > > > > > > > To: user@lists.neo4j.org > > > > > > > Subject: Re: [Neo4j] neo4j-graph-collections > > > > > > > > > > > > > > The RTree in principle should be generalizable, but the current > > > > > > > implementation in neo4j-spatial does make a few assumptions > > specific > > > > to > > > > > > > spatial data, and makes use of spatial envelopes for the tree > > node > > > > bounding > > > > > > > boxes. It is also specific to 2D. We could make a few > > improvements > > > > first, > > > > > > > like generalizing to n-dimensions, replacing the recursive search > > > > with a > > > > > > > traverser and generalizing the bounding boxes to be simple > > > > double-arrays. > > > > > > > Then the only thing left would be to decide if it is ok for it to > > be > > > > based > > > > > > > on n-dim doubles or should be generalized to more types. > > > > > > > > > > > > > > On Tue, Jun 28, 2011 at 11:14 PM, Saikat Kanjilal < > > > > sxk1...@hotmail.com>wrote: > > > > > > > > > > > > > > > I would be interested in helping out with this, let me know > > next > > > > steps. > > > > > > > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > > > > > > On Jun 28, 2011, at 8:49 AM, Niels Hoogeveen < > > > > pd_aficion...@hotmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > A couple of weeks ago Peter Neubauer set up a repository for > > > > in-graph > > > > > > > > datastructures: > > https://github.com/peterneubauer/graph-collections > > > > . > > > > > > > > > At this time of writing only the Btree/Timeline index is part > > of > > > > this > > > > > > > > "component". > > > > > > > > > In my opinion it would be interesting to move the Rtree parts > > of > > > > > > > > neo-spatial to neo4j-graph-collections too. > > > > > > > > > I looked at the code but don't feel competent to seperate out > > > > those > > > > > > > > classes that support generic Rtrees from those classes that are > > > > clearly > > > > > > > > spatial related. > > > > > > > > > Is there any enthusiasm for such a project and if so, who is > > > > willing and > > > > > > > > able to do this? > > > > > > > > > Niels > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > > Neo4j mailing list > > > > > > > > > User@lists.neo4j.org > > > > > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > > Neo4j mailing list > > > > > > > > User@lists.neo4j.org > > > > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > > > > _______________________________________________ > > > > > > > Neo4j mailing list > > > > > > > User@lists.neo4j.org > > > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > _______________________________________________ > > > > > > Neo4j mailing list > > > > > > User@lists.neo4j.org > > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > _______________________________________________ > > > > > Neo4j mailing list > > > > > User@lists.neo4j.org > > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > > > > > > _______________________________________________ > > > > Neo4j mailing list > > > > User@lists.neo4j.org > > > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > > _______________________________________________ > > > Neo4j mailing list > > > User@lists.neo4j.org > > > https://lists.neo4j.org/mailman/listinfo/user > > > > _______________________________________________ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
