The planar implementation is not a full blown topology library like spatial3d and it only aims to provide some very fast implementation for some common use cases. Therefore we might argue that such implementation lives on core. In addition, if we are to add a new spatial tree that supports indexing shapes, where should it go?
1. If we remove all spatial code from core, then the tree might need to be added wherever the spatial code has been moved to. 2. If the tree is added to core, it should contain at least a basic implementation. One final note, the geo3d universe contains references to the planar universe. That only make sense if those shapes are generic and are in core. Ignacio From: Karl Wright [mailto:daddy...@gmail.com] Sent: Sunday, June 24, 2018 7:10 AM To: Lucene/Solr dev <dev@lucene.apache.org> Subject: Re: [DISCUSS] Geo/spatial organization in Lucene Data points: (1) For both geo3d and Robert's implementation (at least!) there exists a public API already. For geo3d, this consists of: drwxrwxrwx 0 root root 512 Jun 19 02:47 geom -rwxrwxrwx 1 root root 5586 Jun 19 02:47 PointInGeo3DShapeQuery.java -rwxrwxrwx 1 root root 4940 Jun 19 02:47 Geo3DPointOutsideDistanceComparator.java -rwxrwxrwx 1 root root 6225 Jun 19 02:47 Geo3DPointDistanceComparator.java -rwxrwxrwx 1 root root 11872 Apr 10 11:59 Geo3DUtil.java -rwxrwxrwx 1 root root 966 Mar 2 17:39 package-info.java -rwxrwxrwx 1 root root 5486 Mar 2 17:39 PointInShapeIntersectVisitor.java -rwxrwxrwx 1 root root 3175 Mar 2 17:39 Geo3DPointSortField.java -rwxrwxrwx 1 root root 3217 Mar 2 17:39 Geo3DPointOutsideSortField.java -rwxrwxrwx 1 root root 8681 Mar 2 17:39 Geo3DPoint.java -rwxrwxrwx 1 root root 22099 Mar 2 17:39 Geo3DDocValuesField.java ... plus shape factories found in geom. There are similar public API classes in Robert's implementation, but they are implemented very differently and work only with 2d points. A fair bit of effort went into insuring that the public api was well thought out, as lightweight as possible, and defensible. (2) Neither Robert's planar, or my 3d implementation, has any external dependencies. (3) There exists a spatial-4j implementation for geo3d as well, in spatial-extras. spatial-extras does have external dependencies. So, as you can see, merging all the packages is possible, but only if you sacrifice backwards compatibility, and only if you accept external dependencies. Merging everything except spatial-extras is also possible but you still need to give up backwards compatibility, and you'd be putting classes together that have to individually signal what spatial universe they belong to. That argues against a solution where all geometric implementations are merged into a single "spatial" package at this point. So my thought is that we maintain multiple spatial-X modules, one for each universe, plus spatial-4j. It may be possible to combine Nicholas's and Robert's 2D universes together, but I'd recommend doing that with great care since Robert spent quite a bit of time performance tuning his stuff. Merging "into core" would seem like a good idea only if there was ONE implementation and ONE universe. Thanks, Karl On Sat, Jun 23, 2018 at 6:11 PM Nicholas Knize <nkn...@gmail.com<mailto:nkn...@gmail.com>> wrote: Hi David. I'm not arguing for or against anything in particular. I was simply communicating the state of things as I saw today. And yes, we have spatial code in five modules; and yes, that's pretty crazy. I was originally of the "keep it simple" opinion that all spatial should live in either the spatial module (dependency free), or spatial-extras (dependencies welcome) and that core should have absolutely no spatial code whatsoever. I still feel pretty strongly about this, but there are some compelling reasons to have a simple LatLonPoint in core at minimum. Namely the one Mike raised - "because it's the best default geo implementation we have to offer for basic usage now." I can't argue with that because at the end of the day I think its good for the Lucene project to have a default spatial capability. That being said, I have been working on a simple default shape implementation that also uses BKD. While sandbox is certainly a good place for this to start, I do struggle with where it will ultimately land. Does it become a good default shape implementation that should go in core like LatLonPoint? Or is it considered "expert" and go in the spatial module (which further fragments the spatial codebase)? At the end of the day it doesn't matter as long as 1. the learning curve to contribute is not too high for the rest of the community, 2. javadocs make it clear where everything lives, and 3. people actually read javadocs. On Fri, Jun 22, 2018 at 3:16 PM David Smiley <david.w.smi...@gmail.com<mailto:david.w.smi...@gmail.com>> wrote: Nick, are you not only arguing for spatial code to be in Lucene core, but also for the "spatial" module to continue to exist? And I believe Adrien still wants some spatial stuff in sandbox so that means spatial code in 5 modules. Five modules... let that that sink in... wow. Gosh that's kinda overwhelming IMO. Karl do you have any opinions about this stuff? I don't know what your opinions are, come to think of it. ~ David On Wed, Jun 20, 2018 at 1:01 PM Nicholas Knize <nkn...@gmail.com<mailto:nkn...@gmail.com>> wrote: If I were to pick between the two, I also have a preference for B. I've also tried to keep this whole spatial organization rather simple: core - simple spatial capabilities needed by the 99% spatial use case (e.g., web mapping). Includes LatLonPoint, polygon & distance search (everything currently in sandbox). Lightweight, and no dependencies or complexities. If one wants simple and fast point search, all you need is the core module. spatial - dependency free. Expands on core spatial to include simple shape searching. Uses internal relations. Everything confined to core and spatial modules. spatial-extras - expanded spatial capabilities. Welcomes third-party dependencies (e.g., S3, SIS, Proj4J). Targets more advanced/expert GIS use-cases. geo3d - trades speed for accuracy. I've always struggled with the name, since it implies 3D shapes/point cloud support. But history has shown considering a name change to be a bike-shedding endeavor. At the end of the day I'm up for whatever makes most sense for everyone here. Lord knows we could use more people helping out on geo. - Nick On Wed, Jun 20, 2018 at 11:40 AM Adrien Grand <jpou...@gmail.com<mailto:jpou...@gmail.com>> wrote: I have a slight preference for B similarly to how StandardAnalyzer is in core and other analyzers are in analysis, but no strong feelings. In any case I agree that both A and B would be much better than the current situation. Le mer. 20 juin 2018 à 18:09, David Smiley <david.w.smi...@gmail.com<mailto:david.w.smi...@gmail.com>> a écrit : I think everyone agrees the current state of spatial code organization in Lucene is not desirable. We have a spatial module that has almost nothing in it, we have mature spatial code in the sandbox that needs to "graduate" somewhere, and we've got a handful of geo utilities in Lucene core (mostly because I didn't notice). No agreement has been reached on what the desired state should be. I'd like to hear opinions on this from members of the community. I am especially interested in listening to people that normally don't seem to speak up about spatial matters. Perhaps Uwe Schindlerand Alan Woodward – I respect both of you guys a ton for your tenure with Lucene and aren't too pushy with your opinions. I can be convinced to change my mind, especially if coming from you two. Of course anyone can respond -- this is an open discussion! As I understand it, there are two proposals loosely defined as follows: (A) Common spatial needs will be met in the "spatial" module. The Lucene "spatial" module, currently in a weird gutted state, should have basically all spatial code currently in sandbox plus all geo stuff in Lucene core. Thus there will be no geo stuff in Lucene core. (B) Common spatial needs will be met by Lucene core. Lucene core should expand it's current "geo" utilities to include the spatial stuff currently in the sandbox module. It'd also take on what little remains in the Lucene spatial module and thus we can remove the spatial module. With either plan if a user has certain advanced/specialized needs they may need to go to spatial3d or spatial-extras modules. These would be untouched in both proposals. I'm in favor of (A) on the grounds that we have modules for special feature areas, and spatial should be no different. My gut estimation is that 75-90% of apps do not have spatial requirements and need not depend on any spatial module. Other modules are probably used more (e.g. queries, suggest, etc.) Respectfully, ~ David p.s. if I mischaracterized any proposal or overlooked another then I'm sorry, please correct me. -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com -- Nicholas Knize | Geospatial Software Guy | Elasticsearch & Apache Lucene | nkn...@apache.org<mailto:nkn...@apache.org> -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com -- Nicholas Knize | Geospatial Software Guy | Elasticsearch & Apache Lucene | nkn...@apache.org<mailto:nkn...@apache.org>
