[
https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-2155:
-------------------------------
Attachment: GeoHashPrefixFilter.patch
Attached is my latest patch for geohash prefix based geospatial search. This
patch is a performance-centric update. GeoHashes are still used, but I'm
indexing a token for every geohash length per point indexed. So a geohash
length of 9 results in 9 tokens. This solves performance issues when a huge
number of points were matching a query. See the below table:
||km||places/query||ms/query (LatLon)||ms/query (geohash)||
|11| 692| 3.8| 5.0|
|44| 4,043| 4.8|6.0|
|230| 57,200| 15.0|17.5|
|1800| 1,405,767| 94.0|71.0|
The LatLon is using a pair of trie doubles at a precisionStep of 8. I tried 6
& 16 but 8 was about right. The GeoHash length (a new configurable option) was
chosen to be 9 which has plenty of precision for most uses (I recall it's a
couple meters or less; I forget). The queries are bounding lat-lon boxes.
What isn't in this performance table is the impact of this new algorithm on
more complicated spatial queries. It's superior to the algorithm that existed
before it and it should also be superior to LatLonType. Grid boxes that are
completely within the query shape get efficiently added in one fell swoop.
Code details:
* Most of the former patch, which included a lot of additions to GeoHashUtils
is no longer present in this new patch. This was basically a rewrite.
* I abstracted use of GeoHashUtils to GridNode.GridReferenceSystem class so
that in the future I can tinker with alternate more efficient encodings without
breaking any code here.
* I needed a shape interface or abstract class and so I decided to embrace &
extend org.apache.lucene.spatial.geometry.shape.Geometry2D instead of having my
own like I did before. I added PointDistanceGeom & MultiGeom.
* There is an extensive random data filter test in SpatialFilterTest that I
added. It's hard to follow but it teased out a few bugs.
Next patch real soon:
* I'm going to modify the build.xml to grab the LGPL licensed JTS library which
has well-tested & high performance geometry code. In particular, I'll use it
to implement a polygon shape. (Already done in another codebase; just needs to
be ported to this patch)
* I'm going to include an alternative query parser to what comes with Solr.
This one will do all of point-distance, lat-lon box, and polygon. (Already done
in another codebase; just needs to be ported to this patch).
Future:
* Replace geohash with something more efficient. Some basic testing suggests
to me I could double-or-better the performance.
* Compatibility with distance sorting / relevancy boosting when not
multi-valued.
I'd really like input from other geospatial birds-of-a-feather in Solr,
especially committers.
As an aside, MongoDB has chosen a similar algorithm.
> Geospatial search using geohash prefixes
> ----------------------------------------
>
> Key: SOLR-2155
> URL: https://issues.apache.org/jira/browse/SOLR-2155
> Project: Solr
> Issue Type: Improvement
> Reporter: David Smiley
> Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch
>
>
> There currently isn't a solution in Solr for doing geospatial filtering on
> documents that have a variable number of points. This scenario occurs when
> there is location extraction (i.e. via a "gazateer") occurring on free text.
> None, one, or many geospatial locations might be extracted from any given
> document and users want to limit their search results to those occurring in a
> user-specified area.
> I've implemented this by furthering the GeoHash based work in Lucene/Solr
> with a geohash prefix based filter. A geohash refers to a lat-lon box on the
> earth. Each successive character added further subdivides the box into a 4x8
> (or 8x4 depending on the even/odd length of the geohash) grid. The first
> step in this scheme is figuring out which geohash grid squares cover the
> user's search query. I've added various extra methods to GeoHashUtils (and
> added tests) to assist in this purpose. The next step is an actual Lucene
> Filter, GeoHashPrefixFilter, that uses these geohash prefixes in
> TermsEnum.seek() to skip to relevant grid squares in the index. Once a
> matching geohash grid is found, the points therein are compared against the
> user's query to see if it matches. I created an abstraction GeoShape
> extended by subclasses named PointDistance... and CartesianBox.... to support
> different queried shapes so that the filter need not care about these details.
> This work was presented at LuceneRevolution in Boston on October 8th.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
