Re: Spatial search with arbitrary rectangle?

2017-08-29 Thread David Smiley
Hi, The "rectangular area" refers to a hypothetical map UI. In this scenario, the UI ought to communicate the lat-lon of each corner. The geofilt and bbox query parsers don't handle that; they only take a point and distance. RE projections: You may or may not need to care depending on exactly

Re: The unified highlighter html escaping. Seems rather extreme...

2017-07-20 Thread David Smiley
The escaping does appear excessive. Please file a bug to the Lucene project in Apache JIRA. On Fri, May 26, 2017 at 11:26 AM Michael Joyner wrote: > Isn't the unified html escaper a rather bit extreme in it's escaping? > > It makes it hard to deal with for simple

Re: Issue: Hit Highlighting Working Inconsistently in Solr 6.6

2017-07-14 Thread David Smiley
Does hl.method=unified help any? Perhaps you need to set hl.fl? or hl.requireFieldMatch=false? (although it should default to false already) On Fri, Jul 14, 2017 at 6:52 PM Vikram Oberoi wrote: > Hi! > > Just wanted to close the loop here. > > I'm pretty sure this has

Re: Not highlighting "and" and "or"?

2017-06-28 Thread David Smiley
Hi Walter, No they are not. Does debug=query show that these words are in your parsed query? On Wed, Jun 28, 2017 at 5:13 PM Walter Underwood wrote: > Is there some special casing in the highlighter to skip query syntax > words? The words “and” and “or” don’t get

Re: Polygon search query working but NOT Multipolygon

2017-06-28 Thread David Smiley
This polygon is fairly rectangular with one side having a ton of points. Nonetheless the query point is clearly far apart from it (it's much lower (smaller 'y' dimension). On Wed, Jun 28, 2017 at 10:17 PM puneeta wrote: > Hi David, > Actually my polygon had too

Re: Spatial Search based on the amount of docs, not the distance

2017-06-28 Thread David Smiley
Deniz didn't mention document-to-document distance sort but he/she didn't say it wasn't that case either. Any way, FYI at the Lucene level with LatLonPoint there is some sophisticated BKD search code to efficiently return the top N distance ordered documents (where you supply N). Although as

Re: Polygon search query working but NOT Multipolygon

2017-06-28 Thread David Smiley
https://lucene.apache.org/solr/guide/6_6/spatial-search.html#SpatialSearch-RptWithGeometrySpatialField > On Jun 28, 2017, at 11:32 AM, puneeta wrote: > > Hi

Re: Solr 5.5 - spatial intersects query returns results outside of search box

2017-06-28 Thread David Smiley
es and put some error checking in my > Solr indexing script to trap for this type of coordinate mismatch. > > -Original Message----- > From: David Smiley [mailto:david.w.smi...@gmail.com] > Sent: Wednesday, June 28, 2017 8:21 AM > To: solr-user@lucene.apache.org > Subjec

Re: Solr 5.5 - spatial intersects query returns results outside of search box

2017-06-28 Thread David Smiley
> On Jun 27, 2017, at 3:28 AM, Leila Gonzales wrote: > > { > >"id": "5230", > >"location_geo": > ["ENVELOPE(-75.0,-75.939723,39.3597224,38.289722)"] > > } This is an unusual rectangle. Remember this is minX, maxX, maxY, minY.

Re: Polygon search query working but NOT Multipolygon

2017-06-28 Thread David Smiley
I suggest using RptWithGeometry field, and with that change remove distErrPct and maxDistErr. See the ref guide, and note the geometry cache option. BTW spatialContextFactory can simply be "jts". If this fixes the issue, then the issue was related to grid approximation. BTW you never quite

Re: Polygon search query working but NOT Multipolygon

2017-06-28 Thread David Smiley
Hi Puneeta, So what does your field type definition look like? I'd imagine you're using RptWithGeometrySpatialField. And what is your Solr version? BTW note the settings here https://locationtech.github.io/spatial4j/apidocs/org/locationtech/spatial4j/context/jts/JtsSpatialContextFactory.html

Re: Issue with highlighter

2017-06-14 Thread David Smiley
> Beware of NOT plus OR in a search. That will certainly produce no highlights. (eg test -results when default op is OR) Seems like a bug to me; the default operator shouldn't matter in that case I think since there is only one clause that has no BooleanQuery.Occur operator and thus the OR/AND

Re: Highlighter not working on some documents

2017-06-11 Thread David Smiley
Probably the most common reason is the default hl.maxAnalyzedChars -- thus your highlightable text might not be in the first 51200 chars of text. The first Solr release with the unified highlighter had an even lower default of 10k chars. On Fri, Jun 9, 2017 at 9:58 PM Phil Scadden

Re: why MULTILINESTRING can contains polygon in solr spatial search

2017-06-02 Thread David Smiley
Hi, Solr 4.7 is old but is probably okay. Is it easy to try a 6.x version? (note Spatial4j java package names have changed). There's also multiple new pertinent options to your scenario:

Re: Spatial Search: can not use FieldCache on a field which is neither indexed nor has doc values: latitudeLongitude_0_coordinate

2017-04-30 Thread David Smiley
Frederick, RE LatLonType: Weird. Is the dynamic field "_coordinate" defined? It should be ensure it has indexed=true on it. I forget if indexed needs to be set on that or on the LLT field that refers to it but to be sure set on both. RE LatLonPointSpatialField: You should use this for sure

Re: DateRangeField and Faceting

2017-04-26 Thread David Smiley
Hi Stephen, I agree that it would be nice if the JSON faceting module worked with DateRangeField. Sadly Solr has several faceting engines (classic, JSON Facets, analytics contrib) and there has yet been any effort to coral them. My sense is that JSON Faceting is where effort should go, and as

Re: Error with polygon search

2017-03-21 Thread David Smiley
Hello Hank, The online version of the reference guide is always for the latest Solr release. I think your configuration would work in the latest release. Prior to Solr 6, the Spatial4J library had a different Java package location: replace "org.locationtech.spatial4j" with

Re: Boolean expression for spatial query

2017-03-02 Thread David Smiley
I recommend the MULTIPOINT approach. BTW if you go the route of multiple OR'ed sub-clauses, I recommend avoiding the _query_ syntax which predates Solr 4.x's (4.2?) ability to embed fully the sub-clauses more naturally; though you need to beware of the gotcha of needing to add a leading space.

Re: Solr 6.4 new SynonymGraphFilter help for multi-word synonyms

2017-02-03 Thread David Smiley
Solr _does_ have a query parser that doesn't suffer from this problem -- SimpleQParser chosen as the string "simple". https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-SimpleQueryParser In this case, see the "WHITESPACE" operator feature which can be toggled. Configure to

Re: How to identify documents failed in a batch request?

2016-12-17 Thread David Smiley
If you enable the "TolerantUpdateProcessor" Solr-side, you can add documents in bulk allowing some to fail and know which did: http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/update/processor/TolerantUpdateProcessorFactory.html On Sat, Dec 17, 2016 at 5:05 PM S G

Re: Search opening hours

2016-11-28 Thread David Smiley
Lets say you wanted to do ranges over some integer. Simply convert those integers to dates, such as java.time.Instant.ofEpochSecond(myInteger).toString(). It's more efficient to convert to seconds (as in this example) as a base instead milliseconds because the internal date oriented tree has

Re: Search opening hours

2016-11-24 Thread David Smiley
I just saw this conversation now. I didn't read every word but I have to ask immediately: does DateRangeField address your needs? https://cwiki.apache.org/confluence/display/solr/Working+with+Dates It was introduced in 5.0. On Wed, Nov 16, 2016 at 4:59 AM O. Klein wrote: >

Re: How-To: Secure Solr by IP Address

2016-11-04 Thread David Smiley
Not to knock the other suggestions, but a benefit to securing Jetty like this is that *everyone* can do this approach. On Fri, Nov 4, 2016 at 9:54 AM john saylor wrote: > hi > > any firewall worth it's name should be able to do this. in fact, that is > one of several

How-To: Secure Solr by IP Address

2016-11-04 Thread David Smiley
I was just researching how to secure Solr by IP address and I finally figured it out. Perhaps this might go in the ref guide but I'd like to share it here anyhow. The scenario is where only "localhost" should have full unfettered access to Solr, whereas everyone else (notably web clients) can

Re: Heatmap in JSON facet API

2016-11-01 Thread David Smiley
I plan on adding this in the near future... hopefully for Solr 6.4. On Mon, Oct 31, 2016 at 7:06 AM Никита Веневитин wrote: > I've built query as described in https://cwiki.apache.org/confluence/x/ZYDxAQ;>Heatmap Faceting, > but I would like to get same results using

Re: Migrating to Solr 6.1.0 from 5.5.0

2016-09-29 Thread David Smiley
Arjun, Your input is a POLYGON -- as seen in the error message. The "Try JTS" was hopefully a clue -- on https://cwiki.apache.org/confluence/display/solr/Spatial+Search search for "JTS" and you should see how to set the spatialContextFactory to JTS, and a mention of needing JTS jar. I'll try

Re: Negative Date Query for Local Params in Solr

2016-09-20 Thread David Smiley
cwiki.apache.org/confluence/display/solr/Working+with+Dates "More DateRangeField Details" mentions "op". {!lucene df=dateRange op=Contains}... would also work. I don't know of any other local-param used in this way. On Tue, Sep 20, 2016 at 11:21 PM David Smiley <david.w.s

Re: Negative Date Query for Local Params in Solr

2016-09-20 Thread David Smiley
cate some references to > catch up. > Thanks a lot ... SRK > > Show original message On Tuesday, September 20, 2016 5:54 PM, David > Smiley <david.w.smi...@gmail.com> wrote: > > > OH! Ok the moment the query no longer starts with "{!", the quer

Re: Negative Date Query for Local Params in Solr

2016-09-20 Thread David Smiley
ot;, "wt": "json", "_": > "1474373612202" } }, "error": { "msg": "Invalid Date in Date Math > String:'[2016-08-26T12:00:12Z'", "code": 400 }} > SRK > > On Tuesday, September 20, 2016 5:34 PM, Davi

Re: Negative Date Query for Local Params in Solr

2016-09-20 Thread David Smiley
It should, I think... what happens? Can you ascertain the nature of the results? ~ David On Tue, Sep 20, 2016 at 5:35 AM Sandeep Khanzode wrote: > For Solr 6.1.0 > This works .. -{!field f=schedule op=Intersects}2016-08-26T12:00:56Z > > This works .. {!field

Re: request SOLR - spatial field with Intersect and Contains functions

2016-09-19 Thread David Smiley
Hi Leo, You should use two spatial fields for this -- one is for an indexed Box/Envelope, and another for an indexed LineString. The indexed box should use either BBoxField or RptWithGeometrySpatialField, and the LineString field should use RptWithGeometrySpatialField. If you have an older

Re: Sorting on DateRangeField?

2016-09-09 Thread David Smiley
Hi Alex, DateRangeField extends some spatial stuff, which has that error message in it, not in DateRangeField proper. You cannot sort on a DateRangeField. If you want to... try adding either one plain docValues field if you just have date instances, or a pair of them to hold a min & max and

Re: Need Help Resolving Unknown Shape Definition Error

2016-08-15 Thread David Smiley
Hello Jennifer, The spatial documentation is largely this page: https://cwiki.apache.org/confluence/display/solr/Spatial+Search (however note the online version is always for the latest Solr release. You can download a PDF versioned against your Solr version). To do polygon searches, you both

Re: error indexing spatial

2016-07-25 Thread David Smiley
Hi tig. Most likely, you didn't repeat the first point as the last. Even though it's redundant, nonetheless this is what WKT (and some other spatial formats) calls for. ~ David On Wed, Jul 20, 2016 at 10:13 PM tkg_cangkul wrote: > hi i try to indexing spatial format to

Re: error rendering solr spatial in geoserver

2016-07-01 Thread David Smiley
g sure that no Solr 5 node reloads the configuration. > > --Ere > > 30.6.2016, 3.46, David Smiley kirjoitti: > > For polygons in 6.0 you need to set > > > spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" > > -- see >

Re: error rendering solr spatial in geoserver

2016-06-29 Thread David Smiley
For polygons in 6.0 you need to set spatialContextFactory="org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" -- see https://cwiki.apache.org/confluence/display/solr/Spatial+Search and the example. And of course as you probably already know, put the JTS jar on Solr's classpath.

Re: Issues with coordinates in Solr during updating of fields

2016-06-13 Thread David Smiley
Zheng, There are a few Solr FieldTypes that are basically composite fields -- a virtual field of other fields. AFAIK they are all spatial related. You don't necessarily need to pay attention to the fact that gps_1_coordinate exists under the hood unless you wish to customize the options on that

Re: Facet heatmaps: cluster coordinates based on average position of docs

2016-05-18 Thread David Smiley
can > use stat component to add average positions of documents in cell? > I think I've seen hand-rolled heatmap capabilities added to Solr (i.e. no custom Solr hacking) that went about it kinda like that. stats.facet on some geohash (or similar), then average lat & average lon. ~ David

Re: Highlighting phone numbers

2016-05-18 Thread David Smiley
Perhaps an easy thing to try is see of the FastVectorHighlighter yields any different results. There are some nuances to the highlighters -- it might. Failing that, this likely due to your analysis chain, and where exactly the offsets point to, which you can see/debug in Solr's analysis screen.

Re: relaxed vs. improved validation in solr.TrieDateField

2016-05-06 Thread David Smiley
Sorry to hear that Uwe Reh. If this is just in your input/index data, then this could be handled with an URP, maybe evan an existing URP. See ParseDateFieldUpdateProcessorFactory which uses the Joda-time API. I am not sure if that will work, I'm a little doubtful in fact since Solr now uses the

Re: Solr - index polygons from csv

2016-04-28 Thread David Smiley
Hi. To use polygons, you need to add JTS, otherwise you get an unsupported shape error. See https://cwiki.apache.org/confluence/display/solr/Apache+Solr+Reference+Guide it involves not only adding a JTS lib to your classpath (ideal spot is WEB-INF/lib ) but also adding a spatialContextFactory

Re: issues doing a spatial query

2016-04-28 Thread David Smiley
Hi. This makes sense to me. The point 49.8,-97.1 is in your query box. The box is lower-left to upper-right, so your box is actually an almost world-wrapping one grabbing all longitudes except -93 to -92. Maybe you mean to switch your left & right. On Sun, Apr 24, 2016 at 8:03 PM GW

Re: Facet heatmaps: cluster coordinates based on average position of docs

2016-04-19 Thread David Smiley
Hi Anton, Perhaps you should request a more detailed / high-res heatmap, and then work with that, perhaps using some clustering technique? I confess I don't work on the UI end of things these days. p.s. I'm on vacation this week; so I don't respond quickly ~ David On Thu, Apr 7, 2016 at 3:43

Re: Seasonal searches in SOLR 5.x

2016-03-22 Thread David Smiley
Hi, I suggest having a "season" field (or whatever you might want to call it) using DateRangeField but simply use a nominal year value. So basically all durations would be within this nominal year. For some docs that span new-years, this might mean 2 durations and that's okay. Also it's okay

Re: Regarding google maps polyline to use IsWithin(POLYGON(())) in solr

2016-03-19 Thread David Smiley
JTS doesn't has any vertex limit on the geometries. So I don't know why your query isn't working. On Wed, Mar 16, 2016 at 1:58 AM Pradeep Chandra < pradeepchandra@gmail.com> wrote: > Hi Sir, > > Let me give some clarification on IsWithin(POLYGON(())) query...It is not > giving any result

Re: Regarding google maps polyline to use IsWithin(POLYGON(())) in solr

2016-03-15 Thread David Smiley
Hi Pradeep, Are you seeing an error when it doesn't work? I believe a shape overlapping itself will cause an error from JTS. If you do see that, then you can ask Spatial4j (used by Lucene/Solr) to attempt to deal with it in a number of ways. See "validationRule":

Re: Price Range Faceting Based on Date Constraints

2015-05-21 Thread David Smiley
Another more modern option, very related to this, is to use DateRangeField in 5.0. You have full 64 bit precision. More info is in the Solr Ref Guide. If Alessandro sticks with RPT, then the best reference to give is this: http://wiki.apache.org/solr/SpatialForTimeDurations ~ David

Re: Solr join + Boost in single query

2015-03-03 Thread David Smiley
No, not without writing something custom anyway. It'd be difficult to make it fast if there's a lot of documents to join on. sraav wrote David, Is it possible to write a query to join two cores and either bring back data from the two cores or to boost on the data coming back from either of

Re: Boosting by calculated distance buckets

2015-02-17 Thread David Smiley
Raav, You may need to actually subscribe to the solr-user list. Nabble seems to not be working to well. p.s. I’m on vacation this week so I can’t be very responsive First of all... it's not clear you actually want to *boost* (since you seem to not care about the relevancy score), it seems you

Re: Boosting by calculated distance buckets

2015-02-14 Thread David Smiley
: boost=recip(geodist(),1,20,20) ~ David Smiley Freelance Apache Lucene/Solr Search Consultant/Developer http://www.linkedin.com/in/davidwsmiley sraav wrote I hit a block when I ran into a use case where I had to boost on ranges of distances calculated at query time. This is the case when

Re: Spatial maxDistErr changes

2014-04-02 Thread David Smiley
Good question Steve, You'll have to re-index right off. ~ David p.s. Sorry I didn't reply sooner; I just switched jobs and reconfigured my mailing list subscriptions Steven Bower wrote If am only indexing point shapes and I want to change the maxDistErr from 0.09 (1m res) to 0.00045

Re: Issue with spatial search

2014-03-11 Thread David Smiley (@MITRE.org)
the error percentage? or maybe better put how does it impact perf? steve On Mon, Mar 10, 2014 at 11:17 PM, David Smiley (@MITRE.org) [hidden email]/user/SendEmail.jtp?type=nodenode=4122855i=0 wrote: Correct, Steve. Alternatively you can also put this option in your query after the end

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
You're going to have to use the Lucene-spatial module directly then. There's SpatialExample.java to get you started. javinsnc wrote David Smiley (@MITRE.org) wrote On 3/10/14, 12:56 PM, javinsnc lt; javiersangrador@ gt; wrote: This is indeed the source of the problem. Why do you

Re: Solr spatial search within the polygon

2014-03-10 Thread David Smiley (@MITRE.org)
Lucene has multiple modules, one of which is spatial. You'll see it in the source tree checkout underneath the lucene directory. Javadocs: http://lucene.apache.org/core/4_7_0/spatial/index.html SpatialExample.java:

Re: Issue with spatial search

2014-03-10 Thread David Smiley (@MITRE.org)
Correct, Steve. Alternatively you can also put this option in your query after the end of the last parenthesis, as in this example from the wiki: fq=geo:IsWithin(POLYGON((-10 30, -40 40, -10 -20, 40 20, 0 0, -10 30))) distErrPct=0 ~ David Steven Bower wrote Only points in the index.. Am I

Re: Adding (spatial) filter query slows down avg response time

2014-03-01 Thread David Smiley (@MITRE.org)
Hi, The reason why it's working faster for you when you put {!geofilt} into your main query is most likely because the particular field type you are using is LatLonType, which works on a per-document basis, and by putting it in the main query it will, by default, leap-frog across the documents

Re: Solr is NoSQL database or not?

2014-03-01 Thread David Smiley (@MITRE.org)
+1 Excellent responses, Jack. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-NoSQL-database-or-not-tp4120554p4120682.html Sent from the Solr - User mailing list archive at

Re: Join Scoring

2014-02-11 Thread David Smiley (@MITRE.org)
Hi Anand. Solr's JOIN query, {!join}, constant-scores. It's simpler and faster and more memory efficient (particularly the worse-case memory use) to implement the JOIN query without scoring, so that's why. Of course, you might want it to score and pay whatever penalty is involved. For that

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Mark, I appreciate all the context. ... Mark Miller-3 wrote What do you mean you are hosed? All the SolrCores should be gone, so why does it matter so much that the collection is gone? In the days before the collections api, if you wanted to then create the collection again, just create a

Re: Removing last replica from a SolrCloud collection

2014-02-02 Thread David Smiley (@MITRE.org)
Thanks; that is *exactly* the behavior I'm talking about. I tried to find an existing issue before posting but missed this one somehow. Ramkumar R. Aiyengar wrote There's already an issue for this, https://issues.apache.org/jira/browse/SOLR-5209, we were once bitten by the same issue, when

Re: Clone (or Restore) Solrcloud

2014-02-02 Thread David Smiley (@MITRE.org)
(since it's not needed anymore; it becomes inactive). I'm not sure why this metadata is recorded because, at least after the split, I can't see why it's pertinent to anything. ~ David David Smiley (@MITRE.org) wrote Hi, I'm attempting to come up with a SolrCloud restore / clone process

Removing last replica from a SolrCloud collection

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, If I issue either a core UNLOAD command, or a collection DELETEREPLICA command, (which both seem pretty much equivalent) it works but if there are no other replicas for the shard, then the metadata for the shard is completely gone in clusterstate.json! That's pretty disconcerting because

Clone (or Restore) Solrcloud

2014-01-31 Thread David Smiley (@MITRE.org)
Hi, I'm attempting to come up with a SolrCloud restore / clone process for either recover to a known good state or to clone the environment for experimentation. At the moment my process involves either creating a new zookeeper environment or at least deleting the existing Collection so that I

Re: Index JTS Point in Solr/Lucene index

2013-10-28 Thread David Smiley (@MITRE.org)
Just follow-ing up with this thread after a round of emails between Shahbaz and I… David Smiley wrote Ooooh, I see your confusion. You looked at code in an UpdateRequestProcessor and expected it to work on the client in SolrJ. It won't work for the reason that the code in the URP

Re: Solr 4.5 spatial search - distance and score

2013-09-13 Thread David Smiley (@MITRE.org)
Hi Weber, Returning the distance separately from the score is really awkward without being able to use geodist() (which is coming in Solr 4.5 for the RPT spatial field). But as you note in SOLR-4255 it is possible. If you modify the Solr example schema so that the 'store' spatial field is of

Re: Change the score of a document based on the *value* of a multifield using dismax

2013-09-03 Thread David Smiley (@MITRE.org)
If you want to alter the score in a customized way based on indexed text data on a per-value basis then index Lucene payloads, and use PayloadTermQuery. See the javadocs for PayloadTermQuery in particular and follow the references. This is a bit dated but read this:

Re: Geo spatial clustering of points

2013-08-23 Thread David Smiley (@MITRE.org)
Hi Chris Jeroen, Tonight I posted some tips on Solr's wiki on this subject: http://wiki.apache.org/solr/SpatialClustering ~ David Chris Atkinson wrote Did you get any resolution for this? I'm about to implement something identical. On 3 Jul 2013 23:03, Jeroen Steggink lt; jeroen@ gt;

Re: Distance sort on a multi-value field

2013-08-22 Thread David Smiley (@MITRE.org)
, David Smiley (@MITRE.org) lt; DSMILEY@ gt; wrote: The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because

Re: How to access latitude and longitude with only LatLonType?

2013-08-22 Thread David Smiley (@MITRE.org)
Hi Quan You claim to be using LatLonType, yet the error you posted makes it clear you are in fact using SpatialRecursivePrefixTreeFieldType (RPT). Regardless of which spatial field you use, it's not clear to me what sort of statistics could be useful on a spatial field. The stats component

Re: spatial search, geofilt does not work

2013-08-20 Thread David Smiley (@MITRE.org)
could use some error msg if the query format is invalid. But, THANKS! David, you probably saved me another half day. Ming- On Mon, Aug 19, 2013 at 10:20 PM, David Smiley (@MITRE.org) DSMILEY@ wrote: Thank goodness for Solr's feature of echo'ing params back in the response

Re: Distance sort on a multi-value field

2013-08-20 Thread David Smiley (@MITRE.org)
The distance sorting code in SOLR-2155 is roughly equivalent to the code that RPT uses (RPT has its lineage in SOLR-2155 after all). I just reviewed it to double-check. It's possible the behavior is slightly better in SOLR-2155 because the cache (a Solr cache) contains normal hard-references

Re: SOLR4 Spatial sorting and query string

2013-08-19 Thread David Smiley (@MITRE.org)
This is a known limitation. From CHANGES.txt: * SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the field is referenced via 'sfield' and the query point is constant. (David Smiley) The reason why that limitation is there relates to the fact that the function query

Re: spatial search, geofilt does not work

2013-08-19 Thread David Smiley (@MITRE.org)
Thank goodness for Solr's feature of echo'ing params back in the response as it helps diagnose problems like this. In your case, the filter query that Solr is seeing isn't what you (seemed) to have given on the command line: fq:!geofilt sfield=author_geo Clearly wrong. Try escaping the braces

Re: Use case of Spatial search

2013-08-19 Thread David Smiley (@MITRE.org)
Shishir, Use the location_rpt type and index circles of the business and the distance they serve with this syntax: field name=myfieldNameCircle(lat,lon d=degreesRadius)/field Your query shape is then simply a point; use bbox query parser with d=0. This approach should scale *great* at query

Re: SOLR4 Spatial sorting and query string

2013-08-15 Thread David Smiley (@MITRE.org)
Hi Roy, You'll have to calculate this client-side. I am aware of this conundrum and I put up a TODO JIRA item for it here months ago: https://issues.apache.org/jira/browse/SOLR-4633It actually shouldn't be that hard to do. ~ David roySolr wrote Hello David, The first months there

Re: SOLR4 Spatial sorting and query string

2013-08-13 Thread David Smiley (@MITRE.org)
Hi Roy. Using the example schema and data, and copying the store field to store_rpt indexed with location_rpt field type, try this query:

Re: Multipoint date ranges with spatial - Invalid Longitude Exception?

2013-08-12 Thread David Smiley (@MITRE.org)
Hi Dan, FYI the main reference page on this technique is here: http://wiki.apache.org/solr/SpatialForTimeDurations (note the slight buffering needed for the query shape). You got the exception you got because you separated the dimensions of your query rectangle using a comma, which got

Re: Performance question on Spatial Search

2013-08-05 Thread David Smiley (@MITRE.org)
From: Steven Bower-2 [via Lucene] ml-node+s472066n4082569...@n3.nabble.commailto:ml-node+s472066n4082569...@n3.nabble.com Date: Monday, August 5, 2013 9:14 AM To: Smiley, David W. dsmi...@mitre.orgmailto:dsmi...@mitre.org Subject: Re: Performance question on Spatial Search So after re-feeding

Re: Ingesting geo data into Solr very slow

2013-07-30 Thread David Smiley (@MITRE.org)
Hi Marta, Presumably you are indexing polygons -- I suspect complex ones. There isn't too much that you can do about this right now other than index them in parallel. I see you are doing this in 2 threads; try 4, or maybe even 6. Also, ensure that maxDistErr is reflective of the smallest

Re: custom field type plugin

2013-07-23 Thread David Smiley (@MITRE.org)
, David Smiley (@MITRE.org) lt; DSMILEY@ gt; wrote: Like Hoss said, you're going to have to solve this using http://wiki.apache.org/solr/SpatialForTimeDurations Using PointType is *not* going to work because your durations are multi-valued per document. It would be useful to create a custom field

Re: custom field type plugin

2013-07-22 Thread David Smiley (@MITRE.org)
. I don't know where the boundary is, but I doubt 10B. You could try and see what happens. I'm working (very slowly on very little spare time) on improving the PrefixTree implementations to scale to such large numbers; I hope something will be available this fall. ~ David Smiley Kevin Stone

Re: SpatialRecursivePrefixTreeFieldType Spatial Searching

2013-06-04 Thread David Smiley (@MITRE.org)
maxDistErr should be like 0.3 based on earlier parts of this discussion since your data is to one of a couple hours of the day, not whole days. If it was whole days, you would use 1. Changing this requires a re-index. So does changing worldBounds if you do so. distErrPct should be 0. Changing

Re: SOLR 4.3.0 - How to make fq optional?

2013-05-28 Thread David Smiley (@MITRE.org)
Your client needs to know to submit the proper filter query conditionally. It's not really a spatial issue, and I disagree with the idea to make bbox (and all other query parsers for that matter) do nothing if not given an expected input. ~ David bbarani wrote I am using the SOLR geospatial

Re: exact match country

2013-05-26 Thread David Smiley (@MITRE.org)
Hi Bill. So it seems you want an exact match to be first even if it is outside the spatial region, right? Your suggested implementation suggests this. And apparently you want to sort by distance, notwithstanding the exact match being first. Although you don't have to do this as two queries, I

Re: multivalue location_rpt field not indexing with JSON format

2013-05-24 Thread David Smiley (@MITRE.org)
Hi Barbra, Solr needs to see a String for each point value, not a 2-element array. Your doc should look like: [{id:054ac6377d6ca4ad387f73b063000910,keywords:[time, trouble, exactly],description:a anno is an anno is an anno,

Re: Restaurant availability from database

2013-05-24 Thread David Smiley (@MITRE.org)
Use this reference: http://wiki.apache.org/solr/SpatialForTimeDurations Alexandre Rafalovitch wrote On Thu, May 23, 2013 at 6:47 PM, Amit Nithian lt; anithian@ gt; wrote: Hossman did a presentation on something similar to this using spatial data at a Solr meetup some months ago.

Re: How to query docs with an indexed polygon field in java?

2013-05-23 Thread David Smiley (@MITRE.org)
Hi Kevenz, kevenz wrote ... String sql = indexType:219 AND geo:Contains(POINT(114.078327401257,22.5424866754136)); ... Then I got an error at java.lang.IllegalArgumentException: missing parens: Contains. Is there any suggestion? First of all, if your query shape is a point, then use

Re: How to improve performance of geodist()

2013-05-13 Thread David Smiley (@MITRE.org)
Hi Nicholas, Given that boosting is generally inherently fuzzy / inexact thing, you can likely get away with using simpler calculations. dist() can do the Euclidean distance (i.e. the Pythagorean theorem). If your data is in just one region of the world, you can project your data into a 2-D

Re: Multi dimensional spatial search

2013-05-11 Thread David Smiley (@MITRE.org)
Hi Kiran. The often-forgotten PointType field type can be configured to hold a variable number of dimensions. See the dimension attribute of the field type's configuration in the example schema. This field type is really just a kind of a macro field type for a configurable number of numeric

Re: Easier way to do this?

2013-04-13 Thread David Smiley (@MITRE.org)
Good question. With geofilt it's kilometers. - Author: http://www.packtpub.com/apache-solr-3-enterprise-search-server/book -- View this message in context: http://lucene.472066.n3.nabble.com/Easier-way-to-do-this-tp4055474p4055784.html Sent from the Solr - User mailing list archive at

Re: Easier way to do this?

2013-04-12 Thread David Smiley (@MITRE.org)
Bill, I responded to the issue you created about this: https://issues.apache.org/jira/browse/SOLR-4704 In summary, use {!geofilt}. ~ David Billnbell wrote I would love for the SOLR spatial 4 to support pt so that I can run # of results around a central point easily like in 3.6. How can I

Re: Support old syntax including geodist

2013-04-12 Thread David Smiley (@MITRE.org)
Hi Bill, FYI see https://issues.apache.org/jira/browse/SOLR-4242 Billnbell wrote Since Spatial Lucene 4 does not seem to support geodist(), even sending d,pt,fq={!geofilt}does not help me = I need to sort. So I end up having to set up the sortsq. Any other ideas on how to support the old

Re: Spatial search question

2013-04-12 Thread David Smiley (@MITRE.org)
Yup, Lance is right. But it won't always work if you have multi-valued data since it wouldn't match a document that had a point both in the ring and the hole. Another approach that internally works faster and addresses the multi-value case is to implement a custom Spatial4j Shape. In this case,

Re: DocValues vs stored fields?

2013-04-01 Thread David Smiley (@MITRE.org)
Otis, DocValues are quite insufficient for true field updates. DocValues is a per-document value storage (hence the name); it's not uninverted/indexed. If you needed to search based on these values (e.g. find all docs that have this value or between these values) then that's not going to work.

Re: Solr 4.1: problems with Spatial Search.

2013-03-09 Thread David Smiley (@MITRE.org)
Rakudten wrote Hello again! Uhm, so if I have understood then if I´m writing/reading from the index at the same time (in other words, indexing operations are executing at the same that that other ones are querying) the performance goes down, isn´t it? Committing is the problem (soft or

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
benchmarks -- especially v4.2 due out soon, but earlier 4.x should be nearly the same. It's weird that you're seeing the stored value coming back in search results as a geohash. In Solr 4 you get precisely what you added. ~ David Harley wrote Hi David Smiley: We use a 3rd party software to load

RE: Migrate Solr 3.4 w/ solr-1255 GeoHash to Solr 4

2013-03-08 Thread David Smiley (@MITRE.org)
going to fix that. ~ David Harley wrote David Smiley: Because we use a 3rd party software.. I checked to see if this would still worked... search query still works. But adding data seems to be broken, likely because of the geohash type. So, below is the log file, which tells me to upgrade

Re: InvalidShapeException when using SpatialRecursivePrefixTreeFieldType with custom worldBounds

2013-03-08 Thread David Smiley (@MITRE.org)
Hi Jon. If you're able to trigger an IndexOutOfBoundsException out of the prefix tree then please file a bug (to the Lucene project, not Solr). I'll look into it when I have time. I need to add a Wiki page on the use of spatial for time ranges; there are some tricks to it. Nevertheless you've

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
. : : Without the sort it works perfectly, and there are no more error messages, : just the one I copypasted, :-( : El 05/03/2013 19:05, David Smiley (@MITRE.org) lt; DSMILEY@ gt; : escribió: : : Hmm; weird. It looks right. Does it work without the sort? -- i.e. does : the : filter

Re: Solr 4.1: problems with Spatial Search.

2013-03-06 Thread David Smiley (@MITRE.org)
is a little bit confusing, but know It works perfectly. Regards, - Luis Cappa 2013/3/6 David Smiley (@MITRE.org) [hidden email]/user/SendEmail.jtp?type=nodenode=4045273i=1 Ah; bingo! The top error in the log is what Solr reports in the HTTP response you reported but it's the message

<    1   2   3   4   >