Hello again! Uhm, so if I have understood then if I´m writing/reading from the index at the same time (in other words, indexing operations are executing at the same that that other ones are querying) the performance goes down, isn´t it?
Refering to the fieldType that I´m using, I´m just testing and I just want to implement the classic scenario where each document has one coordinate and to which execute geo spatial queries "given a circle with X radius". I was testing with a multiValued field because I was wondering to include one or more coordinates for each document due to the requirements of the project that I´m developing: I´m tracking users positions until a top of N positions (coordinates) to be stored in that multiValued field, but I can adapt into just one coordinate per user if that change gets a better performance. In other words, David, and to summary: - I just one to implement the classic scenario "coordinate + circle + radius X". - Due to NRT user position changes, I will write & read into the index continuosly. - I was testing with that fieldType multiValued, but I can change it into another fieldType and into a single value field type if that improves performance. In that case, what would you recommend? - If even changing fieldTypes, multiValues or not, etc. the performance doesn´t improves, what other alternatives would you recommend me to use? I´m a Solr lover from Solr 1.4 version, but I´m familiar to other Spatial Search alternatives technologies/tools/dbs like MongoDB. Any suggestions or recommendations? Best regards, - Luis Cappa 2013/3/6 David Smiley (@MITRE.org) <dsmi...@mitre.org> > Luis, > I should have asked how much data you have when I offered the solution. > > If you have a multi-valued spatial field and you need to get the closest > of potentially many indexed points (and your schema snippet below shows > multiValued=true) then I'm afraid you're stuck with this until the > underlying distance caching mechanism is improved. This is the biggest > limitation of this field type. See this issue for background on the > problem: > https://issues.apache.org/jira/browse/LUCENE-4698 > I suggest "watching" that issue to be notified of changes. I've got a > couple approaches to a solution on the horizon but it unfortunately hasn't > been a priority for my time. > > RE 3-4 seconds… the first time it needs to build the cache (slow) but then > speed-wise it shouldn't be too bad — it depends on how many documents you > actually matched in your query, not how many might be in the system. So > instead please tell me how many documents your search matched (aka > "numFound"). If you're doing simultaneous committing then this approach is > completely un-workable. > > ~ David > > From: "Rakudten [via Lucene]" <ml-node+s472066n4045273...@n3.nabble.com > <mailto:ml-node+s472066n4045273...@n3.nabble.com>> > Date: Wednesday, March 6, 2013 11:48 AM > To: "Smiley, David W." <dsmi...@mitre.org<mailto:dsmi...@mitre.org>> > Subject: Re: Solr 4.1: problems with Spatial Search. > > I´ve doing some performance tests and I´ve noticed that with the new query > syntax that David told me to use the QTime increases a lot. I´ve got an > index with up to 8 million docs and sometimes the query response delays > three, four or more seconds until finishes. Is that normal? > > 2013/3/6 Luis Cappa Banda <[hidden > email]</user/SendEmail.jtp?type=node&node=4045273&i=0>> > > > Hey David, it works! Thank you very much. The true is that the > > docummentation is a little bit confusing, but know It works perfectly. > > > > Regards, > > > > - Luis Cappa > > > > 2013/3/6 David Smiley (@MITRE.org) <[hidden > email]</user/SendEmail.jtp?type=node&node=4045273&i=1>> > > > > Ah; bingo! > >> > >> The top error in the log is what Solr reports in the HTTP response you > >> reported but it's the message of the exception wrapped by it in the logs > >> which is more indicative of the problem: > >> > >> Caused by: org.apache.solr.common.SolrException: A ValueSource isn't > >> directly available from this field. Instead try a query using the > distance > >> as the score. > >> > >> That error message (which I wrote) even contains the solution :-) > >> > >> You're using geodist() against solr.SpatialRecursivePrefixTreeFieldType > >> which isn't supported. You can get the distance but not using that > >> approach. Instead the query itself returns the distance as the score. > In > >> the example schema you'll see a link to documentation about this field > >> type > >> which is this URL: > >> http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 > >> From there click on "Sorting and Relevancy": > >> > >> > http://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4#Sorting_and_Relevancy > >> And you'll see an example query along the lines of what you want: > >> &fl=*,score&sort=score asc&q={! > >> score=distance}geo:"Intersects(Circle(54.729696,-98.525391 d=10))" > >> (the score is the distance in this case) > >> > >> ~ David > >> > >> > >> Rakudten wrote > >> > Hello everyone! > >> > > >> > - I´m using Solr 4.1.0. > >> > > >> > > >> > - Yes, without the sort the query works perfectly. > >> > > >> > > >> > - The fieldType is as follows: > >> > > >> > > >> > <field name="geolocation" type="location_rpt" indexed="true" > >> > stored="true" multiValued="true" /> > >> > * * > >> > > >> > <fieldType name="location_rpt" > >> > class="solr.SpatialRecursivePrefixTreeFieldType" > >> > > >> > > >> > > >> > spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory" > >> > distErrPct="0.025" > >> > maxDistErr="0.000009" > >> > units="degrees" > >> > /> > >> > - The error trace from Solr: > >> > > >> > Mar 6, 2013 10:02:29 AM org.apache.solr.common.SolrException log > >> > SEVERE: org.apache.solr.common.SolrException: sort param could not be > >> > parsed as a query, and is not a field that exists in the index: > >> geodist() > >> > at > org.apache.solr.search.QueryParsing.parseSort(QueryParsing.java:340) > >> > at org.apache.solr.search.QParser.getSort(QParser.java:281) > >> > at > >> > > >> > org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:123) > >> > at > >> > > >> > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) > >> > at > >> > > >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > >> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816) > >> > at > >> > > >> > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:448) > >> > at > >> > > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:269) > >> > at > >> > > >> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) > >> > at > >> > > >> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) > >> > at > >> > > >> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225) > >> > at > >> > > >> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) > >> > at > >> > > >> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168) > >> > at > >> > > >> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) > >> > at > >> > > >> > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927) > >> > at > >> > > >> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) > >> > at > >> > > >> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407) > >> > at > >> > > >> > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001) > >> > at > >> > > >> > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585) > >> > at > >> > > >> > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312) > >> > at > >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > >> > at > >> > > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > >> > at java.lang.Thread.run(Thread.java:662) > >> > Caused by: org.apache.solr.common.SolrException: A ValueSource isn't > >> > directly available from this field. Instead try a query using the > >> distance > >> > as the score. > >> > at > >> > > >> > org.apache.solr.schema.AbstractSpatialFieldType.getValueSource(AbstractSpatialFieldType.java:219) > >> > at > >> > > >> > org.apache.solr.search.function.distance.HaversineConstFunction.parseSfield(HaversineConstFunction.java:168) > >> > at > >> > > >> > org.apache.solr.search.function.distance.HaversineConstFunction.access$200(HaversineConstFunction.java:47) > >> > at > >> > > >> > org.apache.solr.search.function.distance.HaversineConstFunction$1.parse(HaversineConstFunction.java:104) > >> > at > >> > > >> > org.apache.solr.search.FunctionQParser.parseValueSource(FunctionQParser.java:352) > >> > at > org.apache.solr.search.FunctionQParser.parse(FunctionQParser.java:68) > >> > at org.apache.solr.search.QParser.getQuery(QParser.java:142) > >> > at > org.apache.solr.search.QueryParsing.parseSort(QueryParsing.java:271) > >> > ... 22 more > >> > > >> > > >> > I would like to sort by distance and also to return that distance as a > >> fl > >> > parameter inside the response. > >> > > >> > Thank you very much. > >> > > >> > Regards, > >> > > >> > > >> > - Luis Cappa > >> > > >> > > >> > 2013/3/5 Chris Hostetter < > >> > >> > hossman_lucene@ > >> > >> > > > >> > > >> >> > >> >> 1) which version of solr are you using? > >> >> 2) what is the field & fieldtype for "geolocation" > >> >> 2) can you try changing your query to "q={!func}geodist()" to verify > >> that > >> >> the function works at all? > >> >> > >> >> > >> >> > >> >> : Date: Tue, 5 Mar 2013 19:30:33 +0100 > >> >> : From: Luis Cappa Banda < > >> > >> > luiscappa@ > >> > >> > > > >> >> : Reply-To: > >> > >> > solr-user@.apache > >> > >> >> : To: > >> > >> > solr-user@.apache > >> > >> >> : Subject: Re: Solr 4.1: problems with Spatial Search. > >> >> : > >> >> : Without the sort it works perfectly, and there are no more error > >> >> messages, > >> >> : just the one I copy&pasted, :-( > >> >> : El 05/03/2013 19:05, "David Smiley (@MITRE.org)" < > >> > >> > DSMILEY@ > >> > >> > > > >> >> : escribió: > >> >> : > >> >> : > Hmm; weird. It looks right. Does it work without the sort? -- > >> i.e. > >> >> does > >> >> : > the > >> >> : > filter work? Are there more interesting looking error messages > >> >> output > >> >> by > >> >> : > Solr? > >> >> : > > >> >> : > > >> >> : > Rakudten wrote > >> >> : > > Hello! > >> >> : > > > >> >> : > > I´m trying to sort by geodist() distance, but it seems that I > >> >> can´t: > >> >> : > > > >> >> : > > *The query:* > >> >> : > > > >> >> : > > > >> >> : > > >> >> > >> > http://192.168.1.129:8080/geo/zones/select?q=*:*&fq={!geofilt}&sfield=geolocation&pt=38.96442,-3.89047&d=10&wt=json&start=0&rows=10&sort=geodist()%20asc > < > http://192.168.1.129:8080/geo/zones/select?q=*:*&fq=%7B!geofilt%7D&sfield=geolocation&pt=38.96442,-3.89047&d=10&wt=json&start=0&rows=10&sort=geodist()%20asc > > > >> >> : > > > >> >> : > > *The response:* > >> >> : > > * > >> >> : > > * > >> >> : > > > >> >> : > > { > >> >> : > > responseHeader: { > >> >> : > > status: 400, > >> >> : > > QTime: 2, > >> >> : > > params: { > >> >> : > > d: "10", > >> >> : > > sort: "geodist() asc", > >> >> : > > start: "0", > >> >> : > > q: "*:*", > >> >> : > > sfield: "geolocation", > >> >> : > > pt: "38.96442,-3.89047", > >> >> : > > wt: "json", > >> >> : > > fq: "{!geofilt}", > >> >> : > > rows: "10" > >> >> : > > } > >> >> : > > }, > >> >> : > > error: { > >> >> : > > msg: "sort param could not be parsed as a query, and is not a > >> field > >> >> that > >> >> : > > exists in the index: geodist()", > >> >> : > > code: 400 > >> >> : > > } > >> >> : > > } > >> >> : > > > >> >> : > > I´ve read several times the documentation and it seems that > this > >> is > >> >> the > >> >> : > > correct way to search filtering by coordinates and sorting by > >> >> distance, > >> >> : > > but > >> >> : > > I doesn´t work. Any suggestions? > >> >> : > > > >> >> : > > Also I´ve tried to return the distance as a field via fl > params, > >> >> but > >> >> I > >> >> : > > couldn´t. It may help me a lot if anyone could tell me how to > do > >> >> it. > >> >> : > > > >> >> : > > Thank you very much in advance! > >> >> : > > > >> >> : > > > >> >> : > > Best regards, > >> >> : > > > >> >> : > > > >> >> : > > - Luis Cappa > >> >> : > > >> >> : > > >> >> : > > >> >> : > > >> >> : > > >> >> : > ----- > >> >> : > Author: > >> >> : > > >> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > >> >> : > -- > >> >> : > View this message in context: > >> >> : > > >> >> > >> > http://lucene.472066.n3.nabble.com/Solr-4-1-problems-with-Spatial-Search-tp4044868p4044958.html > >> >> : > Sent from the Solr - User mailing list archive at Nabble.com. > >> >> : > > >> >> : > >> >> > >> >> -Hoss > >> > >> > >> > >> > >> > >> ----- > >> Author: > >> http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > >> -- > >> View this message in context: > >> > http://lucene.472066.n3.nabble.com/Solr-4-1-problems-with-Spatial-Search-tp4044868p4045231.html > >> Sent from the Solr - User mailing list archive at Nabble.com. > >> > > > > > > > ________________________________ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Solr-4-1-problems-with-Spatial-Search-tp4044868p4045273.html > To unsubscribe from Solr 4.1: problems with Spatial Search., click here< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4044868&code=RFNNSUxFWUBtaXRyZS5vcmd8NDA0NDg2OHwxMDE2NDI2OTUw > >. > NAML< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml > > > > > > > ----- > Author: > http://www.packtpub.com/apache-solr-3-enterprise-search-server/book > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-4-1-problems-with-Spatial-Search-tp4044868p4045295.html > Sent from the Solr - User mailing list archive at Nabble.com. >
