Re: Geocoding with Solr
Spadez, I've had some success into using the nicely open data of GeoNames.org but that was modestly only to fetch zip-code-to-town-name and long-lat associations, search for postal-codes. It would be lovely that some more best practice examples are distributed on how to handle, e.g., the lon-lat uncertainty. I still do not know how I could take usefully advantage of a user-allowed-browser-geolocation which gives me lon/lat. I've seen article on how to do this with a YUI query thus far. paul Le 29 juil. 2012 à 00:54, Spadez a écrit : I am using Solr as my main search system for my site. Currently, I am using google to turn a place name (such as a postcode or city) into a long / lat co-ordinate. Then I am supplying this long / lat to Solr so it can perform a spacial search. I am really new to this, but I dont like my reliance on google for this. Is it possible to import a database into Solr which linked both cities and postcodes to a co-ordinate, and use Solr for this as well?
Re: Request header is too large.
Thank you, Alexandre. I solved this problem. It's not a database problem but a server problem as the error message clearly stated. I resolved this by adding more tcp transport buffer size in application server. Xuefeng From: Alexandre Rafalovitch arafa...@gmail.com To: Xue-Feng Yang just4l...@yahoo.com; solr-user@lucene.apache.org Sent: Sunday, July 29, 2012 12:10:20 AM Subject: Re: Request header is too large. Where is the error message? On the database side? If it is repeatable, I would just put the two on separate machines and wireshark the http conversation. The problem might become apparent then from visual inspection. Regards, Alex On Jul 28, 2012 1:24 PM, Xue-Feng Yang just4l...@yahoo.com wrote: Hi all, When run DIH indexing data from database, I run into the following error. Anyone knows what is the problem? Thanks, Xufeng /// SEVERE: GRIZZLY0040: Request header is too large. java.nio.BufferOverflowException at com.sun.grizzly.tcp.http11.InternalInputBuffer.fill(InternalInputBuffer.java:765) at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseHeader(InternalInputBuffer.java:669) at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseHeaders(InternalInputBuffer.java:555) at com.sun.grizzly.http.ProcessorTask.parseRequest(ProcessorTask.java:881) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:692) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225) at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) at com.sun.grizzly.ContextTask.run(ContextTask.java:71) at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) at java.lang.Thread.run(Thread.java:662)
Re: Geocoding with Solr
If I wanted to return long/lat from either town city or postcode, how would I structure it? If I was just searching for towns it would be easy, like this: *Town, City, Postcode, Country, Long/Lat* Orpington, Kent, Uk, 1.2/1.1 However, but there will be city entries and postcode, I dont how that would work, would the entry just be for a city for example: *Town, City, Postcode, Country, Long/Lat* null, Kent, Uk, 1.2/1.1 -- View this message in context: http://lucene.472066.n3.nabble.com/Geocoding-with-Solr-tp3997913p3997973.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Geocoding with Solr
Hi there, You may want to check out: SOLR-2073 Geonames.org UpdateProcessor for Spatial SOLR-2074 GeoRSS ResponseWriter SOLR-2075 SpatialQParserPlugin and HostIP adaptor SOLR-2076 Spatial example schema updates SOLR-2077 Spatial example solconfig updates SOLR-2079 Expose HttpServletRequest object from SolrQueryRequest object SOLR-2081 BaseResponseWriter isStreamingDocs causes SingleResponseWriter.end to be called 2x SOLR-2082 Geopost.jar for loading geonames data My student and I submitted those issues a few years ago to handle geocoding in Solr and I think it's still a pretty useful approach. Cheers, Chris On Jul 29, 2012, at 11:34 AM, Spadez wrote: If I wanted to return long/lat from either town city or postcode, how would I structure it? If I was just searching for towns it would be easy, like this: *Town, City, Postcode, Country, Long/Lat* Orpington, Kent, Uk, 1.2/1.1 However, but there will be city entries and postcode, I dont how that would work, would the entry just be for a city for example: *Town, City, Postcode, Country, Long/Lat* null, Kent, Uk, 1.2/1.1 -- View this message in context: http://lucene.472066.n3.nabble.com/Geocoding-with-Solr-tp3997913p3997973.html Sent from the Solr - User mailing list archive at Nabble.com. ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Tools for schema.xml generation and to import from a database
Hi, I'm new to Solr. I've installed 3.6.1 but I'm a little bit confused about what and how to do next. I will use the Jetty version for now. Two poinst I need to know: 1 - I've 2 views that I would like to import to Solr. I think I must do a schema.xml and then import data to that schema. I'm correct with this one? 2 - About tools to autogenerate the schema.xml, there are any? And about tools to import data to the schema, there are any(I'm using Python)? Please give me some clues. Thanks, Best Regards, André.
How config multicore using solr cloud feature
Hi, I'm a new user and our program need use multicore to manage index. I found that Solr 4.0 ALPHA has Solr cloud feature which I could use for load balance in query and sync for update. But the wiki for Solr cloud just tell me how to use single core for sync. For my requirement should use it for multicore synchronized in update. Could someone tell me how to configure it? Thanks.
Re: Tools for schema.xml generation and to import from a database
If you are just starting with SOLR, you might as well jump to 4.0 Alpha. By the time you finished, it will be the production copy. If you want to index stuff from the database, your first step is probably to use DataImportHandler (DIH). Once you get past the basics, you may want to do a custom code, but start from from DIH for faster results. You will want to modify schema.xml. I started by using DIH example and just adding an extra core at first. This might be easier than building a full directory setup from scratch. You also don't actually need to configure schema too much at the beginning. You can start by using dynamic fields. So, if in DIH, you say that your target field is XYZ_i it is automatically picked by as an integer field by SOLR (due to *_i definition that you do need to have). This will not work for fields you want to do aggregation on (e.g. multiple text fields copied into one for easier search), for multilingual text fields, etc. But it will get you going. Oh, and welcome to SOLR. You will like it. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Sun, Jul 29, 2012 at 3:45 PM, Andre Lopes lopes80an...@gmail.com wrote: Hi, I'm new to Solr. I've installed 3.6.1 but I'm a little bit confused about what and how to do next. I will use the Jetty version for now. Two poinst I need to know: 1 - I've 2 views that I would like to import to Solr. I think I must do a schema.xml and then import data to that schema. I'm correct with this one? 2 - About tools to autogenerate the schema.xml, there are any? And about tools to import data to the schema, there are any(I'm using Python)? Please give me some clues. Thanks, Best Regards, André.
java.net.SocketException: Connection reset
I've got SocketException(Connection reset) frequently. This is occurred during distibuted search and logged like below in request server. At First, I thought that the reason of exception is long gc pause time of jvm. So I changed connectionTimeout of the connector in tomcat server.xml to 6ms. However, In gc pause time under 1ms that exception still occurs. Does this exception not caused by gc pause? Requested server is not closed and just sometimes occures stop the world by gc. What's the reason and how can I solve this? 2012-07-30 09:21:36,316 [http-8080-exec-1649] ERROR org.apache.solr.core.SolrCore - org.apache.solr.common.SolrException: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://xxx.xxx.xxx.xxx:8080/solr_jt/jtp01 at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1555) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:445) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:266) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:732) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2262) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://xxx.xxx.xxx.xxx:8080/solr_jt/jtp01 at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:437) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:209) at org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:165) at org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:132) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) ... 3 more Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:168) at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149) at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111) at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264) at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247) at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464) at
