Re: Getting access to current core's conf dir
On Tue, May 5, 2009 at 11:11 AM, Amit Nithian wrote: > I am trying to get at the configuration directory in an implementation of > the SolrEventListener. > > Implement SolrCoreAware and use solrCore.getResourceLoader().getConfigDir() -- Regards, Shalin Shekhar Mangar.
Re: DIH ConcurrentModificationException
This is fixed in trunk. 2009/5/5 Noble Paul നോബിള് नोब्ळ् > hi Walter, > it needs synchronization. I shall open a bug. > > > > On Mon, May 4, 2009 at 7:31 PM, Walter Ferrara > wrote: > > I've got a ConcurrentModificationException during a cron-ed delta import > of > > DIH, I'm using multicore solr nightly from hudson 2009-04-02_08-06-47. > > I don't know if this stacktrace maybe useful to you, but here it is: > > > > java.util.ConcurrentModificationException > >at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown > > Source) > >at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) > >at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) > >at > > > org.apache.solr.handler.dataimport.DataImporter.getStatusMessages(DataImporter.java:384) > >at > > > org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:210 > > ) > >at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) > >at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) > >at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) > >at > > > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) > >at > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > >at > > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > >at > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > >at > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > >at > > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > >at > > > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) > >at > > > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > >at > > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > >at org.mortbay.jetty.Server.handle(Server.java:285) > >at > > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > >at > > > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) > >at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) > >at > org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > >at > org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > >at > > > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) > >at > > > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) > > > > of-course due to the nature of this exception I doubt it can be > reproduced > > easily (this is the only one I've got, and the croned job runned a lot of > > times), but maybe should a synchronized be put somewhere? > > ciao, > > Walter > > > > > > -- > - > Noble Paul | Principal Engineer| AOL | http://aol.com > -- Regards, Shalin Shekhar Mangar.
Re: Getting access to current core's conf dir
Make your class implement the interface SolrCoreAware and you will get a callback with the core. On Tue, May 5, 2009 at 11:11 AM, Amit Nithian wrote: > I am trying to get at the configuration directory in an implementation of > the SolrEventListener. > > 2009/5/4 Noble Paul നോബിള് नोब्ळ् > >> What component is trying to get the SolrCore? if it is implemented as >> SolrCoreAware it gets a callback after the core is completely >> initialized. >> >> On Tue, May 5, 2009 at 4:29 AM, Amit Nithian wrote: >> > How do you get access to the current SolrCore in code? More specifically, >> I >> > am looking for the conf/ directory for the current core (i.e. I have a >> > configuration file housed in SOLR_HOME//conf and would like to >> > get at the "SOLR_HOME//conf" directory.) I was able to do >> this >> > in 1.2 but of course it only had one core so life was easy but I am not >> sure >> > how to do this in 1.3 >> > Thanks! >> > Amit >> > >> >> >> >> -- >> --Noble Paul >> > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: Getting access to current core's conf dir
I am trying to get at the configuration directory in an implementation of the SolrEventListener. 2009/5/4 Noble Paul നോബിള് नोब्ळ् > What component is trying to get the SolrCore? if it is implemented as > SolrCoreAware it gets a callback after the core is completely > initialized. > > On Tue, May 5, 2009 at 4:29 AM, Amit Nithian wrote: > > How do you get access to the current SolrCore in code? More specifically, > I > > am looking for the conf/ directory for the current core (i.e. I have a > > configuration file housed in SOLR_HOME//conf and would like to > > get at the "SOLR_HOME//conf" directory.) I was able to do > this > > in 1.2 but of course it only had one core so life was easy but I am not > sure > > how to do this in 1.3 > > Thanks! > > Amit > > > > > > -- > --Noble Paul >
Re: Core Reload issue
If you change the the conf files and if you reindex the documents it must be reflected are you sure you re-indexed? On Tue, May 5, 2009 at 10:00 AM, Sagar Khetkade wrote: > > Hi, > > I came across a strange problem while reloading the core in multicore > scenario. In the config of one of the core I am making changes in the synonym > and stopword files and then reloading the core. The core gets reloaded but > the changes in stopword and synonym fiels does not get reflected when I query > in. The filters for index and query are the same. I face this problem even if > I reindex the documents. But when I restart the servlet container in which > the solr is embedded I problem does not resurfaces. > My ultimate goal is/was to reflect the changes made in the text files inside > the config folder. > Is this the expected behaviour or some problem at my side. Could anyone > suggest me the possible work around? > > Thanks in advance! > > Regards, > Sagar Khetkade > _ > More than messages–check out the rest of the Windows Live™. > http://www.microsoft.com/india/windows/windowslive/ -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Re: DIH ConcurrentModificationException
hi Walter, it needs synchronization. I shall open a bug. On Mon, May 4, 2009 at 7:31 PM, Walter Ferrara wrote: > I've got a ConcurrentModificationException during a cron-ed delta import of > DIH, I'm using multicore solr nightly from hudson 2009-04-02_08-06-47. > I don't know if this stacktrace maybe useful to you, but here it is: > > java.util.ConcurrentModificationException > at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown > Source) > at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) > at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) > at > org.apache.solr.handler.dataimport.DataImporter.getStatusMessages(DataImporter.java:384) > at > org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:210 > ) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) > at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) > at > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > at > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > at > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > at > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > at > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > at > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) > at > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) > at > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > at org.mortbay.jetty.Server.handle(Server.java:285) > at > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > at > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) > at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > at > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) > at > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) > > of-course due to the nature of this exception I doubt it can be reproduced > easily (this is the only one I've got, and the croned job runned a lot of > times), but maybe should a synchronized be put somewhere? > ciao, > Walter > -- - Noble Paul | Principal Engineer| AOL | http://aol.com
Core Reload issue
Hi, I came across a strange problem while reloading the core in multicore scenario. In the config of one of the core I am making changes in the synonym and stopword files and then reloading the core. The core gets reloaded but the changes in stopword and synonym fiels does not get reflected when I query in. The filters for index and query are the same. I face this problem even if I reindex the documents. But when I restart the servlet container in which the solr is embedded I problem does not resurfaces. My ultimate goal is/was to reflect the changes made in the text files inside the config folder. Is this the expected behaviour or some problem at my side. Could anyone suggest me the possible work around? Thanks in advance! Regards, Sagar Khetkade _ More than messages–check out the rest of the Windows Live™. http://www.microsoft.com/india/windows/windowslive/
Re: Shutting down an instance of EmbeddedSolrServer
hi Eric, there should be a getter for CoreContainer in EmbeddedSolrServer. Open an issue --Noble On Tue, May 5, 2009 at 12:17 AM, Eric Pugh wrote: > Hi all, > > I notice that when I use EmbeddedSolrServer I have to use Control C to stop > the process. I think the way to shut it down is by calling > > coreContainer.shutdown(). > > However, is it possible to get the coreContainer from a SolrServer object? > Right now it is defined as protected final CoreContainer coreContainer;. > > I wanted to do: > > ((EmbeddedSolrServer)solr)getCoreContainer.shutdown(); > > But is seem I need to keep my own reference to the coreContainer? > > Is changing this worth a patch? > > Eric > > - > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com > Free/Busy: http://tinyurl.com/eric-cal > > > > > -- --Noble Paul
Re: Getting access to current core's conf dir
What component is trying to get the SolrCore? if it is implemented as SolrCoreAware it gets a callback after the core is completely initialized. On Tue, May 5, 2009 at 4:29 AM, Amit Nithian wrote: > How do you get access to the current SolrCore in code? More specifically, I > am looking for the conf/ directory for the current core (i.e. I have a > configuration file housed in SOLR_HOME//conf and would like to > get at the "SOLR_HOME//conf" directory.) I was able to do this > in 1.2 but of course it only had one core so life was easy but I am not sure > how to do this in 1.3 > Thanks! > Amit > -- --Noble Paul
Re: Does solrj return result in XML format? If not then how to make it do that.
The we interface returns the data in xml for mat because that is the most readable format.SolrJ uses a compact binary format by default. But you can make it use xml as well. but there is no real use case for using xml with solrJ On Mon, May 4, 2009 at 7:06 PM, ahmed baseet wrote: > As I know when we query solr from solr admin interface we get back the > results in xml format, so thought there must be something similar for solrj > as well, which I'll make to go thru an xml parser at the other end and > display all the results in the browser. Otherwise I've to iterate the > solrdocumentlist and create a list[may be] to put the results and return it > back to the browser which will handle displaying that list/map etc. > > --Ahmed. > > > > On Mon, May 4, 2009 at 5:52 PM, Erik Hatcher > wrote: > >> Just out of curiosity, what's the use case for getting the result back in >> XML from SolrJ? >> >> Erik >> >> >> On May 4, 2009, at 8:13 AM, ahmed baseet wrote: >> >> Can we get the results as received by Solrj in XML format? If yes how to >>> do >>> that. I think there must be some way to make solrj returns results in XML >>> format. >>> I need some pointers in this direction. As I know solrs returns the result >>> in solrdocument format that we've to iterate to extract the fields. Thank >>> you. >>> >>> --Ahmed. >>> >> >> > -- --Noble Paul
Re: Distributed Sizing Question
Hi Todd, It depends what kind of hardware you run this on. 4 instances of 15GB shard 1 @ machines 1-4 behind VIP1 4 instances of 15GB shard 2 @ machines 5-8 behind VIP2 4 instances of 15GB shard 3 @ machines 9-12 behind VIP3 4 instances of 15GB shard 4 @ machines 13-16 behind VIP4 This is what you are building, right? That sounds fine, though you may be able to get away with even just 2 boxes per shard (e.g. machines 1-2 for shard 1 instead of 4 machines), depending on the query rate and their actual latency. In that case you could break the index into even smaller shards, thus lowering your per-machine RAM requirements: 2 instances of 7.5GB shard 1 @ machines 1-2 behind VIP1 2 instances of 7.5GB shard 2 @ machines 3-4 behind VIP2 2 instances of 7.5GB shard 3 @ machines 5-6 behind VIP3 2 instances of 7.5GB shard 4 @ machines 7-8 behind VIP4 2 instances of 7.5GB shard 5 @ machines 9-10 behind VIP5 2 instances of 7.5GB shard 6 @ machines 11-12 behind VIP6 2 instances of 7.5GB shard 7 @ machines 13-14 behind VIP7 2 instances of 7.5GB shard 8 @ machines 15-16 behind VIP88 You should be okay with 8GB of RAM per machine, or even 4 GB perhaps. It depends how much heap you give to the JVM, how big your Solr caches are, etc. And don't forget you can always put a caching HTTP proxy in front of Solr! Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Todd Benge > To: solr-user@lucene.apache.org > Sent: Monday, May 4, 2009 6:35:47 PM > Subject: Distributed Sizing Question > > Hi, > > We're in the process of converting a Lucene deployment of 500 M documents / > 60 G deployment into a Solr Distributed search deployment. > > The primary reason for the change is instability in the Lucene deployment > due to memory constraints. The existing infrastructure is deployed on 48 > machines with all indices on each machine using the MultiSearcher. > > The new deployment will leverage Solr's distributed search model for to > deploy smaller index shards in various clusters. Under average load, the > system should be able to easily handle 8-10 requests per second. > > We're looking for some guidance on best practices for sizing the cluster's > correctly. Our current thought is to divide the indices into 4 equal parts > and build several 4 machine clusters. So each machine will be hosting ~ 15 > G each. > > Has anyone had experience with a similar size deployment? Any suggestions > on the architectural strategy? > > Thanks for the help. > > Todd
Getting access to current core's conf dir
How do you get access to the current SolrCore in code? More specifically, I am looking for the conf/ directory for the current core (i.e. I have a configuration file housed in SOLR_HOME//conf and would like to get at the "SOLR_HOME//conf" directory.) I was able to do this in 1.2 but of course it only had one core so life was easy but I am not sure how to do this in 1.3 Thanks! Amit
Distributed Sizing Question
Hi, We're in the process of converting a Lucene deployment of 500 M documents / 60 G deployment into a Solr Distributed search deployment. The primary reason for the change is instability in the Lucene deployment due to memory constraints. The existing infrastructure is deployed on 48 machines with all indices on each machine using the MultiSearcher. The new deployment will leverage Solr's distributed search model for to deploy smaller index shards in various clusters. Under average load, the system should be able to easily handle 8-10 requests per second. We're looking for some guidance on best practices for sizing the cluster's correctly. Our current thought is to divide the indices into 4 equal parts and build several 4 machine clusters. So each machine will be hosting ~ 15 G each. Has anyone had experience with a similar size deployment? Any suggestions on the architectural strategy? Thanks for the help. Todd
Re: Hierarchal Faceting Field Type
On Mon, May 4, 2009 at 5:28 PM, Nasseam Elkarra wrote: > Hello Koji, > > I applied the patch: > http://issues.apache.org/jira/browse/SOLR-64 > > It's working but I have a question about facet.prefix. When we add a > facet.prefix it filters the facet results but not the query results. Is > there a way to filter both? Use a normal filter (fq parameter) instead of facet.prefix. -Yonik http://www.lucidimagination.com
Re: Hierarchal Faceting Field Type
Hello Koji, I applied the patch: http://issues.apache.org/jira/browse/SOLR-64 It's working but I have a question about facet.prefix. When we add a facet.prefix it filters the facet results but not the query results. Is there a way to filter both? Thank you, Nasseam On Apr 14, 2009, at 3:32 PM, Koji Sekiguchi wrote: Nasseam Elkarra wrote: Background: Set up a system for hierarchal categories using the following scheme: level one# level one#level two# level one#level two#level three# Trying to find the right combination of field type and query to get the desired results. Saw some previous posts about hierarchal facets which helped in the generating the right query but having an issue using the built in text field which ignores our delimiter and the string field which prevents us from doing a start with search. Does anyone have any insight into the field declaration? Any help is appreciated. Thank you. Out of need in my project, I'll get started to work for SOLR-64, expected any day. I'm thinking introducing a field type for hierarchical facet. Koji
Re: Slow Post Performance
I'm using the embedded Jetty. >> I have an index of about 6 million small documents. >> I find that after a few days of on and off posting, searching >> still works fine but posting gets really slow - client http >> connection always times out. I restarted server. Then posting >> speed seemed to have returned to normal. Is this the expected >> behavior? > > Definitely not... what servlet container are you using? > > -Yonik
Shutting down an instance of EmbeddedSolrServer
Hi all, I notice that when I use EmbeddedSolrServer I have to use Control C to stop the process. I think the way to shut it down is by calling coreContainer.shutdown(). However, is it possible to get the coreContainer from a SolrServer object? Right now it is defined as protected final CoreContainer coreContainer;. I wanted to do: ((EmbeddedSolrServer)solr)getCoreContainer.shutdown(); But is seem I need to keep my own reference to the coreContainer? Is changing this worth a patch? Eric - Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com Free/Busy: http://tinyurl.com/eric-cal
Re: offline solr indexing
This should be fine. You won't have to replicate your index, just reopen the searcher when commit is done, that's all. Index updates and searches can be happening at the same time. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Charles Federspiel > To: solr-user@lucene.apache.org > Sent: Saturday, May 2, 2009 12:40:13 PM > Subject: Re: offline solr indexing > > Thanks. I imagine replication to a slave would require a filesystem writable > by that slave. > > I think it helps to realize that indexing is really a function of Content > Management. After some discussion with coworkers, I've learned that our > internal CMS server runs within tomcat and shares a filesystem with our > public app-servers. So I'm hoping to deploy solr both to the tomcat > instance (for indexing) and our other instances (for searching) simply > sharing a Solr home between them. How bad is this? Does updating and > commiting the index interrupt search? It would only affect our internal > instance, but I still need to know all the effects of this setup. > > On Mon, Apr 27, 2009 at 6:32 PM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > > > On Tue, Apr 28, 2009 at 12:38 AM, Charles Federspiel < > > charles.federsp...@gmail.com> wrote: > > > > > Solr Users, > > > Our app servers are setup on read-only filesystems. Is there a way > > > to perform indexing from the command line, then copy the index files to > > the > > > app-server and use Solr to perform search from inside the servlet > > > container? > > > > > > If the filesystem is read-only, then how can you index at all? > > > > But what I think you are describing is the regular master-slave setup that > > we use. A dedicated master on which writes are performed. Multiple slaves > > on > > which searches are performed. The index is replicated to slaves through > > script or the new java based replication. > > > > > > > If the Solr implementation is bound to http requests, can Solr perform > > > searches against an index that I create with Lucene? > > > thank you, > > > > > > It can but it is a little tricky to get the schema and analysis correct > > between your Lucene writer and Solr searcher. > > > > -- > > Regards, > > Shalin Shekhar Mangar. > >
Re: Custom Sorting Based on Relevancy
Or you could collapse search results with SOLR-236. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: David Giffin > To: solr-user@lucene.apache.org > Sent: Monday, May 4, 2009 12:37:29 PM > Subject: Custom Sorting Based on Relevancy > > Hi There, > > I'm working on a sorting issue. Our site currently sorts by creation > date descending, so users list similar products multiple times to show > up at the top of the results. When sorting based on score, we want to > move items by the "same user" with the "same title" down search > results. It would be best if the first item stayed in place based on > score, and each additional item is moved out (rows * repeated > user/title). > > Is custom sorting the best way? or is there something else I'm not > thinking about. At the moment I'm looking at doing roughly the > opposite of the Query Elevate Search component. > > Thanks, > David
Re: Update an existing Solr Index
Correct. For an attempt to allow only field-level updates (without requiring the whole document to be re-analyzed and re-indexed) please see SOLR-139. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Amit Nithian > To: solr-user@lucene.apache.org > Sent: Monday, May 4, 2009 1:03:04 PM > Subject: Re: Update an existing Solr Index > > I believe you do have to re-index the entire document. From what I have > read, this is a Lucene limitation not a Solr one. I have run into this > problem too and it's a pain to deal with. If you have data relevant for > ranking but not searching, then consider storing it elsewhere and write a > custom function to bring the data into Solr. > - Amit > > On Mon, May 4, 2009 at 5:18 AM, ahmed baseet wrote: > > > As I know when you resend another index request with some old ID, old field > > but new content, the old one gets overwritten by the new one. > > @solr-users, Views??? > > > > --Ahmed > > > > > > On Mon, May 4, 2009 at 5:26 PM, appleman1982 > > >wrote: > > > > > > > > Hi All, > > > I have a requirement wherein i want to update an existing index in solr. > > > For example : I have issued an index command in solr as > > > > > > > > > 123 > > > xxx > > > > > > > > > > > > The id field is a unique key here. > > > > > > My requirement is that i should be able to update this inex i.e add > > another > > > field to it without the need to build the entire index again. > > > For example > > > if i issue the following solr command > > > > > > > > > 123 > > > delhi > > > > > > > > > > > > it should give me a merged index like > > > > > > > > > 123 > > > xxx > > > delhi > > > > > > > > > > > > > > > Any pointers or workarounds to achieve this in solr would be highly > > > appreciated. > > > > > > Thanks, Jugesh > > > -- > > > View this message in context: > > > > > http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html > > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > > > >
Re: schema.xml: default values for @indexed and @stored
Attribute values for fields should be inherited from attribute values of their field types. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Michael Ludwig > To: solr-user@lucene.apache.org > Sent: Monday, May 4, 2009 1:08:43 PM > Subject: schema.xml: default values for @indexed and @stored > > From the apache-solr-1.3.0\example\solr\conf\schema.xml file: > > > > class="solr.StrField" /> > > So for both fieldtype/@stored and fieldtype/@indexed, the default is > "true", correct? > > And does the configuration constitute a default for > so that field/@stored and field/@indexed take their effective values > according to field/@type? > > Or do these default to "true" regardless of what's specified in the > respective ? > > Michael Ludwig
Re: Authenticated Indexing Not working
Hi, My error is as below. If authentication error while indexing with authentication enable (Basic GET/POST) org.apache.solr.client.solrj.SolrServerException: org.apache.commons.httpclient.ProtocolException: Unbuffered entity enclosing request can not be repeated. at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:470) at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:242) at org.apache.solr.client.solrj.request.UpdateRequest.process(UpdateRequest.java:259) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:48) at com.infy.icode.SolrAuthenticationTest.index(SolrAuthenticationTest.java:84) http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html If I am indexing without authentication it works fine. I followed procedure listed in above blog. Regards, Allahbaksh On Wed, Apr 29, 2009 at 9:44 PM, Allahbaksh Asadullah < allahbaks...@gmail.com> wrote: > Hi, > I followed the procedure given at > http://blog.comtaste.com/2009/02/securing_your_solr_server_on_t.html > Regards, > Allahbaksh > > On 4/28/09, Shalin Shekhar Mangar wrote: > > On Sun, Apr 26, 2009 at 11:04 AM, Allahbaksh Asadullah < > > allahbaks...@gmail.com> wrote: > > > >> HI Otis, > >> I am using HTTPClient for authentication. When I use the server with > >> Authentication for searching it works fine. But when I use it for > >> indexing it throws error. > >> > > > > What is the error? Is it thrown by Solr or your servlet container? > > > > One difference between a search request and update request with Solrj is > > that a search request uses HTTP GET by default but an update request uses > > an > > HTTP POST by default. Perhaps your authentication scheme is not > configured > > correctly for POST requests? > > > > -- > > Regards, > > Shalin Shekhar Mangar. > > > > > -- > Allahbaksh Mohammedali Asadullah, > Software Engineering & Technology Labs, > Infosys Technolgies Limited, Electronic City, > Hosur Road, Bangalore 560 100, India. > (Board: 91-80-28520261 | Extn: 73927 | Direct: 41173927. > Fax: 91-80-28520362 | Mobile: 91-9845505322. > -- Allahbaksh Mohammedali Asadullah, Software Engineering & Technology Labs, Infosys Technolgies Limited, Electronic City, Hosur Road, Bangalore 560 100, India. (Board: 91-80-28520261 | Extn: 73927 | Direct: 41173927. Fax: 91-80-28520362 | Mobile: 91-9845505322.
Re: Slow Post Performance
On Mon, May 4, 2009 at 1:09 PM, wrote: > I have an index of about 6 million small documents. > I find that after a few days of on and off posting, searching > still works fine but posting gets really slow - client http > connection always times out. I restarted server. Then posting > speed seemed to have returned to normal. Is this the expected > behavior? Definitely not... what servlet container are you using? -Yonik http://www.lucidimagination.com
Slow Post Performance
I have an index of about 6 million small documents. I find that after a few days of on and off posting, searching still works fine but posting gets really slow - client http connection always times out. I restarted server. Then posting speed seemed to have returned to normal. Is this the expected behavior? Should I start considering sharking my data? I didn't think 6 million was a large number for one instance of solr but I'd like to know others' experience. Thanks!
schema.xml: default values for @indexed and @stored
From the apache-solr-1.3.0\example\solr\conf\schema.xml file: So for both fieldtype/@stored and fieldtype/@indexed, the default is "true", correct? And does the configuration constitute a default for so that field/@stored and field/@indexed take their effective values according to field/@type? Or do these default to "true" regardless of what's specified in the respective ? Michael Ludwig
Re[2]: Highlight MoreLikeThis results?
I see. Thanks. > There was a thread about this last week and verdict is currently you > can't highlight MoreLikeThis results. > Thanks, > Matt Weber >> My query returns a number of MoreLikeThis results for a given >> document. I wonder if there is a way to highlight the terms >> in the MoreLikeThis results? Thanks.
Re: Update an existing Solr Index
I believe you do have to re-index the entire document. From what I have read, this is a Lucene limitation not a Solr one. I have run into this problem too and it's a pain to deal with. If you have data relevant for ranking but not searching, then consider storing it elsewhere and write a custom function to bring the data into Solr. - Amit On Mon, May 4, 2009 at 5:18 AM, ahmed baseet wrote: > As I know when you resend another index request with some old ID, old field > but new content, the old one gets overwritten by the new one. > @solr-users, Views??? > > --Ahmed > > > On Mon, May 4, 2009 at 5:26 PM, appleman1982 >wrote: > > > > > Hi All, > > I have a requirement wherein i want to update an existing index in solr. > > For example : I have issued an index command in solr as > > > > > > 123 > > xxx > > > > > > > > The id field is a unique key here. > > > > My requirement is that i should be able to update this inex i.e add > another > > field to it without the need to build the entire index again. > > For example > > if i issue the following solr command > > > > > > 123 > > delhi > > > > > > > > it should give me a merged index like > > > > > > 123 > > xxx > > delhi > > > > > > > > > > Any pointers or workarounds to achieve this in solr would be highly > > appreciated. > > > > Thanks, Jugesh > > -- > > View this message in context: > > > http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > >
Re: Solr autocompletion in rails
You will probably want to use the new TermsComponent in Solr 1.4. See http://wiki.apache.org/solr/TermsComponent . I just recently wrote a blog post about using autocompletion with TermsComponent, a servlet, and jQuery. You can probably follow these instructions, but instead of writing a servlet you can write a rails handler parsing the json output directly. http://www.mattweber.org/2009/05/02/solr-autosuggest-with-termscomponent-and-jquery/ . Thanks, Matt Weber On May 4, 2009, at 9:39 AM, manisha_5 wrote: Hi, I am new to solr. I am using solr server to index the data and make search in a Ruby on rails project.I want to add autocompletion feature. I tried with the xml patch in the schema.xml file of solr, but dont know how to test if the feature is working.also havent been able to integrate the same in the Rails project that is using Solr.Can anyone please provide some help in this regards?? the patch of codes in Schema.xml is : minGramSize="3" maxGramSize="15" /> maxGramSize="100" minGramSize="1" /> -- View this message in context: http://www.nabble.com/Solr-autocompletion-in-rails-tp23372020p23372020.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr autocompletion in rails
Hi, I am new to solr. I am using solr server to index the data and make search in a Ruby on rails project.I want to add autocompletion feature. I tried with the xml patch in the schema.xml file of solr, but dont know how to test if the feature is working.also havent been able to integrate the same in the Rails project that is using Solr.Can anyone please provide some help in this regards?? the patch of codes in Schema.xml is : -- View this message in context: http://www.nabble.com/Solr-autocompletion-in-rails-tp23372020p23372020.html Sent from the Solr - User mailing list archive at Nabble.com.
Custom Sorting Based on Relevancy
Hi There, I'm working on a sorting issue. Our site currently sorts by creation date descending, so users list similar products multiple times to show up at the top of the results. When sorting based on score, we want to move items by the "same user" with the "same title" down search results. It would be best if the first item stayed in place based on score, and each additional item is moved out (rows * repeated user/title). Is custom sorting the best way? or is there something else I'm not thinking about. At the moment I'm looking at doing roughly the opposite of the Query Elevate Search component. Thanks, David
Re: fieldType without tokenizer
On Mon, May 4, 2009 at 9:28 PM, sunnyfr wrote: > > Hi, > > I would like to create a field without tokenizer but I've an error, > You can use KeywordTokenizer which does not do any tokenization. -- Regards, Shalin Shekhar Mangar.
fieldType without tokenizer
Hi, I would like to create a field without tokenizer but I've an error, I tried : But I've : May 4 17:49:41 solr-test jsvc.exec[5786]: May 4, 2009 5:49:41 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: analyzer without class or tokenizer & filter list ^Iat org.apache.solr.schema.IndexSchema.readAnalyzer(IndexSchema.java:808) ^Iat org.apache.solr.schema.IndexSchema.access$100(IndexSchema.java:58) ^Iat org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:425) ^Iat org.apache.solr.schema.IndexSchema$1.create(IndexSchema.java:443) ^Iat org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:141) ^Iat org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:452) ^Iat org.apache.solr.schema.IndexSchema.(IndexSchema.java:95) ^Iat org.apache.solr.core.CoreContainer.create(CoreContainer.java:344) ^Iat org.apache.solr.core.CoreContainer.load(CoreContainer.java:222) ^Iat org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:107) ^Iat org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:69) Thanks for your help ! -- View this message in context: http://www.nabble.com/fieldType-without-tokenizer-tp23371300p23371300.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: autoSuggest
I am not sure you can return the results in order of frequency, you will have to sort the results yourself. Also, for autoSuggest you will want to add the terms.prefix=input term and terms.lower.incl=false so your example will be: /autoSuggest? terms = true &indent = true &terms .fl = title &terms .rows = 5 &terms .lower=simp&terms.lower.incl=false&terms.prefix=simp&omitHeader=true To get results for more multiple words such as "barack obama", you need to set the terms.fl parameter to an untokenized, un-processed field just as you would with a facet. So in your schema.xml, add a new string field, then use a copyfield to copy the value of title into the new field and set terms.fl to the new field you just created after reindexing. Thanks, Matt Weber On May 4, 2009, at 6:46 AM, sunnyfr wrote: Hi, I would like to know how work /autoSuggest. I do have result when I hit : /autoSuggest? terms = true &indent =true&terms.fl=title&terms.rows=5&terms.lower=simp&omitHeader=true I've: 74 129 2 2 1 How can I ask it to suggest first expression which are more frequent in the database ? How can I look for even for two words, ie: I look for "bara" ... make it suggesting "barack obama" ??? thanks a lot, -- View this message in context: http://www.nabble.com/autoSuggest-tp23367848p23367848.html Sent from the Solr - User mailing list archive at Nabble.com. Thanks, Matt Weber eSr Technologies http://www.esr-technologies.com
Re: Highlight MoreLikeThis results?
There was a thread about this last week and verdict is currently you can't highlight MoreLikeThis results. Thanks, Matt Weber On May 4, 2009, at 1:22 AM, jli...@gmail.com wrote: My query returns a number of MoreLikeThis results for a given document. I wonder if there is a way to highlight the terms in the MoreLikeThis results? Thanks.
Re: Does solrj return result in XML format? If not then how to make it do that.
As I know when we query solr from solr admin interface we get back the results in xml format, so thought there must be something similar for solrj as well, which I'll make to go thru an xml parser at the other end and display all the results in the browser. Otherwise I've to iterate the solrdocumentlist and create a list[may be] to put the results and return it back to the browser which will handle displaying that list/map etc. --Ahmed. On Mon, May 4, 2009 at 5:52 PM, Erik Hatcher wrote: > Just out of curiosity, what's the use case for getting the result back in > XML from SolrJ? > >Erik > > > On May 4, 2009, at 8:13 AM, ahmed baseet wrote: > > Can we get the results as received by Solrj in XML format? If yes how to >> do >> that. I think there must be some way to make solrj returns results in XML >> format. >> I need some pointers in this direction. As I know solrs returns the result >> in solrdocument format that we've to iterate to extract the fields. Thank >> you. >> >> --Ahmed. >> > >
autoSuggest
Hi, I would like to know how work /autoSuggest. I do have result when I hit : /autoSuggest?terms=true&indent=true&terms.fl=title&terms.rows=5&terms.lower=simp&omitHeader=true I've: 74 129 2 2 1 How can I ask it to suggest first expression which are more frequent in the database ? How can I look for even for two words, ie: I look for "bara" ... make it suggesting "barack obama" ??? thanks a lot, -- View this message in context: http://www.nabble.com/autoSuggest-tp23367848p23367848.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Does solrj return result in XML format? If not then how to make it do that.
The point of using solrj is that you don't have to do any parsing yourself -- you get access to the results in object form. If you need to do parsing, just grab the xml directly: http://host/solr/select?q=*:*&wt=xml On May 4, 2009, at 9:36 AM, ahmed baseet wrote: As I know when we query solr from solr admin interface we get back the results in xml format, so thought there must be something similar for solrj as well, which I'll make to go thru an xml parser at the other end and display all the results in the browser. Otherwise I've to iterate the solrdocumentlist and create a list[may be] to put the results and return it back to the browser which will handle displaying that list/map etc. --Ahmed. On Mon, May 4, 2009 at 5:52 PM, Erik Hatcher >wrote: Just out of curiosity, what's the use case for getting the result back in XML from SolrJ? Erik On May 4, 2009, at 8:13 AM, ahmed baseet wrote: Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed.
DIH ConcurrentModificationException
I've got a ConcurrentModificationException during a cron-ed delta import of DIH, I'm using multicore solr nightly from hudson 2009-04-02_08-06-47. I don't know if this stacktrace maybe useful to you, but here it is: java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at java.util.LinkedHashMap$EntryIterator.next(Unknown Source) at org.apache.solr.handler.dataimport.DataImporter.getStatusMessages(DataImporter.java:384) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:210 ) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1333) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:232) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) of-course due to the nature of this exception I doubt it can be reproduced easily (this is the only one I've got, and the croned job runned a lot of times), but maybe should a synchronized be put somewhere? ciao, Walter
RE: DC/NOVA Lucene&Solr meetup
Hi, Actually I have not yet reached the search/Lucene part on my project (which consists in selling pastries over the internet) but I would like to participate to benefit for the discussion. When will this gathering occur? Regards, Mohamed -Original Message- From: Erik Hatcher [mailto:e...@ehatchersolutions.com] Sent: Monday, May 04, 2009 8:50 AM To: solr-user@lucene.apache.org; java-u...@lucene.apache.org ; core-u...@hadoop.apache.org ; solr-user@lucene.apache.org Subject: DC/NOVA Lucene&Solr meetup My company is co-sponsoring a Lucene/Solr meetup later this month in the Northern VA / DC area (Reston). Details will be coming out soon. We've got one night of talks planned and considering adding another consecutive night. If you're in the area and have a Lucene (any of the Lucene family of projects, including Mahout, Solr, Tika, etc) project you're eager to discuss for 20-30 minutes, write me offline and tell me about it. Thanks, Erik This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message.
Re: Does solrj return result in XML format? If not then how to make it do that.
use this constructor:
public CommonsHttpSolrServer(String solrServerUrl, HttpClient
httpClient, ResponseParser parser) throws MalformedURLException {
this(new URL(solrServerUrl), httpClient, parser, false);
}
and give it the XMLResponseParser
-- - - -
Is this just helpful for debugging with packet sniffing? the XML
format will be a bit slower then the binary format.
ryan
On May 4, 2009, at 8:22 AM, Erik Hatcher wrote:
Just out of curiosity, what's the use case for getting the result
back in XML from SolrJ?
Erik
On May 4, 2009, at 8:13 AM, ahmed baseet wrote:
Can we get the results as received by Solrj in XML format? If yes
how to do
that. I think there must be some way to make solrj returns results
in XML
format.
I need some pointers in this direction. As I know solrs returns the
result
in solrdocument format that we've to iterate to extract the fields.
Thank
you.
--Ahmed.
DC/NOVA Lucene&Solr meetup
My company is co-sponsoring a Lucene/Solr meetup later this month in the Northern VA / DC area (Reston). Details will be coming out soon. We've got one night of talks planned and considering adding another consecutive night. If you're in the area and have a Lucene (any of the Lucene family of projects, including Mahout, Solr, Tika, etc) project you're eager to discuss for 20-30 minutes, write me offline and tell me about it. Thanks, Erik
Re: Does solrj return result in XML format? If not then how to make it do that.
Just out of curiosity, what's the use case for getting the result back in XML from SolrJ? Erik On May 4, 2009, at 8:13 AM, ahmed baseet wrote: Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed.
Re: Update an existing Solr Index
As I know when you resend another index request with some old ID, old field but new content, the old one gets overwritten by the new one. @solr-users, Views??? --Ahmed On Mon, May 4, 2009 at 5:26 PM, appleman1982 wrote: > > Hi All, > I have a requirement wherein i want to update an existing index in solr. > For example : I have issued an index command in solr as > > > 123 > xxx > > > > The id field is a unique key here. > > My requirement is that i should be able to update this inex i.e add another > field to it without the need to build the entire index again. > For example > if i issue the following solr command > > > 123 > delhi > > > > it should give me a merged index like > > > 123 > xxx > delhi > > > > > Any pointers or workarounds to achieve this in solr would be highly > appreciated. > > Thanks, Jugesh > -- > View this message in context: > http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
Does solrj return result in XML format? If not then how to make it do that.
Can we get the results as received by Solrj in XML format? If yes how to do that. I think there must be some way to make solrj returns results in XML format. I need some pointers in this direction. As I know solrs returns the result in solrdocument format that we've to iterate to extract the fields. Thank you. --Ahmed.
Update an existing Solr Index
Hi All, I have a requirement wherein i want to update an existing index in solr. For example : I have issued an index command in solr as 123 xxx The id field is a unique key here. My requirement is that i should be able to update this inex i.e add another field to it without the need to build the entire index again. For example if i issue the following solr command 123 delhi it should give me a merged index like 123 xxx delhi Any pointers or workarounds to achieve this in solr would be highly appreciated. Thanks, Jugesh -- View this message in context: http://www.nabble.com/Update-an-existing-Solr-Index-tp23366705p23366705.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multiple Facet Dates
Good solution without touching any code! Thank you very much.
I have already used the "key function" for query facets. I use it to send to
the xml response a different word/sentence than the one I use to do the
proper faceting. It has been really useful
Thank you.
Erik Hatcher wrote:
>
>
> On May 1, 2009, at 8:23 PM, Yonik Seeley wrote:
>> On Fri, May 1, 2009 at 8:06 PM, Chris Hostetter
>> wrote:
>>>facet.date.gap = +1MONTH
>>>facet.date = {!key=begin_month}begin_df
>>>f.begin_month.date.start = NOW/MONTH-1YEAR
>>>f.begin_month.date.end = NOW/MONTH+1MONTH
>>
>> Another syntax alternative is to allow start/end as local params:
>>
>> facet.date={!key=begin_month start=NOW/MONTH-1YEAR
>> end=NOW/MONTH+1MONTH}begin_df
>
> +1, I like it.
>
> Erik
>
>
>
--
View this message in context:
http://www.nabble.com/Multiple-Facet-Dates-tp23272868p23366696.html
Sent from the Solr - User mailing list archive at Nabble.com.
Highlight MoreLikeThis results?
My query returns a number of MoreLikeThis results for a given document. I wonder if there is a way to highlight the terms in the MoreLikeThis results? Thanks.
Re: Last modified time for cores, taking into account uncommitted changes
Hi, I use SolrIndexReader.isCurrent() for this purpose On Fri, May 1, 2009 at 1:42 AM, James Brady wrote: > Hi, > The lastModified field the Solr status seems to only be updated when a > commit/optimize operation takes place. > > Is there any way to determine when a core has been changed, including any > uncommitted add operations? > > > Thanks, > James > -- Andrew Klochkov
