Re: Location of Solr Logs
Geeta, have a look on the the following wiki pages: http://wiki.apache.org/solr/SolrLogging http://wiki.apache.org/solr/LoggingInDefaultJettySetup Regards Stefan On Fri, Apr 29, 2011 at 8:35 AM, Grijesh wrote: > You can see solr logs at your servlet container's log file i.e. if you are > using Tomcat it can be found at > [CATALINA_HOME]/logs/catalina.XXX.log > > > -Thanx: > Grijesh > www.gettinhahead.co.in -- > View this message in context: > http://lucene.472066.n3.nabble.com/Location-of-Solr-Logs-tp2877510p2878294.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Spatial Search
I did the String Solution, works great, thank you! Am 29.04.2011 um 00:25 schrieb Jan Høydahl: > 1) Create an extra String field on your bean as Yonik suggests or > 2) Write an UpdateRequestHandler which reads the doubles and creates the > LatLon from that > > -- > Jan Høydahl, search solution architect > Cominvent AS - www.cominvent.com > > On 28. apr. 2011, at 14.44, Yonik Seeley wrote: > >> On Thu, Apr 28, 2011 at 5:15 AM, Jonas Lanzendörfer >> wrote: >>> I am new to solr and try to use the spatial search feature which was added >>> in 3.1. In my schema.xml I have 2 double fields for latitude and longitude. >>> How can I get them into the location field type? I use solrj to fill the >>> index with data. If I would use a location field instead of two double >>> fields, how could I fill this with solrj? I use annotations to link the >>> data from my dto´s to the index fields... >> >> >> I've not used the annotation stuff in SolrJ, but since the value sent >> in must be of the for 10.3,20.4 then >> I guess one would have to have a String field with this value on your object. >> >> >> -Yonik >> http://www.lucenerevolution.org -- Lucene/Solr User Conference, May >> 25-26, San Francisco >
Re: manual background re-indexing
I understand that multicores allows met o create two cores which are just normal solr-homes. That's easy. However I'm only interested to the reload command and, in particular, to reload the underlying index segment files. Do I understand correctly that the following is what I should do? - still make my setting multicore and get the core-admin requesthandler to work, even with one core - attempt the reload with a change of solrconfig or schema - do the reload of data by changing the index-segment-path in the config as an example of the above thanks to clarify paul Le 29 avr. 2011 à 03:12, Erick Erickson a écrit : > You simply create two cores. One in solr/cores/core1 and another in > solr/cores/core2 > They each have a separate conf and data directory,and the index in in > core#/data/index. > > Really, its' just introducing one more level. You can experiment just > by configuring a core > and copying your index to solr/cores/yourcore/data/index. After, of > course, configuring > Solr.xml to understand cores.
Autocomplete(terms) middle of words
Hello, I use the termsComponent to fix some autocomplete on my website. I use the prefix and get the following results: searching for manch: manchester city(10) manchester united(2) When a user search for ches i want the following results: chesterfield united(13) manchester united(2) I want to search in the middle of words. How can i fix that? I have tried the NgramsFilter on index time but i doesn't seems to work with the termsComponent. My current configuration: -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878694.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
NGram will work for you if you want to search in middle of the word .You can also look for wildcard search for that. NGram will increase the size of index while wildcard queries are slow. -Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878748.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Strange performance behaviour when concurrent requests are done
Any suggestion about this issue?-- View this message in context: http://lucene.472066.n3.nabble.com/Strange-performance-behaviour-when-concurrent-requests-are-done-tp505478p2878758.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Strange performance behaviour when concurrent requests are done
You have 4 core machine that can execute 4 threads parallel. After 4 threads it will start context switching between threads. -Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Strange-performance-behaviour-when-concurrent-requests-are-done-tp505478p2878799.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Strange performance behaviour when concurrent requests are done
That's true. But the degradation is so big. If you use lunch concurrent requests to a web app taht doesn't use Solr the time per request won't degradate that much. For me, it looks more like a synchronized is happening somewhere in Solr or Lucene and is causing this.-- View this message in context: http://lucene.472066.n3.nabble.com/Strange-performance-behaviour-when-concurrent-requests-are-done-tp505478p2878856.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
Ok, i try NGrams. My configuration looks like this: i try to run the query: http://localhost:8983/solr/terms?terms.fl=suggestionField&terms.prefix=chest Result: chest cheste chester The result is not what i expected. I think the query is not ok?..-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878877.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
you could use EdgeNGramFilterFactory : http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.EdgeNGramFilterFactory And you should mix front and back ngram process in your analyzer : is it better ? Ludovic. -Jouve France.-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878891.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
Hello , If you are using NGram then do not use TermsComponent, Query normally like http://localhost:8983/solr/select?q=suggestionField:chest It will give you the desired suggestions -Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878894.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
You can do it without NGram with a query like this : http://localhost:8983/solr/terms?terms=true&terms.fl=suggestionField&terms.regex=(.*)chest(.*)&terms.regex.flag=case_insensitive In my case, I had to encode (.*) so replace it with %28.*%29 if needed. It use a regex. I don't know if it has an impact on performance. 2011/4/29 lboutros > you could use EdgeNGramFilterFactory : > > > http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.EdgeNGramFilterFactory > > And you should mix front and back ngram process in your analyzer : > > maxGramSize="15" > side="front"/> > maxGramSize="15" > side="back"/> > > is it better ? > > Ludovic. > -Jouve > France.-- > View this message in context: > http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878891.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Quentin Proust Email : q.pro...@gmail.com Tel : 06.78.81.15.94 http://www.linkedin.com/in/quentinproust
Re: Autocomplete(terms) middle of words
The words are now splitted in the index(nGram). It looks like this: m ma man manc manch manche manches manchest mancheste manchester The termsComponent does not see it as one word(manchester). It gives me the results back in NGrams(m,ma,man etc)-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878916.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
terms.regex doesn´t work for me. Prefix works fine. I use SOLR 1.4.. Is it compatible?-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878948.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: manual background re-indexing
I'm a little confused about your use of the word "reload", see below. But if this doesn't clarify things, or your experiments give strange results, could you please start posting sample configurations so we can be sure we're talking about the same thing? Best Erick On Fri, Apr 29, 2011 at 5:15 AM, Paul Libbrecht wrote: > I understand that multicores allows met o create two cores which are just > normal solr-homes. That's easy. > > However I'm only interested to the reload command and, in particular, to > reload the underlying index segment files. > > Do I understand correctly that the following is what I should do? > - still make my setting multicore and get the core-admin requesthandler to > work, even with one core Great place to start. > - attempt the reload with a change of solrconfig or schema Should work find. A reload will cause that core to understand the changes to the schema > - do the reload of data by changing the index-segment-path in the config as > an example of the above Here's where I'm confused. Changing the index-segment-path in config to point where? I guess you *could* point at your old Solr index directory for, say, core1. But the normal path would be /cores/core1/. Under that directory you should have a bin, conf, and data just like you had under . Reloading doesn't have much to do with the actual contents of the .../data directory, if you relocate stuff you'd have to copy those three directories to the place that corresponds to the path in solrconfig.xml I've never had one core point to the directory and other cores point to the "normal" place, so I can't speak from experience but it seems to me that this should work. > > thanks to clarify > > paul > > > Le 29 avr. 2011 à 03:12, Erick Erickson a écrit : > >> You simply create two cores. One in solr/cores/core1 and another in >> solr/cores/core2 >> They each have a separate conf and data directory,and the index in in >> core#/data/index. >> >> Really, its' just introducing one more level. You can experiment just >> by configuring a core >> and copying your index to solr/cores/yourcore/data/index. After, of >> course, configuring >> Solr.xml to understand cores. > >
Re: Strange performance behaviour when concurrent requests are done
I'd guess that what's happening is that you're pegging the CPU with 4 threads, serving up the absolute maximum that the machine will handle. The thing to look at is the throughput rather than avg response time (although that's going down but not as rapidly, which is still a bit puzzling). Usually, adding more threads can increase throughput by allowing threads to steal some cycles while other threads block on, say, I/O. But if the CPU utilization is 100% adding more threads will not increase throughput, they just wait for each other. So your average response time includes the time threads are waiting, the context switching that Grijesh spoke of. So look at your machine with a performance monitor and see if your CPU is pegged. Or, more accurately, whether your CPU is pegged when 4 threads are active. Best Erick On Fri, Apr 29, 2011 at 7:43 AM, Marc Sturlese wrote: > That's true. But the degradation is so big. If you use lunch concurrent > requests to a web app taht doesn't use Solr the time per request won't > degradate that much. For me, it looks more like a synchronized is happening > somewhere in Solr or Lucene and is causing this.-- > View this message in context: > http://lucene.472066.n3.nabble.com/Strange-performance-behaviour-when-concurrent-requests-are-done-tp505478p2878856.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Autocomplete(terms) middle of words
hi I have question about regex terms. I try to find terms before and after word'ing but can't sand blank char. how can I send through ?? terms?terms=true&terms.fl=content&terms.regex=(.*)( book)&terms.regex.flag=case_insensitive&terms.limit=50-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2879192.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Can the Suggester be updated incrementally?
Good question, you could be correct about that. It's possible that part hasn't been built yet? If not then you could create a patch? On Thu, Apr 28, 2011 at 10:13 PM, Andy wrote: > > --- On Fri, 4/29/11, Jason Rutherglen wrote: > >> It's answered on the wiki site: >> >> "TSTLookup - ternary tree based representation, capable of >> immediate >> data structure updates" >> > > But how to update it? > > The wiki talks about getting data sources from a file or from the main index. > In either case it sounds like the entire data structure will be rebuilt, no? >
Re: Autocomplete(terms) middle of words
@roySolr : terms.regex exits from Solr 3.1. Doesn't seem compatible. @ramires : Did you try with space in your regex. Something like terms.regex=(.*) book (.*) <-- I put space before and after book. If it doesn't work, try to replace space with %20. I didn't try so I don't know if it work. 2011/4/29 roySolr > terms.regex doesn´t work for me. Prefix works fine. I use SOLR 1.4.. Is it > compatible?-- > View this message in context: > http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2878948.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Quentin Proust Email : q.pro...@gmail.com Tel : 06.78.81.15.94 http://www.linkedin.com/in/quentinproust
Re: Autocomplete(terms) middle of words
hi I tried before both %20 and " " terms it didn`t work. Also regex=(.*)(book) delete spaces and merge results like thebook asbook atbook songbook yearbook-- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2879375.html Sent from the Solr - User mailing list archive at Nabble.com.
Problem with Master
Hello, i'm having some issues with replication in my production environment. I have a master and 4 slaves. I had some data indexed and was replicated successfully. We are close to make the production environment public, so i deleted the old data deleting the data folder in the master Then i reloaded the master (from the manager of tomcat) hoping that the slaves will get updated with the new empty index. But when i enter to each slave in the replication page, i see that they have the old index. Even if i manually tell them to replicate, the replication count increases but there is no change in the indexed data. I checked the logs of the master and slaves and i see no error. I do see the /replication request reaching to the master. I put the SolrCore and ReplicationHandler log's level to "FINEST". Still nothing. I went agains the slave with the command=details and i saw a list of ReplicationList and FailedList. And the failed list indicating that the replication is failing. But i don't know why and i don't know where to look for the error. Thanks in advance, this is really urgent. PD: I hope this goes thru the spam filter...
Re: Problem with Master
Just to add more info... this is the result of a Replication / Command=Details I'm really confused by the masterDetails/indexSize being 52 byts (its correct), but the indexSize being 303.8 KB - - 0 15 - 303.8 KB D:\Solr\data\solr\index false true 1301331343628 3 - - 52 bytes D:\Solr\data\solr\index - - 1304086785516 1 - segments_1 true false 1304086785516 1 http://192.168.211.185:8787/solr/replication 00:00:60 Fri Apr 29 12:04:57 ART 2011 Fri Apr 29 12:03:57 ART 2011 - Fri Apr 29 12:03:57 ART 2011 Fri Apr 29 12:02:57 ART 2011 Fri Apr 29 12:01:57 ART 2011 Fri Apr 29 12:00:57 ART 2011 Fri Apr 29 11:59:57 ART 2011 Fri Apr 29 11:58:57 ART 2011 Fri Apr 29 11:57:57 ART 2011 Fri Apr 29 11:56:57 ART 2011 Fri Apr 29 11:55:57 ART 2011 Fri Apr 29 11:54:57 ART 2011 - Fri Apr 29 12:03:57 ART 2011 Fri Apr 29 12:02:57 ART 2011 Fri Apr 29 12:01:57 ART 2011 Fri Apr 29 12:00:57 ART 2011 Fri Apr 29 11:59:57 ART 2011 Fri Apr 29 11:58:57 ART 2011 Fri Apr 29 11:57:57 ART 2011 Fri Apr 29 11:56:57 ART 2011 Fri Apr 29 11:55:57 ART 2011 Fri Apr 29 11:54:57 ART 2011 44794 [solrconfig_slave.xml] 1 1301405968250 0 44792 Fri Apr 29 12:03:57 ART 2011 0 false false This response format is experimental. It is likely to change in the future. On Fri, Apr 29, 2011 at 11:52 AM, Ezequiel Calderara wrote: > Hello, i'm having some issues with replication in my production environment. > > I have a master and 4 slaves. > I had some data indexed and was replicated successfully. > > We are close to make the production environment public, so i deleted > the old data deleting the data folder in the master > Then i reloaded the master (from the manager of tomcat) hoping that > the slaves will get updated with the new empty index. > > But when i enter to each slave in the replication page, i see that > they have the old index. Even if i manually tell them to replicate, > the replication count increases but there is no change in the indexed > data. > > I checked the logs of the master and slaves and i see no error. I do > see the /replication request reaching to the master. > I put the SolrCore and ReplicationHandler log's level to "FINEST". > Still nothing. > > I went agains the slave with the command=details > and i saw a list of ReplicationList and FailedList. And the failed > list indicating that the replication is failing. But i don't know why > and i don't know where to look for the error. > > Thanks in advance, this is really urgent. > > > PD: I hope this goes thru the spam filter... > -- __ Ezequiel. Http://www.ironicnet.com
Re: SOLR startup problem
it seems jar files related to DataImport Handler are not being loaded in memory.Try to copy Data Import Handler related jars to solr_home/lib directory and restart the servlet container. -Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-startup-problem-tp2879789p2880022.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Autocomplete(terms) middle of words
solr-1.4 version does not support terms.regex .So you need to upgrade your version to solr-3.1. -Thanx: Grijesh www.gettinhahead.co.in -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-middle-of-words-tp2878694p2880040.html Sent from the Solr - User mailing list archive at Nabble.com.
querying in Java
Is there any way to query for data that is in any field, i.e. not using a specific field name? For example, when I use the following statements: SolrQuery query = new SolrQuery(); Query.setQuery("ANALYST:John Schummers"); QueryResponse rsp = server.query(query); I get the documents I'm looking for. But I would like to get the same set of documents without using the specific ANALYST field name. I have tried using just "Schummers" as the query, but no documents are returned. The ANALYST field is an indexed field. This message and any enclosures are intended only for the addressee. Please notify the sender by email if you are not the intended recipient. If you are not the intended recipient, you may not use, copy, disclose, or distribute this message or its contents or enclosures to any other person and any such actions may be unlawful. Ball reserves the right to monitor and review all messages and enclosures sent to or from this email address.
Re: querying in Java
Hi Jeff, In that case, it will query w.r.t default field. What is your default search field in the schema? Regards, Anuj On Fri, Apr 29, 2011 at 11:10 PM, Saler, Jeff wrote: > Is there any way to query for data that is in any field, i.e. not using > a specific field name? > > > > For example, when I use the following statements: > > > >SolrQuery query = new SolrQuery(); > >Query.setQuery("ANALYST:John Schummers"); > > QueryResponse rsp = server.query(query); > > > > > > I get the documents I'm looking for. > > > > But I would like to get the same set of documents without using the > specific ANALYST field name. > > I have tried using just "Schummers" as the query, but no documents are > returned. > > The ANALYST field is an indexed field. > > > > > > > This message and any enclosures are intended only for the addressee. > Please > notify the sender by email if you are not the intended recipient. If you > are > not the intended recipient, you may not use, copy, disclose, or distribute > this > message or its contents or enclosures to any other person and any such > actions > may be unlawful. Ball reserves the right to monitor and review all > messages > and enclosures sent to or from this email address.
RE: querying in Java
Thanks for the reply. What I want is for the query to search all fields for the specified value. -Original Message- From: Anuj Kumar [mailto:anujs...@gmail.com] Sent: Friday, April 29, 2011 1:51 PM To: solr-user@lucene.apache.org Subject: Re: querying in Java Hi Jeff, In that case, it will query w.r.t default field. What is your default search field in the schema? Regards, Anuj On Fri, Apr 29, 2011 at 11:10 PM, Saler, Jeff wrote: > Is there any way to query for data that is in any field, i.e. not using > a specific field name? > > > > For example, when I use the following statements: > > > >SolrQuery query = new SolrQuery(); > >Query.setQuery("ANALYST:John Schummers"); > > QueryResponse rsp = server.query(query); > > > > > > I get the documents I'm looking for. > > > > But I would like to get the same set of documents without using the > specific ANALYST field name. > > I have tried using just "Schummers" as the query, but no documents are > returned. > > The ANALYST field is an indexed field. > > > > > > > This message and any enclosures are intended only for the addressee. > Please > notify the sender by email if you are not the intended recipient. If you > are > not the intended recipient, you may not use, copy, disclose, or distribute > this > message or its contents or enclosures to any other person and any such > actions > may be unlawful. Ball reserves the right to monitor and review all > messages > and enclosures sent to or from this email address. This message and any enclosures are intended only for the addressee. Please notify the sender by email if you are not the intended recipient. If you are not the intended recipient, you may not use, copy, disclose, or distribute this message or its contents or enclosures to any other person and any such actions may be unlawful. Ball reserves the right to monitor and review all messages and enclosures sent to or from this email address.
Re: querying in Java
copyField should do the trick: http://wiki.apache.org/solr/SchemaXml#Copy_Fields "A common requirement is to copy or merge all input fields into a single solr field. This can be done as follows:- " hth, rob On Fri, Apr 29, 2011 at 2:06 PM, Saler, Jeff wrote: > Thanks for the reply. What I want is for the query to search all fields > for the specified value. > > -Original Message- > From: Anuj Kumar [mailto:anujs...@gmail.com] > Sent: Friday, April 29, 2011 1:51 PM > To: solr-user@lucene.apache.org > Subject: Re: querying in Java > > Hi Jeff, > > In that case, it will query w.r.t default field. What is your default > search > field in the schema? > > Regards, > Anuj > > On Fri, Apr 29, 2011 at 11:10 PM, Saler, Jeff wrote: > >> Is there any way to query for data that is in any field, i.e. not > using >> a specific field name? >> >> >> >> For example, when I use the following statements: >> >> >> >> SolrQuery query = new SolrQuery(); >> >> Query.setQuery("ANALYST:John Schummers"); >> >> QueryResponse rsp = server.query(query); >> >> >> >> >> >> I get the documents I'm looking for. >> >> >> >> But I would like to get the same set of documents without using the >> specific ANALYST field name. >> >> I have tried using just "Schummers" as the query, but no documents are >> returned. >> >> The ANALYST field is an indexed field. >> >> >> >> >> >> >> This message and any enclosures are intended only for the addressee. >> Please >> notify the sender by email if you are not the intended recipient. If > you >> are >> not the intended recipient, you may not use, copy, disclose, or > distribute >> this >> message or its contents or enclosures to any other person and any such >> actions >> may be unlawful. Ball reserves the right to monitor and review all >> messages >> and enclosures sent to or from this email address. > > > > This message and any enclosures are intended only for the addressee. Please > notify the sender by email if you are not the intended recipient. If you are > not the intended recipient, you may not use, copy, disclose, or distribute > this > message or its contents or enclosures to any other person and any such actions > may be unlawful. Ball reserves the right to monitor and review all messages > and enclosures sent to or from this email address. >
Re: querying in Java
Hi Jeff, In that case, you can create a new index field (set indexed to true and stored to false) and copy all your fields to it using copyField. Also make this new field as your default search field. This will handle your case. Regards, Anuj On Fri, Apr 29, 2011 at 11:36 PM, Saler, Jeff wrote: > Thanks for the reply. What I want is for the query to search all fields > for the specified value. > > -Original Message- > From: Anuj Kumar [mailto:anujs...@gmail.com] > Sent: Friday, April 29, 2011 1:51 PM > To: solr-user@lucene.apache.org > Subject: Re: querying in Java > > Hi Jeff, > > In that case, it will query w.r.t default field. What is your default > search > field in the schema? > > Regards, > Anuj > > On Fri, Apr 29, 2011 at 11:10 PM, Saler, Jeff wrote: > > > Is there any way to query for data that is in any field, i.e. not > using > > a specific field name? > > > > > > > > For example, when I use the following statements: > > > > > > > >SolrQuery query = new SolrQuery(); > > > >Query.setQuery("ANALYST:John Schummers"); > > > > QueryResponse rsp = server.query(query); > > > > > > > > > > > > I get the documents I'm looking for. > > > > > > > > But I would like to get the same set of documents without using the > > specific ANALYST field name. > > > > I have tried using just "Schummers" as the query, but no documents are > > returned. > > > > The ANALYST field is an indexed field. > > > > > > > > > > > > > > This message and any enclosures are intended only for the addressee. > > Please > > notify the sender by email if you are not the intended recipient. If > you > > are > > not the intended recipient, you may not use, copy, disclose, or > distribute > > this > > message or its contents or enclosures to any other person and any such > > actions > > may be unlawful. Ball reserves the right to monitor and review all > > messages > > and enclosures sent to or from this email address. > > > > This message and any enclosures are intended only for the addressee. > Please > notify the sender by email if you are not the intended recipient. If you > are > not the intended recipient, you may not use, copy, disclose, or distribute > this > message or its contents or enclosures to any other person and any such > actions > may be unlawful. Ball reserves the right to monitor and review all > messages > and enclosures sent to or from this email address. >
Baseline vs. Incremental Indexing
Hi, Currently we index new/updated records every 30 minutes (I am referring to this as incremental/partial index) -- i.e., records will be added to an existing index. Are there any benefits in creating a new index (i.e., delete the existing index and create it) from a performance point of view everyday or a week? In other search system I worked with, incremental updates are generated in a small file. When the server is restarted, each update in the small files need to be applied. When there are several small files to be applied, the restart process could take a few minutes to complete so the recommendation was to run baseline process every night. I'm wondering if this is the case with Solr as well? Thanks, Satish
Re: Baseline vs. Incremental Indexing
Hi Satish, I can't think of any benefits you'd reap by complete/full reindexing into a new index. Incremental indexing will be faster. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: Satish Kumar > To: solr-user@lucene.apache.org > Sent: Fri, April 29, 2011 2:58:25 PM > Subject: Baseline vs. Incremental Indexing > > Hi, > > Currently we index new/updated records every 30 minutes (I am referring to > this as incremental/partial index) -- i.e., records will be added to an > existing index. Are there any benefits in creating a new index (i.e., delete > the existing index and create it) from a performance point of view everyday > or a week? > > In other search system I worked with, incremental updates are generated in a > small file. When the server is restarted, each update in the small files > need to be applied. When there are several small files to be applied, the > restart process could take a few minutes to complete so the recommendation > was to run baseline process every night. I'm wondering if this is the case > with Solr as well? > > > Thanks, > Satish >
Re: Baseline vs. Incremental Indexing
The only thing you'd periodically do is optimizing the existing index. > Hi Satish, > > I can't think of any benefits you'd reap by complete/full reindexing into a > new index. Incremental indexing will be faster. > > > Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > - Original Message > > > From: Satish Kumar > > To: solr-user@lucene.apache.org > > Sent: Fri, April 29, 2011 2:58:25 PM > > Subject: Baseline vs. Incremental Indexing > > > > Hi, > > > > Currently we index new/updated records every 30 minutes (I am referring > > to this as incremental/partial index) -- i.e., records will be added to > > an existing index. Are there any benefits in creating a new index > > (i.e., delete the existing index and create it) from a performance point > > of view everyday or a week? > > > > In other search system I worked with, incremental updates are generated > > in a small file. When the server is restarted, each update in the small > > files need to be applied. When there are several small files to be > > applied, the restart process could take a few minutes to complete so > > the recommendation was to run baseline process every night. I'm > > wondering if this is the case with Solr as well? > > > > > > Thanks, > > Satish
Re: Baseline vs. Incremental Indexing
thanks Markus and Otis! This link was helpful: http://wiki.apache.org/solr/SolrPerformanceFactors#Optimization_Considerations On Fri, Apr 29, 2011 at 3:12 PM, Markus Jelsma wrote: > The only thing you'd periodically do is optimizing the existing index. > > > Hi Satish, > > > > I can't think of any benefits you'd reap by complete/full reindexing into > a > > new index. Incremental indexing will be faster. > > > > > > Otis > > > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > > Lucene ecosystem search :: http://search-lucene.com/ > > > > > > > > - Original Message > > > > > From: Satish Kumar > > > To: solr-user@lucene.apache.org > > > Sent: Fri, April 29, 2011 2:58:25 PM > > > Subject: Baseline vs. Incremental Indexing > > > > > > Hi, > > > > > > Currently we index new/updated records every 30 minutes (I am > referring > > > to this as incremental/partial index) -- i.e., records will be added > to > > > an existing index. Are there any benefits in creating a new index > > > (i.e., delete the existing index and create it) from a performance > point > > > of view everyday or a week? > > > > > > In other search system I worked with, incremental updates are > generated > > > in a small file. When the server is restarted, each update in the > small > > > files need to be applied. When there are several small files to be > > > applied, the restart process could take a few minutes to complete so > > > the recommendation was to run baseline process every night. I'm > > > wondering if this is the case with Solr as well? > > > > > > > > > Thanks, > > > Satish >