Re: Advice needed on master-slave configuration
It appears that the only solution (outside of Noble Paul's suggestion for using solr's replication handler) is for me to restart tomcat.To minimize the effect of this downtime, I propose to do the following: a) Generate the new index on the master periodically. When the new index has been created, the apps that do the search, will now be made to point their queries at this master. b) Because queries are now going to the master, tomcat on the slave can be stopped, the index file copied over and tomcat is restarted. c) The apps that do search will be made to point their queries back at the slave. Does anyone see a better approach? Also, when will solr's replication handler release in an official release? Can it be released as a patch on 1.3? It is terribly useful functionality and if there's a way to get it out sooner, I'd sure appreciate it! Thanks, - Bill -- From: Noble Paul ??? ?? [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 10:51 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration If you are using a nightly you can try the new SolrReplication feature http://wiki.apache.org/solr/SolrReplication On Thu, Oct 23, 2008 at 4:32 AM, William Pierce [EMAIL PROTECTED] wrote: Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill -- --Noble Paul
Re: Advice needed on master-slave configuration
I tried the nightly build from 10/18 -- I did the following: a) I downloaded the nightly build of 10/18 (the zip file). b) I unpacked it and copied the war file to my tomcat lib folder. c) I made the relevant changes in the config files per the instructions shown in the wiki. When tomcat starts, I see the error message in tomcat logs... Caused by: java.lang.ClassNotFoundException: solr.ReplicationHandler at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1358) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1204) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:258) ... 36 more Where do I get the nightly bits that will enable me to try this replication handler? Thanks, - Bill -- From: Noble Paul ??? ?? [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 10:51 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration If you are using a nightly you can try the new SolrReplication feature http://wiki.apache.org/solr/SolrReplication On Thu, Oct 23, 2008 at 4:32 AM, William Pierce [EMAIL PROTECTED] wrote: Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill -- --Noble Paul
Re: Advice needed on master-slave configuration
It was committed on 10/21 take the latest 10/23 build http://people.apache.org/builds/lucene/solr/nightly/solr-2008-10-23.zip On Fri, Oct 24, 2008 at 2:27 AM, William Pierce [EMAIL PROTECTED] wrote: I tried the nightly build from 10/18 -- I did the following: a) I downloaded the nightly build of 10/18 (the zip file). b) I unpacked it and copied the war file to my tomcat lib folder. c) I made the relevant changes in the config files per the instructions shown in the wiki. When tomcat starts, I see the error message in tomcat logs... Caused by: java.lang.ClassNotFoundException: solr.ReplicationHandler at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1358) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1204) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:258) ... 36 more Where do I get the nightly bits that will enable me to try this replication handler? Thanks, - Bill -- From: Noble Paul ??? ?? [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 10:51 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration If you are using a nightly you can try the new SolrReplication feature http://wiki.apache.org/solr/SolrReplication On Thu, Oct 23, 2008 at 4:32 AM, William Pierce [EMAIL PROTECTED] wrote: Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill -- --Noble Paul -- --Noble Paul
Re: Advice needed on master-slave configuration
Also, when will solr's replication handler release in an official release? Can it be released as a patch on 1.3? It is terribly useful functionality and if there's a way to get it out sooner, I'd sure appreciate it! It is a possiblity.You can raise a JIRA issue The feature depends on some other core changes which also needs to be included in the patch. Moreover, the logging is changed from java logging to slf4j Thanks, - Bill -- From: Noble Paul ??? ?? [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 10:51 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration If you are using a nightly you can try the new SolrReplication feature http://wiki.apache.org/solr/SolrReplication On Thu, Oct 23, 2008 at 4:32 AM, William Pierce [EMAIL PROTECTED] wrote: Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill -- --Noble Paul -- --Noble Paul
Re: Advice needed on master-slave configuration
Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill
Re: Advice needed on master-slave configuration
If you are using a nightly you can try the new SolrReplication feature http://wiki.apache.org/solr/SolrReplication On Thu, Oct 23, 2008 at 4:32 AM, William Pierce [EMAIL PROTECTED] wrote: Otis, Yes, I had forgotten that Windows will not permit me to overwrite files currently in use. So my copy scripts are failing. Windows will not even allow a rename of a folder containing a file in use so I am not sure how to do this I am going to dig around and see what I can come up with short of stopping/restarting tomcat... Thanks, - Bill -- From: Otis Gospodnetic [EMAIL PROTECTED] Sent: Wednesday, October 22, 2008 2:30 PM To: solr-user@lucene.apache.org Subject: Re: Advice needed on master-slave configuration Normally you don't have to start Q, but only reload Solr searcher when the index has been copied. However, you are on Windows, and its FS has the tendency not to let you delete/overwrite files that another app (Solr/java) has opened. Are you able to copy the index from U to Q? How are you doing it? Are you deleting index files from the index dir on Q that are no longer in the index dir on U? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: William Pierce [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Wednesday, October 22, 2008 5:24:28 PM Subject: Advice needed on master-slave configuration Folks: I have two instances of solr running one on the master (U) and the other on the slave (Q). Q is used for queries only, while U is where updates/deletes are done. I am running on Windows so unfortunately I cannot use the distribution scripts. Every N hours when changes are committed and the index on U is updated, I want to copy the files from the master to the slave.Do I need to halt the solr server on Q while the index is being updated? If not, how do I copy the files into the data folder while the server is running? Any pointers would be greatly appreciated! Thanks! - Bill -- --Noble Paul
