Re: master/slave failure scenario

2009-05-21 Thread Bryan Talbot
hange the master make the VIP point to the new host On Wed, May 13, 2009 at 10:52 PM, nk 11 1...@gmail.com> wrote: This is more interesting.Such a procedure would involve taking down and reconfiguring the slave? On Wed, May 13, 2009 at 7:55 PM, Bryan Talbot wrote: Or ... 1. Promote existin

Re: Howto? Applying a filter across schema fileds using state information

2009-05-18 Thread Bryan Talbot
I needed to do something like this recently as well. I needed to copy a date field (with full precision to the millisecond) to a string field of just MMDD. I didn't see a way to do it in solr core. I ended up doing it in the Data Import Handler during import. I'd rather have code li

Re: replication of lucene-write.lock file

2009-05-15 Thread Bryan Talbot
guess we must ignore the .lock file if it is returned in the list of files. you can raise an issue and we can fix it. --Noble On Fri, May 15, 2009 at 12:38 AM, Bryan Talbot wrote: When using solr 1.4 replication, I see that the lucene-write.lock file is being replicated to slaves. I&#

replication of lucene-write.lock file

2009-05-14 Thread Bryan Talbot
When using solr 1.4 replication, I see that the lucene-write.lock file is being replicated to slaves. I'm importing data from a db every 5 minutes using cron to trigger a DIH delta-import. Replication polls every 60 seconds and the master is configured to take a snapshot (replicateAfter

Re: Replication master+slave

2009-05-14 Thread Bryan Talbot
Nutch - Original Message From: Bryan Talbot To: solr-user@lucene.apache.org Sent: Wednesday, May 13, 2009 10:11:21 PM Subject: Re: Replication master+slave I think the patch I included earlier covers solr core, but it looks like at least some other extensions (DIH) create and use their ow

Re: Replication master+slave

2009-05-13 Thread Bryan Talbot
cene - Solr - Nutch ----- Original Message From: Bryan Talbot To: solr-user@lucene.apache.org Sent: Wednesday, May 13, 2009 11:26:41 AM Subject: Re: Replication master+slave I see that Nobel's final comment in SOLR-1154 is that config files need to be able to include snippets from

Re: master/slave failure scenario

2009-05-13 Thread Bryan Talbot
Or ... 1. Promote existing slave to new master 2. Add new slave to cluster -Bryan On May 13, 2009, at May 13, 9:48 AM, Jay Hill wrote: - Migrate configuration files from old master (or backup) to new master. - Replicate from a slave to the new master. - Resume indexing to new master.

Re: Replication master+slave

2009-05-13 Thread Bryan Talbot
w3.org/2001/XInclude";> -Bryan On May 12, 2009, at May 12, 8:05 PM, Jian Han Guo wrote: I was looking at the same problem, and had a discussion with Noble. You can use a hack to achieve what you want, see https://issues.apache.org/jira/browse/SOLR-1154 Thanks, Jianhan On Tue,

Re: Replication master+slave

2009-05-12 Thread Bryan Talbot
entBuilders would make it possible to modularize configuration files to make this possible? http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#setXIncludeAware(boolean) -Bryan On May 12, 2009, at May 12, 11:43 AM, Shalin Shekhar Mangar wrote: On Tue, May 12

Replication master+slave

2009-05-12 Thread Bryan Talbot
For replication in 1.4, the wiki at http://wiki.apache.org/solr/SolrReplication says that a node can be both the master and a slave: A node can act as both master and slave. In that case both the master and slave configuration lists need to be present inside the ReplicationHandler requestHa

Re: Importing data from Sybase

2009-04-28 Thread Bryan Talbot
Holdability is a JDBC 3.0 feature -- maybe that database driver is old and doesn't support JDBC 3.0? I see from the stack trace that jdbc2 is in the package name ... -Bryan On Apr 28, 2009, at Apr 28, 12:32 PM, ahammad wrote: Hello, I'm trying to index data from a Sybase DB, but w

Re: Garbage Collectors

2009-04-16 Thread Bryan Talbot
If you're using java 5 or 6 jmap is a useful tool in tracking down memory leaks. http://java.sun.com/javase/6/docs/technotes/tools/share/jmap.html jmap -histo:live will print a histogram of all live objects in the heap. Start at the top and work your way down until you find something susp

Re: DataImporter : Java heap space

2009-04-15 Thread Bryan Talbot
I think there is a bug in the 1.4 daily builds of data import handler which is causing the batchSize parameter to be ignored. This was probably introduced with more recent patches to resolve variables. The affected code is in JdbcDataSource.java String bsz = initProps.getProperty("batch

Re: Changing to multicore

2009-01-28 Thread Bryan Talbot
I would think that using a servlet filter to rewrite the URL should be pretty strait forward. You could write your own or use a tool like http://tuckey.org/urlrewrite/ and just configure that. Using something like this, I think the upgrade procedure could be: - install rewrite filter to rewr

Re: Help with Solr 1.3 lockups?

2009-01-16 Thread Bryan Talbot
I think it's pretty easy to check if SOLR is alive. Even from a shell script, a simple command like curl -iIs --url "http://solrhost/solr/select?start=0&rows=0"; | grep -c "HTTP/1.1 200 OK" will return 1 if the response is an HTTP 200. If the return is not 1, then there is a problem. A

Re: Unicode characters that are not legal XML characters

2008-12-23 Thread Bryan Talbot
I believe you can use the following unicode characters in XML documents: U+0009, U+000A, U+000D, [U+0020-U+D7FF], [U+E000-U+FFFD], and [U+1-U+10] One of your documents contains a U0022 character which is an invalid space character for XML. http://www.unicode.org/unicode/reports/tr

Re: Solr - DataImportHandler - Large Dataset results ?

2008-12-12 Thread Bryan Talbot
It only supports streaming if properly enabled which is completely lame: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-implementation-notes.html By default, ResultSets are completely retrieved and stored in memory. In most cases this is the most efficient way to operate, and