Re: Distributed Indexes

2008-02-11 Thread Ruslan Sivak
Basically the index is big is because there is a large number of documents, but each individual document is very small. There is also a lot of redundancy, which, I believe is also why the index size is fairly small. Basically I am using the index to store the user's profile information, and

Re: Distributed Indexes

2008-02-11 Thread Ruslan Sivak
Cedric Ho wrote: On Feb 9, 2008 12:07 AM, Ruslan Sivak <[EMAIL PROTECTED]> wrote: The app does other things then search the index. I'm basically using ColdFusion for the website and have four instances running on two servers for load balancing. Each app does the searches, and the search tim

Re: Distributed Indexes

2008-02-11 Thread Grant Ingersoll
Solr has a strategy using rsync that makes it relatively easy to copy an index around to other servers. It uses rsync to just copy the diffs, so you could easily mirror this in your application. There is no SQL backend for Lucene, but at 4mb you could certainly serialize it as a blob to a

Re: Distributed Indexes

2008-02-10 Thread Cedric Ho
On Feb 9, 2008 12:07 AM, Ruslan Sivak <[EMAIL PROTECTED]> wrote: > The app does other things then search the index. I'm basically using > ColdFusion for the website and have four instances running on two > servers for load balancing. Each app does the searches, and the search > times are small, t

Re: Distributed Indexes

2008-02-10 Thread Ruslan Sivak
So nobody's run into anything like this before? The need to share the index between many copies of the app possibly running on multiple servers? Russ Ruslan Sivak wrote: The app does other things then search the index. I'm basically using ColdFusion for the website and have four instances ru

Re: Distributed Indexes

2008-02-08 Thread Ruslan Sivak
The app does other things then search the index. I'm basically using ColdFusion for the website and have four instances running on two servers for load balancing. Each app does the searches, and the search times are small, the index is small, but it takes a long time to fully create the index

Re: Distributed Indexes

2008-02-07 Thread Erick Erickson
With an index that small, I wonder why you bother with so many copies? What kind of load are you hitting it with and how complex are the queries? Because unless you have *very* high query rate, I'd look at why my queries were taking so long before complexifying things this way. Best Erick On Feb

Re: Distributed Indexes

2008-02-07 Thread Ruslan Sivak
My index is only 4mb. Is there a SQL backend for Lucene? Russ Michael McCandless wrote: If you're able to tell Windows FRS which specific files to copy, then SnapshotDeletionPolicy (in 2.3) should work for this. It basically protects a consistent snapshot of your index, ensuring those fil

Re: Distributed Indexes

2008-02-07 Thread Ruslan Sivak
No, FRS copies the whole directory. It's fairly fast, but if there is a modification on both servers at the same time, there will be issues. Russ Michael McCandless wrote: If you're able to tell Windows FRS which specific files to copy, then SnapshotDeletionPolicy (in 2.3) should work for

Re: Distributed Indexes

2008-02-07 Thread Michael McCandless
If you're able to tell Windows FRS which specific files to copy, then SnapshotDeletionPolicy (in 2.3) should work for this. It basically protects a consistent snapshot of your index, ensuring those files will not be deleted, while not blocking further updates to the index. Mike Ruslan

Re: Distributed Indexes.

2007-10-29 Thread Otis Gospodnetic
Are you looking for something a la http://wiki.apache.org/solr/CollectionDistribution ? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Monday, October 29, 2007 4:05: