Hi Amin, I am also working on similar implementation. But right now I am working on building Lucene Index in parallel using Hadoop.
How are you building your Lucene index (or indexes) ? Have you tried Hadoop? If yes, we can share some design issues. Also, for serving Lucene indexes over cluster, you might be interested in Katta project- katta.sourceforge.net -Tarandeep On Thu, Jun 11, 2009 at 1:04 AM, Amin Mohammed-Coleman <ami...@gmail.com>wrote: > Hi > > we are thinking of implementing a solution like this. All nodes on the > cluster has a local index from which searches are performed. Once an > update > on the index needs to happen the work is sent to a queue where there is a > master who is responsible for updating the central index. The master is > then responsible for propagating/notifying other nodes in the cluster that > their local index is not in sync. The local index worker can perform an > update against the master index. > > We haven't implemented this yet but its the approach we'll take inorder to > deal with clustering. If anyone has any ideas which could improve this or > a > better approach your thoughts will be appreciated. > > Cheers > Amin > > On Wed, Jun 10, 2009 at 10:15 AM, Ian Lea <ian....@gmail.com> wrote: > > > I'd recommend using your favourite queueing service to pass all > > updates to a central process, the one and only process that updates > > the index. If you don't already have a favourite queueing service, > > > http://en.wikipedia.org/wiki/Java_Message_Service#Provider_implementations > > lists several JMS implementations. > > > > > > -- > > Ian. > > > > On Tue, Jun 9, 2009 at 9:02 PM, Newman, Billy<billy.new...@itt.com> > wrote: > > > I am trying to figure out the best way to add to a lucene index across > a > > clustered app server. I cannot grab an IndexWriter for each node in the > > cluster, because I would run into lock file problems. I am not sure if I > > can share one IndexWriter across the cluster because what happens when > two > > or more nodes update the same thing. Anyone have any insight? > > > > > > Thanks. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > >
