Hi, we are using a very cautious method for batch upating.
We have long (hours) running updates on our index, but complete reindexing would even be longer (days). But I guess our strategy could be scaled down to hours or even less. So what we do is, we keep two instances of the index. There is a file which contains a link to the currently used index for reading. The search application accesses the index through a thin transparent API which whatches this file and can switch from one search request to the next, but does keep the IndexReader open as long as no switching is need. When we reindex, we duplicate the "read" instance of the index and then do an selective update on this copied duplicate. Note we need some HD space and time for this. After indexing there is some checking and comparison between old and new instance of the index. Only if this looks successful we toggle the above "pointer" file and the transparent API switches to the new index. In case human recognition still finds the checked index bad, we can still switch back to the second last presumably "good" index. Cheers, Frank >-----Original Message----- >From: Lee Turner [mailto:[EMAIL PROTECTED] >Sent: Tuesday, April 05, 2005 9:38 AM >To: java-user@lucene.apache.org >Subject: Strategies for updating indexes. > > >Hi > > > >I was wondering whether anyone has any experience of >multithreaded updates to indexes. I the web app I am working >on there are additions, updates and deletes that need to >happen to the index throughout the runtime of the application. > Also, the application is run in a cluster with each app >server having its own index. This means that periodically >each app server is going to have to go through a re-indexing >process to make sure that its index has all the changes from >the other app servers in it. This process can take a few >seconds so if another update to the index occurs at this time >it will need to be queued in some way to make sure it happens >after the re-indexing. > > > >I was just wondering if anyone had any pointers for doing this >kind of thing. Any help would be gratefully appreciated. > > > >Many thanks > >Lee > > > > > >Lee Turner | Java Developer | Oyster Partners > >D. +44 (0)20 74461229 >T. +44 (0)20 7446 7500 > >www.oyster.com > > > > >_______________________________________________________________ >__________________________________________________________ >Internet communications are not secure and therefore Oyster >Partners Ltd does not accept legal responsibility for the >contents of this message. Any views or opinions presented are >solely those of the author and do not necessarily represent >those of Oyster Partners Ltd. > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
