Doug wrote: <<The safest approach is to remove files not mentioned in the segments file and update the index with the missing documents. How does your application recover if it crashes during an update?>>
There are around 20 million documents in the orphaned segments, so it would take a very long time to update the index. Is there an "unsafe" way to edit the segments file to add these back? It seems like the missing piece of information I need to do this is the correct segment size -- where can I find that? My application doesn't really have any recovery method if it crashes. Can you tell me what the proper error handling procedure is? If, in fact, these segments were corrupted because the application crashed, what could I have done programmatically to recover once that had happened? -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Thursday, December 23, 2004 1:34 PM To: Lucene Users List Subject: Re: CFS file and file formats Steve Rajavuori wrote: > 1) First of all, there are both CFS files and standard (non-compound) files > in this directory, and all of them have recent update dates, so I assume > they are all being used. My code never explicitly sets the compound file > flag, so I don't know how this happened. This can happen if your application crashes while the index was being updated. In this case these were never entered into the segments file and may be partially written. > 2) Is there a way to force all files into compound mode? For example, if I > set the compound setting, then call optimize, will that recreate everything > into the CFS format? It should. Except, on Windows not all old CFS file will be deleted immediately, but may instead be listed in the 'deleteable' file for a while. > 3) There are several other large .CFS files in this directory that I think > have somehow become detached from the index. They have recent update dates > -- however, the last time I ran optimize these were not touched, and they > are not being updated now. I know these segments have valid data, because > now when I search I am missing large chunks of data -- which I assume is in > these detached segments. So my thought is to edit the 'segments' file to > make Lucene recognize these again -- but I need to know the correct segment > size in order to do this. So how do I determine what the correct segment > size should be? These could also be the result of crashes. In this case they may be partially written. The safest approach is to remove files not mentioned in the segments file and update the index with the missing documents. How does your application recover if it crashes during an update? Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
