Hi Everybody... First of all let me start of by saying that I am very grateful to all ZF contributors.
While testing Zend_Search_Lucene for production use I have found what I believe is a major bug in concurrency: If two threads are updating the index at the same time the index is easily corrupted and unusable. Some specific situations that cause Exceptions + index corruption: - two different threads adding and/or deleting different documents at the same time - calling the $index->optimize() command while another thread is adding and/or deleting documents. An exception thrown, usually something like: 'Zend_Search_Lucene_Exception' with message 'File '[some_index_file_in_the_data_directory]' is not readable.'... The index remains unusable/corrupted for all future requests. I am not familiar with the deep inner workings of Zend_Search_Lucene but I believe there to be a bug in how flock is being used by Zend/Search/Lucene/Storage/File/Filesystem.php I have tested the latest code from SVN with no luck (and the code from release 1.03 fails for different bugs/reasons). I know some of you may suggest that my filesystem or kernel may not support the flock feature - but I checked that as well. PHP's flock() function works as expected (tested on a fresh standard Fedora Core 4). It should also be noted that certain aspects of the locking *do* work: - if a single thread is writing (updating) the index, and other threads are just reading (searching) the index, the writing/reading threads seem to block each other properly, without any problems. I would hate to see developers affected by this major (IMHO) problem. Especially if your indexes are large and take several minutes or hours to rebuild. To help the ZF contributors I can post a working example that demonstrates all of the above. Thanks - DM -- View this message in context: http://www.nabble.com/Warning%3A-Significant-issues-with-Zend_Search_Lucene-%28index-corruption-is-easy...%29-tp14994669s16154p14994669.html Sent from the Zend Framework mailing list archive at Nabble.com.
