[ https://issues.apache.org/jira/browse/LUCENE-5580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrien Grand updated LUCENE-5580: --------------------------------- Attachment: LUCENE-5580.patch Here is a patch that verifies checksums on stored fields when doing bulk merges. > Always verify stored fields' checksum on merge > ---------------------------------------------- > > Key: LUCENE-5580 > URL: https://issues.apache.org/jira/browse/LUCENE-5580 > Project: Lucene - Core > Issue Type: Improvement > Reporter: Adrien Grand > Assignee: Adrien Grand > Fix For: 4.8 > > Attachments: LUCENE-5580.patch > > > I have seen a couple of index corruptions over the last months, and most of > them happened on stored fields. The explanation might just be that since > stored fields are usually most of the index size, they are just more likely > to be corrupted due to a hardware/operating-system failure, but it might be > as well a sneaky bug on our side. > Lucene recently added checksums to index files, and you can enable integrity > verification upon merge, but this comes with a cost since you need to read > all index files twice instead of once. If you are merging a very large > segment and your merges are I/O-bound, this might be noticeable. > I would like to implement integrity checks for stored fields on merges on the > fly, so that the stored fields files need to be read only once. -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org