Source: xapian-core Version: 1.4.18-3 Severity: critical Tags: patch upstream Justification: causes serious data loss Control: fixed -1 1.4.22-1
Xapian database corruption on disk full is possible. It doesn't happen in every case as ENOSPC needs to happen on a particular operation during the commit but then not happen on a repeat attempt at that operation. Filing systems with more complex freespace handling may be more susceptible to this (it was reported on btrfs). Only glass databases are affected - the commit code for the older chert format is significantly different. I've filed this with severity based on the worst case where the data in the Xapian database is not simply an index of data from elsewhere. This is the situation for user-applied tags in notmuch, for example. In many cases the Xapian database is an index of data stored elsewhere so can be recreated from scratch (albeit this may take some time). This was fixed upstream in 1.4.22, which is the version now in unstable and testing. It looks like all previous 1.4.x releases are affected. I'm intending to submit a stable update for this, and will see if the LTS team are interested too. The upstream patch is a single line change: https://git.xapian.org/?p=xapian;a=commitdiff;h=9f9aad17893bde4acb3a98e60dde397c346fcd9a Details from upstream commit message: If renaming to switch the new version file live fails (e.g. due to ENOSPC) we discard the changes, try to write and switch to a different new version file with an increased revision (on failure of this too we close the database), and throw DatabaseError. Unfortunately the roll-back of state is not complete, and if switching to the different new version file succeeds that bad state persists on disk. Thanks to Uwe Kleine-König for reporting upstream and coming up with the idea to reproduce using strace to inject a rename() failure - this is a simple reproducer: rm -rf enospc.db strace -e inject=rename:error=ENOSPC:when=2 examples/simpleindex enospc.db < INSTALL xapian-check enospc.db Cheers, Olly
