Hello, This is the series I mentioned with the trunc_indirect regression fix: it closes the window in which a freed block could be reused before the transaction that freed it commits.
ext2_free_blocks clears the bitmap bit at once, so the allocator can hand a block to a new owner while the journal still holds copies of its old contents, in the freeing transaction, in the committing one, or in older checkpoint transactions. A home write of such a copy then lands on the new owner's data, and a crash before the free commits gives the block back to its old owner after it was overwritten. The pre-notify regression was one way into this window; this series closes the window itself. Patch 1 does what ext3 and ext4 do: a freed block is allocated again only after its free commits. The bitmap and free counts still change at free time; the block is also marked in an in-memory busy bitmap, which ext2_new_block skips, and the commit hands the blocks back once the old copies are marked written. Without a journal nothing changes: freed blocks are reusable at once, as before. Patch 2 removes a workaround that this makes unnecessary. The orphan list cleared i_size, i_blocks and i_block[] of an orphan inode on disk, so that a reused block could not be reached through its block map. That leaked the blocks of every file still open at shutdown until a full fsck, since recovery and e2fsck saw an orphan with no blocks. Like ext3 and ext4, the block map now stays on disk and only i_dtime is used for the list. Unjournaled paths are not affected by this change. Known limitations, also noted as XXX comments in ext2_new_block: - When every free block is busy, ext2_new_block waits for the committing transaction to hand its blocks back. The caller can hold node locks there, such as alloc_lock, so a pager thread that joined that transaction and waits for the same lock could deadlock it. This needs a nearly full filesystem; I have not hit it despite trying to! - When the only busy blocks belong to the caller's own transaction, nothing can be waited for, and the caller gets ENOSPC although the space comes back at the next commit. - Older transactions still in the log can hold copies of a freed block, which a replay after a crash writes over its new owner. That needs revoke records, which a later series adds. Tested on 32-bit and 64-bit Hurd images: - Repeated apt upgrades of an old Debian image (most of the Hurd libraries replaced), ending with a clean halt and also with qemu killed from the host: e2fsck clean every time. After a clean halt e2fsck now clears the orphans of files that were still open with no pass 5 differences, where before their blocks leaked. - tar -xf and rm -rf of a Linux kernel tree, with qemu killed in the middle of each: the journal replays and e2fsck finds nothing else. - A 128 MB filesystem filled to ENOSPC, then 300 rounds of parallel delete-and-rewrite plus two appenders on one file, alongside a loop of mkdir, many small creates, chmod -R and rm -rf, with the translator killed at random points: e2fsck clean after replay, and every surviving copy of the reference file intact. - And also I ran this for the week as my daily driver on which I develop next patches, and it is stable and working as expected. Milos Nikic (2): ext2fs: Keep freed blocks busy until their free commits ext2fs: Keep the block map of orphan inodes on disk ext2fs/balloc.c | 206 ++++++++++++++++++++++++++++++++++++++++------ ext2fs/ext2fs.h | 8 ++ ext2fs/getblk.c | 3 +- ext2fs/inode.c | 38 +++++---- ext2fs/journal.c | 164 ++++++++++++++++++++++++++++-------- ext2fs/journal.h | 13 ++- ext2fs/orphan.c | 19 ++--- ext2fs/truncate.c | 19 ++--- 8 files changed, 362 insertions(+), 108 deletions(-) -- 2.56.0
