commit 2d60b848f2503f28d840ceae174d07eb149ccce9 Author: Tomohiro Kusumi <tkus...@netbsd.org> Date: Sat Jun 4 20:54:35 2022 +0900
usr.sbin/makefs: Add HAMMER2 support This commit adds HAMMER2 image creation support for makefs(8). It runs newfs_hammer2(8) and then sys/vfs/hammer2 logic in userspace to create HAMMER2 image from a given directory. This commit splits newfs_hammer2(8) into newfs and mkfs part simlarly to newfs_msdos(8), so that makefs(8) can use newfs functionality. The entire sys/vfs/hammer2 (with exception of unneeded hammer2_{bulkfree,ccms,iocom,ioctl,msgops,synchro}.[hc] and reusable hammer2_disk.h) is copied to usr.sbin/makefs with below modification. It intends to have minimum amount of diff against sys/vfs/hammer2. * Header includes are modified so that it compiles in userspace. * VFS and other kernel functions are usually implemented as simple stub functions in hammer2_compat.h and hammer2_buf.c, but some are commented out. * Kernel functions such as kprintf, kmalloc, kprintf, kstrdup, etc are implemented using corresponding libc functions. * Lock primitives are basically NOP, and they (should) never block as makefs(8) is a single thread program. * struct vnode and struct buf (the ones defined locally in makefs(8), not sys/sys/*) have new struct members only used by HAMMER2 to emulate VFS behavior required by HAMMER2. * Since makefs(8) is write-only, VOP_{NRESOLVE,NCREATE,NMKDIR,NLINK, NSYMLINK,WRITE,STRATEGY} are implemented, but other VOPs just return EOPNOTSUPP. * VOP_{INACTIVE,RECLAIM} may be implemented and used in future to better emulate VFS behavior to address current limitation. * VOP_WRITE is modified to directly call VOP_STRATEGY function. * The XOP kernel thread is modified to act as a regular function called from VOPs, along with simplified admin code. It currently has following limitations. * multi-volumes is unsupported, simply due to makefs(8) only taking 1 image file path. * Not necessarily a limitation, but it only supports populating 1 PFS, which is "DATA" by default. Other PFSes if any won't have anything under the root PFS inode. * makefs(8) process gets killed by OOM for a directory with *extremely* large number of files, depending on available memory. This is due to the way it currently tries to flush all chains in a single VFS_SYNC. Supporting multiple VFS_SYNC calls by checking available memory along the way gives chance to free unused vnodes/inodes and chains. This may be implemented in future. This limitation is specific to HAMMER2, as all other makefs(8) filesystems are not CoW, meaning they allow in-place write based objects creation from a top directory to bottom whereas HAMMER2 flushes chains in bottom-up direction. Summary of changes: sbin/newfs_hammer2/Makefile | 2 +- .../{newfs_hammer2.c => mkfs_hammer2.c} | 508 +- sbin/newfs_hammer2/mkfs_hammer2.h | 77 + sbin/newfs_hammer2/newfs_hammer2.c | 784 +-- usr.sbin/makefs/Makefile | 5 +- usr.sbin/makefs/ffs/buf.c | 52 +- usr.sbin/makefs/ffs/buf.h | 15 + usr.sbin/makefs/hammer2.c | 824 +++ usr.sbin/makefs/hammer2.h | 48 + usr.sbin/makefs/hammer2/Makefile.inc | 22 + usr.sbin/makefs/hammer2/hammer2.h | 2091 +++++++ usr.sbin/makefs/hammer2/hammer2_admin.c | 1248 ++++ usr.sbin/makefs/hammer2/hammer2_buf.c | 177 + usr.sbin/makefs/hammer2/hammer2_chain.c | 6143 ++++++++++++++++++++ usr.sbin/makefs/hammer2/hammer2_cluster.c | 760 +++ usr.sbin/makefs/hammer2/hammer2_compat.h | 929 +++ usr.sbin/makefs/hammer2/hammer2_flush.c | 1550 +++++ usr.sbin/makefs/hammer2/hammer2_freemap.c | 1284 ++++ usr.sbin/makefs/hammer2/hammer2_inode.c | 1864 ++++++ usr.sbin/makefs/hammer2/hammer2_io.c | 918 +++ usr.sbin/makefs/hammer2/hammer2_lz4.c | 527 ++ {sys/vfs => usr.sbin/makefs}/hammer2/hammer2_lz4.h | 0 .../makefs}/hammer2/hammer2_lz4_encoder.h | 0 usr.sbin/makefs/hammer2/hammer2_ondisk.c | 703 +++ usr.sbin/makefs/hammer2/hammer2_strategy.c | 1610 +++++ usr.sbin/makefs/hammer2/hammer2_subr.c | 482 ++ usr.sbin/makefs/hammer2/hammer2_vfsops.c | 3139 ++++++++++ usr.sbin/makefs/hammer2/hammer2_vnops.c | 2871 +++++++++ usr.sbin/makefs/hammer2/hammer2_xops.c | 1636 ++++++ .../makefs}/hammer2/zlib/hammer2_zlib.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_adler32.c | 0 .../makefs/hammer2/zlib/hammer2_zlib_deflate.c | 1210 ++++ .../makefs}/hammer2/zlib/hammer2_zlib_deflate.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_inffast.c | 0 .../makefs}/hammer2/zlib/hammer2_zlib_inffast.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_inffixed.h | 0 .../makefs/hammer2/zlib/hammer2_zlib_inflate.c | 1052 ++++ .../makefs}/hammer2/zlib/hammer2_zlib_inflate.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_inftrees.c | 0 .../makefs}/hammer2/zlib/hammer2_zlib_inftrees.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_trees.c | 0 .../makefs}/hammer2/zlib/hammer2_zlib_trees.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_zconf.h | 0 .../makefs}/hammer2/zlib/hammer2_zlib_zutil.c | 0 .../makefs}/hammer2/zlib/hammer2_zlib_zutil.h | 0 usr.sbin/makefs/makefs.8 | 58 +- usr.sbin/makefs/makefs.c | 1 + usr.sbin/makefs/makefs.h | 1 + 48 files changed, 31546 insertions(+), 1045 deletions(-) copy sbin/newfs_hammer2/{newfs_hammer2.c => mkfs_hammer2.c} (80%) create mode 100644 sbin/newfs_hammer2/mkfs_hammer2.h create mode 100644 usr.sbin/makefs/hammer2.c create mode 100644 usr.sbin/makefs/hammer2.h create mode 100644 usr.sbin/makefs/hammer2/Makefile.inc create mode 100644 usr.sbin/makefs/hammer2/hammer2.h create mode 100644 usr.sbin/makefs/hammer2/hammer2_admin.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_buf.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_chain.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_cluster.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_compat.h create mode 100644 usr.sbin/makefs/hammer2/hammer2_flush.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_freemap.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_inode.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_io.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_lz4.c copy {sys/vfs => usr.sbin/makefs}/hammer2/hammer2_lz4.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/hammer2_lz4_encoder.h (100%) create mode 100644 usr.sbin/makefs/hammer2/hammer2_ondisk.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_strategy.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_subr.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_vfsops.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_vnops.c create mode 100644 usr.sbin/makefs/hammer2/hammer2_xops.c copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_adler32.c (100%) create mode 100644 usr.sbin/makefs/hammer2/zlib/hammer2_zlib_deflate.c copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_deflate.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inffast.c (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inffast.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inffixed.h (100%) create mode 100644 usr.sbin/makefs/hammer2/zlib/hammer2_zlib_inflate.c copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inflate.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inftrees.c (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_inftrees.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_trees.c (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_trees.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_zconf.h (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_zutil.c (100%) copy {sys/vfs => usr.sbin/makefs}/hammer2/zlib/hammer2_zlib_zutil.h (100%) http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/2d60b848f2503f28d840ceae174d07eb149ccce9 -- DragonFly BSD source repository