Re: [PATCH] build: export liberofs

2024-06-07 Thread Gao Xiang
Hi, On 2024/6/7 14:49, ComixHe wrote: export liberofs.a for linking to liberofsfuse in the project and provide pc files for developer's convenience. Signed-off-by: ComixHe Currently liberofs interface is still unstable. We tend to address this in erofs-utils v1.9 cycle. If you folks really

Re: [PATCH] erofs: support external crypto for decompression

2024-06-07 Thread kernel test robot
: ee78a17615ad0cfdbbc27182b1047cd36c9d4d5f patch link: https://lore.kernel.org/r/20240606-erofs-decompression-v1-1-ec5f31396e04%40amlogic.com patch subject: [PATCH] erofs: support external crypto for decompression config: arm64-randconfig-004-20240607 (https://download.01.org/0day-ci/archive/20240607

Re: [PATCH] erofs-utils: support virtual files

2024-06-07 Thread Gao Xiang
On 2024/6/6 19:18, Hongzhen Luo wrote: The current erofs-utils I/O implementation is through file descriptors. The new `erofs_vfile` provides a more flexible way to perform I/Os. Signed-off-by: Hongzhen Luo Applied with minor fixes. Thanks, Gao Xiang

[PATCH 2/2] erofs-utils: lib: split erofs_iflush()

2024-06-07 Thread Gao Xiang
So that external programs can directly use it. Signed-off-by: Gao Xiang --- include/erofs/inode.h | 1 + lib/inode.c | 25 - 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/include/erofs/inode.h b/include/erofs/inode.h index 5d6bc98..46d989c 100

[PATCH 1/2] erofs-utils: move erofs_writesb() into lib/

2024-06-07 Thread Gao Xiang
So that external programs can directly use it. Signed-off-by: Gao Xiang --- include/erofs/internal.h | 4 +++ lib/super.c | 52 mkfs/main.c | 64 +++- 3 files changed, 60 insertions(+), 60 deletions(-

[PATCH v4 07/22] xfs: make EOF allocation simpler

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner Currently the allocation at EOF is broken into two cases - when the offset is zero and when the offset is non-zero. When the offset is non-zero, we try to do exact block allocation for contiguous extent allocation. When the offset is zero, the allocation is simply an aligned al

[PATCH v4 00/22] block atomic writes for xfs

2024-06-07 Thread John Garry via Linux-erofs
This series expands atomic write support to filesystems, specifically XFS. Extent alignment is based on new feature forcealign. Flag FS_XFLAG_ATOMICWRITES is added as an enabling flag for atomic writes. XFS can be formatted for atomic writes as follows: mkfs.xfs -i forcealign=1 -d extsize=16384 -

[PATCH v4 18/22] iomap: Atomic write support

2024-06-07 Thread John Garry via Linux-erofs
Support atomic writes by producing a single BIO with REQ_ATOMIC flag set. We rely on the FS to guarantee extent alignment, such that an atomic write should never straddle two or more extents. The FS should also check for validity of an atomic write length/alignment. Signed-off-by: John Garry ---

[PATCH v4 01/22] fs: Add generic_atomic_write_valid_size()

2024-06-07 Thread John Garry via Linux-erofs
Add a generic helper for FSes to validate that an atomic write is appropriately sized (along with the other checks). Signed-off-by: John Garry --- include/linux/fs.h | 12 1 file changed, 12 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 069cbab62700..e13d

[PATCH v4 19/22] xfs: Support FS_XFLAG_ATOMICWRITES for forcealign

2024-06-07 Thread John Garry via Linux-erofs
Add initial support for FS_XFLAG_ATOMICWRITES for forcealign enabled. Current kernel support for atomic writes is based on HW support (for atomic writes). As such, it is required to ensure extent alignment with atomic_write_unit_max so that an atomic write can result in a single HW-compliant IO op

[PATCH v4 12/22] xfs: Update xfs_inode_alloc_unitsize_fsb() for forcealign

2024-06-07 Thread John Garry via Linux-erofs
For forcealign enabled, the allocation unit size is in ip->i_extsize, and this must never be zero. Signed-off-by: John Garry --- fs/xfs/xfs_inode.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index a6cd5f1f2680..9c0ae5c3682e 100644 --- a/fs/xfs/x

[PATCH v4 13/22] xfs: Unmap blocks according to forcealign

2024-06-07 Thread John Garry via Linux-erofs
For when forcealign is enabled, blocks in an inode need to be unmapped according to extent alignment, like what is already done for rtvol. Signed-off-by: John Garry --- fs/xfs/libxfs/xfs_bmap.c | 33 - 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a

[PATCH v4 09/22] xfs: align args->minlen for forced allocation alignment

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner If args->minlen is not aligned to the constraints of forced alignment, we may do minlen allocations that are not aligned when we approach ENOSPC. Avoid this by always aligning args->minlen appropriately. If alignment of minlen results in a value smaller than the alignment const

[PATCH v4 14/22] xfs: Only free full extents for forcealign

2024-06-07 Thread John Garry via Linux-erofs
Like we already do for rtvol, only free full extents for forcealign in xfs_free_file_space(). Signed-off-by: John Garry --- fs/xfs/xfs_bmap_util.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 56b80a7c0992..ee767a

[PATCH v4 04/22] xfs: only allow minlen allocations when near ENOSPC

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner When we are near ENOSPC and don't have enough free space for an args->maxlen allocation, xfs_alloc_space_available() will trim args->maxlen to equal the available space. However, this function has only checked that there is enough contiguous free space for an aligned args->minl

[PATCH v4 02/22] iomap: Allow filesystems set IO block zeroing size

2024-06-07 Thread John Garry via Linux-erofs
Allow filesystems to set the io_block_size for sub-fs block size zeroing, as in future we will want to extend this feature to support zeroing of block sizes of larger than the inode block size. The value in io_block_size does not have to be a power-of-2, so fix up zeroing code to handle that. Sig

[PATCH v4 10/22] xfs: Introduce FORCEALIGN inode flag

2024-06-07 Thread John Garry via Linux-erofs
From: "Darrick J. Wong" Add a new inode flag to require that all file data extent mappings must be aligned (both the file offset range and the allocated space itself) to the extent size hint. Having a separate COW extent size hint is no longer allowed. The goal here is to enable sysadmins and u

[PATCH v4 03/22] xfs: Use extent size granularity for iomap->io_block_size

2024-06-07 Thread John Garry via Linux-erofs
Currently iomap->io_block_size is set to the i_blocksize() value for the inode. Expand the sub-fs block size zeroing to now cover RT extents, by calling setting iomap->io_block_size as xfs_inode_alloc_unitsize(). In xfs_iomap_write_unwritten(), update the unwritten range fsb to cover this extent

[PATCH v4 20/22] xfs: Support atomic write for statx

2024-06-07 Thread John Garry via Linux-erofs
Support providing info on atomic write unit min and max for an inode. For simplicity, currently we limit the min at the FS block size, but a lower limit could be supported in future. This is required by iomap DIO. The atomic write unit min and max is limited by the guaranteed extent alignment for

[PATCH v4 16/22] xfs: Enable file data forcealign feature

2024-06-07 Thread John Garry via Linux-erofs
From: "Darrick J. Wong" Enable this feature. Signed-off-by: "Darrick J. Wong" Signed-off-by: John Garry --- fs/xfs/libxfs/xfs_format.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h index b48cd75d34a6..42e1f80206ab

[PATCH v4 21/22] xfs: Validate atomic writes

2024-06-07 Thread John Garry via Linux-erofs
Validate that an atomic write adheres to length/offset rules. Since we require extent alignment for atomic writes, this effectively also enforces that the BIO which iomap produces is aligned. For an IOCB with IOCB_ATOMIC set to get as far as xfs_file_dio_write(), FMODE_CAN_ATOMIC_WRITE will need t

[PATCH v4 06/22] xfs: simplify extent allocation alignment

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner We currently align extent allocation to stripe unit or stripe width. That is specified by an external parameter to the allocation code, which then manipulates the xfs_alloc_args alignment configuration in interesting ways. The args->alignment field specifies extent start align

[PATCH v4 05/22] xfs: always tail align maxlen allocations

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner When we do a large allocation, the core free space allocation code assumes that args->maxlen is aligned to args->prod/args->mod. hence if we get a maximum sized extent allocated, it does not do tail alignment of the extent. However, this assumes that nothing modifies args->max

[PATCH v4 17/22] fs: Add FS_XFLAG_ATOMICWRITES flag

2024-06-07 Thread John Garry via Linux-erofs
Add a flag indicating that a regular file is enabled for atomic writes. This is a file attribute that mirrors an ondisk inode flag. Actual support for untorn file writes (for now) depends on both the iflag and the underlying storage devices, which we can only really check at statx and pwritev2()

[PATCH v4 15/22] xfs: Don't revert allocated offset for forcealign

2024-06-07 Thread John Garry via Linux-erofs
In xfs_bmap_process_allocated_extent(), for when we found that we could not provide the requested length completely, the mapping is moved so that we can provide as much as possible for the original request. For forcealign, this would mean ignoring alignment guaranteed, so don't do this. Signed-of

[PATCH v4 08/22] xfs: introduce forced allocation alignment

2024-06-07 Thread John Garry via Linux-erofs
From: Dave Chinner When forced allocation alignment is specified, the extent will be aligned to the extent size hint size rather than stripe alignment. If aligned allocation cannot be done, then the allocation is failed rather than attempting non-aligned fallbacks. Note: none of the per-inode fo

[PATCH v4 11/22] xfs: Do not free EOF blocks for forcealign

2024-06-07 Thread John Garry via Linux-erofs
For when forcealign is enabled, we want the EOF to be aligned as well, so do not free EOF blocks. Signed-off-by: John Garry --- fs/xfs/xfs_bmap_util.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index e5d893f93522..56b

[PATCH v4 22/22] xfs: Support setting FMODE_CAN_ATOMIC_WRITE

2024-06-07 Thread John Garry via Linux-erofs
For when an inode is enabled for atomic writes, set FMODE_CAN_ATOMIC_WRITE flag. Only direct IO is currently supported, so check for that also. We rely on the block layer to reject atomic writes which exceed the bdev request_queue limits, so don't bother checking any such thing here. Signed-off-b