Re: [f2fs-dev] [PATCH v3] f2fs: add F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE

2020-12-02 Thread Chao Yu
On 2020/12/3 14:56, Daeho Jeong wrote: From: Daeho Jeong Added two ioctl to decompress/compress explicitly the compression enabled file in "compress_mode=user" mount option. Using these two ioctls, the users can make a control of compression and decompression of their files. Signed-off-by: Da

Re: [f2fs-dev] [PATCH v2 1/2] f2fs: add compress_mode mount option

2020-12-02 Thread Chao Yu
On 2020/12/1 12:08, Daeho Jeong wrote: From: Daeho Jeong We will add a new "compress_mode" mount option to control file compression mode. This supports "fs" and "user". In "fs" mode (default), f2fs does automatic compression on the compression enabled files. In "user" mode, f2fs disables the au

Re: [f2fs-dev] [PATCH v3] f2fs: avoid race condition for shinker count

2020-12-02 Thread Chao Yu
On 2020/12/3 14:07, Jaegeuk Kim wrote: On 11/11, Jaegeuk Kim wrote: Light reported sometimes shinker gets nat_cnt < dirty_nat_cnt resulting in wrong do_shinker work. Let's avoid to get stale data by using nat_tree_lock. Reported-by: Light Hsieh Signed-off-by: Jaegeuk Kim --- v3: - fix to us

[f2fs-dev] [PATCH v3] f2fs: add F2FS_IOC_DECOMPRESS_FILE and F2FS_IOC_COMPRESS_FILE

2020-12-02 Thread Daeho Jeong
From: Daeho Jeong Added two ioctl to decompress/compress explicitly the compression enabled file in "compress_mode=user" mount option. Using these two ioctls, the users can make a control of compression and decompression of their files. Signed-off-by: Daeho Jeong --- v3: changed error conditio

Re: [f2fs-dev] [PATCH v7 2/2] f2fs: add F2FS_IOC_SET_COMPRESS_OPTION ioctl

2020-12-02 Thread Chao Yu
On 2020/12/3 14:08, Jaegeuk Kim wrote: On 12/02, Jaegeuk Kim wrote: On 12/03, Chao Yu wrote: Jaegeuk, not sure, is it too late to merge this cleanup into original patch? Let me merge this on top of tree. Thanks. :) I integrated this into the original patch since I had to revise this commit.

[f2fs-dev] [PATCH v6] f2fs: compress: support compress level

2020-12-02 Thread Chao Yu
Expand 'compress_algorithm' mount option to accept parameter as format of :, by this way, it gives a way to allow user to do more specified config on lz4 and zstd compression level, then f2fs compression can provide higher compress ratio. In order to set compress level for lz4 algorithm, it needs

Re: [f2fs-dev] [PATCH v3] f2fs: avoid race condition for shinker count

2020-12-02 Thread Jaegeuk Kim
On 11/11, Jaegeuk Kim wrote: > Light reported sometimes shinker gets nat_cnt < dirty_nat_cnt resulting in > wrong do_shinker work. Let's avoid to get stale data by using nat_tree_lock. > > Reported-by: Light Hsieh > Signed-off-by: Jaegeuk Kim > --- > v3: > - fix to use NM_I(sbi) > > fs/f2fs/s

Re: [f2fs-dev] [PATCH v7 2/2] f2fs: add F2FS_IOC_SET_COMPRESS_OPTION ioctl

2020-12-02 Thread Jaegeuk Kim
On 12/02, Jaegeuk Kim wrote: > On 12/03, Chao Yu wrote: > > Jaegeuk, not sure, is it too late to merge this cleanup into original patch? > > Let me merge this on top of tree. Thanks. :) I integrated this into the original patch since I had to revise this commit. :( ("f2fs: avoid race condition fo

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Nick Terrell via Linux-f2fs-devel
> On Dec 2, 2020, at 9:03 PM, Michał Mirosław wrote: > > On Thu, Dec 03, 2020 at 03:59:21AM +, Nick Terrell wrote: >> On Dec 2, 2020, at 7:14 PM, Michał Mirosław wrote: >>> On Thu, Dec 03, 2020 at 01:42:03AM +, Nick Terrell wrote: On Dec 2, 2020, at 5:16 PM, Michał Mirosław

Re: [f2fs-dev] [PATCH v5] f2fs: compress: support compress level

2020-12-02 Thread Jaegeuk Kim
On 12/03, Chao Yu wrote: > Expand 'compress_algorithm' mount option to accept parameter as format of > :, by this way, it gives a way to allow user to do more > specified config on lz4 and zstd compression level, then f2fs compression > can provide higher compress ratio. > > In order to set compre

Re: [f2fs-dev] [PATCH v7 2/2] f2fs: add F2FS_IOC_SET_COMPRESS_OPTION ioctl

2020-12-02 Thread Jaegeuk Kim
On 12/03, Chao Yu wrote: > Jaegeuk, not sure, is it too late to merge this cleanup into original patch? Let me merge this on top of tree. Thanks. :) > > From a5c63ec58e0cda6eb5d186b46942eea46422b7a9 Mon Sep 17 00:00:00 2001 > From: Chao Yu > Date: Thu, 3 Dec 2020 10:04:26 +0800 > Subject: [PATC

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Michał Mirosław
On Thu, Dec 03, 2020 at 03:59:21AM +, Nick Terrell wrote: > On Dec 2, 2020, at 7:14 PM, Michał Mirosław wrote: > > On Thu, Dec 03, 2020 at 01:42:03AM +, Nick Terrell wrote: > >> On Dec 2, 2020, at 5:16 PM, Michał Mirosław > >> wrote: > >>> On Wed, Dec 02, 2020 at 12:32:40PM -0800, Nick T

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Nick Terrell via Linux-f2fs-devel
> On Dec 2, 2020, at 7:14 PM, Michał Mirosław wrote: > > On Thu, Dec 03, 2020 at 01:42:03AM +, Nick Terrell wrote: >> >> >>> On Dec 2, 2020, at 5:16 PM, Michał Mirosław wrote: >>> >>> On Wed, Dec 02, 2020 at 12:32:40PM -0800, Nick Terrell wrote: From: Nick Terrell This

Re: [f2fs-dev] [PATCH v7 2/2] f2fs: add F2FS_IOC_SET_COMPRESS_OPTION ioctl

2020-12-02 Thread Daeho Jeong
Oh, by the control logic changed in the previous patch, we don't need to use f2fs_is_compress_algorithm_valid() anymore. Looks good~ 2020년 12월 3일 (목) 오전 11:49, Chao Yu 님이 작성: > > Jaegeuk, not sure, is it too late to merge this cleanup into original patch? > > From a5c63ec58e0cda6eb5d186b46942eea4

[f2fs-dev] [PATCH] f2fs: compress: deny setting unsupported compress algorithm

2020-12-02 Thread Chao Yu
If kernel doesn't support certain kinds of compress algorithm, deny to set them as compress algorithm of f2fs via 'compress_algorithm=%s' mount option. Signed-off-by: Chao Yu --- fs/f2fs/super.c | 21 - 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/fs/f2fs/su

[f2fs-dev] [PATCH v5] f2fs: compress: support compress level

2020-12-02 Thread Chao Yu
Expand 'compress_algorithm' mount option to accept parameter as format of :, by this way, it gives a way to allow user to do more specified config on lz4 and zstd compression level, then f2fs compression can provide higher compress ratio. In order to set compress level for lz4 algorithm, it needs

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Michał Mirosław
On Thu, Dec 03, 2020 at 01:42:03AM +, Nick Terrell wrote: > > > > On Dec 2, 2020, at 5:16 PM, Michał Mirosław wrote: > > > > On Wed, Dec 02, 2020 at 12:32:40PM -0800, Nick Terrell wrote: > >> From: Nick Terrell > >> > >> This patch: > >> - Moves `include/linux/zstd.h` -> `lib/zstd/zstd.h`

Re: [f2fs-dev] [PATCH v7 2/2] f2fs: add F2FS_IOC_SET_COMPRESS_OPTION ioctl

2020-12-02 Thread Chao Yu
Jaegeuk, not sure, is it too late to merge this cleanup into original patch? From a5c63ec58e0cda6eb5d186b46942eea46422b7a9 Mon Sep 17 00:00:00 2001 From: Chao Yu Date: Thu, 3 Dec 2020 10:04:26 +0800 Subject: [PATCH] f2fs: remove f2fs_is_compress_algorithm_valid() for cleanup No logic changes.

[f2fs-dev] [PATCH v2 1/9] ext4: remove ext4_dir_open()

2020-12-02 Thread Eric Biggers
From: Eric Biggers Since encrypted directories can be opened and searched without their key being available, and each readdir and ->lookup() tries to set up the key, trying to set up the key in ->open() too isn't really useful. Just remove it so that directories don't need an ->open() method any

[f2fs-dev] [PATCH v2 5/9] fscrypt: introduce fscrypt_prepare_readdir()

2020-12-02 Thread Eric Biggers
From: Eric Biggers The last remaining use of fscrypt_get_encryption_info() from filesystems is for readdir (->iterate_shared()). Every other call is now in fs/crypto/ as part of some other higher-level operation. We need to add a new argument to fscrypt_get_encryption_info() to indicate whether

[f2fs-dev] [PATCH v2 3/9] ubifs: remove ubifs_dir_open()

2020-12-02 Thread Eric Biggers
From: Eric Biggers Since encrypted directories can be opened and searched without their key being available, and each readdir and ->lookup() tries to set up the key, trying to set up the key in ->open() too isn't really useful. Just remove it so that directories don't need an ->open() method any

[f2fs-dev] [PATCH v2 7/9] fscrypt: move fscrypt_require_key() to fscrypt_private.h

2020-12-02 Thread Eric Biggers
From: Eric Biggers fscrypt_require_key() is now only used by files in fs/crypto/. So reduce its visibility to fscrypt_private.h. This is also a prerequsite for unexporting fscrypt_get_encryption_info(). Reviewed-by: Andreas Dilger Signed-off-by: Eric Biggers --- fs/crypto/fscrypt_private.h

[f2fs-dev] [PATCH v2 6/9] fscrypt: move body of fscrypt_prepare_setattr() out-of-line

2020-12-02 Thread Eric Biggers
From: Eric Biggers In preparation for reducing the visibility of fscrypt_require_key() by moving it to fscrypt_private.h, move the call to it from fscrypt_prepare_setattr() to an out-of-line function. Reviewed-by: Andreas Dilger Signed-off-by: Eric Biggers --- fs/crypto/hooks.c | 8 +++

[f2fs-dev] [PATCH v2 0/9] Allow deleting files with unsupported encryption policy

2020-12-02 Thread Eric Biggers
Currently it's impossible to delete files that use an unsupported encryption policy, as the kernel will just return an error when performing any operation on the top-level encrypted directory, even just a path lookup into the directory or opening the directory for readdir. It's desirable to return

[f2fs-dev] [PATCH v2 9/9] fscrypt: allow deleting files with unsupported encryption policy

2020-12-02 Thread Eric Biggers
From: Eric Biggers Currently it's impossible to delete files that use an unsupported encryption policy, as the kernel will just return an error when performing any operation on the top-level encrypted directory, even just a path lookup into the directory or opening the directory for readdir. Mor

[f2fs-dev] [PATCH v2 8/9] fscrypt: unexport fscrypt_get_encryption_info()

2020-12-02 Thread Eric Biggers
From: Eric Biggers Now that fscrypt_get_encryption_info() is only called from files in fs/crypto/ (due to all key setup now being handled by higher-level helper functions instead of directly by filesystems), unexport it and move its declaration to fscrypt_private.h. Reviewed-by: Andreas Dilger

[f2fs-dev] [PATCH v2 4/9] ext4: don't call fscrypt_get_encryption_info() from dx_show_leaf()

2020-12-02 Thread Eric Biggers
From: Eric Biggers The call to fscrypt_get_encryption_info() in dx_show_leaf() is too low in the call tree; fscrypt_get_encryption_info() should have already been called when starting the directory operation. And indeed, it already is. Moreover, the encryption key is guaranteed to already be av

[f2fs-dev] [PATCH v2 2/9] f2fs: remove f2fs_dir_open()

2020-12-02 Thread Eric Biggers
From: Eric Biggers Since encrypted directories can be opened and searched without their key being available, and each readdir and ->lookup() tries to set up the key, trying to set up the key in ->open() too isn't really useful. Just remove it so that directories don't need an ->open() method any

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Nick Terrell via Linux-f2fs-devel
> On Dec 2, 2020, at 5:16 PM, Michał Mirosław wrote: > > On Wed, Dec 02, 2020 at 12:32:40PM -0800, Nick Terrell wrote: >> From: Nick Terrell >> >> This patch: >> - Moves `include/linux/zstd.h` -> `lib/zstd/zstd.h` >> - Adds a new API in `include/linux/zstd.h` that is functionally >> equivale

Re: [f2fs-dev] [PATCH v6 1/3] lib: zstd: Add kernel-specific API

2020-12-02 Thread Michał Mirosław
On Wed, Dec 02, 2020 at 12:32:40PM -0800, Nick Terrell wrote: > From: Nick Terrell > > This patch: > - Moves `include/linux/zstd.h` -> `lib/zstd/zstd.h` > - Adds a new API in `include/linux/zstd.h` that is functionally > equivalent to the in-use subset of the current API. Functions are > rena

Re: [f2fs-dev] [PATCH v2 1/2] f2fs: add compress_mode mount option

2020-12-02 Thread Chao Yu
Alright, that makes sense. :) On 2020/12/3 9:23, Daeho Jeong wrote: Yep, you're right~ :) 2020년 12월 3일 (목) 오전 10:16, Chao Yu 님이 작성: On 2020/12/2 18:54, Daeho Jeong wrote: We might use compress_extension=*,compress_option=user. In this option, we're gonna allocate all the writes in cold zone.

Re: [f2fs-dev] [PATCH v2 1/2] f2fs: add compress_mode mount option

2020-12-02 Thread Daeho Jeong
Yep, you're right~ :) 2020년 12월 3일 (목) 오전 10:16, Chao Yu 님이 작성: > > On 2020/12/2 18:54, Daeho Jeong wrote: > > We might use compress_extension=*,compress_option=user. > > In this option, we're gonna allocate all the writes in cold zone. > > Oh, so all files in data partition will be tagged as comp

Re: [f2fs-dev] [PATCH v2 1/2] f2fs: add compress_mode mount option

2020-12-02 Thread Chao Yu
On 2020/12/2 18:54, Daeho Jeong wrote: We might use compress_extension=*,compress_option=user. In this option, we're gonna allocate all the writes in cold zone. Oh, so all files in data partition will be tagged as compressed file, but the compressing time will be controlled by user, then, only

Re: [f2fs-dev] Using secure erase in f2fs

2020-12-02 Thread Daeho Jeong
Hi Mikhail, As you noticed, f2fs already sends discard commands on released blocks. Do you want to fill the blocks with zero data, instead of sending discard commands? 2020년 12월 3일 (목) 오전 4:24, Mikhail Novosyolov via Linux-f2fs-devel 님이 작성: > > 02.12.2020 21:24, Mikhail Novosyolov пишет: > > Hell

Re: [f2fs-dev] [PATCH v7 1/8] block: ensure bios are not split in middle of crypto data unit

2020-12-02 Thread Satya Tangirala via Linux-f2fs-devel
On Wed, Nov 25, 2020 at 02:12:35PM -0800, Eric Biggers wrote: > On Wed, Nov 18, 2020 at 12:38:15AM +, Satya Tangirala wrote: > > > > +/** > > > > + * update_aligned_sectors_and_segs() - Ensures that *@aligned_sectors > > > > is aligned > > > > + *to @bio_sec

Re: [f2fs-dev] [PATCH 0/9] Allow deleting files with unsupported encryption policy

2020-12-02 Thread Eric Biggers
On Tue, Nov 24, 2020 at 04:23:27PM -0800, Eric Biggers wrote: > Currently it's impossible to delete files that use an unsupported > encryption policy, as the kernel will just return an error when > performing any operation on the top-level encrypted directory, even just > a path lookup into the dir

[f2fs-dev] [PATCH v6 2/3] lib: zstd: Add decompress_sources.h for decompress_unzstd

2020-12-02 Thread Nick Terrell
From: Nick Terrell Adds decompress_sources.h which includes every .c file necessary for zstd decompression. This is used in decompress_unzstd.c so the internal structure of the library isn't exposed. This allows us to upgrade the zstd library version without modifying any callers. Instead we jus

[f2fs-dev] [PATCH v6 0/3] Update to zstd-1.4.6

2020-12-02 Thread Nick Terrell
From: Nick Terrell Please pull from g...@github.com:terrelln/linux.git tags/zstd-1.4.6-v6 to get these changes. Alternatively the patchset is included. This patchset upgrades the zstd library to the latest upstream release. The current zstd version in the kernel is a modified version of upst

Re: [f2fs-dev] Using secure erase in f2fs

2020-12-02 Thread Mikhail Novosyolov via Linux-f2fs-devel
02.12.2020 21:24, Mikhail Novosyolov пишет: > Hello, f2fs developers! > > I saw commit 9af846486d781a63 "f2fs: add F2FS_IOC_SEC_TRIM_FILE ioctl" in > Linux kernel > https://github.com/torvalds/linux/commit/9af846486d781a63de025a5f502c515268e48790# > > I saw usage of it in Android only > https://an

[f2fs-dev] Using secure erase in f2fs

2020-12-02 Thread Mikhail Novosyolov via Linux-f2fs-devel
Hello, f2fs developers! I saw commit 9af846486d781a63 "f2fs: add F2FS_IOC_SEC_TRIM_FILE ioctl" in Linux kernel https://github.com/torvalds/linux/commit/9af846486d781a63de025a5f502c515268e48790# I saw usage of it in Android only https://android.googlesource.com/platform/system/vold/+/master/secdi

[f2fs-dev] [PATCH v4] f2fs: compress: support compress level

2020-12-02 Thread Chao Yu
From: Chao Yu Expand 'compress_algorithm' mount option to accept parameter as format of :, by this way, it gives a way to allow user to do more specified config on lz4 and zstd compression level, then f2fs compression can provide higher compress ratio. In order to set compress level for lz4 algo

[f2fs-dev] [PATCH v5] f2fs: compress: add compress_inode to cache compressed blocks

2020-12-02 Thread Chao Yu
From: Chao Yu Support to use address space of inner inode to cache compressed block, in order to improve cache hit ratio of random read. Signed-off-by: Chao Yu --- v5: - rebase to last dev branch. Documentation/filesystems/f2fs.rst | 3 + fs/f2fs/compress.c | 198