[f2fs-dev] [PATCH v6] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-11 Thread Tiezhu Yang 
There exists many similar and duplicate codes to check "." and "..",
so introduce is_dot_or_dotdot helper to make the code more clean.

Signed-off-by: Tiezhu Yang 
---

v6:
  - do not use the helper function in fs/namei.c
  - add extra check "len >= 1" in is_dot_or_dotdot()

v5:
  - remove "qname" variable in fscrypt_fname_disk_to_usr()
  - modify "len < 2" to "len == 1" in is_dot_or_dotdot()

v4:
  - rename is_dot_dotdot() to is_dot_or_dotdot()

v3:
  - use "name" and "len" as arguments instead of qstr
  - move is_dot_dotdot() to include/linux/namei.h

v2:
  - use the better performance implementation of is_dot_dotdot
  - make it static inline and move it to include/linux/fs.h

 fs/crypto/fname.c | 17 +++--
 fs/ecryptfs/crypto.c  | 12 +---
 fs/f2fs/f2fs.h| 11 ---
 fs/f2fs/hash.c|  3 ++-
 include/linux/namei.h | 10 ++
 5 files changed, 16 insertions(+), 37 deletions(-)

diff --git a/fs/crypto/fname.c b/fs/crypto/fname.c
index 3da3707..bb41f5d 100644
--- a/fs/crypto/fname.c
+++ b/fs/crypto/fname.c
@@ -11,21 +11,11 @@
  * This has not yet undergone a rigorous security audit.
  */
 
+#include 
 #include 
 #include 
 #include "fscrypt_private.h"
 
-static inline bool fscrypt_is_dot_dotdot(const struct qstr *str)
-{
-   if (str->len == 1 && str->name[0] == '.')
-   return true;
-
-   if (str->len == 2 && str->name[0] == '.' && str->name[1] == '.')
-   return true;
-
-   return false;
-}
-
 /**
  * fname_encrypt() - encrypt a filename
  *
@@ -252,10 +242,9 @@ int fscrypt_fname_disk_to_usr(struct inode *inode,
const struct fscrypt_str *iname,
struct fscrypt_str *oname)
 {
-   const struct qstr qname = FSTR_TO_QSTR(iname);
struct fscrypt_digested_name digested_name;
 
-   if (fscrypt_is_dot_dotdot(&qname)) {
+   if (is_dot_or_dotdot(iname->name, iname->len)) {
oname->name[0] = '.';
oname->name[iname->len - 1] = '.';
oname->len = iname->len;
@@ -323,7 +312,7 @@ int fscrypt_setup_filename(struct inode *dir, const struct 
qstr *iname,
memset(fname, 0, sizeof(struct fscrypt_name));
fname->usr_fname = iname;
 
-   if (!IS_ENCRYPTED(dir) || fscrypt_is_dot_dotdot(iname)) {
+   if (!IS_ENCRYPTED(dir) || is_dot_or_dotdot(iname->name, iname->len)) {
fname->disk_name.name = (unsigned char *)iname->name;
fname->disk_name.len = iname->len;
return 0;
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index f91db24..c3bcbf0 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -1991,16 +1991,6 @@ int ecryptfs_encrypt_and_encode_filename(
return rc;
 }
 
-static bool is_dot_dotdot(const char *name, size_t name_size)
-{
-   if (name_size == 1 && name[0] == '.')
-   return true;
-   else if (name_size == 2 && name[0] == '.' && name[1] == '.')
-   return true;
-
-   return false;
-}
-
 /**
  * ecryptfs_decode_and_decrypt_filename - converts the encoded cipher text 
name to decoded plaintext
  * @plaintext_name: The plaintext name
@@ -2027,7 +2017,7 @@ int ecryptfs_decode_and_decrypt_filename(char 
**plaintext_name,
 
if ((mount_crypt_stat->flags & ECRYPTFS_GLOBAL_ENCRYPT_FILENAMES) &&
!(mount_crypt_stat->flags & ECRYPTFS_ENCRYPTED_VIEW_ENABLED)) {
-   if (is_dot_dotdot(name, name_size)) {
+   if (is_dot_or_dotdot(name, name_size)) {
rc = ecryptfs_copy_filename(plaintext_name,
plaintext_name_size,
name, name_size);
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 5a888a0..3d5e684 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2767,17 +2767,6 @@ static inline bool f2fs_cp_error(struct f2fs_sb_info 
*sbi)
return is_set_ckpt_flags(sbi, CP_ERROR_FLAG);
 }
 
-static inline bool is_dot_dotdot(const struct qstr *str)
-{
-   if (str->len == 1 && str->name[0] == '.')
-   return true;
-
-   if (str->len == 2 && str->name[0] == '.' && str->name[1] == '.')
-   return true;
-
-   return false;
-}
-
 static inline bool f2fs_may_extent_tree(struct inode *inode)
 {
struct f2fs_sb_info *sbi = F2FS_I_SB(inode);
diff --git a/fs/f2fs/hash.c b/fs/f2fs/hash.c
index 5bc4dcd..ef155c2 100644
--- a/fs/f2fs/hash.c
+++ b/fs/f2fs/hash.c
@@ -15,6 +15,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "f2fs.h"
 
@@ -82,7 +83,7 @@ static f2fs_hash_t __f2fs_dentry_hash(const struct qstr 
*name_info,
if (fname && !fname->disk_name.name)
return cpu_to_le32(fname->hash);
 
-   if (is_dot_dotdot(name_info))
+   if (is_dot_or_dotdot(name, len))
return 0;
 
/* Initialize the default seed for the hash checksum functions */
diff --git a/include/lin

[f2fs-dev] [PATCH AUTOSEL 4.14 48/58] f2fs: choose hardlimit when softlimit is larger than hardlimit in f2fs_statfs_project()

2019-12-11 Thread Sasha Levin 
From: Chengguang Xu 

[ Upstream commit 909110c060f22e65756659ec6fa957ae75777e00 ]

Setting softlimit larger than hardlimit seems meaningless
for disk quota but currently it is allowed. In this case,
there may be a bit of comfusion for users when they run
df comamnd to directory which has project quota.

For example, we set 20M softlimit and 10M hardlimit of
block usage limit for project quota of test_dir(project id 123).

[root@hades f2fs]# repquota -P -a
*** Report for project quotas on device /dev/nvme0n1p8
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
Project used soft hard grace used soft hard grace
--
0 -- 4 0 0 1 0 0
123 +- 10248 20480 10240 2 0 0

The result of df command as below:

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 20M 11M 10M 51% /mnt/f2fs

Even though it looks like there is another 10M free space to use,
if we write new data to diretory test(inherit project id),
the write will fail with errno(-EDQUOT).

After this patch, the df result looks like below.

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 10M 10M 0 100% /mnt/f2fs

Signed-off-by: Chengguang Xu 
Reviewed-by: Chao Yu 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/super.c | 20 ++--
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index e4aabfc21bd43..8635df6cba553 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -912,9 +912,13 @@ static int f2fs_statfs_project(struct super_block *sb,
return PTR_ERR(dquot);
spin_lock(&dq_data_lock);
 
-   limit = (dquot->dq_dqb.dqb_bsoftlimit ?
-dquot->dq_dqb.dqb_bsoftlimit :
-dquot->dq_dqb.dqb_bhardlimit) >> sb->s_blocksize_bits;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_bsoftlimit)
+   limit = dquot->dq_dqb.dqb_bsoftlimit;
+   if (dquot->dq_dqb.dqb_bhardlimit &&
+   (!limit || dquot->dq_dqb.dqb_bhardlimit < limit))
+   limit = dquot->dq_dqb.dqb_bhardlimit;
+
if (limit && buf->f_blocks > limit) {
curblock = dquot->dq_dqb.dqb_curspace >> sb->s_blocksize_bits;
buf->f_blocks = limit;
@@ -923,9 +927,13 @@ static int f2fs_statfs_project(struct super_block *sb,
 (buf->f_blocks - curblock) : 0;
}
 
-   limit = dquot->dq_dqb.dqb_isoftlimit ?
-   dquot->dq_dqb.dqb_isoftlimit :
-   dquot->dq_dqb.dqb_ihardlimit;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_isoftlimit)
+   limit = dquot->dq_dqb.dqb_isoftlimit;
+   if (dquot->dq_dqb.dqb_ihardlimit &&
+   (!limit || dquot->dq_dqb.dqb_ihardlimit < limit))
+   limit = dquot->dq_dqb.dqb_ihardlimit;
+
if (limit && buf->f_files > limit) {
buf->f_files = limit;
buf->f_ffree =
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 4.14 20/58] f2fs: fix to update dir's i_pino during cross_rename

2019-12-11 Thread Sasha Levin 
From: Chao Yu 

[ Upstream commit 2a60637f06ac94869b2e630eaf837110d39bf291 ]

As Eric reported:

RENAME_EXCHANGE support was just added to fsstress in xfstests:

commit 65dfd40a97b6bbbd2a22538977bab355c5bc0f06
Author: kaixuxia 
Date:   Thu Oct 31 14:41:48 2019 +0800

fsstress: add EXCHANGE renameat2 support

This is causing xfstest generic/579 to fail due to fsck.f2fs reporting errors.
I'm not sure what the problem is, but it still happens even with all the
fs-verity stuff in the test commented out, so that the test just runs fsstress.

generic/579 23s ... [10:02:25]
[7.745370] run fstests generic/579 at 2019-11-04 10:02:25
_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
(see /results/f2fs/results-default/generic/579.full for details)
 [10:02:47]
Ran: generic/579
Failures: generic/579
Failed 1 of 1 tests
Xunit report: /results/f2fs/results-default/result.xml

Here's the contents of 579.full:

_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
*** fsck.f2fs output ***
[ASSERT] (__chk_dots_dentries:1378)  --> Bad inode number[0x24] for '..', 
parent parent ino is [0xd10]

The root cause is that we forgot to update directory's i_pino during
cross_rename, fix it.

Fixes: 32f9bc25cbda0 ("f2fs: support ->rename2()")
Signed-off-by: Chao Yu 
Tested-by: Eric Biggers 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/namei.c | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index b80e7db3b55b5..b13383948fca3 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -862,7 +862,8 @@ static int f2fs_rename(struct inode *old_dir, struct dentry 
*old_dentry,
if (!old_dir_entry || whiteout)
file_lost_pino(old_inode);
else
-   F2FS_I(old_inode)->i_pino = new_dir->i_ino;
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_inode->i_ctime = current_time(old_inode);
@@ -1027,7 +1028,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(old_dir, old_entry, old_page, new_inode);
 
down_write(&F2FS_I(old_inode)->i_sem);
-   file_lost_pino(old_inode);
+   if (!old_dir_entry)
+   file_lost_pino(old_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_dir->i_ctime = current_time(old_dir);
@@ -1042,7 +1047,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(new_dir, new_entry, new_page, old_inode);
 
down_write(&F2FS_I(new_inode)->i_sem);
-   file_lost_pino(new_inode);
+   if (!new_dir_entry)
+   file_lost_pino(new_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(new_inode, old_dir->i_ino);
up_write(&F2FS_I(new_inode)->i_sem);
 
new_dir->i_ctime = current_time(new_dir);
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 4.19 67/79] f2fs: choose hardlimit when softlimit is larger than hardlimit in f2fs_statfs_project()

2019-12-11 Thread Sasha Levin 
From: Chengguang Xu 

[ Upstream commit 909110c060f22e65756659ec6fa957ae75777e00 ]

Setting softlimit larger than hardlimit seems meaningless
for disk quota but currently it is allowed. In this case,
there may be a bit of comfusion for users when they run
df comamnd to directory which has project quota.

For example, we set 20M softlimit and 10M hardlimit of
block usage limit for project quota of test_dir(project id 123).

[root@hades f2fs]# repquota -P -a
*** Report for project quotas on device /dev/nvme0n1p8
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
Project used soft hard grace used soft hard grace
--
0 -- 4 0 0 1 0 0
123 +- 10248 20480 10240 2 0 0

The result of df command as below:

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 20M 11M 10M 51% /mnt/f2fs

Even though it looks like there is another 10M free space to use,
if we write new data to diretory test(inherit project id),
the write will fail with errno(-EDQUOT).

After this patch, the df result looks like below.

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 10M 10M 0 100% /mnt/f2fs

Signed-off-by: Chengguang Xu 
Reviewed-by: Chao Yu 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/super.c | 20 ++--
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 7a9cc64f5ca37..662c7de58b990 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1148,9 +1148,13 @@ static int f2fs_statfs_project(struct super_block *sb,
return PTR_ERR(dquot);
spin_lock(&dquot->dq_dqb_lock);
 
-   limit = (dquot->dq_dqb.dqb_bsoftlimit ?
-dquot->dq_dqb.dqb_bsoftlimit :
-dquot->dq_dqb.dqb_bhardlimit) >> sb->s_blocksize_bits;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_bsoftlimit)
+   limit = dquot->dq_dqb.dqb_bsoftlimit;
+   if (dquot->dq_dqb.dqb_bhardlimit &&
+   (!limit || dquot->dq_dqb.dqb_bhardlimit < limit))
+   limit = dquot->dq_dqb.dqb_bhardlimit;
+
if (limit && buf->f_blocks > limit) {
curblock = dquot->dq_dqb.dqb_curspace >> sb->s_blocksize_bits;
buf->f_blocks = limit;
@@ -1159,9 +1163,13 @@ static int f2fs_statfs_project(struct super_block *sb,
 (buf->f_blocks - curblock) : 0;
}
 
-   limit = dquot->dq_dqb.dqb_isoftlimit ?
-   dquot->dq_dqb.dqb_isoftlimit :
-   dquot->dq_dqb.dqb_ihardlimit;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_isoftlimit)
+   limit = dquot->dq_dqb.dqb_isoftlimit;
+   if (dquot->dq_dqb.dqb_ihardlimit &&
+   (!limit || dquot->dq_dqb.dqb_ihardlimit < limit))
+   limit = dquot->dq_dqb.dqb_ihardlimit;
+
if (limit && buf->f_files > limit) {
buf->f_files = limit;
buf->f_ffree =
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 4.19 05/79] f2fs: fix to update time in lazytime mode

2019-12-11 Thread Sasha Levin 
From: Chao Yu 

[ Upstream commit fe1897eaa6646f5a64a4cee0e6473ed9887d324b ]

generic/018 reports an inconsistent status of atime, the
testcase is as below:
- open file with O_SYNC
- write file to construct fraged space
- calc md5 of file
- record {a,c,m}time
- defrag file --- do nothing
- umount & mount
- check {a,c,m}time

The root cause is, as f2fs enables lazytime by default, atime
update will dirty vfs inode, rather than dirtying f2fs inode (by set
with FI_DIRTY_INODE), so later f2fs_write_inode() called from VFS will
fail to update inode page due to our skip:

f2fs_write_inode()
if (is_inode_flag_set(inode, FI_DIRTY_INODE))
return 0;

So eventually, after evict(), we lose last atime for ever.

To fix this issue, we need to check whether {a,c,m,cr}time is
consistent in between inode cache and inode page, and only skip
f2fs_update_inode() if f2fs inode is not dirty and time is
consistent as well.

Signed-off-by: Chao Yu 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/f2fs.h  | 23 +++
 fs/f2fs/inode.c |  6 +-
 2 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 34e48bcf50874..72d154e71bb56 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2578,6 +2578,20 @@ static inline void clear_file(struct inode *inode, int 
type)
f2fs_mark_inode_dirty_sync(inode, true);
 }
 
+static inline bool f2fs_is_time_consistent(struct inode *inode)
+{
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time, &inode->i_atime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 1, &inode->i_ctime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 2, &inode->i_mtime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 3,
+   &F2FS_I(inode)->i_crtime))
+   return false;
+   return true;
+}
+
 static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
 {
bool ret;
@@ -2595,14 +2609,7 @@ static inline bool f2fs_skip_inode_update(struct inode 
*inode, int dsync)
i_size_read(inode) & ~PAGE_MASK)
return false;
 
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time, &inode->i_atime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 1, &inode->i_ctime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 2, &inode->i_mtime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 3,
-   &F2FS_I(inode)->i_crtime))
+   if (!f2fs_is_time_consistent(inode))
return false;
 
down_read(&F2FS_I(inode)->i_sem);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index 540d45759621a..a01be7d8db867 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -614,7 +614,11 @@ int f2fs_write_inode(struct inode *inode, struct 
writeback_control *wbc)
inode->i_ino == F2FS_META_INO(sbi))
return 0;
 
-   if (!is_inode_flag_set(inode, FI_DIRTY_INODE))
+   /*
+* atime could be updated without dirtying f2fs inode in lazytime mode
+*/
+   if (f2fs_is_time_consistent(inode) &&
+   !is_inode_flag_set(inode, FI_DIRTY_INODE))
return 0;
 
/*
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 4.19 28/79] f2fs: fix to update dir's i_pino during cross_rename

2019-12-11 Thread Sasha Levin 
From: Chao Yu 

[ Upstream commit 2a60637f06ac94869b2e630eaf837110d39bf291 ]

As Eric reported:

RENAME_EXCHANGE support was just added to fsstress in xfstests:

commit 65dfd40a97b6bbbd2a22538977bab355c5bc0f06
Author: kaixuxia 
Date:   Thu Oct 31 14:41:48 2019 +0800

fsstress: add EXCHANGE renameat2 support

This is causing xfstest generic/579 to fail due to fsck.f2fs reporting errors.
I'm not sure what the problem is, but it still happens even with all the
fs-verity stuff in the test commented out, so that the test just runs fsstress.

generic/579 23s ... [10:02:25]
[7.745370] run fstests generic/579 at 2019-11-04 10:02:25
_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
(see /results/f2fs/results-default/generic/579.full for details)
 [10:02:47]
Ran: generic/579
Failures: generic/579
Failed 1 of 1 tests
Xunit report: /results/f2fs/results-default/result.xml

Here's the contents of 579.full:

_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
*** fsck.f2fs output ***
[ASSERT] (__chk_dots_dentries:1378)  --> Bad inode number[0x24] for '..', 
parent parent ino is [0xd10]

The root cause is that we forgot to update directory's i_pino during
cross_rename, fix it.

Fixes: 32f9bc25cbda0 ("f2fs: support ->rename2()")
Signed-off-by: Chao Yu 
Tested-by: Eric Biggers 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/namei.c | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index 6b23dcbf52f45..0ace2c2e3de93 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -948,7 +948,8 @@ static int f2fs_rename(struct inode *old_dir, struct dentry 
*old_dentry,
if (!old_dir_entry || whiteout)
file_lost_pino(old_inode);
else
-   F2FS_I(old_inode)->i_pino = new_dir->i_ino;
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_inode->i_ctime = current_time(old_inode);
@@ -1103,7 +1104,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(old_dir, old_entry, old_page, new_inode);
 
down_write(&F2FS_I(old_inode)->i_sem);
-   file_lost_pino(old_inode);
+   if (!old_dir_entry)
+   file_lost_pino(old_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_dir->i_ctime = current_time(old_dir);
@@ -1118,7 +1123,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(new_dir, new_entry, new_page, old_inode);
 
down_write(&F2FS_I(new_inode)->i_sem);
-   file_lost_pino(new_inode);
+   if (!new_dir_entry)
+   file_lost_pino(new_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(new_inode, old_dir->i_ino);
up_write(&F2FS_I(new_inode)->i_sem);
 
new_dir->i_ctime = current_time(new_dir);
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 5.4 112/134] f2fs: choose hardlimit when softlimit is larger than hardlimit in f2fs_statfs_project()

2019-12-11 Thread Sasha Levin 
From: Chengguang Xu 

[ Upstream commit 909110c060f22e65756659ec6fa957ae75777e00 ]

Setting softlimit larger than hardlimit seems meaningless
for disk quota but currently it is allowed. In this case,
there may be a bit of comfusion for users when they run
df comamnd to directory which has project quota.

For example, we set 20M softlimit and 10M hardlimit of
block usage limit for project quota of test_dir(project id 123).

[root@hades f2fs]# repquota -P -a
*** Report for project quotas on device /dev/nvme0n1p8
Block grace time: 7days; Inode grace time: 7days
Block limits File limits
Project used soft hard grace used soft hard grace
--
0 -- 4 0 0 1 0 0
123 +- 10248 20480 10240 2 0 0

The result of df command as below:

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 20M 11M 10M 51% /mnt/f2fs

Even though it looks like there is another 10M free space to use,
if we write new data to diretory test(inherit project id),
the write will fail with errno(-EDQUOT).

After this patch, the df result looks like below.

[root@hades f2fs]# df -h /mnt/f2fs/test
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p8 10M 10M 0 100% /mnt/f2fs

Signed-off-by: Chengguang Xu 
Reviewed-by: Chao Yu 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/super.c | 20 ++--
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 1443cee158633..a2af155567b80 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -1213,9 +1213,13 @@ static int f2fs_statfs_project(struct super_block *sb,
return PTR_ERR(dquot);
spin_lock(&dquot->dq_dqb_lock);
 
-   limit = (dquot->dq_dqb.dqb_bsoftlimit ?
-dquot->dq_dqb.dqb_bsoftlimit :
-dquot->dq_dqb.dqb_bhardlimit) >> sb->s_blocksize_bits;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_bsoftlimit)
+   limit = dquot->dq_dqb.dqb_bsoftlimit;
+   if (dquot->dq_dqb.dqb_bhardlimit &&
+   (!limit || dquot->dq_dqb.dqb_bhardlimit < limit))
+   limit = dquot->dq_dqb.dqb_bhardlimit;
+
if (limit && buf->f_blocks > limit) {
curblock = dquot->dq_dqb.dqb_curspace >> sb->s_blocksize_bits;
buf->f_blocks = limit;
@@ -1224,9 +1228,13 @@ static int f2fs_statfs_project(struct super_block *sb,
 (buf->f_blocks - curblock) : 0;
}
 
-   limit = dquot->dq_dqb.dqb_isoftlimit ?
-   dquot->dq_dqb.dqb_isoftlimit :
-   dquot->dq_dqb.dqb_ihardlimit;
+   limit = 0;
+   if (dquot->dq_dqb.dqb_isoftlimit)
+   limit = dquot->dq_dqb.dqb_isoftlimit;
+   if (dquot->dq_dqb.dqb_ihardlimit &&
+   (!limit || dquot->dq_dqb.dqb_ihardlimit < limit))
+   limit = dquot->dq_dqb.dqb_ihardlimit;
+
if (limit && buf->f_files > limit) {
buf->f_files = limit;
buf->f_ffree =
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 5.4 099/134] f2fs: Fix deadlock in f2fs_gc() context during atomic files handling

2019-12-11 Thread Sasha Levin 
From: Sahitya Tummala 

[ Upstream commit 677017d196ba2a4cfff13626b951cc9a206b8c7c ]

The FS got stuck in the below stack when the storage is almost
full/dirty condition (when FG_GC is being done).

schedule_timeout
io_schedule_timeout
congestion_wait
f2fs_drop_inmem_pages_all
f2fs_gc
f2fs_balance_fs
__write_node_page
f2fs_fsync_node_pages
f2fs_do_sync_file
f2fs_ioctl

The root cause for this issue is there is a potential infinite loop
in f2fs_drop_inmem_pages_all() for the case where gc_failure is true
and when there an inode whose i_gc_failures[GC_FAILURE_ATOMIC] is
not set. Fix this by keeping track of the total atomic files
currently opened and using that to exit from this condition.

Fix-suggested-by: Chao Yu 
Signed-off-by: Chao Yu 
Signed-off-by: Sahitya Tummala 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/f2fs.h|  1 +
 fs/f2fs/file.c|  1 +
 fs/f2fs/segment.c | 21 +++--
 3 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index f078cd20dab88..9046432b87c2d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -1289,6 +1289,7 @@ struct f2fs_sb_info {
unsigned int gc_mode;   /* current GC state */
unsigned int next_victim_seg[2];/* next segment in victim 
section */
/* for skip statistic */
+   unsigned int atomic_files;  /* # of opened atomic file */
unsigned long long skipped_atomic_files[2]; /* FG_GC and BG_GC */
unsigned long long skipped_gc_rwsem;/* FG_GC only */
 
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 29bc0a542759a..8ed8e4328bd1a 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1890,6 +1890,7 @@ static int f2fs_ioc_start_atomic_write(struct file *filp)
spin_lock(&sbi->inode_lock[ATOMIC_FILE]);
if (list_empty(&fi->inmem_ilist))
list_add_tail(&fi->inmem_ilist, &sbi->inode_list[ATOMIC_FILE]);
+   sbi->atomic_files++;
spin_unlock(&sbi->inode_lock[ATOMIC_FILE]);
 
/* add inode in inmem_list first and set atomic_file */
diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
index 8087095814819..7d85784012678 100644
--- a/fs/f2fs/segment.c
+++ b/fs/f2fs/segment.c
@@ -288,6 +288,8 @@ void f2fs_drop_inmem_pages_all(struct f2fs_sb_info *sbi, 
bool gc_failure)
struct list_head *head = &sbi->inode_list[ATOMIC_FILE];
struct inode *inode;
struct f2fs_inode_info *fi;
+   unsigned int count = sbi->atomic_files;
+   unsigned int looped = 0;
 next:
spin_lock(&sbi->inode_lock[ATOMIC_FILE]);
if (list_empty(head)) {
@@ -296,22 +298,26 @@ next:
}
fi = list_first_entry(head, struct f2fs_inode_info, inmem_ilist);
inode = igrab(&fi->vfs_inode);
+   if (inode)
+   list_move_tail(&fi->inmem_ilist, head);
spin_unlock(&sbi->inode_lock[ATOMIC_FILE]);
 
if (inode) {
if (gc_failure) {
-   if (fi->i_gc_failures[GC_FAILURE_ATOMIC])
-   goto drop;
-   goto skip;
+   if (!fi->i_gc_failures[GC_FAILURE_ATOMIC])
+   goto skip;
}
-drop:
set_inode_flag(inode, FI_ATOMIC_REVOKE_REQUEST);
f2fs_drop_inmem_pages(inode);
+skip:
iput(inode);
}
-skip:
congestion_wait(BLK_RW_ASYNC, HZ/50);
cond_resched();
+   if (gc_failure) {
+   if (++looped >= count)
+   return;
+   }
goto next;
 }
 
@@ -327,13 +333,16 @@ void f2fs_drop_inmem_pages(struct inode *inode)
mutex_unlock(&fi->inmem_lock);
}
 
-   clear_inode_flag(inode, FI_ATOMIC_FILE);
fi->i_gc_failures[GC_FAILURE_ATOMIC] = 0;
stat_dec_atomic_write(inode);
 
spin_lock(&sbi->inode_lock[ATOMIC_FILE]);
if (!list_empty(&fi->inmem_ilist))
list_del_init(&fi->inmem_ilist);
+   if (f2fs_is_atomic_file(inode)) {
+   clear_inode_flag(inode, FI_ATOMIC_FILE);
+   sbi->atomic_files--;
+   }
spin_unlock(&sbi->inode_lock[ATOMIC_FILE]);
 }
 
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 5.4 009/134] f2fs: fix to update time in lazytime mode

2019-12-11 Thread Sasha Levin 
From: Chao Yu 

[ Upstream commit fe1897eaa6646f5a64a4cee0e6473ed9887d324b ]

generic/018 reports an inconsistent status of atime, the
testcase is as below:
- open file with O_SYNC
- write file to construct fraged space
- calc md5 of file
- record {a,c,m}time
- defrag file --- do nothing
- umount & mount
- check {a,c,m}time

The root cause is, as f2fs enables lazytime by default, atime
update will dirty vfs inode, rather than dirtying f2fs inode (by set
with FI_DIRTY_INODE), so later f2fs_write_inode() called from VFS will
fail to update inode page due to our skip:

f2fs_write_inode()
if (is_inode_flag_set(inode, FI_DIRTY_INODE))
return 0;

So eventually, after evict(), we lose last atime for ever.

To fix this issue, we need to check whether {a,c,m,cr}time is
consistent in between inode cache and inode page, and only skip
f2fs_update_inode() if f2fs inode is not dirty and time is
consistent as well.

Signed-off-by: Chao Yu 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/f2fs.h  | 23 +++
 fs/f2fs/inode.c |  6 +-
 2 files changed, 20 insertions(+), 9 deletions(-)

diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 4024790028aab..f078cd20dab88 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2704,6 +2704,20 @@ static inline void clear_file(struct inode *inode, int 
type)
f2fs_mark_inode_dirty_sync(inode, true);
 }
 
+static inline bool f2fs_is_time_consistent(struct inode *inode)
+{
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time, &inode->i_atime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 1, &inode->i_ctime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 2, &inode->i_mtime))
+   return false;
+   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 3,
+   &F2FS_I(inode)->i_crtime))
+   return false;
+   return true;
+}
+
 static inline bool f2fs_skip_inode_update(struct inode *inode, int dsync)
 {
bool ret;
@@ -2721,14 +2735,7 @@ static inline bool f2fs_skip_inode_update(struct inode 
*inode, int dsync)
i_size_read(inode) & ~PAGE_MASK)
return false;
 
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time, &inode->i_atime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 1, &inode->i_ctime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 2, &inode->i_mtime))
-   return false;
-   if (!timespec64_equal(F2FS_I(inode)->i_disk_time + 3,
-   &F2FS_I(inode)->i_crtime))
+   if (!f2fs_is_time_consistent(inode))
return false;
 
down_read(&F2FS_I(inode)->i_sem);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index db4fec30c30df..386ad54c13c3a 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -615,7 +615,11 @@ int f2fs_write_inode(struct inode *inode, struct 
writeback_control *wbc)
inode->i_ino == F2FS_META_INO(sbi))
return 0;
 
-   if (!is_inode_flag_set(inode, FI_DIRTY_INODE))
+   /*
+* atime could be updated without dirtying f2fs inode in lazytime mode
+*/
+   if (f2fs_is_time_consistent(inode) &&
+   !is_inode_flag_set(inode, FI_DIRTY_INODE))
return 0;
 
if (!f2fs_is_checkpoint_ready(sbi))
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[f2fs-dev] [PATCH AUTOSEL 5.4 047/134] f2fs: fix to update dir's i_pino during cross_rename

2019-12-11 Thread Sasha Levin 
From: Chao Yu 

[ Upstream commit 2a60637f06ac94869b2e630eaf837110d39bf291 ]

As Eric reported:

RENAME_EXCHANGE support was just added to fsstress in xfstests:

commit 65dfd40a97b6bbbd2a22538977bab355c5bc0f06
Author: kaixuxia 
Date:   Thu Oct 31 14:41:48 2019 +0800

fsstress: add EXCHANGE renameat2 support

This is causing xfstest generic/579 to fail due to fsck.f2fs reporting errors.
I'm not sure what the problem is, but it still happens even with all the
fs-verity stuff in the test commented out, so that the test just runs fsstress.

generic/579 23s ... [10:02:25]
[7.745370] run fstests generic/579 at 2019-11-04 10:02:25
_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
(see /results/f2fs/results-default/generic/579.full for details)
 [10:02:47]
Ran: generic/579
Failures: generic/579
Failed 1 of 1 tests
Xunit report: /results/f2fs/results-default/result.xml

Here's the contents of 579.full:

_check_generic_filesystem: filesystem on /dev/vdc is inconsistent
*** fsck.f2fs output ***
[ASSERT] (__chk_dots_dentries:1378)  --> Bad inode number[0x24] for '..', 
parent parent ino is [0xd10]

The root cause is that we forgot to update directory's i_pino during
cross_rename, fix it.

Fixes: 32f9bc25cbda0 ("f2fs: support ->rename2()")
Signed-off-by: Chao Yu 
Tested-by: Eric Biggers 
Signed-off-by: Jaegeuk Kim 
Signed-off-by: Sasha Levin 
---
 fs/f2fs/namei.c | 15 ---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index 4faf06e8bf899..a1c507b0b4ac4 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -981,7 +981,8 @@ static int f2fs_rename(struct inode *old_dir, struct dentry 
*old_dentry,
if (!old_dir_entry || whiteout)
file_lost_pino(old_inode);
else
-   F2FS_I(old_inode)->i_pino = new_dir->i_ino;
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_inode->i_ctime = current_time(old_inode);
@@ -1141,7 +1142,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(old_dir, old_entry, old_page, new_inode);
 
down_write(&F2FS_I(old_inode)->i_sem);
-   file_lost_pino(old_inode);
+   if (!old_dir_entry)
+   file_lost_pino(old_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(old_inode, new_dir->i_ino);
up_write(&F2FS_I(old_inode)->i_sem);
 
old_dir->i_ctime = current_time(old_dir);
@@ -1156,7 +1161,11 @@ static int f2fs_cross_rename(struct inode *old_dir, 
struct dentry *old_dentry,
f2fs_set_link(new_dir, new_entry, new_page, old_inode);
 
down_write(&F2FS_I(new_inode)->i_sem);
-   file_lost_pino(new_inode);
+   if (!new_dir_entry)
+   file_lost_pino(new_inode);
+   else
+   /* adjust dir's i_pino to pass fsck check */
+   f2fs_i_pino_write(new_inode, old_dir->i_ino);
up_write(&F2FS_I(new_inode)->i_sem);
 
new_dir->i_ctime = current_time(new_dir);
-- 
2.20.1



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH v5] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-11 Thread Gao Xiang via Linux-f2fs-devel 
Hi Matthew,

On Wed, Dec 11, 2019 at 05:40:14AM -0800, Matthew Wilcox wrote:
> On Wed, Dec 11, 2019 at 03:17:11PM +0800, Gao Xiang wrote:
> > > static inline bool is_dot_or_dotdot(const unsigned char *name, size_t len)
> > > {
> > > if (len >= 1 && unlikely(name[0] == '.')) {
> > 
> > 
> > And I suggest drop "unlikely" here since files start with prefix
> > '.' (plus specical ".", "..") are not as uncommon as you expected...
> 
> They absolutely are uncommon.  Even if you just consider
> /home/willy/kernel/linux/.git/config, only one of those six path elements
> starts with a '.'.

Okay, I think it depends on userdata and access patterns.
I admit I have no statistics on all those callers.

Just considering introducing an inline helper for cleanup, except for
lookup_one_len_common() (since it's on an error path), others were all
without unlikely() before.

Ignore my words if it seems unreasonable or unlikely() is an improvement
in this patch and sorry for annoying.

Thanks,
Gao Xiang



___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [PATCH v5] fs: introduce is_dot_or_dotdot helper for cleanup

2019-12-11 Thread Matthew Wilcox 
On Wed, Dec 11, 2019 at 03:17:11PM +0800, Gao Xiang wrote:
> > static inline bool is_dot_or_dotdot(const unsigned char *name, size_t len)
> > {
> > if (len >= 1 && unlikely(name[0] == '.')) {
> 
> 
> And I suggest drop "unlikely" here since files start with prefix
> '.' (plus specical ".", "..") are not as uncommon as you expected...

They absolutely are uncommon.  Even if you just consider
/home/willy/kernel/linux/.git/config, only one of those six path elements
starts with a '.'.


___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel