Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-04-13 Thread Wang Yugui
Hi, This patch works. From:Dennis Zhou To: Wang Yugui Cc: Vlastimil Babka , linux...@kvack.org, linux-btrfs@vger.kernel.org Date:Thu, 8 Apr 2021 13:48:33 + Subject: Re: unexpected -ENOMEM from percpu_counter_init() Ah. Can you try the

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-11 Thread Wang Yugui
Hi, > On Sun, Apr 11, 2021 at 11:20:00PM +0800, Wang Yugui wrote: > > Hi, Dennis Zhou > > > > > Hi, > > > > > > > On Sat, Apr 10, 2021 at 11:29:17PM +0800, Wang Yugui wrote: > > > > > Hi, Dennis Zhou > > > > >

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-11 Thread Wang Yugui
Hi, Dennis Zhou > Hi, > > > On Sat, Apr 10, 2021 at 11:29:17PM +0800, Wang Yugui wrote: > > > Hi, Dennis Zhou > > > > > > Thanks for your ncie answer. > > > but still a few questions. > > > > > > > Percpu is not really chea

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-10 Thread Wang Yugui
Hi, > On Sat, Apr 10, 2021 at 11:29:17PM +0800, Wang Yugui wrote: > > Hi, Dennis Zhou > > > > Thanks for your ncie answer. > > but still a few questions. > > > > > Percpu is not really cheap memory to allocate because it has a > > > amplific

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-10 Thread Wang Yugui
17Mi 181Gi 175Gi Swap:0B 0B 0B vm.min_free_kbytes is auto configed to 4Gi(4194304) # write files with the size >= memory size *3 #for((i=0;i<10;++i));do dd if=/dev/zero bs=1M count=64K of=/nodetmp/${i}.txt; free -h; done any advice or patch to let the value of 'free' a little bigger? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/04/10

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-09 Thread Wang Yugui
even if 1 chunk type dropped below, the other chunk > > type might have enough pages. I'm queuing this for 5.12 and will send it > > out assuming it does fix your problem. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/04/09 > +CC btrfs > > On 4/1/21 12:51 PM, Wa

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-09 Thread Wang Yugui
ication pipeline. # nproc 40 # top top - 15:43:06 up 10:16, 1 user, load average: 41.39, 37.90, 35.98 Tasks: 488 total, 3 running, 485 sleeping, 0 stopped, 0 zombie %Cpu(s): 99.6 us, 0.1 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.3 hi, 0.0 si, 0.0 st MiB Mem : 58.3/193384.1 [|

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-09 Thread Wang Yugui
nough, it will save all the buffer to temp file, so it is sometimes high-IO-load too(write 60G or more to file). xfstests(generic/476) is just high-IO-load, cpu/memory load is NOT high. so xfstests(generic/476) maybe easy than our application pipeline. Although there is yet not a simple reproducer for another problem happend here, but there is a little high chance that something is wrong in btrfs/mm/fs-buffer. > but another problem(os freezed without call trace, PANIC without OOPS?, > the reason is yet unkown) still happen. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/04/09

Re: [PATCH] btrfs-progs: utils: fix btrfs_wipe_existing_sb probe bug

2021-04-08 Thread Wang Yugui
aad-457f-a53c-110592d8941f" ... Without this patch, 'mkfs.btrfs -f /dev/nvme0n1 /dev/sdb' have some issue too when /dev/nvme0n1 and /dev/sdb have some partitions. some blkid of partition is still left in the output of blkid. 'blockdev --rereadpt' will let them disappear. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/04/09

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-08 Thread Wang Yugui
Hi, > On Fri, Apr 09, 2021 at 08:08:00AM +0800, Wang Yugui wrote: > > Hi, > > > > > > kernel: at least 5.10.26/5.10.27/5.10.28 > > > > > > > > This problem is triggered by our application, NOT xfstests. > > > > But our applicaito

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-08 Thread Wang Yugui
by the way, this problem still happen in kernel 5.10.28+this patch. Is this is a PANIC without OOPS? any guide for troubleshooting please. > problem: > OS/VGA console is freezed , and no call trace is outputed. > Just some info is outputed to IPMI/dell iDRAC >2 | 04/03/2021 | 11:35

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-08 Thread Wang Yugui
Hi, > On Thu, Apr 08, 2021 at 07:28:01AM +0800, Wang Yugui wrote: > > Hi, > > > > > > > > upper caller: > > > > > > nofs_flag = memalloc_nofs_save(); > > > > > > ret = btrfs_drew_lock_init(&root->snapshot_lock);

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-07 Thread Wang Yugui
u attempts an atomic > allocation. If it cannot find anything already allocated it isn't happy. > This was done before memalloc_nofs_{save/restore}() were pervasive. > > Percpu should probably try to allocate some pages if possible even if > nofs is set. Thanks. I will wait

Re: unexpected -ENOMEM from percpu_counter_init()

2021-04-07 Thread Wang Yugui
Hi, > +CC btrfs > > On 4/1/21 12:51 PM, Wang Yugui wrote: > > Hi, > > > > an unexpected -ENOMEM from percpu_counter_init() happened when xfstest > > with kernel 5.11.10 and 5.10.27 > > Is there a dmesg log showing allocation failure or

Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-03-30 Thread Wang Yugui
H, > On 30.03.21 г. 9:24, Wang Yugui wrote: > > Hi, Nikolay Borisov > > > > With a lot of dump_stack()/printk inserted around ENOMEM in btrfs code, > > we find out the call stack for ENOMEM. > > see the file -btrfs-dump_stack-when-ENOMEM.patch > > >

Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-03-28 Thread Wang Yugui
ened in test. 2) switch to use mempool_t for btrfs_path see 0001-btrfs-switch-to-mempool_t-for-btrfs_path.patch this problem yet not happen in test. But the memory alloc failure is difficult to test, we need more review. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/28

Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-03-27 Thread Wang Yugui
kmem_cache_zalloc() with __GFP_NOFAIL just like xfs or use mempool with pre-alloc to prevent fail? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/27 > Hi, > > these callstack have a same root failure. > struct btrfs_path *btrfs_alloc_path(void) > { > return

Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-03-27 Thread Wang Yugui
, skinny-metadata, no-holes Runtime features: free-space-tree Checksum: crc32c Number of devices: 1 Devices: IDSIZE PATH 110.00GiB /dev/sdb1 Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/27 > Hi, > > SSD/SAS is easy than SSD/NVMe to reproduce thi

Re: xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10)

2021-03-27 Thread Wang Yugui
ERR(new_root); btrfs_abort_transaction(trans, ret); goto fail; } Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/27 > Hi, > > xfstests generic/476 failed on btrfs(errno=-12 Out of memory, kernel 5.11.10) > > The hardware of this server: > CPU: Xeon(R) CPU E5-2660 v

Re: [PATCH 5/9] btrfs: use a bit to track the existence of tree mod log users

2021-03-12 Thread Wang Yugui
OD_LOG_USERS at btrfs_free_tree_block()'? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/13 > From: Filipe Manana > > The tree modification log functions are called very frequently, basically > they are called everytime a btree is modified (a pointer added or removed > to a node,

Re: BTRFS error (device sda1): bdev /dev/sdb1 errs: wr 2702175, rd 2719033, flush 0, corrupt 6, gen 0

2021-03-12 Thread Wang Yugui
Hi, If there is some partition size change after filesystem created, 'btrfs filesystem resize max' will help. See also https://lore.kernel.org/linux-btrfs/42d37a6393db7ad5d83bc167459c8...@steev.me.uk/T/#t Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/12 > Hello

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

2021-03-08 Thread Xiaoguang Wang
hi, First thanks for your patchset. I'd like to know whether your patchset pass fstests? Thanks. Regards, Xiaoguang Wang This patchset is attempt to add CoW support for fsdax, and take XFS, which has both reflink and fsdax feature, as an example. Changes from V1: - Factor some h

Re: btrfs fails to mount on kernel 5.11.4 but works on 5.10.19

2021-03-07 Thread Wang Yugui
d-rhel/rhel-7/issues/129 Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/08 > Dear all > > (please cc) > > not sure this is the right mailing list, but I cannot boot into 5.11.4 > it gives me > devid 9 uui . > failed to read the system array:

Re: [PATCH] btrfs-progs: fix false alert on tree block crossing 64K page boundary

2021-03-05 Thread Wang Yugui
Hi, It passed the test. Thanks a lot. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/06 > [BUG] > When btrfs-check is executed on even newly created fs, it can report > tree blocks crossing 64K page boundary like this: > > Opening filesystem to check... > Chec

Re: [PATCH 1/2] btrfs-progs: check: detect and warn about tree blocks cross 64K page boundary

2021-03-02 Thread Wang Yugui
primary 4 47448064s 63176703s 15728640s primary 5 63176704s 78905343s 15728640s primary Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/02 > > > On 2021/3/2 下午4:48, Wang Yugui wrote: > > Hi, Qu Wenruo > > > > This warning

Re: [PATCH 1/2] btrfs-progs: check: detect and warn about tree blocks cross 64K page boundary

2021-03-02 Thread Wang Yugui
Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 1048576kiB63963136kiB 62914560kiB btrfsprimary Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/03/02 > For the incoming subpage support, there is a

Re: xfstest btrfs/154 failed at kernel 5.4.100

2021-02-25 Thread Wang Yugui
Hi, btrfs/154 passed with with the backport of this patch. btrfs: correctly calculate item size used when item key collision happens Thanks a lot. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/25 > > > On 25.02.21 г. 9:53 ч., Wang Yugui wrote: > > Hi, > >

xfstest btrfs/154 failed at kernel 5.4.100

2021-02-24 Thread Wang Yugui
y->d_name.name, 10158 new_dentry->d_name.len, 0, index); 10159 if (ret) { 10160 btrfs_abort_transaction(trans, ret); 10161 goto out_fail; 10162 } Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/25

Re: [PATCH 1/7] fsdax: Output address in dax_iomap_pfn() and rename it

2021-02-21 Thread Xiaoguang Wang
NULL, pfnp); + kaddr, pfnp); if (length < 0) { rc = length; goto out; } + if (!pfnp) Should this be "if (!*pfnp)"? Regards, Xiaoguang Wang + goto out_check_addr; rc = -EINVAL;

Re: 5.11 free space tree remount warning

2021-02-20 Thread Wang Yugui
supports changing free space tree only from ro to rw 2, no space_cache=v2 in /etc/fstab but space_cache=v2 is reported in /proc/mounts systemd-shutdown ro remount / with the param based on /proc/mount Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/21 > Hi, > > systemd doe

Re: error in backport of 'btrfs: fix possible free space tree corruption with online conversion'

2021-02-19 Thread Wang Yugui
Hi, > On 2021-02-19 04:17, Wang Yugui wrote: > > Hi, Josef Bacik > > > > We noticed an error in 5.10.x backport of 'btrfs: fix possible free > > space tree corruption with online conversion' > > > > It is wrong in 5.10.13, but right in 5

error in backport of 'btrfs: fix possible free space tree corruption with online conversion'

2021-02-18 Thread Wang Yugui
ACE_TREE_UNTRUSTED, }; the usage sample of this enum: set_bit(BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED, &fs_info->flags); Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/19

Re: [PATCH 5.10.x] btrfs: fix crash after non-aligned direct IO write with O_DSYNC

2021-02-12 Thread Wang Yugui
blem? the iomap issue for commit 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround") is already fixed in 5.10? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/13 > Fixes: 0eb79294dbe328 ("btrfs: dio iomap DSYNC workaround") > CC: sta...@vger.kernel.org # 5.10 (an

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-04 Thread Wang Yugui
Hi, > On 4.02.21 г. 13:34 ч., Wang Yugui wrote: > > Hi, > > > >> On 4.02.21 г. 5:17 ч., Wang Yugui wrote: > >>> Hi, > >>> > >>> I tried to run btrfs misc-next(5.11-rc6 +81patches) based on linux LTS > >>>

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-04 Thread Wang Yugui
Hi, > On 4.02.21 г. 5:17 ч., Wang Yugui wrote: > > Hi, > > > > I tried to run btrfs misc-next(5.11-rc6 +81patches) based on linux LTS > > 5.10.12 with the same other kernel components and the same kernel config. > > > > Better dbench(sync open) re

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-03 Thread Wang Yugui
-helper.patch # drop 0001-block-remove-i_bdev.patch # add 0001-btrfs-bdev_nr_sectors.patch Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/04 > Hi, > > There is the dbench(sync open) test result of misc-next(5.11-rc6 +81patches) > > 1, the MaxLat is changed from 1900ms level

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-03 Thread Wang Yugui
te 56 sec latency 7.016 ms 32189209 480.82 MB/sec execute 57 sec latency 6.789 ms 32192072 480.93 MB/sec execute 58 sec latency 7.305 ms 32195054 481.00 MB/sec execute 59 sec latency 7.432 ms Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/03 > > >

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-02 Thread Wang Yugui
2239.973/3.960=565.6. For QoS, can we have an option to tune the value of MaxLat / AvgLat of WriteX to less than 100? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/02 > Hi, Filipe Manana > > > On Tue, Feb 2, 2021 at 5:42 AM Wang Yugui wrote: > > > &g

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-02 Thread Wang Yugui
Hi, Filipe Manana > On Tue, Feb 2, 2021 at 5:42 AM Wang Yugui wrote: > > > > Hi, Filipe Manana > > > > The dbench result with these patches is very good. thanks a lot. > > > > This is the dbench(synchronous mode) result , and then a question. > &g

Re: [PATCH 6/6] btrfs: do not block inode logging for so long during transaction commit

2021-02-01 Thread Wang Yugui
62260 0.750 1659.364 Throughput 461.856 MB/sec (sync open) 32 clients 32 procs max_latency=1872.118 ms + stat -f -c %T /btrfs/ btrfs + uname -r 5.10.12-4.el7.x86_64 Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/02/02 > From: Filipe Manana > > Early on

Re: [PATCH v2 1/3] btrfs-progs: add Kernel TLS to btrfs send/receive

2021-01-03 Thread Wang Yugui
(pid of btrfs-5.9 send') is slow, and then 'ERROR: crc32 mismatch in command' happen in server side. A little difficult to understand. It seems thread-sync problem in client side? Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/01/03 > Hi, Sheng > > I test it again w

Re: [PATCH v2 1/3] btrfs-progs: add Kernel TLS to btrfs send/receive

2021-01-02 Thread Wang Yugui
resident)k 0inputs+0outputs (0major+313minor)pagefaults 0swaps + sleep 1 + cat time.send At subvol /archive/movie2 0.60user 231.95system 3:19.91elapsed 116%CPU (0avgtext+0avgdata 5768maxresident)k 66606224inputs+8outputs (0major+394minor)pagefaults 0swaps Best Regards Wang Yugui (wangyu...@e16-tech.com)

Re: [PATCH] btrfs-progs: align btrfs receive buffer to enable fast CRC

2021-01-02 Thread Wang Yugui
_SEND_BUF_SIZE] > + __attribute__((aligned(sizeof(unsigned long; > > int cmd; Can we move 'int cmd' to before 'char read_buf' too? There is a hole between 'int fd' and 'char read_buf' after the addiatioin of '__attri

Re: [PATCH v2 1/3] btrfs-progs: add Kernel TLS to btrfs send/receive

2021-01-02 Thread Wang Yugui
0 That is to say, for high speed network/disk config, the current bottleneck of btrfs send/receive is the CRC process in btrfs receive side. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/01/02 > From: Sheng Mao > > Currently, btrfs send outputs to a pipe or a file; > btr

Re: [PATCH 3/3] btrfs-progs: add TLS arguments to send/receive

2020-12-31 Thread Wang Yugui
hmark this for 10G Gbps or 40Gbs. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2021/01/01 > Thank you! Happy new year! > > Regards, > Sheng > > > On Dec 31, 2020, at 04:16, Wang Yugui wrote: > > > > Hi, Sheng Mao > > > > some feedback. >

Re: [PATCH 3/3] btrfs-progs: add TLS arguments to send/receive

2020-12-31 Thread Wang Yugui
sed > +as PEM format; if parsing fails, file content is treated as binary key. > + > +--tls-mode :: > +Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm. Best Regards Wang Yugui (wangyu...@e16-tech.com) 2020/12/31

Re: [PATCH v2 1/4] btrfs: Remove userspace transaction ioctls

2018-02-05 Thread Wang Shilong
These ioctl are originally introduced by Sage Weil for Ceph use? Not sure whether it still useful, Cc Sage just in case. 在 2018年2月5日,下午5:41,Nikolay Borisov 写道: Commit 3558d4f88ec8 ("btrfs: Deprecate userspace transaction ioctls") marked the beginning of the end of userspace transaction. Thi

[PATCH] btrfs: use correct string length

2018-01-08 Thread Xiongfeng Wang
From: Xiongfeng Wang gcc-8 reports fs/btrfs/ioctl.c: In function 'btrfs_ioctl': ./include/linux/string.h:245:9: warning: '__builtin_strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] We need one less byte or call strlcpy() to make it a nul-termin

Re: [PATCH 0/5] btrfs-progs: resolve property and enhance defragment

2017-11-01 Thread Wang Shilong
在 2017年11月2日,上午11:44,Su Yue 写道: Sorry, the patchset does not work as expected. Please ignore it. On 11/02/2017 11:23 AM, Su Yue wrote: > The patchset adds an option '--compress-force' to work with > 'btrfs fi defrag -c'. Then no-compression files will be set with > compression property specifie

Re: RedHat 7.4 Release Notes: "Btrfs has been deprecated" - wut?

2017-08-04 Thread Wang Shilong
Hi Qu, On Fri, Aug 4, 2017 at 10:05 PM, Qu Wenruo wrote: > > > On 2017年08月02日 16:38, Brendan Hide wrote: >> >> The title seems alarmist to me - and I suspect it is going to be >> misconstrued. :-/ >> >> From the release notes at >> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise

Re: [PATCH] Btrfs: avoid unnecessarily locking inode when clearing a range

2017-08-03 Thread Wang Shilong
us, we don't need to >> lock and unlock the inode. >> >> Signed-off-by: Filipe Manana > > > Thanks Filipe, looks like it goes all the way back to: > > commit 47059d930f0e002ff851beea87d738146804726d > Author: Wang Shilong > Date: Thu Jul 3 18:22:07 2

Re: RedHat 7.4 Release Notes: "Btrfs has been deprecated" - wut?

2017-08-02 Thread Wang Shilong
I haven't seen active btrfs developers from some time, Redhat looks put most of their efforts on XFS, It is time to switch to SLES/opensuse! On Wed, Aug 2, 2017 at 4:38 PM, Brendan Hide wrote: > The title seems alarmist to me - and I suspect it is going to be > misconstrued. :-/ > > From the rel

Re: [PULL] Btrfs for 4.13, part 1

2017-06-24 Thread Wang Shilong
Sorry for bikeshedding. On Fri, Jun 23, 2017 at 11:16 PM, David Sterba wrote: > Hi, > > this is the main batch for 4.13. There are some user visible changes, see > below. The core updates improve error handling (mostly related to bios), with > the usual incremental work on the GFP_NOFS (mis)use r

Re: [PATCH 0/3] introduce type based delalloc metadata reserve to fix some false enospc issues

2016-11-22 Thread Wang Xiaoguang
hello, Any comments about this patchset? The first two patches almost don't do any functional change which I think could be review firstly. btrfs: improve inode's outstanding_extents computation btrfs: introduce type based delalloc metadata reserve Regards, Xiaoguang Wang

Re: [PATCH] btrfs: limit the number of asynchronous delalloc pages to reasonable value

2016-11-22 Thread Wang Xiaoguang
hello, On 11/22/2016 01:59 AM, Chris Mason wrote: On 11/08/2016 04:30 AM, Wang Xiaoguang wrote: The current limit of number of asynchronous delalloc pages is (10 * SZ_1M). For 4K page, the total ram bytes would be 40G, very big value, I think in most cases, this limit will not work, here I

Re: [PATCH] btrfs: limit the number of asynchronous delalloc pages to reasonable value

2016-11-22 Thread Wang Xiaoguang
hello, On 11/22/2016 12:39 AM, David Sterba wrote: On Tue, Nov 08, 2016 at 05:30:58PM +0800, Wang Xiaoguang wrote: The current limit of number of asynchronous delalloc pages is (10 * SZ_1M). For 4K page, the total ram bytes would be 40G, very big value, I think in most cases, this limit will

Re: [RFC] btrfs: make max inline data can be equal to sectorsize

2016-11-22 Thread Wang Xiaoguang
hello, On 11/19/2016 04:58 AM, Chris Mason wrote: On 11/16/2016 11:10 AM, David Sterba wrote: On Mon, Nov 14, 2016 at 09:55:34AM +0800, Qu Wenruo wrote: At 11/12/2016 04:22 AM, Liu Bo wrote: On Tue, Oct 11, 2016 at 02:47:42PM +0800, Wang Xiaoguang wrote: If we use mount option &qu

[PATCH 1/3] btrfs: improve inode's outstanding_extents computation

2016-11-11 Thread Wang Xiaoguang
data. So btrfs_dirty_pages() won't need to handle cases cross 2 extents. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/inode.c | 65 ++-- fs/btrfs/ioctl.c | 6 ++ 3 files changed, 62 insertions(+), 11 deletions(-)

[PATCH 2/3] btrfs: introduce type based delalloc metadata reserve

2016-11-11 Thread Wang Xiaoguang
More types will follow soon. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 34 ++--- fs/btrfs/extent-tree.c | 50 +-- fs/btrfs/file.c | 22 ++--- fs/btrfs/free-space-cache.c | 6 ++- fs/btrfs/inode-

[PATCH 3/3] btrfs: Introduce COMPRESS reserve type to fix false enospc for compression

2016-11-11 Thread Wang Xiaoguang
eserved_extents = 1023 And in Step 5) we reserve correct amount of metadata space. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/extent-tree.c | 2 ++ fs/btrfs/extent_io.c | 61 -- fs/btrfs/extent_io

[PATCH 0/3] introduce type based delalloc metadata reserve to fix some false enospc issues

2016-11-11 Thread Wang Xiaoguang
will also pass. I have also run whole fstests multiple times, no regression occurs, thanks. Wang Xiaoguang (3): btrfs: improve inode's outstanding_extents computation btrfs: introduce type based delalloc metadata reserve btrfs: Introduce COMPRESS reserve type to fix false enospc for

[PATCH] btrfs: limit the number of asynchronous delalloc pages to reasonable value

2016-11-08 Thread Wang Xiaoguang
The current limit of number of asynchronous delalloc pages is (10 * SZ_1M). For 4K page, the total ram bytes would be 40G, very big value, I think in most cases, this limit will not work, here I set limit of the number of asynchronous delalloc pages to SZ_1M(4GB ram bytes). Signed-off-by: Wang

[PATCH 2/2] btrfs: increase tickets_id even for failed metadata request

2016-11-07 Thread Wang Xiaoguang
Not functional change, it just makes codes logic more reasonable, then at least tickets_id can reflect the number of metadata requests we already handled. Signed-off-by: Wang Xiaoguang --- fs/btrfs/extent-tree.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs

[PATCH 1/2] btrfs: add necessary comments about tickets_id

2016-11-07 Thread Wang Xiaoguang
Tickets_id's name may result in some misunderstandings, it just indicates the next ticket will be handled and is not stored per ticket. Fixes: ce12965 ("btrfs: introduce tickets_id to determine whether asynchronous metadata reclaim work makes progress") Signed-off-by: Wang Xia

Re: [PATCH] generic: create and delete files repeatedly to exercise ENOSPC behaviour

2016-11-02 Thread Wang Xiaoguang
hi, On 11/02/2016 09:27 AM, Dave Chinner wrote: On Tue, Nov 01, 2016 at 07:19:30PM +0800, Wang Xiaoguang wrote: In btrfs, sometimes though the number of created files' consumed disk space are not larger than fs's free space, we can still get some ENOSPC error, it may be that btrfs do

Re: [PATCH] generic: create and delete files repeatedly to exercise ENOSPC behaviour

2016-11-02 Thread Wang Xiaoguang
hi Eryu, There has already be a generic/102 doing this test... Thanks for you kindly review and sorry for wasting your time. Regards, Xiaoguang Wang On 11/01/2016 08:26 PM, Eryu Guan wrote: On Tue, Nov 01, 2016 at 07:19:30PM +0800, Wang Xiaoguang wrote: In btrfs, sometimes though the number

[PATCH v5 2/2] generic: make 17[1-4] work well when btrfs compression is enabled

2016-11-01 Thread Wang Xiaoguang
When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc error occurs. Note _fill_fs is copied from tests/generic/256, but with some minor modifications. Signed-off-by:

[PATCH v5 1/2] common/populate: use _require_xfs_io_command() in right place

2016-11-01 Thread Wang Xiaoguang
ommon/populate, so here I choose to put _require_xfs_io_command "falloc" or "punch" in helper function which really use falloc and fpunch. And xfs/120 uses fpunch, add _require_xfs_io_command "fpunch". Signed-off-by: Wang Xiaoguang --- common/populate | 7 --- tes

[PATCH] generic: create and delete files repeatedly to exercise ENOSPC behaviour

2016-11-01 Thread Wang Xiaoguang
lse enospc error will not always happen even in kernel without my fixing patch). Currently only in btrfs, I get this ENOSPC error, xfs and ext4 work well. Signed-off-by: Wang Xiaoguang --- tests/generic/389 | 78 +++ tests/generic/389.out | 2

Re: [PATCH 2/2] btrfs: fix false enospc for compression

2016-11-01 Thread Wang Xiaoguang
hi, I have rebased these 2 patches against Linux 4.9-rc2, sorry for being late. After applying these patches, Stefan does not see any ENOSPC error :) If you have free time, please have a check, thanks. Regards, Xiaoguang Wang On 11/01/2016 06:18 PM, Wang Xiaoguang wrote: When testing btrfs

[PATCH 2/2] btrfs: fix false enospc for compression

2016-11-01 Thread Wang Xiaoguang
MPRESS flag to mark a data range that will go through compression path. With this patch, we can fix these false enospc error for compression. Signed-off-by: Wang Xiaoguang Tested-by: Holger Hoffstätte Tested-by: Stefan Priebe --- fs/btrfs/ctree.h | 31 ++-- fs/btrfs/extent-tree.c

[PATCH 1/2] btrfs: improve inode's outstanding_extents computation

2016-11-01 Thread Wang Xiaoguang
data. So btrfs_dirty_pages() won't need to handle cases cross 2 extents. Signed-off-by: Wang Xiaoguang Tested-by: Holger Hoffstätte Tested-by: Stefan Priebe --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/inode.c | 65 ++-- fs/btrfs/ioctl.c | 6

Re: [PATCH v4] generic: make 17[1-4] work well when btrfs compression is enabled

2016-11-01 Thread Wang Xiaoguang
hi Darrick, Common/populate needs xfs_io supports falloc and fpunch, so I didn't put _fill_fs() in common/populate. Regards, Xiaoguang Wang On 11/01/2016 04:45 PM, Wang Xiaoguang wrote: When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs

[PATCH v4] generic: make 17[1-4] work well when btrfs compression is enabled

2016-11-01 Thread Wang Xiaoguang
When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc error occurs. Note _fill_fs is copied from tests/generic/256, but with some minor modifications. Signed-off-by:

Re: [PATCH v3] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-28 Thread Wang Xiaoguang
hi, On 10/27/2016 07:25 PM, Eryu Guan wrote: On Wed, Oct 26, 2016 at 05:52:11PM +0800, Wang Xiaoguang wrote: When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc error o

Re: [PATCH v3] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-28 Thread Wang Xiaoguang
hi, On 10/28/2016 01:13 AM, Darrick J. Wong wrote: On Wed, Oct 26, 2016 at 05:52:11PM +0800, Wang Xiaoguang wrote: When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc

[PATCH v3] btrfs: imporve delayed refs iterations

2016-10-26 Thread Wang Xiaoguang
el] [k] _raw_spin_unlock_irq For normal files, this patch also gives help, at least we do not need to iterate whole list to found BTRFS_ADD_DELAYED_REF nodes. Signed-off-by: Wang Xiaoguang Reviewed-by: Liu Bo Tested-by: Holger Hoffstätte --- V2: remove useless "if" statment and use A

[PATCH v3] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-26 Thread Wang Xiaoguang
When enabling btrfs compression, original codes can not fill fs correctly, here we introduce _fill_fs() in common/rc, which'll keep creating and writing files until enospc error occurs. Note _fill_fs is copied from tests/generic/256, but with some minor modifications. Signed-off-by:

Re: [RFC] btrfs: make max inline data can be equal to sectorsize

2016-10-26 Thread Wang Xiaoguang
hi, On 10/11/2016 02:47 PM, Wang Xiaoguang wrote: If we use mount option "-o max_inline=sectorsize", say 4096, indeed even for a fresh fs, say nodesize is 16k, we can not make the first 4k data completely inline, I found this conditon causing this issue: !compressed_size &

[PATCH v2] btrfs: cleanup: use already calculated value in btrfs_should_throttle_delayed_refs()

2016-10-26 Thread Wang Xiaoguang
Signed-off-by: Wang Xiaoguang --- fs/btrfs/extent-tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 9aa6d2c..3c8f0ec 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2823,7 +2823,7 @@ int

[PATCH] btrfs: minor codes cleanup

2016-10-25 Thread Wang Xiaoguang
Signed-off-by: Wang Xiaoguang --- V1: Just one small codes cleanup, if you think it's not appropriate to make a individual patch for it, please ignore it :) --- fs/btrfs/extent-tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/e

Re: BTRFS: space_info 4 has 18446742286429913088 free, is not full

2016-10-25 Thread Wang Xiaoguang
hi, On 10/24/2016 01:47 AM, Stefan Priebe - Profihost AG wrote: Hello list, just wanted to report that my ENOSPC errors are gone. Thanks to wang for his great patches. but the space_info corruption still occours. On every umount i see: [93022.166222] BTRFS: space_info 4 has 208952672256 free

Re: [PATCH 2/2] btrfs: fix false enospc for compression

2016-10-25 Thread Wang Xiaoguang
hi, On 10/19/2016 10:23 PM, David Sterba wrote: On Mon, Oct 17, 2016 at 05:01:46PM +0800, Wang Xiaoguang wrote: [..] int btrfs_set_extent_delalloc(struct inode *inode, u64 start, u64 end, - struct extent_state **cached_state

[PATCH v2] btrfs: imporve delayed refs iterations

2016-10-25 Thread Wang Xiaoguang
el] [k] _raw_spin_unlock_irq For normal files, this patch also gives help, at least we do not need to iterate whole list to found BTRFS_ADD_DELAYED_REF nodes. Signed-off-by: Wang Xiaoguang Reviewed-by: Liu Bo Tested-by: Holger Hoffstätte --- V2: remove useless "if" statment and use A

Re: [PATCH] btrfs: imporve delayed refs iterations

2016-10-25 Thread Wang Xiaoguang
hi, On 10/25/2016 03:00 AM, Liu Bo wrote: On Fri, Oct 21, 2016 at 05:05:07PM +0800, Wang Xiaoguang wrote: This issue was found when I tried to delete a heavily reflinked file, when deleting such files, other transaction operation will not have a chance to make progress, for example

[PATCH] btrfs: imporve delayed refs iterations

2016-10-21 Thread Wang Xiaoguang
el] [k] _raw_spin_unlock_irq For normal files, this patch also gives help, at least we do not need to iterate whole list to found BTRFS_ADD_DELAYED_REF nodes. Signed-off-by: Wang Xiaoguang --- fs/btrfs/delayed-ref.c | 14 ++ fs/btrfs/delayed-ref.h | 8 fs/btrfs/disk-io.c

Re: [PATCH] btrfs: pass correct args to btrfs_async_run_delayed_refs()

2016-10-18 Thread Wang Xiaoguang
hi, On 10/18/2016 06:32 PM, Holger Hoffstätte wrote: On Tue, 18 Oct 2016 15:56:13 +0800, Wang Xiaoguang wrote: In btrfs_truncate_inode_items()->btrfs_async_run_delayed_refs(), we swap the arg2 and arg3 wrongly, fix this. Signed-off-by: Wang Xiaoguang --- fs/btrfs/inode.c | 4 ++-- 1 f

[PATCH] btrfs: pass correct args to btrfs_async_run_delayed_refs()

2016-10-18 Thread Wang Xiaoguang
In btrfs_truncate_inode_items()->btrfs_async_run_delayed_refs(), we swap the arg2 and arg3 wrongly, fix this. Signed-off-by: Wang Xiaoguang --- fs/btrfs/inode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2b790bd..2f13

Re: [PATCH 2/2] btrfs: fix false enospc for compression

2016-10-17 Thread Wang Xiaoguang
hi, On 10/14/2016 09:59 PM, Holger Hoffstätte wrote: On 10/06/16 04:51, Wang Xiaoguang wrote: When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my

[PATCH v2] btrfs: introduce priority based delalloc shrink mechanism

2016-10-13 Thread Wang Xiaoguang
e, and as the priority increases, we will try wo write more delalloc bytes, meanwhile if "reclaim_priority == 0" returns true, we'll also wait all current ordered extents to finish. Signed-off-by: Wang Xiaoguang --- fs/btrfs/extent-tree.c | 63 +++

Re: [PATCH 1/2] btrfs: introduce priority based delalloc shrink mechanism

2016-10-12 Thread Wang Xiaoguang
hi, On 10/13/2016 01:20 AM, Josef Bacik wrote: On 10/12/2016 05:03 AM, Wang Xiaoguang wrote: Since commit b02441999efcc6152b87cd58e7970bb7843f76cf, we don't wait all ordered extents, but I run into some enospc errors when doing large file create and delete tests, it's because shrin

[PATCH] btrfs: make file clone aware of fatal signals

2016-10-12 Thread Wang Xiaoguang
Indeed this just make the behavior similar to xfs when process has fatal signals pending, and it'll make fstests/generic/298 happy. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ioctl.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 7f

[PATCH 1/2] btrfs: introduce priority based delalloc shrink mechanism

2016-10-12 Thread Wang Xiaoguang
a we want to reserve. Meanwhile if "reclaim_priority >= 3" returns true, we'll also wait all current ordered extents to finish. Signed-off-by: Wang Xiaoguang --- fs/btrfs/extent-tree.c | 49 include/trace/events/btrfs.h |

[PATCH 2/2] btrfs: try to satisfy metadata requests if wen can overcommit

2016-10-12 Thread Wang Xiaoguang
In shrink_delalloc(), if can_overcommit() returns true, shrink_delalloc() will give up shrinking delalloc bytes, in this case we should check whether some tickcts' requests can overcommit, if some can, we can satisfy them timely and directly. Signed-off-by: Wang Xiaoguang --- fs/btrfs/e

Re: [PATCH 1/2] btrfs: try to satisfy metadata requests when every flush_space() returns

2016-10-12 Thread Wang Xiaoguang
hi, On 10/07/2016 09:16 PM, Josef Bacik wrote: On 09/21/2016 02:59 AM, Wang Xiaoguang wrote: In flush_space()->shrink_delalloc(), if can_overcommit() returns true, though no bytes added to space_info, we still may satisfy some requests. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ext

Re: [RFC] btrfs: make max inline data can be equal to sectorsize

2016-10-11 Thread Wang Xiaoguang
hi, On 10/11/2016 11:49 PM, Chris Murphy wrote: On Tue, Oct 11, 2016 at 12:47 AM, Wang Xiaoguang wrote: If we use mount option "-o max_inline=sectorsize", say 4096, indeed even for a fresh fs, say nodesize is 16k, we can not make the first 4k data completely inline, I found thi

Re: [PATCH 2/2] btrfs: fix false enospc for compression

2016-10-11 Thread Wang Xiaoguang
, Xiaoguang Wang On 10/06/2016 10:51 AM, Wang Xiaoguang wrote: When testing btrfs compression, sometimes we got ENOSPC error, though fs still has much free space, xfstests generic/171, generic/172, generic/173, generic/174, generic/175 can reveal this bug in my test environment when compression is enabled

[RFC] btrfs: make max inline data can be equal to sectorsize

2016-10-10 Thread Wang Xiaoguang
If it retuns true, we'll not make data inline. For 4k sectorsize, 0~4094 dara range, we can make it inline, but 0~4095, it can not. I don't think this limition is useful, so here remove it which will make max inline data can be equal to sectorsize. Signed-off-by: Wang Xiaoguang --- fs/b

Re: [PATCH] generic/175: disable inline data feature for btrfs

2016-10-10 Thread Wang Xiaoguang
hi, On 10/11/2016 07:09 AM, Dave Chinner wrote: On Mon, Oct 10, 2016 at 01:06:47PM +0800, Wang Xiaoguang wrote: For btrfs, if compression is enabled, it may generate inline data for a blocksize data range, this inline data is stored in fs tree, will not have a individual extent, try to reflink

Re: BTRFS: space_info 4 has 18446742286429913088 free, is not full

2016-10-10 Thread Wang Xiaoguang
hi, On 10/11/2016 04:06 AM, Stefan Priebe - Profihost AG wrote: Dear Wang, Am 06.10.2016 um 05:04 schrieb Wang Xiaoguang: Hi, On 09/29/2016 03:27 PM, Stefan Priebe - Profihost AG wrote: Am 29.09.2016 um 09:13 schrieb Wang Xiaoguang: I found that compress sometime report ENOSPC error even

  1   2   3   4   5   6   7   8   9   10   >