Re: [PATCH v2] shared/298: Wire btrfs support in get_free_sectors

2019-02-21 Thread Nikolay Borisov
On 8.02.19 г. 13:44 ч., Nikolay Borisov wrote: > Add support for btrfs in shared/298. Achieve this by introducing 2 > new awk scripts that parse relevant btrfs structures and print holes. > Additionally modify the test to create larger - 3gb filesystem in the > case of btrfs. This is needed so t

[RFC PATCH] btrfs: Introduce developer-oriented check to ensure all tree blocks are written back before writing super blocks

2019-02-21 Thread Qu Wenruo
There are a lot of error reports complaining about transid error in the mail list. Under most case, the on-disk transid is lower than expected transid. This may indicate that some tree blocks are not written back to disk before writing super blocks. This patch will add a safe net for developers,

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Marek Szyprowski
fig I can gather more logs if needed, just let me which kernel option to enable. Reverting this commit on top of next-20190218 as well as current linux-next (tested with next-20190221) fixes this issue and makes the system bootable again. > --- > block/bio.c | 22

Re: [PATCH v3 1/3] btrfs: kill __btrfs_set_prop()

2019-02-21 Thread Anand Jain
On 2/21/19 3:00 AM, David Sterba wrote: On Tue, Feb 12, 2019 at 10:18:49AM +0800, Anand Jain wrote: btrfs_set_prop() is a redirect to __btrfs_set_prop() with the transaction handler equal to NULL. And __btrfs_set_prop() inturn diectly uses trans to do_setxattr() which when trans is NULL creat

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Ming Lei
e in the device > tree) > > 3. Rootfs: Ext4 > > 4. Kernel config: arch/arm/configs/exynos_defconfig > > I can gather more logs if needed, just let me which kernel option to > enable. Reverting this commit on top of next-20190218 as well as current > linux-next (tested with next-20190221) fixes this issue and makes the > system bootable again. Could you test the patch in following link and see if it can make a difference? https://marc.info/?l=linux-aio&m=155070355614541&w=2 Thanks, Ming

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Marek Szyprowski
Board: ARM 32bit Samsung Exynos4412-based Odroid U3 (device tree >> source: arch/arm/boot/dts/exynos4412-odroidu3.dts) >> >> 2. Block device: MMC/SDHCI/SDHCI-S3C with SD card >> (drivers/mmc/host/sdhci-s3c.c driver, sdhci_2 device node in the device >> tree) &g

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Ming Lei
tailed information about > >> my test system: > >> > >> 1. Board: ARM 32bit Samsung Exynos4412-based Odroid U3 (device tree > >> source: arch/arm/boot/dts/exynos4412-odroidu3.dts) > >> > >> 2. Block device: MMC/SDHCI/SDHCI-S3C with SD card

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Marek Szyprowski
t;>>> CPU3: stopping >>>> >>>> I would like to help debugging and fixing this issue, but I don't really >>>> have idea where to start. Here are some more detailed information about >>>> my test system: >>>> >>>> 1

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Ming Lei
020 bea77998 b6d52368 6050 ffff > >>>> CPU3: stopping > >>>> > >>>> I would like to help debugging and fixing this issue, but I don't really > >>>> have idea where to start. Here are some more detailed information a

Re: [PATCH V15 14/18] block: enable multipage bvecs

2019-02-21 Thread Marek Szyprowski
>>>>>> b6e8d0b8 >>>>>> 3fc0: bea77a18 b6f92010 b6e8d0b8 0001 b6e8d0c8 0001 b6e8c000 >>>>>> bea77b60 >>>>>> 3fe0: 0020 bea77998 b6d52368 6050 >>>>>> CPU3: stopping >>&g

Re: [PATCH v2 3/6] btrfs: Make compress_file_range take only struct async_cow

2019-02-21 Thread Nikolay Borisov
On 20.02.19 г. 17:27 ч., Johannes Thumshirn wrote: > Hmm "... and num_added" for the Subject? Nah, that's obvious, the focus here is instead of taking 4-5 more arguments they are replaced with async_cow.

[PATCH v3 4/6] btrfs: Replace clear_extent_bit with unlock_extent

2019-02-21 Thread Nikolay Borisov
Signed-off-by: Nikolay Borisov Reviewed-by: Johannes Thumshirn --- fs/btrfs/inode.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index d566e15f8c58..c857f3c2bfc4 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1204,8 +1204,7 @

[PATCH v3 5/6] btrfs: Set iotree only once in submit_compressed_extents

2019-02-21 Thread Nikolay Borisov
The inode never changes so it's sufficient to dereference it and get the iotree only once, before the execution of the main loop. No functional changes, only the size of the function is decreased: add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-44 (-44) Function ol

[PATCH v3 3/6] btrfs: Make compress_file_range take only struct async_chunk

2019-02-21 Thread Nikolay Borisov
All context this function needs is held within struct async_chunk. Currently we not only pass the struct but also every individual member. This is redundant, simplify it by only passing struct async_chunk and leaving it to compress_file_range to extract the values it requires. No functional changes

[PATCH v3 0/6] Compress path cleanups

2019-02-21 Thread Nikolay Borisov
Another day, another revision, hopefully this is the last one. Changes since v2: * Reworked patch 1 and made async_cow be the top-level context struct and each chunk is tracked by async_chunk. This cleansup the pointer gymnastics I was doing in the previous version (Johannes) * Reworded

[PATCH v3 2/6] btrfs: Remove fs_info from struct async_chunk

2019-02-21 Thread Nikolay Borisov
The associated btrfs_work already contains a reference to the fs_info so use that instead of passing it via async_chunk. No functional changes. Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/inode.c b/fs

[PATCH v3 6/6] btrfs: Factor out common extent locking code in submit_compressed_extents

2019-02-21 Thread Nikolay Borisov
Irrespective of whether the compress code fell back to uncompressed or a compressed extent has to be submitted, the extent range is always locked. So factor out the common lock_extent call at the beginning of the loop. No functional changes just removes one duplicate lock_extent call. Signed-off-b

[PATCH v3 1/6] btrfs: Refactor cow_file_range_async

2019-02-21 Thread Nikolay Borisov
This commit changes the implementation of cow_file_range_async in order to get rid of the BUG_ON in the middle of the loop. Additionally it reworks the inner loop in the hopes of making it more understandable. The idea is to make async_cow be a top-level structured, shared amongst all chunks being

Re: [PATCH v3 3/6] btrfs: Make compress_file_range take only struct async_chunk

2019-02-21 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH v3 2/6] btrfs: Remove fs_info from struct async_chunk

2019-02-21 Thread Johannes Thumshirn
Looks good, Reviewed-by: Johannes Thumshirn -- Johannes ThumshirnSUSE Labs Filesystems jthumsh...@suse.de+49 911 74053 689 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG N

Re: [PATCH v2 3/6] btrfs: Make compress_file_range take only struct async_cow

2019-02-21 Thread Johannes Thumshirn
On 21/02/2019 12:55, Nikolay Borisov wrote: > > > On 20.02.19 г. 17:27 ч., Johannes Thumshirn wrote: >> Hmm "... and num_added" for the Subject? > > Nah, that's obvious, the focus here is instead of taking 4-5 more > arguments they are replaced with async_cow. > Fair enough -- Johannes Thums

Re: [PATCH v3 1/6] btrfs: Refactor cow_file_range_async

2019-02-21 Thread Johannes Thumshirn
On 21/02/2019 12:57, Nikolay Borisov wrote: > > static int cow_file_range_async(struct inode *inode, struct page > *locked_page, > @@ -1190,45 +1201,68 @@ static int cow_file_range_async(struct inode *inode, > struct page *locked_page, > unsigned int write_flags)

Re: [PATCH v3 1/6] btrfs: Refactor cow_file_range_async

2019-02-21 Thread Nikolay Borisov
On 21.02.19 г. 15:15 ч., Johannes Thumshirn wrote: > On 21/02/2019 12:57, Nikolay Borisov wrote: >> >> static int cow_file_range_async(struct inode *inode, struct page >> *locked_page, >> @@ -1190,45 +1201,68 @@ static int cow_file_range_async(struct inode *inode, >> struct page *locked_pag

[bug report] btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock

2019-02-21 Thread Dan Carpenter
[ This one is not actually your fault. It's the core code which is confusing. -dan ] Hello Qu Wenruo, The patch b72c3aba09a5: "btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock" from Aug 21, 2018, leads to the following static checker warning: fs/btrfs/exte

Re: [RFC PATCH] btrfs: Introduce developer-oriented check to ensure all tree blocks are written back before writing super blocks

2019-02-21 Thread Nikolay Borisov
On 21.02.19 г. 10:22 ч., Qu Wenruo wrote: > There are a lot of error reports complaining about transid error in the > mail list. > > Under most case, the on-disk transid is lower than expected transid. > This may indicate that some tree blocks are not written back to disk > before writing super

Re: [RFC PATCH] btrfs: Introduce developer-oriented check to ensure all tree blocks are written back before writing super blocks

2019-02-21 Thread Qu Wenruo
On 2019/2/21 下午10:25, Nikolay Borisov wrote: > > > On 21.02.19 г. 10:22 ч., Qu Wenruo wrote: >> There are a lot of error reports complaining about transid error in the >> mail list. >> >> Under most case, the on-disk transid is lower than expected transid. >> This may indicate that some tree b

Re: [bug report] btrfs: locking: Add extra check in btrfs_init_new_buffer() to avoid deadlock

2019-02-21 Thread Qu Wenruo
On 2019/2/21 下午10:13, Dan Carpenter wrote: > [ This one is not actually your fault. It's the core code which is > confusing. -dan ] > > Hello Qu Wenruo, > > The patch b72c3aba09a5: "btrfs: locking: Add extra check in > btrfs_init_new_buffer() to avoid deadlock" from Aug 21, 2018, leads > to

Re: [RFC PATCH] btrfs: Introduce developer-oriented check to ensure all tree blocks are written back before writing super blocks

2019-02-21 Thread Nikolay Borisov
On 21.02.19 г. 16:32 ч., Qu Wenruo wrote: > > > On 2019/2/21 下午10:25, Nikolay Borisov wrote: >> >> >> On 21.02.19 г. 10:22 ч., Qu Wenruo wrote: >>> There are a lot of error reports complaining about transid error in the >>> mail list. >>> >>> Under most case, the on-disk transid is lower than

Re: [PATCH] fs/btrfs: init csum_list before possible free

2019-02-21 Thread Dan Robertson
> Thanks, it's been there since the first merge of scrub, but hard to hit. Yeah, very hard to hit. Debated submitting a patch, but the fix seemed simple enough, so figured I'd send one in. Side note: I noticed most of the time ENOMEM is hit in fs/btrfs BUG or BUG_ON is called. Would it be better

Re: [PATCH v3 1/6] btrfs: Refactor cow_file_range_async

2019-02-21 Thread Johannes Thumshirn
On 21/02/2019 14:25, Nikolay Borisov wrote: > > > On 21.02.19 г. 15:15 ч., Johannes Thumshirn wrote: >> On 21/02/2019 12:57, Nikolay Borisov wrote: >>> >>> static int cow_file_range_async(struct inode *inode, struct page >>> *locked_page, >>> @@ -1190,45 +1201,68 @@ static int cow_file_range_

Re: [PATCH v3 1/6] btrfs: Refactor cow_file_range_async

2019-02-21 Thread Nikolay Borisov
On 21.02.19 г. 17:07 ч., Johannes Thumshirn wrote: > On 21/02/2019 14:25, Nikolay Borisov wrote: >> >> >> On 21.02.19 г. 15:15 ч., Johannes Thumshirn wrote: >>> On 21/02/2019 12:57, Nikolay Borisov wrote: static int cow_file_range_async(struct inode *inode, struct page *locked

Re: [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication

2019-02-21 Thread Zygo Blaxell
On Wed, Feb 20, 2019 at 04:54:09PM +, Filipe Manana wrote: > On Wed, Feb 20, 2019 at 4:42 PM Zygo Blaxell > wrote: > > > > On Thu, Jan 31, 2019 at 04:39:22PM +, Filipe Manana wrote: > > > On Thu, Dec 13, 2018 at 4:08 PM David Sterba wrote: > > > > > > > > On Wed, Dec 12, 2018 at 06:05:58P

Re: [PATCH v2] shared/298: Wire btrfs support in get_free_sectors

2019-02-21 Thread Eryu Guan
On Fri, Feb 08, 2019 at 01:44:04PM +0200, Nikolay Borisov wrote: > Add support for btrfs in shared/298. Achieve this by introducing 2 > new awk scripts that parse relevant btrfs structures and print holes. > Additionally modify the test to create larger - 3gb filesystem in the > case of btrfs. This

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-21 Thread Dave Chinner
On Wed, Feb 20, 2019 at 04:47:24PM -0700, Keith Busch wrote: > On Sun, Feb 17, 2019 at 06:42:59PM -0500, Ric Wheeler wrote: > > I think the variability makes life really miserable for layers above it. > > > > Might be worth constructing some tooling that we can use to validate or > > shame vendors

[PATCH] btrfs: zstd ensure reclaim timer is properly cleaned up

2019-02-21 Thread Dennis Zhou
The timer function, zstd_reclaim_timer_fn(), reschedules itself under certain conditions. Switch to del_timer_sync() to ensure that the timer function hasn't rescheduled itself. Signed-off-by: Dennis Zhou --- fs/btrfs/zstd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-21 Thread Jeff Mahoney
On 2/20/19 6:47 PM, Keith Busch wrote: > On Sun, Feb 17, 2019 at 06:42:59PM -0500, Ric Wheeler wrote: >> I think the variability makes life really miserable for layers above it. >> >> Might be worth constructing some tooling that we can use to validate or >> shame vendors over - testing things like

[PATCH] btrfs: Fix possible NULL pointer dereference in btrfs selftest

2019-02-21 Thread Qu Wenruo
When CONFIG_BTRFS_FS_RUN_SANITY_TESTS is enabled, btrfs will run selftest at module load time. During selftest, we allocate extent buffer using alloc_test_extent_buffer(), instead of alloc_test_extent_buffer(). The problem is, unlike alloc_extent_buffer(), alloc_test_extent_buffer() can return NU

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-21 Thread Martin K. Petersen
Keith, > With respect to fs block sizes, one thing making discards suck is that > many high capacity SSDs' physical page sizes are larger than the fs > block size, and a sub-page discard is worse than doing nothing. That ties into the whole zeroing as a side-effect thing. The devices really ne

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-21 Thread Martin K. Petersen
Jeff, > We've always been told "don't worry about what the internal block size > is, that only matters to the FTL." That's obviously not true, but > when devices only report a 512 byte granularity, we believe them and > will issue discard for the smallest size that makes sense for the file > sy

Re: [LSF/MM TOPIC] More async operations for file systems - async discard?

2019-02-21 Thread Roman Mamedov
On Thu, 21 Feb 2019 22:01:24 -0500 "Martin K. Petersen" wrote: > Consequently, many of the modern devices that claim to support discard > to make us software folks happy (or to satisfy a purchase order > requirements) complete the commands without doing anything at all. > We're simply wasting que

Re: [PATCH v2] shared/298: Wire btrfs support in get_free_sectors

2019-02-21 Thread Nikolay Borisov
On 21.02.19 г. 19:10 ч., Eryu Guan wrote: > On Fri, Feb 08, 2019 at 01:44:04PM +0200, Nikolay Borisov wrote: >> Add support for btrfs in shared/298. Achieve this by introducing 2 >> new awk scripts that parse relevant btrfs structures and print holes. >> Additionally modify the test to create la

Re: [PATCH] btrfs: Fix possible NULL pointer dereference in btrfs selftest

2019-02-21 Thread Dan Carpenter
I'm sorry, I feel bad for passing this work on to you when you didn't introduce the problems at all. I think you're doing to the right thing to change it all to error pointers, and most of callers expect that but there are a couple that need to be changed: btrfs_test_qgroups() and run_test(). re

Re: [PATCH] btrfs: Fix possible NULL pointer dereference in btrfs selftest

2019-02-21 Thread Qu Wenruo
On 2019/2/22 下午3:00, Dan Carpenter wrote: > I'm sorry, I feel bad for passing this work on to you when you didn't > introduce the problems at all. No problem at all. Who doesn't like to send out clean up patches? > > I think you're doing to the right thing to change it all to error > pointers