Re: [PATCH 0/6] add sanity check for extent inline ref type

2017-05-30 Thread Ivan Sizov
2017-05-26 3:26 GMT+03:00 Liu Bo : >Patch 6 adds scrub support to detect the corruption, so users can be noticed when they do scrub on a regular basis. >I'm not sure in the real world what may result in this corruption I've caught this type of corruption in the wild. The big rsync backup always en

[PATCH] Btrfs: fix invalid extent maps due to hole punching

2017-05-30 Thread fdmanana
From: Filipe Manana While punching a hole in a range that is not aligned with the sector size (currently the same as the page size) we can end up leaving an extent map in memory with a length that is smaller then the sector size, which is not expected and can lead to problems. This issue is easil

[PATCH] generic: test for hole punching beyond eof followed by buffered writes

2017-05-30 Thread fdmanana
From: Filipe Manana Test that if we punch a hole in a file that goes beyond the file's size and that if after we do some buffered write operations that cover different parts of the hole, no warnings are emmitted in syslog/dmesg and the file's content is correct after remounting the filesystem. T

Debugging Deadlocks?

2017-05-30 Thread Sargun Dhillon
We've been running BtrFS for a couple months now in production on several clusters. We're running on Canonical's 4.8 kernel, and currently, in the process of moving to our own patchset atop vanilla 4.10+. I'm glad to say it's been a fairly good experience for us. Bar some performance issues, it's b

Re: [PATCH 09/10] xfs: nowait aio support

2017-05-30 Thread Goldwyn Rodrigues
On 05/29/2017 03:33 AM, Christoph Hellwig wrote: > On Sun, May 28, 2017 at 09:38:26PM -0500, Goldwyn Rodrigues wrote: >> >> >> On 05/28/2017 04:31 AM, Christoph Hellwig wrote: >>> Despite my previous reviewed-by tag this will need another fix: >>> >>> xfs_file_iomap_begin needs to return EAGAIN i

Re: [PATCH v2] btrfs-progs: btrfs-convert: Add larger device support

2017-05-30 Thread David Sterba
On Mon, May 15, 2017 at 02:06:54PM +0530, Lakshmipathi.G wrote: > On Mon, May 15, 2017 at 09:40:29AM +0800, Qu Wenruo wrote: > > >bug: https://bugzilla.kernel.org/show_bug.cgi?id=194795 > > > > Errr, it seems that you forgot to update ext2_open_fs() to update how we get > > cctx->block_counts. > >

Re: [PATCH 0/6] add sanity check for extent inline ref type

2017-05-30 Thread Liu Bo
On Tue, May 30, 2017 at 05:05:09PM +0300, Ivan Sizov wrote: > 2017-05-26 3:26 GMT+03:00 Liu Bo : > >Patch 6 adds scrub support to detect the corruption, so users can be > noticed when they do scrub on a regular basis. > >I'm not sure in the real world what may result in this corruption > > I've ca

Re: [PATCH v4 00/20] Btrfs-progs offline scrub

2017-05-30 Thread David Sterba
On Thu, May 25, 2017 at 02:21:45PM +0800, Qu Wenruo wrote: > For any one who wants to try it, it can be get from my repo: > https://github.com/adam900710/btrfs-progs/tree/offline_scrub > Qu Wenruo (20): > btrfs-progs: raid56: Introduce raid56 header for later recovery usage > btrfs-progs: raid5

Re: [PATCH 0/6] add sanity check for extent inline ref type

2017-05-30 Thread Ivan Sizov
2017-05-30 21:02 GMT+03:00 Liu Bo : > On Tue, May 30, 2017 at 05:05:09PM +0300, Ivan Sizov wrote: >> 2017-05-26 3:26 GMT+03:00 Liu Bo : >> >Patch 6 adds scrub support to detect the corruption, so users can be >> noticed when they do scrub on a regular basis. >> >I'm not sure in the real world what

Re: [PATCH] Btrfs: fix invalid extent maps due to hole punching

2017-05-30 Thread Omar Sandoval
On Sun, May 28, 2017 at 05:31:53PM +0100, fdman...@kernel.org wrote: > From: Filipe Manana [snip] Hey, Filipe, I saw this warning and tried to apply your patch, but it doesn't apply cleanly (seems to conflict with 9986277e0e4c ("Btrfs: handle only applicable errors returned by btrfs_get_extent"

Re: [RFC PATCH v3.2 5/6] btrfs: qgroup: Introduce extent changeset for qgroup reserve functions

2017-05-30 Thread Qu Wenruo
At 05/29/2017 11:51 PM, David Sterba wrote: On Fri, May 19, 2017 at 08:32:18AM +0800, Qu Wenruo wrote: At 05/18/2017 09:45 PM, David Sterba wrote: On Thu, May 18, 2017 at 08:24:26AM +0800, Qu Wenruo wrote: +static inline void extent_changeset_init(struct extent_changeset *changeset) +{ +

Re: [PATCH 2/5] btrfs-progs: Enhance chunk item validation check

2017-05-30 Thread Qu Wenruo
At 05/30/2017 01:39 AM, David Sterba wrote: On Mon, May 15, 2017 at 04:27:39PM +0800, Qu Wenruo wrote: btrfs_check_chunk_valid() doesn't check if 1) chunk flag has conflicting flags For example chunk type DATA|METADATA|RAID1|RAID10 is completely invalid, while current check_chunk_valid

Re: [PATCH 3/3] btrfs: check namelen before read/memcmp_extent_buffer

2017-05-30 Thread Su Yue
On 05/29/2017 11:43 PM, David Sterba wrote: This patch adds the name length verification to many places and in some of them it looks unnecessary, as the directory item passes sanity checks already. The verification should always happen when we read the input, ie from disk, after search_slot etc

[PATCH v2 2/6] btrfs-progs: Enhance chunk item validation check

2017-05-30 Thread Qu Wenruo
btrfs_check_chunk_valid() doesn't check if 1) chunk flag has conflicting flags For example chunk type DATA|METADATA|RAID1|RAID10 is completely invalid, while current check_chunk_valid() can't detect it. 2) num_stripes is invalid for RAID10 Num_stripes 5 is not valid for RAID10. This patch

[PATCH v2 4/6] btrfs-progs: Introduce function to get correct stripe length

2017-05-30 Thread Qu Wenruo
Introduce a new function, btrfs_get_chunk_stripe_len() to get correct stripe length. This is very handy for lowmem mode, which checks the mapping between device extent and chunk item. Signed-off-by: Qu Wenruo --- volumes.c | 44 volumes.h | 3 +++ 2

[PATCH v2 0/6] Lowmem mode check check fix for mulit-device

2017-05-30 Thread Qu Wenruo
This patchset will fix a false alert in lowmem mode, which doesn't handle RAID0/5/6/10 chunk well. (the 5th patch) Along the lowmem fix, also enhance and cleanup some chunk verification code, as lowmem mode and original mode are using different chunk verification. (the 1st~4th patch) Finally, cle

[PATCH v2 3/6] btrfs-progs: check: Reuse btrfs_check_chunk_valid in lowmem mode

2017-05-30 Thread Qu Wenruo
Before this patch, btrfs check lowmem mode manually checks found chunk item, even we already have the generic chunk validation checker, btrfs_check_chunk_valid(). This patch will use btrfs_check_chunk_valid() to replace open-coded chunk validation checker in check_chunk_item(). Signed-off-by: Qu

[PATCH v2 6/6] btrfs-progs: test: Introduce functions to prepare and cleanup loop device

2017-05-30 Thread Qu Wenruo
Introduce new helpers, prepare_loop_dev() and cleanup_loop_dev() to prepare and cleanup loop device. Signed-off-by: Qu Wenruo --- tests/common | 38 ++ .../misc-tests/006-image-on-missing-device/test.sh | 12 ++- tests/misc-tests/011-

[PATCH v2 1/6] btrfs-progs: Cleanup open-coded btrfs_chunk_item_size

2017-05-30 Thread Qu Wenruo
In btrfs_check_chunk_valid() we calculates chunk item using open code. use btrfs_chunk_item_size() to replace them. Signed-off-by: Qu Wenruo --- volumes.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/volumes.c b/volumes.c index b350e259..62e23aee 100644 --- a/vol

[PATCH v2 5/6] btrfs-progs: lowmem check: Fix false alert on missing chunk or dev extent

2017-05-30 Thread Qu Wenruo
When checking chunk or dev extent, lowmem mode uses chunk length as dev extent length, and if they mismatch, report missing chunk or dev extent like: -- ERROR: chunk[256 4324327424) stripe 0 did not find the related dev extent ERROR: chunk[256 4324327424) stripe 1 did not find the related dev e

Re: Debugging Deadlocks?

2017-05-30 Thread Duncan
Sargun Dhillon posted on Tue, 30 May 2017 09:12:39 -0700 as excerpted: > We've been running BtrFS for a couple months now in production on > several clusters. We're running on Canonical's 4.8 kernel, and > currently, in the process of moving to our own patchset atop vanilla > 4.10+. I'm glad to sa