Re: btrfs send yields "ERROR: send ioctl failed with -5: Input/output error"

1. I guess you should be able to dump tree details via 'btrfs-debug-tree' and then map the extent/data (from scrub offline output) and track it back to inode-object. Store output of both btrfs-debug-tree and scrub-offline in different files and then play around with grep to extract required data.

[PATCH v2 5/6] btrfs: qgroup: Update trace events to use new separate rsv types

Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c| 36 +--- include/trace/events/btrfs.h | 17 - 2 files changed, 33 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index

[PATCH v2 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv to restore different types of reservation. This patch only updates the header and needed modification to pass compile. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 16 ++-- fs/btrfs/qgroup.h

[PATCH v2 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

Introduce helpers to: 1) Get total reserved space For limit calculation 2) Add/release reserved space for given type With underflow detection and warning 3) Add/release reserved space according to child qgroup Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 68

[PATCH v2 6/6] btrfs: qgroup: Cleanup the remaining old reservation counters

So qgroup is switched to new separate types reservation system. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 13 - fs/btrfs/qgroup.h | 1 - 2 files changed, 14 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 0a38e65f55f7..4063004ae9d5

[PATCH v2 4/6] btrfs: qgroup: Fix wrong qgroup reservation update for relationship modification

When modifying qgroup relationship, for qgroup which only owns exclusive extents, we will go through quick update path. In quick update path, we will adding/substracting exclusive and reference number for parent qgroup, since the source (child) qgroup only have exclusive extents, destination

[PATCH v2 3/6] btrfs: qgroup: Make qgroup_reserve and its callers to use separate reservation type

Since most caller of qgroup_reserve() is already defined by type, converting qgroup_reserve() is quite an easy work. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 20 +--- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/qgroup.c

[PATCH v2 0/6] btrfs: qgroup: Separate qgroup reservation types

This patchset will separate original qgroup->reserved into a new structure, btrfs_qgroup_rsv, which has different reservation for different types. Currently it will only includes data and meta type. The main advantage is: 1) Better underflow detection Now it can detection which reservation

Re: btrfs send yields "ERROR: send ioctl failed with -5: Input/output error"

I apologize for the bad line wrapping on the last post...will be setting up mutt soon. This is the final result for the offline scrub: Doing offline scrub [O] [681/683] Scrub result: Tree bytes scrubbed: 5234491392 Tree extents scrubbed: 638975 Data bytes scrubbed: 4353723572224 Data extents

Re: [PATCH 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

On 2017年10月25日 00:22, Edmund Nadolski wrote: > > > On 10/24/2017 06:05 AM, Qu Wenruo wrote: >> >> >> On 2017年10月24日 19:07, Nikolay Borisov wrote: >>> >>> >>> On 24.10.2017 11:39, Qu Wenruo wrote: Introduce helpers to: 1) Get total reserved space For limit calculation

Re: [PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

On 2017年10月25日 01:11, Edmund Nadolski wrote: > > > On 10/24/2017 02:39 AM, Qu Wenruo wrote: >> When modifying qgroup relationship, for qgroup which only owns exclusive >> extents, we will go through quick update path. >> >> In quick update path, we will just adding/removing exclusive and

Next btrfs development cycle open - 4.16

Hi, a friendly reminder of the timetable and what's expected at this phase. 4.13 - current 4.14 - upcoming, urgent regression fixes only 4.15 - development closed, pull request in prep, fixes or regressions only 4.16 - development open, until 4.15-rc5

Re: [PATCH v8 0/6] Btrfs: populate heuristic with code

On Sun, Oct 22, 2017 at 04:44:04PM +0300, Timofey Titovets wrote: > 2017-10-20 16:45 GMT+03:00 David Sterba : > > On Fri, Oct 20, 2017 at 01:48:01AM +0300, Timofey Titovets wrote: > >> 2017-10-19 18:39 GMT+03:00 David Sterba : > >> > On Fri, Sep 29, 2017 at

Re: super_total_bytes 32004083023872 mismatch with fs_devices total_rw_bytes 64008166047744

The mentioning of the device scan scode and the fact the total_bytes is double made me try hashing the raid from the fstab. So i booted, run the "inspect-internal dump-super" that confirmed that it is in order. # grep -i total_bytes hashed-inspect-internal total_bytes 32004083023872

Re: [PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

On 10/24/2017 02:39 AM, Qu Wenruo wrote: > When modifying qgroup relationship, for qgroup which only owns exclusive > extents, we will go through quick update path. > > In quick update path, we will just adding/removing exclusive and reference > number. > > However we did the opposite for

RE: SLES 11 SP4: can't mount btrfs

> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org > [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Austin S. > Hemmelgarn > Sent: Tuesday, October 24, 2017 4:05 PM > To: Lentes, Bernd ; Btrfs ML > >

Re: [PATCH 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

On 10/24/2017 06:05 AM, Qu Wenruo wrote: > > > On 2017年10月24日 19:07, Nikolay Borisov wrote: >> >> >> On 24.10.2017 11:39, Qu Wenruo wrote: >>> Introduce helpers to: >>> >>> 1) Get total reserved space >>>For limit calculation >>> >>> 2) Increase reserved space for given type >>> 2)

Re: SLES 11 SP4: can't mount btrfs

On 2017-10-24 10:12, Andrei Borzenkov wrote: On Tue, Oct 24, 2017 at 2:53 PM, Austin S. Hemmelgarn wrote: SLES (and OpenSUSE in general) does do something special though, they use subvolumes and qgroups to replicate multiple independent partitions (which is a serious

Re: SLES 11 SP4: can't mount btrfs

On Tue, Oct 24, 2017 at 2:53 PM, Austin S. Hemmelgarn wrote: > > SLES (and OpenSUSE in general) does do something special though, they use > subvolumes and qgroups to replicate multiple independent partitions (which > is a serious pain in the arse), and they have

Re: SLES 11 SP4: can't mount btrfs

On 2017-10-24 09:28, Lentes, Bernd wrote: -Original Message- From: Austin S. Hemmelgarn [mailto:ahferro...@gmail.com] Sent: Tuesday, October 24, 2017 1:53 PM To: Adam Borowski ; Lentes, Bernd Cc: Btrfs ML

Re: super_total_bytes 32004083023872 mismatch with fs_devices total_rw_bytes 64008166047744

On 2017年10月24日 19:44, Konstantin V. Gavrilenko wrote: > answers inline marked with KVG: > > Yours sincerely, > Konstantin V. Gavrilenko > > > > > - Original Message - > From: "Qu Wenruo" > To: "Konstantin V. Gavrilenko" , "Linux fs

RE: SLES 11 SP4: can't mount btrfs

> -Original Message- > From: Austin S. Hemmelgarn [mailto:ahferro...@gmail.com] > Sent: Tuesday, October 24, 2017 1:53 PM > To: Adam Borowski ; Lentes, Bernd > > Cc: Btrfs ML > Subject: Re: SLES 11

Re: [PATCH v2] btrfs: Fix memory barriers usage with device stats counters

- Nikolay Borisov wrote: > Commit addc3fa74e5b ("Btrfs: Fix the problem that the dirty flag of dev stats > is cleared") > reworked the way device stats changes are tracked. A new atomic dev_stats_ccnt > counter was introduced which is incremented every time any of the

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 2017年10月24日 20:29, Jeff Mahoney wrote: > On 10/24/17 7:51 AM, Qu Wenruo wrote: >> >> >> On 2017年10月24日 19:00, Nikolay Borisov wrote: >>> >>> >>> On 24.10.2017 11:39, Qu Wenruo wrote: Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv to restore different types

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 10/24/17 8:29 AM, Jeff Mahoney wrote: > On 10/24/17 7:51 AM, Qu Wenruo wrote: >> >> >> On 2017年10月24日 19:00, Nikolay Borisov wrote: >>> nit: Why not BTRFS_QGROUP_RSV_TYPES_MAX = 2; >> >> My original plan is just as the same as yours. >> >> However I still remember I did it before and David

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 2017年10月24日 20:23, Nikolay Borisov wrote: > > > On 24.10.2017 14:51, Qu Wenruo wrote: >> >> >> On 2017年10月24日 19:00, Nikolay Borisov wrote: >>> >>> >>> On 24.10.2017 11:39, Qu Wenruo wrote: Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv to restore

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 10/24/17 7:51 AM, Qu Wenruo wrote: > > > On 2017年10月24日 19:00, Nikolay Borisov wrote: >> >> >> On 24.10.2017 11:39, Qu Wenruo wrote: >>> Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv >>> to restore different types of reservation. >>> >>> This patch only updates the

Re: [PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

On 24.10.2017 15:19, Qu Wenruo wrote: > > > On 2017年10月24日 20:01, Nikolay Borisov wrote: >> >> >> On 24.10.2017 11:39, Qu Wenruo wrote: >>> When modifying qgroup relationship, for qgroup which only owns exclusive >>> extents, we will go through quick update path. >>> >>> In quick update path,

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 24.10.2017 14:51, Qu Wenruo wrote: > > > On 2017年10月24日 19:00, Nikolay Borisov wrote: >> >> >> On 24.10.2017 11:39, Qu Wenruo wrote: >>> Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv >>> to restore different types of reservation. >>> >>> This patch only updates

Re: [PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

On 2017年10月24日 20:01, Nikolay Borisov wrote: > > > On 24.10.2017 11:39, Qu Wenruo wrote: >> When modifying qgroup relationship, for qgroup which only owns exclusive >> extents, we will go through quick update path. >> >> In quick update path, we will just adding/removing exclusive and

Re: [PATCH 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

On 2017年10月24日 19:07, Nikolay Borisov wrote: > > > On 24.10.2017 11:39, Qu Wenruo wrote: >> Introduce helpers to: >> >> 1) Get total reserved space >>For limit calculation >> >> 2) Increase reserved space for given type >> 2) Decrease reserved space for given type >> >> Signed-off-by: Qu

Re: [PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

On 24.10.2017 11:39, Qu Wenruo wrote: > When modifying qgroup relationship, for qgroup which only owns exclusive > extents, we will go through quick update path. > > In quick update path, we will just adding/removing exclusive and reference > number. > > However we did the opposite for qgroup

Re: SLES 11 SP4: can't mount btrfs

On 2017-10-21 14:07, Adam Borowski wrote: On Sat, Oct 21, 2017 at 01:46:06PM +0200, Lentes, Bernd wrote: - Am 21. Okt 2017 um 4:31 schrieb Duncan 1i5t5.dun...@cox.net: Lentes, Bernd posted on Fri, 20 Oct 2017 20:40:15 +0200 as excerpted: Is it generally possible to restore a btrfs

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 2017年10月24日 19:00, Nikolay Borisov wrote: > > > On 24.10.2017 11:39, Qu Wenruo wrote: >> Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv >> to restore different types of reservation. >> >> This patch only updates the header and needed modification to pass >>

Re: [PATCH 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

On 24.10.2017 11:39, Qu Wenruo wrote: > Introduce helpers to: > > 1) Get total reserved space >For limit calculation > > 2) Increase reserved space for given type > 2) Decrease reserved space for given type > > Signed-off-by: Qu Wenruo > --- > fs/btrfs/qgroup.c | 66 >

Re: [PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

On 24.10.2017 11:39, Qu Wenruo wrote: > Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv > to restore different types of reservation. > > This patch only updates the header and needed modification to pass > compile. > > Signed-off-by: Qu Wenruo > --- >

[PATCH v2] btrfs: Fix memory barriers usage with device stats counters

Commit addc3fa74e5b ("Btrfs: Fix the problem that the dirty flag of dev stats is cleared") reworked the way device stats changes are tracked. A new atomic dev_stats_ccnt counter was introduced which is incremented every time any of the device stats counters are changed. This serves as a flag

What is the purpose of EXTENT_PAGE_MAPPED

EXTENT_PAGE_MAPPED gets set in set_page_extent_mapped(), but I don't see it being cross checked anytime. What is the purpose of setting it? -- Goldwyn -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo

Re: super_total_bytes 32004083023872 mismatch with fs_devices total_rw_bytes 64008166047744

On 2017年10月24日 17:20, Konstantin V. Gavrilenko wrote: > Hi list, > > having installed the recent kernel version I am no longer able to mount the > btrfs partition with compression on the first attempt. Previously on > 4.10.0-37-generic everything was working fine, once I switched to >

super_total_bytes 32004083023872 mismatch with fs_devices total_rw_bytes 64008166047744

Hi list, having installed the recent kernel version I am no longer able to mount the btrfs partition with compression on the first attempt. Previously on 4.10.0-37-generic everything was working fine, once I switched to 4.13.9-041309-generic I started getting the following error while trying

Re: [PATCH] fstests: btrfs/143: make test case more reliable

On 23.10.2017 23:57, Liu Bo wrote: > Currently drop_caches is used to invalidate file's page cache so that > buffered read can hit disk, but the problem is that it may also > invalidate metadata's page cache, so the test case may not get read > errors (and repair) if reading metadata has

[PATCH] btrfs: Remove redundant mirror_num arg

The following callpath is always invoked with mirror_num set to 0, so let's remove it as an argument and directly pass 0 to __do_redpage. No functional change extent_readpages __extent_readpages __do_contiguous_readpages __do_readpage Signed-off-by: Nikolay Borisov

[PATCH 6/6] btrfs: qgroup: Cleanup the remaining old reservation counters

So qgroup is switched to new separate types reservation system. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 13 - fs/btrfs/qgroup.h | 1 - 2 files changed, 14 deletions(-) diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 2b587056b797..c5936fc75a4d

[PATCH 3/6] btrfs: qgroup: Make qgroup_reserve and its callers to use separate reservation type

Since most caller of qgroup_reserve() is already defined by type, converting qgroup_reserve() is quite an easy work. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 20 +--- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/qgroup.c

[PATCH 1/6] btrfs: qgroup: Skeleton to support separate qgroup reservation type

Instead of single qgroup->reserved, use a new structure btrfs_qgroup_rsv to restore different types of reservation. This patch only updates the header and needed modification to pass compile. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 16 ++-- fs/btrfs/qgroup.h

[PATCH 2/6] btrfs: qgroup: Introduce helpers to update and access new qgroup rsv

Introduce helpers to: 1) Get total reserved space For limit calculation 2) Increase reserved space for given type 2) Decrease reserved space for given type Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 66 +++ 1 file

[PATCH 4/6] btrfs: qgroup: Fix wrong qgroup reservation inheritance for relationship update

When modifying qgroup relationship, for qgroup which only owns exclusive extents, we will go through quick update path. In quick update path, we will just adding/removing exclusive and reference number. However we did the opposite for qgroup reservation from the very beginning. In fact, we

[PATCH 0/6] btrfs: qgroup: Separate qgroup reservation types

This patchset will separate original qgroup->reserved into a new structure, btrfs_qgroup_rsv, which has different reservation for different types. Currently it will only includes data and meta type. The main advantage is: 1) Better underflow detection Now it can detection which reservation

[PATCH 5/6] btrfs: qgroup: Update trace events to use new separate rsv types

To make the trace point cleaner, remove the @fs_info parameter, and integrate @fs_info to struct qgroup. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c| 32 ++-- include/trace/events/btrfs.h | 17 - 2 files changed, 30

[PATCH v2] Btrfs: add write_flags for compression bio

Compression code path has only flaged bios with REQ_OP_WRITE no matter where the bios come from, but it could be a sync write if fsync starts this writeback or a normal writeback write if wb kthread starts a periodic writeback. It breaks the rule that sync writes and writeback writes need to be

[PATCH v3] Btrfs: free btrfs_device in place

It's pointless to defer it to a kthread helper as we're not under a special context. For reference, commit 1f78160ce1b1 ("Btrfs: using rcu lock in the reader side of devices list") introduced RCU freeing for device structures. Signed-off-by: Liu Bo Reviewed-by: Anand Jain

Re: btrfs send yields "ERROR: send ioctl failed with -5: Input/output error"

Yes, it is finding much more than just one error. >From dmesg [89520.441354] BTRFS warning (device sdn): csum failed ino 4708 off 27529216 csum 2615801759 expected csum 874979996 $ sudo btrfs scrub start --offline --progress /dev/sdn ERROR: data at bytenr 68431499264 mirror 1 csum mismatch, have