[PATCH v1.1 4/6] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2018-10-18 Thread Qu Wenruo
To allow delayed subtree swap rescan, btrfs needs to record per-root info about which tree blocks get swapped. So this patch introduces per-root btrfs_qgroup_swapped_blocks structure, which records which tree blocks get swapped. The designed workflow will be: 1) Record the subtree root block

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-10-18 Thread Su Yue
[Bad format in previous reply, send again] On 10/18/18 10:41 PM, Christoph Anton Mitterer wrote: Hey. So I'm back from a longer vacation and had now the time to try out your patches from below: On Wed, 2018-09-05 at 15:04 +0800, Su Yue wrote: I found the errors should blame to something

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-10-18 Thread Su Yue
On 10/18/18 10:41 PM, Christoph Anton Mitterer wrote: Hey. So I'm back from a longer vacation and had now the time to try out your patches from below: On Wed, 2018-09-05 at 15:04 +0800, Su Yue wrote: I found the errors should blame to something about inode_extref check in lowmem mode. I

Re: reproducible builds with btrfs seed feature

2018-10-18 Thread Anand Jain
On 10/19/2018 02:02 AM, Chris Murphy wrote: On Tue, Oct 16, 2018 at 10:08 PM, Anand Jain wrote: So a possible solution for the reproducible builds: usual mkfs.btrfs dev Write the data unmount; create btrfs-image with uuid/fsid/time sanitized; mark it as a seed (RO).

Re: [PATCH 4/6] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2018-10-18 Thread Qu Wenruo
On 2018/10/19 上午12:20, David Sterba wrote: > On Thu, Oct 18, 2018 at 07:17:27PM +0800, Qu Wenruo wrote: >> +void btrfs_qgroup_clean_swapped_blocks(struct btrfs_root *root) >> +{ >> +struct btrfs_qgroup_swapped_blocks *swapped_blocks; >> +struct btrfs_qgroup_swapped_block *cur, *next; >>

Re: reproducible builds with btrfs seed feature

2018-10-18 Thread Chris Murphy
On Tue, Oct 16, 2018 at 10:08 PM, Anand Jain wrote: > > So a possible solution for the reproducible builds: >usual mkfs.btrfs dev >Write the data >unmount; create btrfs-image with uuid/fsid/time sanitized; mark it as a > seed (RO). >check/verify the hash of the image. Gotcha.

Re: [PATCH 18/42] btrfs: move the dio_sem higher up the callchain

2018-10-18 Thread David Sterba
On Fri, Oct 12, 2018 at 03:32:32PM -0400, Josef Bacik wrote: > --- a/fs/btrfs/tree-log.c > +++ b/fs/btrfs/tree-log.c > @@ -4374,7 +4374,6 @@ static int btrfs_log_changed_extents(struct > btrfs_trans_handle *trans, > > INIT_LIST_HEAD(); > > - down_write(>dio_sem); I'll add

Re: [PATCH 4/6] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2018-10-18 Thread David Sterba
On Thu, Oct 18, 2018 at 07:17:27PM +0800, Qu Wenruo wrote: > +void btrfs_qgroup_clean_swapped_blocks(struct btrfs_root *root) > +{ > + struct btrfs_qgroup_swapped_blocks *swapped_blocks; > + struct btrfs_qgroup_swapped_block *cur, *next; > + int i; > + > + swapped_blocks =

Re: fsck lowmem mode only: ERROR: errors found in fs roots

2018-10-18 Thread Christoph Anton Mitterer
Hey. So I'm back from a longer vacation and had now the time to try out your patches from below: On Wed, 2018-09-05 at 15:04 +0800, Su Yue wrote: > I found the errors should blame to something about inode_extref check > in lowmem mode. > I have writeen three patches to detect and report errors

Backref error

2018-10-18 Thread Hegyi László
Hello guys! I have a 2TB disk formatted to btrfs.My notebook broke last may, so I haven't used it in a long time, only to backup a few files, but from a windows pc(using windows btrfs driver), and maybe (i don't remember) put a few files to the disk.Then my thinkpadlinux-bt...@vger.kernel.org

Re: Spurious mount point

2018-10-18 Thread Andrei Borzenkov
16.10.2018 0:33, Chris Murphy пишет: > On Mon, Oct 15, 2018 at 3:26 PM, Anton Shepelev wrote: >> Chris Murphy to Anton Shepelev: >> How can I track down the origin of this mount point: /dev/sda2 on /home/hana type btrfs

Re: CRC mismatch

2018-10-18 Thread Austin S. Hemmelgarn
On 18/10/2018 08.02, Anton Shepelev wrote: I wrote: What may be the reason of a CRC mismatch on a BTRFS file in a virutal machine: csum failed ino 175524 off 1876295680 csum 451760558 expected csum 1446289185 Shall I seek the culprit in the host machine on in the guest one? Supposing the

Re: CRC mismatch

2018-10-18 Thread Anton Shepelev
I wrote: >What may be the reason of a CRC mismatch on a BTRFS file in >a virutal machine: > >csum failed ino 175524 off 1876295680 csum 451760558 >expected csum 1446289185 > >Shall I seek the culprit in the host machine on in the >guest one? Supposing the host machine healty, what >operations on

[PATCH 3/6] btrfs: qgroup: Refactor btrfs_qgroup_trace_subtree_swap()

2018-10-18 Thread Qu Wenruo
Refactor btrfs_qgroup_trace_subtree_swap() into qgroup_trace_subtree_swap(), which only needs two extent buffer and some other bool to control the behavior. Also, allow depending functions to accept parameter @exec_post to determine whether we need to trigger backref walk. This provides the

[PATCH 4/6] btrfs: qgroup: Introduce per-root swapped blocks infrastructure

2018-10-18 Thread Qu Wenruo
To allow delayed subtree swap rescan, btrfs needs to record per-root info about which tree blocks get swapped. So this patch introduces per-root btrfs_qgroup_swapped_blocks structure, which records which tree blocks get swapped. The designed workflow will be: 1) Record the subtree root block

[PATCH 5/6] btrfs: qgroup: Use delayed subtree rescan for balance

2018-10-18 Thread Qu Wenruo
Before this patch, qgroup code trace the whole subtree of file and reloc trees unconditionally. This makes qgroup numbers consistent, but it could cause tons of unnecessary extent trace, which cause a lot of overhead. However for subtree swap of balance, since both subtree contains the same

[PATCH 1/6] btrfs: qgroup: Allow btrfs_qgroup_extent_record::old_roots unpopulated at insert time

2018-10-18 Thread Qu Wenruo
Commit fb235dc06fac ("btrfs: qgroup: Move half of the qgroup accounting time out of commit trans") makes btrfs_qgroup_extent_record::old_roots populated at insert time. It's OK for most cases as btrfs_qgroup_extent_record is inserted at delayed ref head insert time, which has a less restrict lock

[PATCH 2/6] btrfs: relocation: Commit transaction before dropping btrfs_root::reloc_root

2018-10-18 Thread Qu Wenruo
Currently only relocation code cares about btrfs_root::reloc_root, and they have the method to sync btrfs_root::reloc_root without screwing things up. However qgroup code doesn't really have the ability to keep btrfs_root::reloc_root reliable. Currently if someone outside of relocation code want

[PATCH 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-10-18 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/linux/tree/qgroup_balance_skip_trees Which is still based on v4.19-rc1, but with previous submitted patches as dependency. This patch address the heavy load subtree scan, but delaying it until we're going to modify the

[PATCH 6/6] btrfs: qgroup: Cleanup old subtree swap code

2018-10-18 Thread Qu Wenruo
Since it's replaced by new delayed subtree swap code, remove the original code. The cleanup is small since most of its core function is still used by delayed subtree swap trace. Signed-off-by: Qu Wenruo --- fs/btrfs/qgroup.c | 94 ---

Re: Conversion to btrfs raid1 profile on added ext device renders some systems unable to boot into converted rootfs

2018-10-18 Thread Qu Wenruo
On 2018/10/18 下午2:16, Tony Prokott wrote: > On Wed, 17 Oct 2018 17:57:25 -0700 Qu Wenruo > wrote > ... > > > But after chrooting to update-initramfs and cataloging resulting image > content, usb_storage and uas were present under /lib/modules/xxx already, and > failing systems

Re: Untar on empty partition returns ENOSPACE

2018-10-18 Thread Henk Slager
On Thu, Oct 18, 2018 at 6:04 AM Jean-Denis Girard wrote: > > Hi list, > > My goal is to duplicate some SD cards, to prepare 50 similar Raspberry Pi. > > First, I made a tar of my master SD card (unmounted). Then I made a > script, which creates 2 partitions (50 MB for boot, 14 GB for /), >

Re: Conversion to btrfs raid1 profile on added ext device renders some systems unable to boot into converted rootfs

2018-10-18 Thread Tony Prokott
On Wed, 17 Oct 2018 17:57:25 -0700 Qu Wenruo wrote ... > > But after chrooting to update-initramfs and cataloging resulting image > > content, usb_storage and uas were present under /lib/modules/xxx already, > > and failing systems still just busybox without a real rootfs rather