Re: [GIT PULL v2] zstd support (lib, btrfs, squashfs, nocrypto)

2017-09-11 Thread Phillip Lougher
On Mon, Sep 11, 2017 at 8:24 PM, Chris Mason wrote: > Hi Linus, > > Nick Terrell's patch series to add zstd support to the kernel has been > floating around for a while. After talking with Dave Sterba, Herbert > and Phillip, we decided to send the whole thing in as one pull request. As the Squas

Re: btrfs_remove_chunk call trace?

2017-09-11 Thread Rich Rauenzahn
On 9/11/2017 1:35 PM, Duncan wrote: Rich Rauenzahn posted on Sun, 10 Sep 2017 22:45:50 -0700 as excerpted: ...and can it be related to the Samsung 840 SSD's not supporting NCQ Trim? (Although I can't tell which device this trace is from -- it could be a mechanical Western Digital.) On Sun,

Re: Storage and snapshots as historical yearly

2017-09-11 Thread Andrei Borzenkov
11.09.2017 21:17, Senén Vidal Blanco пишет: > I am trying to implement a system that stores the data in a unit (A) with > BTRFS format that is untouchable and that future files and folders created or > modified are stored in another physical unit (B) with BTRFS format. > Each year the new files w

Re: send | receive: received snapshot is missing recent files

2017-09-11 Thread Andrei Borzenkov
11.09.2017 20:53, Axel Burri пишет: > On 2017-09-08 06:44, Dave wrote: >> I'm referring to the link below. Using "btrfs subvolume snapshot -r" >> copies the Received UUID from the source into the new snapshot. The >> btrbk FAQ entry suggests otherwise. Has something changed? > > I don't think some

Re: [PATCH] btrfs: send: fix error code if an unknown inode type is found

2017-09-11 Thread Tsutomu Itoh
Hi David, On 2017/09/12 1:58, David Sterba wrote: > There are two types for "not supported", but the correct one is > EOPNOTSUPP, let's use that one. The error message in the error case is > not very helpful, enhance it at least with the inode number. When I first posted a similar patch, you said

Fwd: [PATCH] btrfs: remove BTRFS_FS_QUOTA_DISABLING flag

2017-09-11 Thread Misono, Tomohiro
Hello, Does anyone have a comment on this? Regards, Tomohiro Forwarded Message Subject: [PATCH] btrfs: remove BTRFS_FS_QUOTA_DISABLING flag Date: Wed, 30 Aug 2017 16:33:16 +0900 From: Misono, Tomohiro To: linux-btrfs@vger.kernel.org Currently, "btrfs quota enable" would fail

[PATCH] btrfs-progs: cleanup whitespaces of btrfs-completion

2017-09-11 Thread Misono, Tomohiro
Clean up spaces and use only tabs. No functional change. Signed-off-by: Tomohiro Misono --- btrfs-completion | 188 +++ 1 file changed, 94 insertions(+), 94 deletions(-) diff --git a/btrfs-completion b/btrfs-completion index 1f00add..ae683f4 1

Re: Storage and snapshots as historical yearly

2017-09-11 Thread Pete
On 09/11/2017 07:49 PM, Austin S. Hemmelgarn wrote: > Unfortunately, I don't know of any overlay mount implementation that > works correctly and reliably with BTRFS.  I know for a fact that > OverlayFS (the upstream in-kernel implementation) does not work, and I > believe that AUFS3 and UnionFS (t

Re: Please help with exact actions for raid1 hot-swap

2017-09-11 Thread Duncan
Austin S. Hemmelgarn posted on Mon, 11 Sep 2017 11:11:01 -0400 as excerpted: > On 2017-09-11 09:16, Marat Khalili wrote: >> Patrik, Duncan, thank you for the help. The `btrfs replace start >> /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't >> try to reboot yet, still have gr

[PATCH] Btrfs: compress_file_range() remove dead variable num_bytes

2017-09-11 Thread Timofey Titovets
Remove dead assigment of num_bytes Also as num_bytes only used in will_compress block as copy of total_in just replace that with total_in and drop num_bytes entire Signed-off-by: Timofey Titovets --- fs/btrfs/inode.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a

[PATCH 06/10] btrfs: remove delayed_ref_node from ref_head

2017-09-11 Thread josef
From: Josef Bacik This is just excessive information in the ref_head, and makes the code complicated. It is a relic from when we had the heads and the refs in the same tree, which is no longer the case. With this removal I've cleaned up a bunch of the cruft around this old assumption as well.

[PATCH 07/10] btrfs: remove type argument from comp_tree_refs

2017-09-11 Thread josef
From: Josef Bacik We can get this from the ref we've passed in. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 7cb1172..7374710 100644 --- a/fs/btrfs/dela

[PATCH 09/10] btrfs: add a comp_refs() helper

2017-09-11 Thread josef
From: Josef Bacik Instead of open-coding the delayed ref comparisons, add a helper to do the comparisons generically and use that everywhere. We compare sequence numbers last for following patches. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 54

[PATCH 04/10] btrfs: move ref_mod modification into the if (ref) logic

2017-09-11 Thread josef
From: Josef Bacik We only use this logic if our ref isn't a ref_head, so move it up into the if (ref) case since we know that this is a normal ref and not a delayed ref head. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff

[PATCH 05/10] btrfs: move all ref head cleanup to the helper function

2017-09-11 Thread josef
From: Josef Bacik We do a couple different cleanup operations on the ref head. We adjust counters, we'll free any reserved space if we didn't end up using the ref, and we clear the pending csum bytes. Move all these disparate things into cleanup_ref_head and clean up the logic in __btrfs_run_de

[PATCH 08/10] btrfs: switch args for comp_*_refs

2017-09-11 Thread josef
From: Josef Bacik Because seriously? ref2 and then ref1? Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 7374710..c5de6c7 100644 --- a/fs/btrfs/delayed-ref.c

[PATCH 10/10] btrfs: track refs in a rb_tree instead of a list

2017-09-11 Thread josef
From: Josef Bacik If we get a significant amount of delayed refs for a single block (think modifying multiple snapshots) we can end up spending an ungodly amount of time looping through all of the entries trying to see if they can be merged. This is because we only add them to a list, so we have

[PATCH 02/10] btrfs: move extent_op cleanup to a helper

2017-09-11 Thread josef
From: Josef Bacik Move the extent_op cleanup for an empty head ref to a helper function to help simplify __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 77 ++ 1 file changed, 40 insertions(+), 37 deletions(-) d

[PATCH 03/10] btrfs: breakout empty head cleanup to a helper

2017-09-11 Thread josef
From: Josef Bacik Move this code out to a helper function to further simplivy __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 80 -- 1 file changed, 45 insertions(+), 35 deletions(-) diff --git a/fs/btrfs/extent

[PATCH 01/10] btrfs: add a helper to return a head ref

2017-09-11 Thread josef
From: Josef Bacik Simplify the error handling in __btrfs_run_delayed_refs by breaking out the code used to return a head back to the delayed_refs tree for processing into a helper function. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 30 +++--- 1 file change

[RFC][0/10] Cleanup and enhance delayed refs

2017-09-11 Thread josef
I'm leaving for Linux Plumbers tomorrow so I wanted to get this out for people to look at and see if there were any general comments. These patches have only been compiled, not tested in any way, so I wouldn't recommend trying them. They do a few things 1) Cleanup empty delayed ref head handling

[PATCH v2] Btrfs: clear_dirty only on pages in compression range

2017-09-11 Thread Timofey Titovets
At now while compressing data range code touch dirty page status on whole range on each BTRFS_MAX_UNCOMPRESSED (128kb) iteration, that's costs time As we care only about page status in range that will be compressed in current iteration, let's touch dirty status only for actual compression range v

Re: btrfs_remove_chunk call trace?

2017-09-11 Thread Duncan
Rich Rauenzahn posted on Sun, 10 Sep 2017 22:45:50 -0700 as excerpted: > ...and can it be related to the Samsung 840 SSD's not supporting NCQ > Trim? (Although I can't tell which device this trace is from -- it > could be a mechanical Western Digital.) > > On Sun, Sep 10, 2017 at 10:16 PM, Rich

Re: [PATCH v2] btrfs-progs: Add zstd support

2017-09-11 Thread David Sterba
On Fri, Sep 08, 2017 at 01:29:23PM -0700, Nick Terrell wrote: > Adds zstd support to the btrfs program. An optional dependency on libzstd > >= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and > defaults to detecting if libzstd is present using `pkg-config'. Adds tests > for

Re: "Unable to find ref byte nr ...." (4.11 somehow fishy?)

2017-09-11 Thread Clemens Eisserer
Hi again, > I have exactly the same errors, also on 4.11 kernel, but RAID-1 with 4 disks. After re-formating my backup-disk back in July, I today got exactly the same error running 4.12.9 (stack trace at the end of the mail) using the following mount options: "ro,nosuid,nodev,relatime,compress-fo

[GIT PULL v2] zstd support (lib, btrfs, squashfs, nocrypto)

2017-09-11 Thread Chris Mason
Hi Linus, Nick Terrell's patch series to add zstd support to the kernel has been floating around for a while. After talking with Dave Sterba, Herbert and Phillip, we decided to send the whole thing in as one pull request. Herbert had asked about the crypto patch when we discussed the pull, but I

Re: PATCH - btrfs-progs-4.13: cmds-inspect-tree-stats.h: No such file or directory

2017-09-11 Thread David Sterba
On Sat, Sep 09, 2017 at 08:13:51AM +0200, Uli Heller wrote: > From edc11bf1463e41f5c503655f67c3036bd952eb72 Mon Sep 17 00:00:00 2001 > From: Uli Heller > Date: Sat, 9 Sep 2017 08:02:51 +0200 > Subject: [PATCH] Removed missing header file, fixes compilation error Applied, thanks. -- To unsubscrib

Re: [PATCH] btrfs-progs: update btrfs-completion

2017-09-11 Thread David Sterba
On Mon, Sep 11, 2017 at 02:22:43PM +0900, Misono, Tomohiro wrote: > This patch updates btrfs-completion: > - add "filesystem du" and "rescure zero-log" > - restrict _btrfs_mnts to show btrfs type only > - add more completion in last case statements > > (This file contains both spaces/tabs and m

Re: Storage and snapshots as historical yearly

2017-09-11 Thread Austin S. Hemmelgarn
On 2017-09-11 14:17, Senén Vidal Blanco wrote: I am trying to implement a system that stores the data in a unit (A) with BTRFS format that is untouchable and that future files and folders created or modified are stored in another physical unit (B) with BTRFS format. Each year the new files will b

Storage and snapshots as historical yearly

2017-09-11 Thread Senén Vidal Blanco
I am trying to implement a system that stores the data in a unit (A) with BTRFS format that is untouchable and that future files and folders created or modified are stored in another physical unit (B) with BTRFS format. Each year the new files will be moved to store A and start over. The idea i

Re: send | receive: received snapshot is missing recent files

2017-09-11 Thread Axel Burri
On 2017-09-08 06:44, Dave wrote: > I'm referring to the link below. Using "btrfs subvolume snapshot -r" > copies the Received UUID from the source into the new snapshot. The > btrbk FAQ entry suggests otherwise. Has something changed? I don't think something has changed, the description for the re

Re: [PATCH] btrfs: Clean up dead code in root-tree

2017-09-11 Thread David Sterba
On Sat, Sep 09, 2017 at 03:29:34PM +0100, Christos Gkekas wrote: > The value of variable 'can_recover' is never used after being set, thus > it should be removed. Seems it has never been used since introduction in commit 68a7342c51c95042 . Reviewed-by: David Sterba > > Signed-off-by: Christos G

[PATCH] btrfs: send: fix error code if an unknown inode type is found

2017-09-11 Thread David Sterba
There are two types for "not supported", but the correct one is EOPNOTSUPP, let's use that one. The error message in the error case is not very helpful, enhance it at least with the inode number. The unknown type seems to be the block device (S_IFBLK)(as it's the only unhandled case. Adding this t

[PATCH] btrfs: avoid null pointer dereference on fs_info when calling btrfs_crit

2017-09-11 Thread Colin King
From: Colin Ian King There are checks on fs_info in __btrfs_panic to avoid dereferencing a null fs_info, however, there is a call to btrfs_crit that may also dereference a null fs_info. Fix this by adding a check to see if fs_info is null and only print the s_id if fs_info is non-null. Detected

Re: Please help with exact actions for raid1 hot-swap

2017-09-11 Thread Austin S. Hemmelgarn
On 2017-09-11 09:16, Marat Khalili wrote: Patrik, Duncan, thank you for the help. The `btrfs replace start /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't try to reboot yet, still have grub/efi/several mdadm partitions to copy). It also worked much faster than mdadm woul

Re: [PATCH] btrfs-progs: Output time elapsed for each major tree it checked

2017-09-11 Thread Marc MERLIN
On Mon, Sep 11, 2017 at 02:26:23PM +0900, Qu Wenruo wrote: > Marc reported that "btrfs check --repair" runs much faster than "btrfs > check", which is quite weird. > > This patch will add time elapsed for each major tree it checked, for > both original mode and lowmem mode, so we can have a clue w

Re: Please help with exact actions for raid1 hot-swap

2017-09-11 Thread Marat Khalili
Patrik, Duncan, thank you for the help. The `btrfs replace start /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't try to reboot yet, still have grub/efi/several mdadm partitions to copy). It also worked much faster than mdadm would take, apparently only moving 126GB used,

Re: Please help with exact actions for raid1 hot-swap

2017-09-11 Thread Austin S. Hemmelgarn
On 2017-09-10 02:33, Marat Khalili wrote: It doesn't need replaced disk to be readable, right? Then what prevents same procedure to work without a spare bay? In theory, nothing. In practice, there are reliability issues with mounting a filesystem degraded (and you should be avoiding running

Re: test if a subvolume is a snapshot?

2017-09-11 Thread Austin S. Hemmelgarn
On 2017-09-08 16:54, Tomasz Kłoczko wrote: On 8 September 2017 at 20:06, Austin S. Hemmelgarn wrote: [..] If you don't like awk you can use jq, sed, perl, python, ruby or whatever you have/like/want. And which command is more readable? Something like the above, or: btrfs device stats --forma

Re: BTRFS Deduplication

2017-09-11 Thread shally verma
On Mon, Sep 11, 2017 at 2:55 PM, Qu Wenruo wrote: > > > On 2017年09月11日 17:14, Qu Wenruo wrote: >> >> >> >> On 2017年09月11日 16:57, shally verma wrote: >>> >>> On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo >>> wrote: On 2017年09月11日 15:54, shally verma wrote: > > > On Mon

Re: BTRFS Deduplication

2017-09-11 Thread Qu Wenruo
On 2017年09月11日 17:14, Qu Wenruo wrote: On 2017年09月11日 16:57, shally verma wrote: On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On 2017年09月11日 14:05, shally verma wrote: I was going

Re: btrfs is slow while looping in search_bitmap <-btrfs_find_space_for_alloc

2017-09-11 Thread Stefan Priebe - Profihost AG
Am 05.09.2017 um 07:58 schrieb Stefan Priebe - Profihost AG: > Hello, > > while expecting slow btrfs volumes i switched to kernel v4.13 and to > space_cache=v2. ... > > Is btrfs trying to hard to find free space? Even nobody replied - i reply to myself. I could completely "fix" this by using ssd_

Re: speed up big btrfs volumes with ssds

2017-09-11 Thread Stefan Priebe - Profihost AG
Hello, Am 04.09.2017 um 20:32 schrieb Stefan Priebe - Profihost AG: > Am 04.09.2017 um 15:28 schrieb Timofey Titovets: >> 2017-09-04 15:57 GMT+03:00 Stefan Priebe - Profihost AG >> : >>> Am 04.09.2017 um 12:53 schrieb Henk Slager: On Sun, Sep 3, 2017 at 8:32 PM, Stefan Priebe - Profihost AG

Re: BTRFS Deduplication

2017-09-11 Thread Qu Wenruo
On 2017年09月11日 16:57, shally verma wrote: On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On 2017年09月11日 14:05, shally verma wrote: I was going through BTRFS Deduplication page (https:/

Re: BTRFS Deduplication

2017-09-11 Thread shally verma
On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: > > > On 2017年09月11日 15:54, shally verma wrote: >> >> On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo >> wrote: >>> >>> >>> >>> On 2017年09月11日 14:05, shally verma wrote: I was going through BTRFS Deduplication page (https://btrfs.

Re: Workqueue: events_unbound btrfs_async_reclaim_metadata_space [btrfs]

2017-09-11 Thread Marco Lorenzo Crociani
On 08/09/2017 05:24, Duncan wrote: Marco Lorenzo Crociani posted on Thu, 07 Sep 2017 16:37:08 +0200 as excerpted: This e-mail (including any attachments) is private and confidential, and may be privileged. It is for the exclusive use of the intended recipient(s). If you have received this

Re: Help me understand what is going on with my RAID1 FS

2017-09-11 Thread FLJ
Thanks everyone for the helpful and detailed responses. Now that you confirmed that everything is fine with my FS, I'm all relaxed because I can for sure live with the output of df. On Mon, Sep 11, 2017 at 5:29 AM, Andrei Borzenkov wrote: > 10.09.2017 23:17, Dmitrii Tcvetkov пишет: Drive1

Re: BTRFS Deduplication

2017-09-11 Thread Qu Wenruo
On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On 2017年09月11日 14:05, shally verma wrote: I was going through BTRFS Deduplication page (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read "As such, xfs_io, is able to perform ded

Re: BTRFS Deduplication

2017-09-11 Thread shally verma
On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: > > > On 2017年09月11日 14:05, shally verma wrote: >> >> I was going through BTRFS Deduplication page >> (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read >> >> "As such, xfs_io, is able to perform deduplication on a BTRFS file >> s

btrfs send fails with ioctl error 524

2017-09-11 Thread Alessandro Polverini
Hello, I've opened a bug report here: https://bugzilla.kernel.org/show_bug.cgi?id=196903 because I'm unable to transfer snapshots any more, I get the error: > ERROR: send ioctl failed with -524: Unknown error 524 I don't know how to solve the problem, I'm stuck. Any help would be very welcome