Re: Storage and snapshots as historical yearly

On 09/11/2017 07:49 PM, Austin S. Hemmelgarn wrote: > Unfortunately, I don't know of any overlay mount implementation that > works correctly and reliably with BTRFS.  I know for a fact that > OverlayFS (the upstream in-kernel implementation) does not work, and I > believe that AUFS3 and UnionFS

Re: send | receive: received snapshot is missing recent files

11.09.2017 20:53, Axel Burri пишет: > On 2017-09-08 06:44, Dave wrote: >> I'm referring to the link below. Using "btrfs subvolume snapshot -r" >> copies the Received UUID from the source into the new snapshot. The >> btrbk FAQ entry suggests otherwise. Has something changed? > > I don't think

Re: Storage and snapshots as historical yearly

11.09.2017 21:17, Senén Vidal Blanco пишет: > I am trying to implement a system that stores the data in a unit (A) with > BTRFS format that is untouchable and that future files and folders created or > modified are stored in another physical unit (B) with BTRFS format. > Each year the new files

Re: [PATCH] btrfs: send: fix error code if an unknown inode type is found

Hi David, On 2017/09/12 1:58, David Sterba wrote: > There are two types for "not supported", but the correct one is > EOPNOTSUPP, let's use that one. The error message in the error case is > not very helpful, enhance it at least with the inode number. When I first posted a similar patch, you

Fwd: [PATCH] btrfs: remove BTRFS_FS_QUOTA_DISABLING flag

Hello, Does anyone have a comment on this? Regards, Tomohiro Forwarded Message Subject: [PATCH] btrfs: remove BTRFS_FS_QUOTA_DISABLING flag Date: Wed, 30 Aug 2017 16:33:16 +0900 From: Misono, Tomohiro To: linux-btrfs@vger.kernel.org Currently,

[PATCH] btrfs-progs: cleanup whitespaces of btrfs-completion

Clean up spaces and use only tabs. No functional change. Signed-off-by: Tomohiro Misono --- btrfs-completion | 188 +++ 1 file changed, 94 insertions(+), 94 deletions(-) diff --git a/btrfs-completion

Re: BTRFS Deduplication

On Mon, Sep 11, 2017 at 2:55 PM, Qu Wenruo wrote: > > > On 2017年09月11日 17:14, Qu Wenruo wrote: >> >> >> >> On 2017年09月11日 16:57, shally verma wrote: >>> >>> On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo >>> wrote: On 2017年09月11日

Re: Help me understand what is going on with my RAID1 FS

Thanks everyone for the helpful and detailed responses. Now that you confirmed that everything is fine with my FS, I'm all relaxed because I can for sure live with the output of df. On Mon, Sep 11, 2017 at 5:29 AM, Andrei Borzenkov wrote: > 10.09.2017 23:17, Dmitrii

Re: BTRFS Deduplication

On 2017年09月11日 17:14, Qu Wenruo wrote: On 2017年09月11日 16:57, shally verma wrote: On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On

Re: BTRFS Deduplication

On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: > > > On 2017年09月11日 15:54, shally verma wrote: >> >> On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo >> wrote: >>> >>> >>> >>> On 2017年09月11日 14:05, shally verma wrote: I was going

Re: btrfs is slow while looping in search_bitmap <-btrfs_find_space_for_alloc

Am 05.09.2017 um 07:58 schrieb Stefan Priebe - Profihost AG: > Hello, > > while expecting slow btrfs volumes i switched to kernel v4.13 and to > space_cache=v2. ... > > Is btrfs trying to hard to find free space? Even nobody replied - i reply to myself. I could completely "fix" this by using

Re: BTRFS Deduplication

On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On 2017年09月11日 14:05, shally verma wrote: I was going through BTRFS Deduplication page (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read "As such,

Re: BTRFS Deduplication

On 2017年09月11日 16:57, shally verma wrote: On Mon, Sep 11, 2017 at 1:42 PM, Qu Wenruo wrote: On 2017年09月11日 15:54, shally verma wrote: On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: On 2017年09月11日 14:05, shally verma wrote: I was

Re: speed up big btrfs volumes with ssds

Hello, Am 04.09.2017 um 20:32 schrieb Stefan Priebe - Profihost AG: > Am 04.09.2017 um 15:28 schrieb Timofey Titovets: >> 2017-09-04 15:57 GMT+03:00 Stefan Priebe - Profihost AG >> : >>> Am 04.09.2017 um 12:53 schrieb Henk Slager: On Sun, Sep 3, 2017 at 8:32 PM,

Re: Workqueue: events_unbound btrfs_async_reclaim_metadata_space [btrfs]

On 08/09/2017 05:24, Duncan wrote: Marco Lorenzo Crociani posted on Thu, 07 Sep 2017 16:37:08 +0200 as excerpted: This e-mail (including any attachments) is private and confidential, and may be privileged. It is for the exclusive use of the intended recipient(s). If you have received this

Re: test if a subvolume is a snapshot?

On 2017-09-08 16:54, Tomasz Kłoczko wrote: On 8 September 2017 at 20:06, Austin S. Hemmelgarn wrote: [..] If you don't like awk you can use jq, sed, perl, python, ruby or whatever you have/like/want. And which command is more readable? Something like the above, or:

Re: Please help with exact actions for raid1 hot-swap

On 2017-09-10 02:33, Marat Khalili wrote: It doesn't need replaced disk to be readable, right? Then what prevents same procedure to work without a spare bay? In theory, nothing. In practice, there are reliability issues with mounting a filesystem degraded (and you should be avoiding running

Re: Please help with exact actions for raid1 hot-swap

Patrik, Duncan, thank you for the help. The `btrfs replace start /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't try to reboot yet, still have grub/efi/several mdadm partitions to copy). It also worked much faster than mdadm would take, apparently only moving 126GB

Re: [PATCH] btrfs-progs: Output time elapsed for each major tree it checked

On Mon, Sep 11, 2017 at 02:26:23PM +0900, Qu Wenruo wrote: > Marc reported that "btrfs check --repair" runs much faster than "btrfs > check", which is quite weird. > > This patch will add time elapsed for each major tree it checked, for > both original mode and lowmem mode, so we can have a clue

[PATCH v2 0/7] Mkfs: Rework --rootdir to a more generic behavior

mkfs.btrfs --rootdir provides user a method to generate btrfs with pre-written content while without the need of root privilege to mount the fs. However the code is quite old and doesn't get much review or test. This makes some strange behavior, from customized chunk allocation (which uses the

[PATCH v2 2/7] btrfs-progs: Fix one-byte overlap bug in free_block_group_cache

free_block_group_cache() calls clear_extent_bits() with wrong end, which is one byte larger than the correct range. This will cause the next adjacent cache state be split. And due to the split, private pointer (which points to block group cache) will be reset to NULL. This is very hard to detect

Re: BTRFS Deduplication

On 2017年09月11日 14:05, shally verma wrote: I was going through BTRFS Deduplication page (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read "As such, xfs_io, is able to perform deduplication on a BTRFS file system," .. following this, I followed on to xfs_io link

BTRFS Deduplication

I was going through BTRFS Deduplication page (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read "As such, xfs_io, is able to perform deduplication on a BTRFS file system," .. following this, I followed on to xfs_io link https://linux.die.net/man/8/xfs_io As I understand, these

[PATCH v2 1/7] btrfs-progs: Refactor find_next_chunk() to get rid of parameter root and objectid

Function find_next_chunk() is used to find next chunk start position, which should only do search on chunk tree and objectid is fixed to BTRFS_FIRST_CHUNK_TREE_OBJECTID. So refactor the parameter list to get rid of @root, which should be get from fs_info->chunk_root, and @objectid, which is fixed

[PATCH v2 7/7] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

Add extra limitation explained for --rootdir option, including: 1) Size limitation Now I decide to follow "mkfs.ext4 -d" behavior, so we user is responsible to make sure the block device/file is large enough. 2) Read permission If user can't read the content, mkfs will just fail. So

[PATCH v2 6/7] btrfs-progs: mkfs: Workaround BUG_ON caused by rootdir option

--rootdir option will start a transaction to fill the fs, however if something goes wrong, from ENOSPC to lack of permission, we won't commit transaction and cause BUG_ON trigger by uncommitted transaction: -- extent buffer leak: start 29392896 len 16384 extent_io.c:579: free_extent_buffer:

[PATCH v2 4/7] btrfs-progs: mkfs: Update allocation info before verbose output

Since new --rootdir can allocate chunk, it will modify the chunk allocation result. This patch will update allocation info before verbose output to reflect such info. Signed-off-by: Qu Wenruo --- mkfs/main.c | 33 + 1 file changed, 33

[PATCH v2 3/7] btrfs-progs: mkfs: Rework rootdir option to avoid custom chunk layout

mkfs.btrfs --rootdir uses its own custom chunk layout. This provides the possibility to limit the filesystem to a minimal size. However this custom chunk allocation has several problems. The most obvious problem is that it will allocate chunk from device offset 0. Both kernel and normal mkfs will

[PATCH v2 5/7] btrfs-progs: Avoid BUG_ON for chunk allocation when ENOSPC happens

When passing directory larger than block device using --rootdir parameter, we get the following backtrace: -- extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28 ./mkfs.btrfs(+0x1a05d)[0x557939e6b05d] ./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8]

Re: BTRFS Deduplication

On Mon, Sep 11, 2017 at 12:16 PM, Qu Wenruo wrote: > > > On 2017年09月11日 14:05, shally verma wrote: >> >> I was going through BTRFS Deduplication page >> (https://btrfs.wiki.kernel.org/index.php/Deduplication) and I read >> >> "As such, xfs_io, is able to perform

btrfs send fails with ioctl error 524

Hello, I've opened a bug report here: https://bugzilla.kernel.org/show_bug.cgi?id=196903 because I'm unable to transfer snapshots any more, I get the error: > ERROR: send ioctl failed with -524: Unknown error 524 I don't know how to solve the problem, I'm stuck. Any help would be very

Re: Please help with exact actions for raid1 hot-swap

On 2017-09-11 09:16, Marat Khalili wrote: Patrik, Duncan, thank you for the help. The `btrfs replace start /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't try to reboot yet, still have grub/efi/several mdadm partitions to copy). It also worked much faster than mdadm

[PATCH] btrfs: avoid null pointer dereference on fs_info when calling btrfs_crit

From: Colin Ian King There are checks on fs_info in __btrfs_panic to avoid dereferencing a null fs_info, however, there is a call to btrfs_crit that may also dereference a null fs_info. Fix this by adding a check to see if fs_info is null and only print the s_id if

Re: Storage and snapshots as historical yearly

On 2017-09-11 14:17, Senén Vidal Blanco wrote: I am trying to implement a system that stores the data in a unit (A) with BTRFS format that is untouchable and that future files and folders created or modified are stored in another physical unit (B) with BTRFS format. Each year the new files will

Re: [PATCH] btrfs-progs: update btrfs-completion

On Mon, Sep 11, 2017 at 02:22:43PM +0900, Misono, Tomohiro wrote: > This patch updates btrfs-completion: > - add "filesystem du" and "rescure zero-log" > - restrict _btrfs_mnts to show btrfs type only > - add more completion in last case statements > > (This file contains both spaces/tabs and

Re: PATCH - btrfs-progs-4.13: cmds-inspect-tree-stats.h: No such file or directory

On Sat, Sep 09, 2017 at 08:13:51AM +0200, Uli Heller wrote: > From edc11bf1463e41f5c503655f67c3036bd952eb72 Mon Sep 17 00:00:00 2001 > From: Uli Heller > Date: Sat, 9 Sep 2017 08:02:51 +0200 > Subject: [PATCH] Removed missing header file, fixes compilation error

[GIT PULL v2] zstd support (lib, btrfs, squashfs, nocrypto)

Hi Linus, Nick Terrell's patch series to add zstd support to the kernel has been floating around for a while. After talking with Dave Sterba, Herbert and Phillip, we decided to send the whole thing in as one pull request. Herbert had asked about the crypto patch when we discussed the pull, but

[PATCH] btrfs: send: fix error code if an unknown inode type is found

There are two types for "not supported", but the correct one is EOPNOTSUPP, let's use that one. The error message in the error case is not very helpful, enhance it at least with the inode number. The unknown type seems to be the block device (S_IFBLK)(as it's the only unhandled case. Adding this

Re: [PATCH] btrfs: Clean up dead code in root-tree

On Sat, Sep 09, 2017 at 03:29:34PM +0100, Christos Gkekas wrote: > The value of variable 'can_recover' is never used after being set, thus > it should be removed. Seems it has never been used since introduction in commit 68a7342c51c95042 . Reviewed-by: David Sterba > >

Storage and snapshots as historical yearly

I am trying to implement a system that stores the data in a unit (A) with BTRFS format that is untouchable and that future files and folders created or modified are stored in another physical unit (B) with BTRFS format. Each year the new files will be moved to store A and start over. The idea

Re: send | receive: received snapshot is missing recent files

On 2017-09-08 06:44, Dave wrote: > I'm referring to the link below. Using "btrfs subvolume snapshot -r" > copies the Received UUID from the source into the new snapshot. The > btrbk FAQ entry suggests otherwise. Has something changed? I don't think something has changed, the description for the

Re: Please help with exact actions for raid1 hot-swap

Austin S. Hemmelgarn posted on Mon, 11 Sep 2017 11:11:01 -0400 as excerpted: > On 2017-09-11 09:16, Marat Khalili wrote: >> Patrik, Duncan, thank you for the help. The `btrfs replace start >> /dev/sdb7 /dev/sdd7 /mnt/data` worked without a hitch (though I didn't >> try to reboot yet, still have

[PATCH v2] Btrfs: clear_dirty only on pages in compression range

At now while compressing data range code touch dirty page status on whole range on each BTRFS_MAX_UNCOMPRESSED (128kb) iteration, that's costs time As we care only about page status in range that will be compressed in current iteration, let's touch dirty status only for actual compression range

[RFC][0/10] Cleanup and enhance delayed refs

I'm leaving for Linux Plumbers tomorrow so I wanted to get this out for people to look at and see if there were any general comments. These patches have only been compiled, not tested in any way, so I wouldn't recommend trying them. They do a few things 1) Cleanup empty delayed ref head

[PATCH 01/10] btrfs: add a helper to return a head ref

From: Josef Bacik Simplify the error handling in __btrfs_run_delayed_refs by breaking out the code used to return a head back to the delayed_refs tree for processing into a helper function. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 30

[PATCH 05/10] btrfs: move all ref head cleanup to the helper function

From: Josef Bacik We do a couple different cleanup operations on the ref head. We adjust counters, we'll free any reserved space if we didn't end up using the ref, and we clear the pending csum bytes. Move all these disparate things into cleanup_ref_head and clean up the logic

[PATCH 08/10] btrfs: switch args for comp_*_refs

From: Josef Bacik Because seriously? ref2 and then ref1? Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index 7374710..c5de6c7

[PATCH 09/10] btrfs: add a comp_refs() helper

From: Josef Bacik Instead of open-coding the delayed ref comparisons, add a helper to do the comparisons generically and use that everywhere. We compare sequence numbers last for following patches. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 54

[PATCH 04/10] btrfs: move ref_mod modification into the if (ref) logic

From: Josef Bacik We only use this logic if our ref isn't a ref_head, so move it up into the if (ref) case since we know that this is a normal ref and not a delayed ref head. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 6 ++ 1 file changed, 2

[PATCH 06/10] btrfs: remove delayed_ref_node from ref_head

From: Josef Bacik This is just excessive information in the ref_head, and makes the code complicated. It is a relic from when we had the heads and the refs in the same tree, which is no longer the case. With this removal I've cleaned up a bunch of the cruft around this old

[PATCH 07/10] btrfs: remove type argument from comp_tree_refs

From: Josef Bacik We can get this from the ref we've passed in. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-ref.c b/fs/btrfs/delayed-ref.c index

[PATCH 02/10] btrfs: move extent_op cleanup to a helper

From: Josef Bacik Move the extent_op cleanup for an empty head ref to a helper function to help simplify __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 77 ++ 1 file changed, 40

[PATCH 10/10] btrfs: track refs in a rb_tree instead of a list

From: Josef Bacik If we get a significant amount of delayed refs for a single block (think modifying multiple snapshots) we can end up spending an ungodly amount of time looping through all of the entries trying to see if they can be merged. This is because we only add them to a

[PATCH 03/10] btrfs: breakout empty head cleanup to a helper

From: Josef Bacik Move this code out to a helper function to further simplivy __btrfs_run_delayed_refs. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 80 -- 1 file changed, 45 insertions(+), 35

[PATCH] Btrfs: compress_file_range() remove dead variable num_bytes

Remove dead assigment of num_bytes Also as num_bytes only used in will_compress block as copy of total_in just replace that with total_in and drop num_bytes entire Signed-off-by: Timofey Titovets --- fs/btrfs/inode.c | 10 +++--- 1 file changed, 3 insertions(+), 7

Re: "Unable to find ref byte nr ...." (4.11 somehow fishy?)

Hi again, > I have exactly the same errors, also on 4.11 kernel, but RAID-1 with 4 disks. After re-formating my backup-disk back in July, I today got exactly the same error running 4.12.9 (stack trace at the end of the mail) using the following mount options:

Re: [PATCH v2] btrfs-progs: Add zstd support

On Fri, Sep 08, 2017 at 01:29:23PM -0700, Nick Terrell wrote: > Adds zstd support to the btrfs program. An optional dependency on libzstd > >= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and > defaults to detecting if libzstd is present using `pkg-config'. Adds tests > for

Re: btrfs_remove_chunk call trace?

Rich Rauenzahn posted on Sun, 10 Sep 2017 22:45:50 -0700 as excerpted: > ...and can it be related to the Samsung 840 SSD's not supporting NCQ > Trim? (Although I can't tell which device this trace is from -- it > could be a mechanical Western Digital.) > > On Sun, Sep 10, 2017 at 10:16 PM, Rich