Re: BTRFS recovery not possible

2019-06-19 Thread Qu Wenruo
On 2019/6/20 上午7:45, Zygo Blaxell wrote: > On Sun, Jun 16, 2019 at 12:05:21AM +0200, Claudius Winkel wrote: >> Thanks for the Help >> >> I get my data back. >> >> But now I`m thinking... how did it come so far? >> >> Was it luks the dm-crypt? > > dm-crypt is fine. dm-crypt is not a magical tool

Re: BTRFS recovery not possible

2019-06-19 Thread Zygo Blaxell
On Sun, Jun 16, 2019 at 12:05:21AM +0200, Claudius Winkel wrote: > Thanks for the Help > > I get my data back. > > But now I`m thinking... how did it come so far? > > Was it luks the dm-crypt? dm-crypt is fine. dm-crypt is not a magical tool for creating data loss in Linux storage stacks. I'v

Re: [PATCH 0/4] Spring cleaning

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 05:04:36PM +0300, Nikolay Borisov wrote: > Just a couple of patches that remove unneeded code. The first one does away > with an always true if construct and the rest remove code which has been #if 0 > for quite some time. > > Nikolay Borisov (4): > btrfs-progs: Remove r

Re: [PATCH 3/4] btrfs: Remove old send implementation

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 05:04:39PM +0300, Nikolay Borisov wrote: > Commit ba23855cdc89 ("btrfs-progs: send: use splice syscall instead of > read/write to transfer buffer") changed the send implementation to use > splice(). The old read/write implementation hasn't be used for at least > 3 years, it'

Re: [PATCH 2/4] btrfs-progs: Remove commented code

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 05:04:38PM +0300, Nikolay Borisov wrote: > This piece of code has been commented since 2009, given the number of > changes that have happened it's unlikely it could be made to work or > is needed at all. Just delete it. I has been ifdefed out in the infamous monster commit

[PATCH 3/4] btrfs: migrate the delalloc space stuff to it's own home

2019-06-19 Thread Josef Bacik
We have code for data and metadata reservations for delalloc. There's quite a bit of code here, and it's used in a lot of places so I've separated it out to it's own file. inode.c and file.c are already pretty large, and this code is complicated enough to live in its own space. Signed-off-by: Jo

[PATCH 0/4] btrfs: move random block rsv code around

2019-06-19 Thread Josef Bacik
We have some specialized block rsvs, like the inode rsv and the delayed refs rsv, that have their own special oddities. These don't fit in extent-tree.c, so migrate them to different places. Most of them go to existing files, but the delalloc stuff gets its own file because it's a lot of code, an

[PATCH 4/4] btrfs: move the subvolume reservation stuff out of extent-tree.c

2019-06-19 Thread Josef Bacik
This is just two functions, put it in root-tree.c since it involves root items. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 54 fs/btrfs/root-tree.c | 56 ++ 2 files changed, 56 inserti

[PATCH 2/4] btrfs: migrate btrfs_trans_release_chunk_metadata

2019-06-19 Thread Josef Bacik
Move this into transaction.c with the rest of the transaction related code. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 - fs/btrfs/extent-tree.c | 18 -- fs/btrfs/transaction.c | 18 ++ fs/btrfs/transaction.h | 1 + 4 files changed, 19 insertions(

[PATCH 1/4] btrfs: migrate the delayed refs rsv code

2019-06-19 Thread Josef Bacik
These belong with the delayed refs related code, not in extent-tree.c. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 9 --- fs/btrfs/delayed-ref.c | 177 + fs/btrfs/delayed-ref.h | 10 +++ fs/btrfs/extent-tree.c | 175 -

[PATCH 6/8] btrfs: migrate the block-rsv code to block-rsv.c

2019-06-19 Thread Josef Bacik
This moves everything out of extent-tree.c to block-rsv.c. Signed-off-by: Josef Bacik --- fs/btrfs/Makefile | 3 +- fs/btrfs/block-rsv.c | 257 + fs/btrfs/extent-tree.c | 249 --- 3 files changed

[PATCH 5/8] btrfs: stop using block_rsv_release_bytes everywhere

2019-06-19 Thread Josef Bacik
block_rsv_release_bytes() is the internal to the block_rsv code, and shouldn't be called directly by anything else. Switch all users to the exported helpers. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/

[PATCH 8/8] btrfs: export and migrate use_block_rsv/unuse_block_rsv

2019-06-19 Thread Josef Bacik
Move these into block_rsv.c/h respectively, and export them for use by alloc_tree_block. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.c | 82 fs/btrfs/block-rsv.h | 13 ++- fs/btrfs/extent-tree.c | 92 ++---

[PATCH 4/8] btrfs: cleanup the target logic in __btrfs_block_rsv_release

2019-06-19 Thread Josef Bacik
This works for all callers already, but if we wanted to use the helper for the global_block_rsv it would end up trying to refill itself. Fix the logic to be able to be used no matter which block rsv is passed in to this helper. Signed-off-by: Josef Bacik --- fs/btrfs/extent-tree.c | 14

[PATCH 7/8] btrfs: migrate the global_block_rsv helpers to block-rsv.c

2019-06-19 Thread Josef Bacik
These helpers belong in block-rsv.c Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.c | 90 + fs/btrfs/block-rsv.h | 3 ++ fs/btrfs/extent-tree.c | 99 +++--- 3 files changed, 98 insertions(+), 94 dele

[PATCH 3/8] btrfs: export __btrfs_block_rsv_release

2019-06-19 Thread Josef Bacik
The delalloc reserve stuff calls this directly because it cares about the qgroup accounting stuff, so export it so we can move it around. Fix btrfs_block_rsv_release() to just be a static inline since it just calls __btrfs_block_rsv_release() with NULL for the qgroup stuff. Signed-off-by: Josef B

[PATCH 0/8] btrfs: move the block_rsv code out of extent-tree.c

2019-06-19 Thread Josef Bacik
This patchset depends on the space_info migration patchset. This is the next logical chunk of things to move out of extent-tree.c With these sets of patches we're down below 10k loc in extent-tree.c. This chunk was much easier to move as we had exported a lot of these functions already. There i

[PATCH 1/8] btrfs: move btrfs_block_rsv definitions into it's own header

2019-06-19 Thread Josef Bacik
Prep work for separating out all of the block_rsv related code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.h | 81 ++ fs/btrfs/ctree.h | 70 +-- fs/btrfs/extent-tree.c | 1 +

[PATCH 2/8] btrfs: export btrfs_block_rsv_add_bytes

2019-06-19 Thread Josef Bacik
This is used in a few places, we need to make sure it's exported so we can move it around. Signed-off-by: Josef Bacik --- fs/btrfs/block-rsv.h | 2 ++ fs/btrfs/extent-tree.c | 18 +- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/block-rsv.h b/fs/btrf

Re: "no space left on device" from tar on ppc64le

2019-06-19 Thread Jean-Denis Girard
Hi Rich, Le 18/06/2019 à 13:19, Chris Murphy a écrit : > On Tue, Jun 18, 2019 at 4:23 PM Rich Turner wrote: >> >> tar: ./lib/modules/4.4.73-7-default/kernel/drivers/md/faulty.ko: Cannot >> open: No space left on device > > If this really is a 4.4.73 based kernel, I expect the report is out of >

Re: [PATCH] btrfs: Simplify update space_info in __reserve_metadata_bytes()

2019-06-19 Thread Nikolay Borisov
On 19.06.19 г. 17:12 ч., Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Simplification. > We don't need an if-else-if structure where we can use a > simple OR since both conditions are performing the > same action. The short-circuit for OR will ensure that if > the first condition is t

Re: [PATCH] btrfs: reorder struct btrfs_key for better alignment

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 09:50:58PM +0800, Qu Wenruo wrote: > > > On 2019/6/19 下午9:39, David Sterba wrote: > > On Wed, Jun 19, 2019 at 09:37:39AM +0800, Qu Wenruo wrote: > >>> struct btrfs_key { > >>> __u64 objectid; > >>> - __u8 type; > >>> __u64 offset; > >>> + __u8 type; > >>> } __attribu

[PATCH] btrfs: Simplify update space_info in __reserve_metadata_bytes()

2019-06-19 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Simplification. We don't need an if-else-if structure where we can use a simple OR since both conditions are performing the same action. The short-circuit for OR will ensure that if the first condition is true, can_overcommit() is not called. Signed-off-by: Goldwyn Rodrig

[PATCH 4/4] btrfs-progs: check: Remove duplicated and commented functions

2019-06-19 Thread Nikolay Borisov
Commit 756105181e57 ("btrfs-progs: check: supplement extent backref list with rbtree") changed the backref implementation to use rb tree and also commented the old implementations. It's been almost 2 years since that change and it's unlikely the old version will ever be used, so just remove it. Si

[PATCH 3/4] btrfs: Remove old send implementation

2019-06-19 Thread Nikolay Borisov
Commit ba23855cdc89 ("btrfs-progs: send: use splice syscall instead of read/write to transfer buffer") changed the send implementation to use splice(). The old read/write implementation hasn't be used for at least 3 years, it's time to remove it. Signed-off-by: Nikolay Borisov --- cmds-send.c |

[PATCH 0/4] Spring cleaning

2019-06-19 Thread Nikolay Borisov
Just a couple of patches that remove unneeded code. The first one does away with an always true if construct and the rest remove code which has been #if 0 for quite some time. Nikolay Borisov (4): btrfs-progs: Remove redundant if btrfs-progs: Remove commented code btrfs: Remove old send imp

[PATCH 1/4] btrfs-progs: Remove redundant if

2019-06-19 Thread Nikolay Borisov
'pin' is always true in __free_extent so there is no point in checking it. Just remove the if and unindent the code. Signed-off-by: Nikolay Borisov --- extent-tree.c | 13 + 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/extent-tree.c b/extent-tree.c index c6516b2ba445

[PATCH 2/4] btrfs-progs: Remove commented code

2019-06-19 Thread Nikolay Borisov
This piece of code has been commented since 2009, given the number of changes that have happened it's unlikely it could be made to work or is needed at all. Just delete it. Signed-off-by: Nikolay Borisov --- extent-tree.c | 257 -- 1 file changed,

Re: [PATCH] btrfs: reorder struct btrfs_key for better alignment

2019-06-19 Thread Qu Wenruo
On 2019/6/19 下午9:39, David Sterba wrote: > On Wed, Jun 19, 2019 at 09:37:39AM +0800, Qu Wenruo wrote: >>> struct btrfs_key { >>> __u64 objectid; >>> - __u8 type; >>> __u64 offset; >>> + __u8 type; >>> } __attribute__ ((__packed__)); >> >> And why not remove the packed attribute? >

Re: [PATCH] btrfs: reorder struct btrfs_key for better alignment

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 09:37:39AM +0800, Qu Wenruo wrote: > > struct btrfs_key { > > __u64 objectid; > > - __u8 type; > > __u64 offset; > > + __u8 type; > > } __attribute__ ((__packed__)); > > And why not remove the packed attribute? Because of this (stack usage changes): do_reloc

Re: [PATCH] btrfs: Evaluate io_tree in find_lock_delalloc_range()

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 06:36:08AM -0500, Goldwyn Rodrigues wrote: > On 9:05 19/06, Nikolay Borisov wrote: > > > > > > On 19.06.19 г. 3:35 ч., Goldwyn Rodrigues wrote: > > > Simplification. > > > No point passing the tree variable when it can be evaluated > > > from inode. > > > > > > Signed-of

Re: updating btrfs-debugfs to python3?

2019-06-19 Thread Hans van Kranenburg
On 6/19/19 1:30 PM, David Sterba wrote: > On Wed, Jun 19, 2019 at 12:21:34PM +0200, Hans van Kranenburg wrote: >> On 6/19/19 12:13 PM, Holger Hoffstätte wrote: >>> On 6/19/19 5:20 AM, Chris Murphy wrote: This is on Fedora Rawhide (which is planned to become Fedora 31 due in October), whic

Re: [PATCH 5/6] btrfs: use raid_attr for minimum stripe count in btrfs_calc_avail_data_space

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 10:51:14AM +0300, Nikolay Borisov wrote: > > > On 18.06.19 г. 21:00 ч., David Sterba wrote: > > Minimum stripe count matches the minimum devices required for a given > > profile. The open coded assignments match the raid_attr table. > > > > What's changed here is the mean

[PATCH 2/2] generic/059: also test that the file's mtime and ctime are updated

2019-06-19 Thread fdmanana
From: Filipe Manana Test as well that hole punch operations that affect a single file block also update the file's mtime and ctime. This is motivated by a bug a found in btrfs which is fixed by the following patch for the linux kernel: "Btrfs: add missing inode version, ctime and mtime updates

[PATCH] Btrfs: add missing inode version, ctime and mtime updates when punching hole

2019-06-19 Thread fdmanana
From: Filipe Manana If the range for which we are punching a hole covers only part of a page, we end up updating the inode item but we skip the update of the inode's iversion, mtime and ctime. Fix that by ensuring we update those properties of the inode. A patch for fstests test case generic/059

[PATCH 1/2] generic: test dir fsync after deleting dentry post eviction of its inode

2019-06-19 Thread fdmanana
From: Filipe Manana Test that if we fsync a file, evict its inode, unlink it and then fsync its parent directory, after a power failure the file does not exists. This is motivated by a bug found in btrfs, which is fixed by the following patch for the linux kernel: "Btrfs: fix fsync not persist

[PATCH] Btrfs: fix fsync not persisting dentry deletions due to inode evictions

2019-06-19 Thread fdmanana
From: Filipe Manana In order to avoid searches on a log tree when unlinking an inode, we check if the inode being unlinked was logged in the current transaction, as well as the inode of its parent directory. When any of the inodes are logged, we proceed to delete directory items and inode referen

Re: [PATCH 6/6] btrfs: lift bio_set_dev from bio allocation helpers

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 10:02:53AM +0300, Nikolay Borisov wrote: > > > On 18.06.19 г. 21:00 ч., David Sterba wrote: > > The block device is passed around for the only purpose to set it in new > > bios. Move the assignment one level up. This is a preparatory patch for > > further bdev cleanups. >

Re: Rebalancing raid1 after adding a device

2019-06-19 Thread Supercilious Dude
On Tue, 18 Jun 2019 at 19:45, Hugo Mills wrote: > >It would be really great if there was an ioctl that allowed you to > say things like "take the chunks of this block group and put them on > devices 2, 4 and 5 in RAID-5", because you could do a load of > optimisation with reshaping the FS in u

Re: [PATCH 2/6] btrfs: use common helpers for extent IO state insertion messages

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 09:54:16AM +0300, Nikolay Borisov wrote: > > > On 18.06.19 г. 21:00 ч., David Sterba wrote: > > Print the error messages using the helpers that also print the > > filesystem identification. > > > > Signed-off-by: David Sterba > > --- > > fs/btrfs/extent_io.c | 11 ++

Re: [PATCH] btrfs: Evaluate io_tree in find_lock_delalloc_range()

2019-06-19 Thread Goldwyn Rodrigues
On 9:05 19/06, Nikolay Borisov wrote: > > > On 19.06.19 г. 3:35 ч., Goldwyn Rodrigues wrote: > > Simplification. > > No point passing the tree variable when it can be evaluated > > from inode. > > > > Signed-off-by: Goldwyn Rodrigues > > The patch is good, however, there are several calls to

Re: updating btrfs-debugfs to python3?

2019-06-19 Thread David Sterba
On Wed, Jun 19, 2019 at 12:21:34PM +0200, Hans van Kranenburg wrote: > On 6/19/19 12:13 PM, Holger Hoffstätte wrote: > > On 6/19/19 5:20 AM, Chris Murphy wrote: > >> This is on Fedora Rawhide (which is planned to become Fedora 31 due in > >> October), which no longer provides python2. > >> > >> $ .

Re: [PATCH 11/19] btrfs: introduce submit buffer

2019-06-19 Thread Damien Le Moal
On 2019/06/18 22:34, Josef Bacik wrote: > On Mon, Jun 17, 2019 at 03:16:05AM +, Damien Le Moal wrote: >> The block I/O scheduler reorders requests in LBA order, but that happens for >> a >> newly inserted request against pending requests. If there are no pending >> requests because all request

Re: updating btrfs-debugfs to python3?

2019-06-19 Thread Hans van Kranenburg
On 6/19/19 12:13 PM, Holger Hoffstätte wrote: > On 6/19/19 5:20 AM, Chris Murphy wrote: >> This is on Fedora Rawhide (which is planned to become Fedora 31 due in >> October), which no longer provides python2. >> >> $ ./btrfs-debugfs -b / >> /usr/bin/env: ‘python2’: No such file or directory >> $ >>

Re: updating btrfs-debugfs to python3?

2019-06-19 Thread Holger Hoffstätte
On 6/19/19 5:20 AM, Chris Murphy wrote: This is on Fedora Rawhide (which is planned to become Fedora 31 due in October), which no longer provides python2. $ ./btrfs-debugfs -b / /usr/bin/env: ‘python2’: No such file or directory $ I expect other distros are going to follow as Python 2.7 EOL is

Re: Rebalancing raid1 after adding a device

2019-06-19 Thread Stéphane Lesimple
Le 19 juin 2019 05:27:21 Andrei Borzenkov a écrit : 18.06.2019 21:45, Hugo Mills пишет: ... Is there a way to ask the block group allocator to prefer writing to a specific device during a balance? Something like -ddestdevid=N? This would just be a hint to the allocator and the usual const

Re: [PATCH 5/6] btrfs: use raid_attr for minimum stripe count in btrfs_calc_avail_data_space

2019-06-19 Thread Nikolay Borisov
On 18.06.19 г. 21:00 ч., David Sterba wrote: > Minimum stripe count matches the minimum devices required for a given > profile. The open coded assignments match the raid_attr table. > > What's changed here is the meaning for RAID5/6. Previously their > min_stripes would be 1, while newly it's de

Re: [PATCH 6/6] btrfs: lift bio_set_dev from bio allocation helpers

2019-06-19 Thread Nikolay Borisov
On 18.06.19 г. 21:00 ч., David Sterba wrote: > The block device is passed around for the only purpose to set it in new > bios. Move the assignment one level up. This is a preparatory patch for > further bdev cleanups. > > Signed-off-by: David Sterba Reviewed-by: Nikolay Borisov Albeit I'd g