Re: [PATCH RESEND v12] btrfs: introduce feature to forget a btrfs device

2019-01-02 Thread Anand Jain
On 01/03/2019 12:00 AM, David Sterba wrote: On Wed, Jan 02, 2019 at 01:49:31PM +0800, Anand Jain wrote: Support for a new command 'btrfs dev forget [dev]' is proposed here to undo the effects of 'btrfs dev scan [dev]'. For this purpose this patch proposes to use ioctl #5 as it was empty.

Re: BTRFS corruption: open_ctree failed

2019-01-02 Thread Tomasz Chmielewski
On 2019-01-03 16:27, Andrea Gelmini wrote: On Thu, Jan 03, 2019 at 11:52:05AM +0900, Tomasz Chmielewski wrote: Did you use 4.19.x kernels earlier than 4.19.8? They had a bug which would corrupt filesystems (mostly ext4 users would be reporting it, but I saw it with other filesystems, like xfs

[PATCH v2 5/5] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Qu Wenruo
Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") creates uuid tree at mkfs time. However it doesn't populate uuid tree correctly nor just create an empty root. It uses create_tree(), which just copies the content of fs root, containing meaningless INODE_ITEM: v4.15 mkfs (no uuid

[PATCH v2 4/5] btrfs-progs: uuid: Port btrfs_uuid_tree_add() function

2019-01-02 Thread Qu Wenruo
This function provide the offline ability to add new uuid tree entry. Also port fs_info->uuid and its initialization and cleanup code to support uuid tree. Signed-off-by: Qu Wenruo --- ctree.h | 7 +- disk-io.c | 19 +++- uuid-tree.c | 65 +

[PATCH v2 2/5] btrfs-progs: mkfs: Create data reloc tree from scratch

2019-01-02 Thread Qu Wenruo
For data reloc tree creation, we copy its content from fs tree just for its INODE_ITEM, INODE_REF and dirid. This hides the detail and is not obvious for why we're copying from fs root. This patch will create data reloc tree from scratch by: - Create root Including root item and new tree root.

[PATCH v2 1/5] btrfs-progs: Export btrfs_create_tree() and move it to disk-io.c

2019-01-02 Thread Qu Wenruo
Just as how kernel uses it. This provides the basis for later uuid creation. Signed-off-by: Qu Wenruo --- disk-io.c | 72 +++ disk-io.h | 4 ++- free-space-tree.c | 72 --- 3 files changed,

[PATCH v2 0/5] Fix incorrectly created uuid tree

2019-01-02 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/uuid_tree_support Which is based on v4.19.1 tag. Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") creates uuid tree at mkfs time. However it doesn't populate uuid tree correctly nor just cre

[PATCH v2 3/5] btrfs-progs: uuid: Port kernel btrfs_uuid_tree_lookup()

2019-01-02 Thread Qu Wenruo
Although we have btrfs_uuid_tree_lookup_any(), it's an online function utilizing tree search ioctl, not an offline search function. This patch will port kernel btrfs_uuid_tree_lookup() into btrfs-progs for later proper uuid tree initialization. Signed-off-by: Qu Wenruo --- uuid-tree.c | 73

Re: BTRFS corruption: open_ctree failed

2019-01-02 Thread Andrea Gelmini
On Thu, Jan 03, 2019 at 11:52:05AM +0900, Tomasz Chmielewski wrote: > Did you use 4.19.x kernels earlier than 4.19.8? > > They had a bug which would corrupt filesystems (mostly ext4 users would be > reporting it, but I saw it with other filesystems, like xfs and btrfs, too): Well, just for the re

Re: BTRFS corruption: open_ctree failed

2019-01-02 Thread Chris Murphy
On Wed, Jan 2, 2019 at 5:26 PM b11g wrote: > > Hi all, > > I have several BTRFS success-stories, and I've been an happy user for quite a > long time now. I was therefore surprised to face a BTRFS corruption on a > system I'd just installed. > > I use NixOS, unstable branch (linux kernel 4.19.12)

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Qu Wenruo
On 2019/1/2 下午6:00, Qu Wenruo wrote: > > > On 2019/1/2 下午5:13, Nikolay Borisov wrote: >> >> >> On 27.12.18 г. 9:13 ч., Qu Wenruo wrote: >>> Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") >>> creates uuid tree at mkfs time. >>> >>> However it doesn't populate uuid tree corre

Re: BTRFS corruption: open_ctree failed

2019-01-02 Thread Tomasz Chmielewski
I have several BTRFS success-stories, and I've been an happy user for quite= a long time now. I was therefore surprised to face a BTRFS corruption on a= system I'd just installed. I use NixOS, unstable branch (linux kernel 4.19.12). The system runs on a S= SD with an ext4 boot partition, a simp

Re: [PATCH RFC 0/2] Use new incompat feature BG_TREE to hugely reduce mount time

2019-01-02 Thread Qu Wenruo
On 2019/1/3 上午8:57, Hans van Kranenburg wrote: > Hi, > > On 1/3/19 1:14 AM, Qu Wenruo wrote: >> >> >> On 2019/1/3 上午12:21, David Sterba wrote: >>> On Fri, Dec 28, 2018 at 05:28:13PM +0800, Qu Wenruo wrote: On 2018/12/28 下午5:15, Nikolay Borisov wrote: > On 28.12.18 г. 10:37 ч., Qu Wenru

Re: [PATCH RFC 0/2] Use new incompat feature BG_TREE to hugely reduce mount time

2019-01-02 Thread Hans van Kranenburg
Hi, On 1/3/19 1:14 AM, Qu Wenruo wrote: > > > On 2019/1/3 上午12:21, David Sterba wrote: >> On Fri, Dec 28, 2018 at 05:28:13PM +0800, Qu Wenruo wrote: >>> On 2018/12/28 下午5:15, Nikolay Borisov wrote: On 28.12.18 г. 10:37 ч., Qu Wenruo wrote: > This patchset can be fetched from: > http

BTRFS corruption: open_ctree failed

2019-01-02 Thread b11g
Hi all, I have several BTRFS success-stories, and I've been an happy user for quite a long time now. I was therefore surprised to face a BTRFS corruption on a system I'd just installed. I use NixOS, unstable branch (linux kernel 4.19.12). The system runs on a SSD with an ext4 boot partition, a

Re: [PATCH RFC 0/2] Use new incompat feature BG_TREE to hugely reduce mount time

2019-01-02 Thread Qu Wenruo
On 2019/1/3 上午12:21, David Sterba wrote: > On Fri, Dec 28, 2018 at 05:28:13PM +0800, Qu Wenruo wrote: >> On 2018/12/28 下午5:15, Nikolay Borisov wrote: >>> On 28.12.18 г. 10:37 ч., Qu Wenruo wrote: This patchset can be fetched from: https://github.com/adam900710/linux/tree/bg_tree W

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Su Yue
On 2019/1/3 12:31 AM, David Sterba wrote: On Thu, Dec 27, 2018 at 07:28:40PM +0800, Su Yue wrote: +} + +int create_uuid_tree(struct btrfs_trans_handle *trans) +{ + struct btrfs_fs_info *fs_info = trans->fs_info; + struct btrfs_root *uuid_root = fs_info->uuid_root; + struct b

Re: [PATCH] Btrfs: do not overwrite error return value in the balance ioctl

2019-01-02 Thread David Sterba
On Mon, Dec 17, 2018 at 09:24:43AM +, Filipe Manana wrote: > On Mon, Dec 17, 2018 at 8:25 AM Anand Jain wrote: > > > > > > > > On 12/15/2018 03:45 AM, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > If the call to btrfs_balance() failed we would overwrite the error > > > retu

Re: [PATCH v2] Btrfs: do not overwrite scrub error with fault error in scrub ioctl

2019-01-02 Thread David Sterba
On Fri, Dec 14, 2018 at 07:50:17PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > If scrub returned an error and then the copy_to_user() call did not > succeed, we would overwrite the error returned by scrub with -EFAULT. > Fix that by removing the call to copy_to_user() as there is

Re: [PATCH] Btrfs: do not overwrite error return value in scrub progress ioctl

2019-01-02 Thread David Sterba
On Mon, Dec 17, 2018 at 09:33:43AM +0200, Nikolay Borisov wrote: > > > On 14.12.18 г. 21:45 ч., fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If the call to btrfs_scrub_progress() failed we would overwrite the error > > returned to user space with -EFAULT if the call to copy_to_use

Re: [PATCH 3/4] btrfs: Refactor retval handling of btrfs_lookup_file_extent in btrfs_get_extent

2019-01-02 Thread Nikolay Borisov
On 2.01.19 г. 19:05 ч., David Sterba wrote: > This is repeating code and could be simplified ... to the original code. It does and this is intentional. What I've strived to do is make the idea of the code obvious and not try to reduce the total line of code. It is a massive improvement given t

Re: [PATCH 1/1] btrfs: add kernel scrub log messages

2019-01-02 Thread David Sterba
On Mon, Dec 17, 2018 at 11:15:12AM +0800, Anand Jain wrote: > scrub kernel messages helps debug and audit, add them to the log. > > Signed-off-by: Anand Jain > --- > fs/btrfs/scrub.c | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c > index 902819

Re: [PATCH 0/4] Misc cleanups of btrfs_get_extent

2019-01-02 Thread David Sterba
On Mon, Dec 17, 2018 at 10:35:58AM +0200, Nikolay Borisov wrote: > Here is a series that hopefully makes the code a bit more obvious. First > patch > gives a variable a more appropraite name, since it's going to be used only > for > holding an extent type and not the type of a found item. > >

Re: [PATCH 3/4] btrfs: Refactor retval handling of btrfs_lookup_file_extent in btrfs_get_extent

2019-01-02 Thread David Sterba
On Mon, Dec 17, 2018 at 10:36:01AM +0200, Nikolay Borisov wrote: > The code which executes if path->slots[0] points to an item not > belonging to the inode we are interested in or is not na extent at all > could really trigger if btrfs_lookup_file_extent returned > 0 and > subsequently path->slots[

Re: [PATCH 11/20] Btrfs: drop useless LIST_HEAD

2019-01-02 Thread David Sterba
On Sun, Dec 23, 2018 at 09:57:06AM +0100, Julia Lawall wrote: > Drop LIST_HEAD where the variable it declares is never used. > > The uses were removed in 3fd0a5585eb9 ("Btrfs: Metadata ENOSPC > handling for balance"), but not the declaration. > > The semantic patch that fixes this problem is as f

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread David Sterba
On Thu, Dec 27, 2018 at 07:28:40PM +0800, Su Yue wrote: > > +} > > + > > +int create_uuid_tree(struct btrfs_trans_handle *trans) > > +{ > > + struct btrfs_fs_info *fs_info = trans->fs_info; > > + struct btrfs_root *uuid_root = fs_info->uuid_root; > > + struct btrfs_key key; > > + int ret; >

Re: [PATCH RFC 0/2] Use new incompat feature BG_TREE to hugely reduce mount time

2019-01-02 Thread David Sterba
On Fri, Dec 28, 2018 at 05:28:13PM +0800, Qu Wenruo wrote: > On 2018/12/28 下午5:15, Nikolay Borisov wrote: > > On 28.12.18 г. 10:37 ч., Qu Wenruo wrote: > >> This patchset can be fetched from: > >> https://github.com/adam900710/linux/tree/bg_tree > >> Which is based on v4.20-rc1 tag. > >> > >> This

Re: [PATCH RESEND v12] btrfs: introduce feature to forget a btrfs device

2019-01-02 Thread David Sterba
On Wed, Jan 02, 2019 at 01:49:31PM +0800, Anand Jain wrote: > Support for a new command 'btrfs dev forget [dev]' is proposed here > to undo the effects of 'btrfs dev scan [dev]'. For this purpose > this patch proposes to use ioctl #5 as it was empty. > IOW(BTRFS_IOCTL_MAGIC, 5, ..) > This pat

Re: [PATCH] btrfs-progs: Update backup roots when writing super blocks

2019-01-02 Thread Nikolay Borisov
On 2.01.19 г. 12:08 ч., Qu Wenruo wrote: > > > On 2019/1/2 下午4:57, Nikolay Borisov wrote: >> >> >> On 25.12.18 г. 7:28 ч., Qu Wenruo wrote: >>> The code is mostly ported from kernel with minimal change. >>> >>> Since btrfs-progs doesn't support replaying log, there is some code >>> unnecessary

Re: compress-force not really forcing compression?

2019-01-02 Thread Austin S. Hemmelgarn
On 12/23/2018 1:16 AM, Adam Borowski wrote: On Sun, Dec 23, 2018 at 12:24:02AM +, Paul Jones wrote: IMHO the more pertinent question is : If a file has portions which are not easily compressible does that imply all future writes are also incompressible. IMO no, so I think what will be prude

Re: [PATCH] btrfs-progs: Update backup roots when writing super blocks

2019-01-02 Thread Nikolay Borisov
On 2.01.19 г. 12:08 ч., Qu Wenruo wrote: > > > On 2019/1/2 下午4:57, Nikolay Borisov wrote: >> >> >> On 25.12.18 г. 7:28 ч., Qu Wenruo wrote: >>> The code is mostly ported from kernel with minimal change. >>> >>> Since btrfs-progs doesn't support replaying log, there is some code >>> unnecessary

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Qu Wenruo
On 2019/1/2 下午6:07, Nikolay Borisov wrote: > > > On 2.01.19 г. 12:00 ч., Qu Wenruo wrote: >> > > > +} +/* + * Create a tree containing an root inode + * + * Caller must ensure at the time of calling, fs tree only contains 2 items + * (one for INODE

Re: [PATCH] btrfs-progs: Update backup roots when writing super blocks

2019-01-02 Thread Qu Wenruo
On 2019/1/2 下午4:57, Nikolay Borisov wrote: > > > On 25.12.18 г. 7:28 ч., Qu Wenruo wrote: >> The code is mostly ported from kernel with minimal change. >> >> Since btrfs-progs doesn't support replaying log, there is some code >> unnecessary for btrfs-progs, but to keep the code the same, that

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Nikolay Borisov
On 2.01.19 г. 12:00 ч., Qu Wenruo wrote: > >>> +} >>> >>> +/* >>> + * Create a tree containing an root inode >>> + * >>> + * Caller must ensure at the time of calling, fs tree only contains 2 items >>> + * (one for INODE_ITEM and one for INODE_REF) >>> + */ >>> +int create_inode_tree(struc

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Qu Wenruo
On 2019/1/2 下午5:13, Nikolay Borisov wrote: > > > On 27.12.18 г. 9:13 ч., Qu Wenruo wrote: >> Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") >> creates uuid tree at mkfs time. >> >> However it doesn't populate uuid tree correctly nor just create an empty >> root. >> It uses

Re: [PATCH 3/3] btrfs-progs: Create uuid tree with proper contents

2019-01-02 Thread Nikolay Borisov
On 27.12.18 г. 9:13 ч., Qu Wenruo wrote: > Commit 2a496a5b8b74 ("btrfs-progs: mkfs: precreate the uuid tree") > creates uuid tree at mkfs time. > > However it doesn't populate uuid tree correctly nor just create an empty > root. > It uses create_tree(), which just copies the content of fs root,

Re: [PATCH] btrfs-progs: Update backup roots when writing super blocks

2019-01-02 Thread Nikolay Borisov
On 25.12.18 г. 7:28 ч., Qu Wenruo wrote: > The code is mostly ported from kernel with minimal change. > > Since btrfs-progs doesn't support replaying log, there is some code > unnecessary for btrfs-progs, but to keep the code the same, that > unnecessary code is kept as it. > > Now "btrfs chec