[PATCH] btrfs-progs: Add a basic monitoring script.

2018-03-23 Thread Austin S. Hemmelgarn
This adds a minimalistic Python 3 based monitoring script to the standard tools called `btrfs-status`. The script itself depends only on a working Python 3 installation, the `btrfs` command, and a regular `blkid` command (the version of `blkid` in Busybox may not work, currently untested). When c

[PATCH 1/2] btrfs-progs: Fix DIR_ITEM checking in lowmem

2018-03-23 Thread Nikolay Borisov
When checking the validity of a DIR_ITEM item the index variable is explicitly set to -1 so that the index check in find_inode_ref() is ignored. This is necessary due to possible name collisions in DIR_ITEMS (i.e multiple files with the same crc32c for their names, resulting in the identical key->o

[PATCH 2/2] btrfs-progs: Add test for collision DIR_ITEM handling

2018-03-23 Thread Nikolay Borisov
Verify that if we have an otherwise clean filesystem, containging collided DIR_ITEM, btrfs check lowmem's mode can correctly handle those and not produce any false positives. This if fixed by commit titled: "btrfs-progs: Fix DIR_ITEM checking in lowmem" Signed-off-by: Nikolay Borisov --- .

Re: [PATCH v2] btrfs: Validate child tree block's level and first key

2018-03-23 Thread Jeff Mahoney
On 3/23/18 7:50 AM, Qu Wenruo wrote: > > > On 2018年03月23日 19:14, Nikolay Borisov wrote: >> >> >> On 23.03.2018 03:31, Qu Wenruo wrote: >>> We have several reports about node pointer points to incorrect child >>> tree blocks, which could have even wrong owner and level but still with >>> valid gen

[PATCH 1/2] btrfs: verify checksum when superblock is read for mount

2018-03-23 Thread Anand Jain
During mount context, we aren't verifying the superblock checksum for all the devices, instead, we verify it only for the struct btrfs_fs_device::latest_bdev. This patch fixes it by moving the checksum verification code from the function open_ctree() into the function btrfs_read_dev_one_super(). B

[PATCH 0/2] verify superblock checksum

2018-03-23 Thread Anand Jain
Here are the threads/context [1] in which we read the superblock(s). And this patchset will make sure the superblock csum are checked when they are read in the respective context as show below [1]. This patchset is on top of [PATCH v2.1] btrfs: check if the fsid in the primary sb and copy sb are

[PATCH 2/2] btrfs: verify checksum when superblock is read for scan

2018-03-23 Thread Anand Jain
During the scan context, we aren't verifying if the superblock- checksum is correct for the primary and its copies. This patch fixes it by adding the checksum verification function btrfs_check_super_csum() in the function btrfs_read_disk_super(). It would fail the scan only if the primary SB csum

[PATCH v2.1] btrfs: check if the fsid in the primary sb and copy sb are same

2018-03-23 Thread Anand Jain
During the btrfs dev scan make sure that other copies of superblock contain the same fsid as the primary SB. So that we bring to the user notice if the superblock has been overwritten. mkfs.btrfs -fq /dev/sdc mkfs.btrfs -fq /dev/sdb dd if=/dev/sdb of=/dev/sdc count=4K skip=64K seek=64K obs=1 ib

Re: Status of host managed SMR device support

2018-03-23 Thread David Sterba
On Thu, Mar 22, 2018 at 10:41:34PM +, Mason Fan wrote: > I read from a SNIA's slide that btrfs will support host managed SMR > device natively, and also saw it in "Features Currently in Development > or Planned for Future Implementation" on the wiki. Does anyone know > any further information?

Re: [PATCH v2] btrfs: Validate child tree block's level and first key

2018-03-23 Thread Qu Wenruo
On 2018年03月23日 19:14, Nikolay Borisov wrote: > > > On 23.03.2018 03:31, Qu Wenruo wrote: >> We have several reports about node pointer points to incorrect child >> tree blocks, which could have even wrong owner and level but still with >> valid generation and checksum. >> >> Although btrfs chec

Re: Hard reset during balance - How to continue?

2018-03-23 Thread Qu Wenruo
On 2018年03月23日 19:04, james harvey wrote: > If a system unexpectedly reboots during a balance, what's the best next step? Normally, nothing. Btrfs CoW is pretty safe for most case. > > From an ISO, read-only operations/mounting looks fine. > > Didn't want to make any writes until I ask, becau

Re: [PATCH v2] btrfs: Validate child tree block's level and first key

2018-03-23 Thread Nikolay Borisov
On 23.03.2018 03:31, Qu Wenruo wrote: > We have several reports about node pointer points to incorrect child > tree blocks, which could have even wrong owner and level but still with > valid generation and checksum. > > Although btrfs check could handle it and print error message like: > leaf pa

Hard reset during balance - How to continue?

2018-03-23 Thread james harvey
If a system unexpectedly reboots during a balance, what's the best next step? >From an ISO, read-only operations/mounting looks fine. Didn't want to make any writes until I ask, because I see some reports of bad things happening with a reboot during a balance, and don't want to mis-step. btrfs c

Re: [PATCH v2] btrfs: check if the fsid in the primary sb and copy sb are same

2018-03-23 Thread Anand Jain
Pls, ignore this. I need to handle the mapped page in a better way. Thanks, Anand On 03/22/2018 09:01 PM, Anand Jain wrote: During the btrfs dev scan make sure that other copies of superblock contain the same fsid as the primary SB. So that we bring to the users notice if the superblock has

Re: [PATCH] btrfs-progs: remove BTRFS_CRC32_SIZE definition

2018-03-23 Thread Qu Wenruo
On 2018年03月23日 16:20, Misono Tomohiro wrote: > The kernel code no longer has BTRFS_CRC32_SIZE and only uses > btrfs_csum_sizes[]. So, update the progs code as well. > > Suggested-by: Qu Wenruo > Signed-off-by: Tomohiro Misono > --- > convert/common.c | 2 +- > convert/main.c | 2 +- > ctr

Re: [PATCH] btrfs: Allow rmdir(2) to delete a subvolume

2018-03-23 Thread Nikolay Borisov
On 23.03.2018 10:28, Misono Tomohiro wrote: > This patch changes the behavior of rmdir(2) to allow it to delete > an empty subvolume. > > In order to do that the core logic of subvolume deletion is moved from > ioctl.c to inode.c and named as btrfs_delete_subvolume(), which is > also called in t

[PATCH] btrfs: Allow rmdir(2) to delete a subvolume

2018-03-23 Thread Misono Tomohiro
This patch changes the behavior of rmdir(2) to allow it to delete an empty subvolume. In order to do that the core logic of subvolume deletion is moved from ioctl.c to inode.c and named as btrfs_delete_subvolume(), which is also called in the btrfs_rmdir() if a directory is an empty subvolume. No

[PATCH] btrfs-progs: remove BTRFS_CRC32_SIZE definition

2018-03-23 Thread Misono Tomohiro
The kernel code no longer has BTRFS_CRC32_SIZE and only uses btrfs_csum_sizes[]. So, update the progs code as well. Suggested-by: Qu Wenruo Signed-off-by: Tomohiro Misono --- convert/common.c | 2 +- convert/main.c | 2 +- ctree.h | 3 +-- image/main.c | 4 ++-- mkfs/common.c

Re: [PATCH] btrfs: ctree.h: Fix wrong comment position about csum size

2018-03-23 Thread Misono Tomohiro
On 2018/03/20 17:50, Qu Wenruo wrote: > > > On 2018年03月20日 14:47, Misono, Tomohiro wrote: >> >> Signed-off-by: Tomohiro Misono > > Reviewed-by: Qu Wenruo > > BTW this reminds me that, btrfs-progs is still using BTRFS_CRC32_SIZE > macro which the original comment is for. > > It may be a good

[PATCH v2] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of, FS_TREE

2018-03-23 Thread Misono Tomohiro
Currently, the top-level subvolume lacks the UUID. As a result, both non-snapshot subvolume and snapshot of top-level subvolume do not have Parent UUID and cannot be distinguisued. Therefore "fi show" of top-level lists all the subvolumes which lacks the UUID in "Snapshot(s)" filed. Also, it lacks

Re: [PATCH] btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of FS_TREE

2018-03-23 Thread Misono Tomohiro
On 2018/03/22 2:48, David Sterba wrote: > On Mon, Mar 19, 2018 at 05:16:42PM +0900, Misono, Tomohiro wrote: >> Currently, the top-level subvolume lacks the UUID. As a result, both >> non-snapshot subvolume and snapshot of top-level subvolume do not have >> Parent UUID and cannot be distinguisued. T