Re: [PATCH 1/4] vfs: introduce function to map unique ino/dev pairs

2018-07-31 Thread Amir Goldstein
On Wed, Aug 1, 2018 at 2:21 AM, Mark Fasheh wrote: > > On Tue, Jul 31, 2018 at 02:10:42PM -0700, Mark Fasheh wrote: >> diff --git a/fs/stat.c b/fs/stat.c >> index f8e6fb2c3657..80ea42505219 100644 >> --- a/fs/stat.c >> +++ b/fs/stat.c >> @@ -84,6 +84,29 @@ int vfs_getattr_nosec(const struct path *

Re: [PATCH 4/4] locks: map correct ino/dev pairs when exporting to userspace

2018-07-31 Thread Amir Goldstein
On Wed, Aug 1, 2018 at 12:10 AM, Mark Fasheh wrote: > /proc/locks does not always print the correct inode number/device pair. > Update lock_get_status() to use vfs_map_unique_ino_dev() to get the real, > unique values for userspace. > > Signed-off-by: Mark Fasheh > --- > fs/locks.c | 12

Re: Unmountable root partition

2018-07-31 Thread Chris Murphy
On Tue, Jul 31, 2018 at 12:03 PM, Cerem Cem ASLAN wrote: > 3. mount -t btrfs /dev/mapper/foo--vg-root /mnt/foo > Gives the following error: > > mount: wrong fs type, bad option, bad superblock on ... > > 4. dmesg | tail > Outputs the following: > > > [17755.840916] sd 3:0:0:0: [sda] tag#0 FAILED

Re: Unmountable root partition

2018-07-31 Thread Cerem Cem ASLAN
That might be the case. Can't we recover anything with some data loss? 2018-08-01 2:04 GMT+03:00 Hans van Kranenburg < hans.van.kranenb...@mendix.com>: > Hi, > > On 07/31/2018 08:03 PM, Cerem Cem ASLAN wrote: > > Hi, > > > > I'm having trouble with my server setup, which contains a BTRFS root > >

BTRFS and databases

2018-07-31 Thread MegaBrutal
Hi all, I know it's a decade-old question, but I'd like to hear your thoughts of today. By now, I became a heavy BTRFS user. Almost everywhere I use BTRFS, except in situations when it is obvious there is no benefit (e.g. /var/log, /boot). At home, all my desktop, laptop and server computers are m

[PATCH 0/7] Some unused parameter cleanup

2018-07-31 Thread Lu Fengqi
PATCH 1-2 remove all unused fs_info parameter from delayed-inode.c PATCH 3-5 remove all unused fs_info parameter from root-tree.c PATCH 6-7 some cleanup for btrfs_unlink_subvol Lu Fengqi (7): btrfs: Remove fs_info from btrfs_insert_delayed_dir_index btrfs: Remove fs_info from btrfs_delete_dela

[PATCH 4/7] btrfs: Remove fs_info from btrfs_del_root_ref

2018-07-31 Thread Lu Fengqi
It can be referenced from the passed transaction handle. Signed-off-by: Lu Fengqi --- fs/btrfs/ctree.h | 7 +++ fs/btrfs/inode.c | 8 +++- fs/btrfs/root-tree.c | 9 - 3 files changed, 10 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h inde

[PATCH 6/7] btrfs: Remove root parameter from btrfs_unlink_subvol

2018-07-31 Thread Lu Fengqi
It can be fetched by BTRFS_I(dir)->root. Signed-off-by: Lu Fengqi --- fs/btrfs/inode.c | 25 ++--- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 407d068d4208..b8c131b82978 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs

[PATCH 2/7] btrfs: Remove fs_info from btrfs_delete_delayed_dir_index

2018-07-31 Thread Lu Fengqi
It can be referenced from the passed transaction handle. Signed-off-by: Lu Fengqi --- fs/btrfs/delayed-inode.c | 6 +++--- fs/btrfs/delayed-inode.h | 1 - fs/btrfs/inode.c | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed

[PATCH 1/7] btrfs: Remove fs_info from btrfs_insert_delayed_dir_index

2018-07-31 Thread Lu Fengqi
It can be referenced from the passed transaction handle. Signed-off-by: Lu Fengqi --- fs/btrfs/delayed-inode.c | 4 +--- fs/btrfs/delayed-inode.h | 1 - fs/btrfs/dir-item.c | 4 ++-- 3 files changed, 3 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-i

[PATCH 5/7] btrfs: Remove fs_info from btrfs_add_root_ref

2018-07-31 Thread Lu Fengqi
It can be referenced from the passed transaction handle. Signed-off-by: Lu Fengqi --- fs/btrfs/ctree.h | 7 +++ fs/btrfs/inode.c | 3 +-- fs/btrfs/ioctl.c | 3 +-- fs/btrfs/root-tree.c | 9 - fs/btrfs/transaction.c | 2 +- 5 files changed, 10 insertions(+), 14 del

[PATCH 3/7] btrfs: Remove fs_info from btrfs_del_root

2018-07-31 Thread Lu Fengqi
It can be referenced from the passed transaction handle. Signed-off-by: Lu Fengqi --- fs/btrfs/ctree.h | 2 +- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/free-space-tree.c | 2 +- fs/btrfs/qgroup.c | 2 +- fs/btrfs/root-tree.c | 4 ++-- 5 files changed, 6 insertions(+),

[PATCH 7/7] btrfs: Remove redundant btrfs_release_path from btrfs_unlink_subvol

2018-07-31 Thread Lu Fengqi
Although it is safe to call this on paths that no locks or extent buffers held, removing the redundant btrfs_release_path is reasonable. Signed-off-by: Lu Fengqi --- fs/btrfs/inode.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b8c131b82978..7e56a2

Re: [PATCH v2 4/6] btrfs: Introduce mount time chunk <-> dev extent mapping check

2018-07-31 Thread Su Yue
On 08/01/2018 10:37 AM, Qu Wenruo wrote: This patch will introduce chunk <-> dev extent mapping check, to protect us against invalid dev extents or chunks. Since chunk mapping is the fundamental infrastructure of btrfs, extra check at mount time could prevent a lot of unexpected behavior (BUG

Re: [PATCH v2 6/6] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-31 Thread Su Yue
On 08/01/2018 10:37 AM, Qu Wenruo wrote: [BUG] For certains crafted image, whose csum root leaf has missing backref, if we try to trigger write with data csum, it could cause deadlock with the following kernel WARN_ON(): -- WARNING: CPU: 1 PID: 41 at fs/btrfs/locking.c:230 btrfs_tree_lock+

Re: [PATCH v2 1/6] btrfs: Check each block group has corresponding chunk at mount time

2018-07-31 Thread Su Yue
On 08/01/2018 10:37 AM, Qu Wenruo wrote: A crafted btrfs with incorrect chunk<->block group mapping, it could leads to a lot of unexpected behavior. Although the crafted image can be catched by block group item checker added in "[PATCH] btrfs: tree-checker: Verify block_group_item", if one cr

[PATCH v2 6/6] btrfs: locking: Allow btrfs_tree_lock() to return error to avoid deadlock

2018-07-31 Thread Qu Wenruo
[BUG] For certains crafted image, whose csum root leaf has missing backref, if we try to trigger write with data csum, it could cause deadlock with the following kernel WARN_ON(): -- WARNING: CPU: 1 PID: 41 at fs/btrfs/locking.c:230 btrfs_tree_lock+0x3e2/0x400 CPU: 1 PID: 41 Comm: kworker/u4:1

[PATCH v2 5/6] btrfs: Exit gracefully when failed to add chunk map

2018-07-31 Thread Qu Wenruo
It's completely possible that a crafted btrfs image contains overlapping chunks. Although we can't detect such problem by tree-checker, but it's not a catastrophic problem, current extent map can already detect such problem and return -EEXIST. We just only need to exit gracefully so btrfs can ref

[PATCH v2 4/6] btrfs: Introduce mount time chunk <-> dev extent mapping check

2018-07-31 Thread Qu Wenruo
This patch will introduce chunk <-> dev extent mapping check, to protect us against invalid dev extents or chunks. Since chunk mapping is the fundamental infrastructure of btrfs, extra check at mount time could prevent a lot of unexpected behavior (BUG_ON). Reported-by: Xu Wen Link: https://bugz

[PATCH v2 1/6] btrfs: Check each block group has corresponding chunk at mount time

2018-07-31 Thread Qu Wenruo
A crafted btrfs with incorrect chunk<->block group mapping, it could leads to a lot of unexpected behavior. Although the crafted image can be catched by block group item checker added in "[PATCH] btrfs: tree-checker: Verify block_group_item", if one crafted a valid enough block group item which ca

[PATCH v2 0/6] btrfs: Enhanced validation check for fuzzed images

2018-07-31 Thread Qu Wenruo
The branch can be fetched from the following git repo: https://github.com/adam900710/linux/tree/tree_checker_enhance It's based on v4.18-rc1, with 3 patches already merged into misc-next. This patchset introduced the following enhanced validation check: 1) chunk/block group/dev extent cross check

[PATCH v2 3/6] btrfs: Remove unused function btrfs_account_dev_extents_size()

2018-07-31 Thread Qu Wenruo
This function is never used by any one in kernel, just remove it. Signed-off-by: Qu Wenruo --- fs/btrfs/volumes.c | 85 -- fs/btrfs/volumes.h | 2 -- 2 files changed, 87 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index b33bf291

[PATCH v2 2/6] btrfs: Verify every chunk has corresponding block group at mount time

2018-07-31 Thread Qu Wenruo
If a crafted btrfs has missing block group items, it could cause unexpected behavior and breaks our expectation on 1:1 chunk<->block group mapping. Although we added block group -> chunk mapping check, we still need chunk -> block group mapping check. This patch will do extra check to ensure each

Re: Expected Received UUID

2018-07-31 Thread Hans van Kranenburg
Hi, Can you please report the kernel versions you are using on every system (uname -a)? I would indeed expect the Received UUID value for C to have the same uuid as the original UUID of A, so the b00e8ba1 one. The send part, generating the send stream is done by the running kernel. The receive p

Re: [PATCH 1/4] vfs: introduce function to map unique ino/dev pairs

2018-07-31 Thread Mark Fasheh
On Tue, Jul 31, 2018 at 02:10:42PM -0700, Mark Fasheh wrote: > diff --git a/fs/stat.c b/fs/stat.c > index f8e6fb2c3657..80ea42505219 100644 > --- a/fs/stat.c > +++ b/fs/stat.c > @@ -84,6 +84,29 @@ int vfs_getattr_nosec(const struct path *path, struct > kstat *stat, > } > EXPORT_SYMBOL(vfs_geta

Re: Unmountable root partition

2018-07-31 Thread Hans van Kranenburg
Hi, On 07/31/2018 08:03 PM, Cerem Cem ASLAN wrote: > Hi,  > > I'm having trouble with my server setup, which contains a BTRFS root > partition on top of LVM on top of LUKS partition.  > > Yesterday server was shut down unexpectedly. I booted the system with a > pendrive which contains Debian 4.9

Re: [PATCH 2/4] nfs: check for NULL vfsmount in nfs_getattr

2018-07-31 Thread Mark Fasheh
Hi Al, On Tue, Jul 31, 2018 at 11:16:05PM +0100, Al Viro wrote: > On Tue, Jul 31, 2018 at 02:10:43PM -0700, Mark Fasheh wrote: > > ->getattr from inside the kernel won't always have a vfsmount, check for > > this. > > Really? Where would that happen? It happens in my first patch. FWIW, I'm not

Re: [PATCH 2/4] nfs: check for NULL vfsmount in nfs_getattr

2018-07-31 Thread Al Viro
On Tue, Jul 31, 2018 at 02:10:43PM -0700, Mark Fasheh wrote: > ->getattr from inside the kernel won't always have a vfsmount, check for > this. Really? Where would that happen? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.ker

[PATCH 1/4] vfs: introduce function to map unique ino/dev pairs

2018-07-31 Thread Mark Fasheh
There are places in the VFS where we export an ino/dev pair to userspace. /proc/PID/maps is a good example - we directly expose inode->i_ino and inode->i_sb->s_dev to userspace there. Many filesystems don't put a unique value in inode->i_ino and instead rely on ->getattr to provide the real inode

[PATCH 2/4] nfs: check for NULL vfsmount in nfs_getattr

2018-07-31 Thread Mark Fasheh
->getattr from inside the kernel won't always have a vfsmount, check for this. Signed-off-by: Mark Fasheh --- fs/nfs/inode.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c index b65aee481d13..7a3d90a7cc92 100644 --- a/fs/nfs/inode.c +++

[PATCH 4/4] locks: map correct ino/dev pairs when exporting to userspace

2018-07-31 Thread Mark Fasheh
/proc/locks does not always print the correct inode number/device pair. Update lock_get_status() to use vfs_map_unique_ino_dev() to get the real, unique values for userspace. Signed-off-by: Mark Fasheh --- fs/locks.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git

[RFC PATCH 0/4] vfs: map unique ino/dev pairs for user space

2018-07-31 Thread Mark Fasheh
Hi, These patches are follow-up to this conversation on fsdevel which provides a good break-down of the core problem: https://www.spinics.net/lists/linux-fsdevel/msg128003.html That e-mail was in turn a follow-up to the following patch series: https://lwn.net/Articles/753917/ which tried to so

[PATCH 3/4] proc: use vfs helper to get ino/dev pairs for maps file

2018-07-31 Thread Mark Fasheh
Proc writes ino/dev into /proc/PID/maps which it gets from i_ino and s_dev. The problem with this is that i_ino and s_dev are not guaranteed to be unique - we can have duplicates in the maps file for otherwise distinct inodes. This breaks any software expecting them to be unique, including lsof(8)

Re: Expected Received UUID

2018-07-31 Thread Cerem Cem ASLAN
Not an answer, but exactly same case: https://unix.stackexchange.com/questions/377914/how-to-test-if-two-btrfs-snapshots-are-identical 2018-07-31 20:42 GMT+03:00 Gaurav Sanghavi : > Hello everyone, > > While researching BTRFS for a project that involves backing up snapshots > from device A to dev

Unmountable root partition

2018-07-31 Thread Cerem Cem ASLAN
Hi, I'm having trouble with my server setup, which contains a BTRFS root partition on top of LVM on top of LUKS partition. Yesterday server was shut down unexpectedly. I booted the system with a pendrive which contains Debian 4.9.18 and tried to mount the BTRFS root partition manually. 1. crypts

Expected Received UUID

2018-07-31 Thread Gaurav Sanghavi
Hello everyone, While researching BTRFS for a project that involves backing up snapshots from device A to device B before consolidating backups from device B to device C, I noticed that received UUID on snapshot on device C is not the same as received UUID for the same snapshot on device B. Here a

Re: [PATCH] btrfs: revert fs_devices state on error of btrfs_init_new_device()

2018-07-31 Thread Filipe Manana
On Tue, Jul 31, 2018 at 11:12 AM, Anand Jain wrote: > > > On 07/27/2018 08:04 AM, Naohiro Aota wrote: >> >> When btrfs hits error after modifying fs_devices in >> btrfs_init_new_device() (such as btrfs_add_dev_item() returns error), it >> leaves everything as is, but frees allocated btrfs_device.

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-31 Thread Filipe Manana
On Tue, Jul 31, 2018 at 11:17 AM, robbieko wrote: > Filipe Manana 於 2018-07-30 20:34 寫到: > >> On Mon, Jul 30, 2018 at 12:28 PM, Filipe Manana >> wrote: >>> >>> On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana >>> wrote: On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: > >

Re: [PATCH] Btrfs: fix data lose with snapshot when nospace

2018-07-31 Thread robbieko
Filipe Manana 於 2018-07-30 20:34 寫到: On Mon, Jul 30, 2018 at 12:28 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 12:08 PM, Filipe Manana wrote: On Mon, Jul 30, 2018 at 11:21 AM, robbieko wrote: From: Robbie Ko Commit e9894fd3e3b3 ("Btrfs: fix snapshot vs nocow writting") modified the n

Re: [PATCH] btrfs: revert fs_devices state on error of btrfs_init_new_device()

2018-07-31 Thread Anand Jain
On 07/27/2018 08:04 AM, Naohiro Aota wrote: When btrfs hits error after modifying fs_devices in btrfs_init_new_device() (such as btrfs_add_dev_item() returns error), it leaves everything as is, but frees allocated btrfs_device. As a result, fs_devices->devices and fs_devices->alloc_list contai

Re: [PATCH 1/2] btrfs: Introduce mount time chunk <-> dev extent mapping check

2018-07-31 Thread Qu Wenruo
On 2018年07月16日 21:06, David Sterba wrote: > On Mon, Jul 09, 2018 at 02:42:02PM +0800, Qu Wenruo wrote: >> This patch will introduce chunk <-> dev extent mapping check, to protect >> us against invalid dev extents or chunks. >> >> Since chunk mapping is the fundamental infrastructure of btrfs, ext

[PATCH] btrfs: replace: Reset on-disk dev stats value after replace

2018-07-31 Thread Misono Tomohiro
on-disk devs stats value is updated in btrfs_run_dev_stats(), which is called during commit transaction, if device->dev_stats_ccnt is not zero. Since current replace operation does not touch dev_stats_ccnt, on-disk dev stats value is not updated. Therefore "btrfs device stats" may return old devic