Re: Problems with btrfs

On Fri, Apr 27, 2018 at 6:20 PM, Qu Wenruo wrote: > > > On 2018年04月28日 02:38, David C. Partridge wrote: >> I'm running Ubuntu 16.04. I rebooted my server today as it wasn't >> responding. >> >> When I rebooted the root FS was read only. >> >> I booted a live Ubuntu CD and

Re: [PATCH 2/2] btrfs: add balance args info during start and resume

On 04/27/2018 02:46 PM, Nikolay Borisov wrote: On 26.04.2018 11:01, Anand Jain wrote: Balance args info is an important information to be reviewed on the system under audit. So this patch adds that. Would have been good to add example output to the change log. Here are the samples (will

Re: [PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume

On Fri, Apr 27, 2018 at 06:26:35PM +0200, David Sterba wrote: > On Fri, Apr 27, 2018 at 05:02:45PM +0900, Misono Tomohiro wrote: > > Add btrfs test that checks "rmdir" or "rm -r" command can delete a > > subvolume like an ordinary drectory. > > > > This behavior has been restricted long time but

Re: NVMe SSD + compression - benchmarking

On 2018年04月28日 01:41, Brendan Hide wrote: > Hey, all > > I'm following up on the queries I had last week since I have installed > the NVMe SSD into the PCI-e adapter. I'm having difficulty knowing > whether or not I'm doing these benchmarks correctly. > > As a first test, I put together a

Re: Problems with btrfs

On 2018年04月28日 02:38, David C. Partridge wrote: > I'm running Ubuntu 16.04. I rebooted my server today as it wasn't > responding. > > When I rebooted the root FS was read only. > > I booted a live Ubuntu CD and checked the drive with the results shown in > attachment btrfs-check.log. > > The

Re: Inconsistent behavior of fsync in btrfs

Thanks Chris and Ted for putting down the expected fsync behaviour of btrfs and ext4 clearly. This sort of information is not documented anywhere; it would be really useful if all major filesystems explicitly stated what fsync behavior to expect. Filesystems definitely seem to provide more

Re: Inconsistent behavior of fsync in btrfs

On Fri, Apr 27, 2018 at 2:53 PM, Theodore Y. Ts'o wrote: > It seems to me that it would be desirable if all of the major file > systems have roughly the same minimum guarantee for fsync(2), so that > application writers don't have to make file-system specific > assumptions. In

Re: Inconsistent behavior of fsync in btrfs

On Fri, Apr 27, 2018 at 11:33:29AM -0600, Chris Mason wrote: > My goal for the fsync tree log was to make it just do the right thing most > of the time. We mostly got there, thanks to a ton of fixes and test cases > from Filipe. > > fsync(some file) -- all the names for this file will exist,

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On 4/27/18 12:40 PM, David Sterba wrote: > On Fri, Apr 27, 2018 at 12:02:13PM -0400, Jeff Mahoney wrote: +static void queue_rescan_worker(struct btrfs_fs_info *fs_info) +{ >>> >>> And this had to be moved upwards as there was earlier use of >>> btrfs_queue_work that matched following the

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On Fri, Apr 27, 2018 at 11:56 AM, David Sterba wrote: > On Thu, Apr 26, 2018 at 03:23:49PM -0400, je...@suse.com wrote: >> From: Jeff Mahoney >> >> Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) >> fixed the issue with

Re: [PATCH 3/3] btrfs-progs: build: use m4_flatten instead of m4_chomp

On 4/27/18 2:56 PM, je...@suse.com wrote: > From: Jeff Mahoney > > Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking) > started m4_chomp to strip the newlines from the version file. m4_chomp > was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63. >

[PATCH 3/3] btrfs-progs: build: use m4_flatten instead of m4_chomp

From: Jeff Mahoney Commit 2e1932e6a38 (btrfs-progs: build: simplify version tracking) started m4_chomp to strip the newlines from the version file. m4_chomp was introduced in autoconf 2.64 but SLE11 ships with autoconf 2.63. For purposes of just stripping the newline, m4_flatten

[PATCH 2/3] btrfs-progs: build: detect whether -std=gnu90 is supported

From: Jeff Mahoney Older versions of gcc don't support -std=gnu90 so btrfs-progs won't build at all on older distros. We can detect whether the compiler supports -std=gnu90 and fall back to -std=gnu89 if it doesn't. Signed-off-by: Jeff Mahoney --- Makefile

[PATCH 1/3] btrfs-progs: convert: fix support for e2fsprogs < 1.42

From: Jeff Mahoney Commit 324d4c1857a (btrfs-progs: convert: Add larger device support) introduced new dependencies on the 64-bit API provided by e2fsprogs. That API was introduced in v1.42 (along with bigalloc). This patch maps the following to their equivalents in e2fsprogs <

[PATCH 0/3] btrfs-progs: build on SLE11

From: Jeff Mahoney This patch set allows btrfs-progs to build without further patching on a stock SLE11 SP4 install. The major issues were: - new dependencies on e2fsprogs 1.42 - use of -std=gnu90, which gcc 4.3.4 doesn't support - use of m4_chomp, introduced in autoconf 2.64.

Problems with btrfs

I'm running Ubuntu 16.04. I rebooted my server today as it wasn't responding. When I rebooted the root FS was read only. I booted a live Ubuntu CD and checked the drive with the results shown in attachment btrfs-check.log. The error was still there after completing the btrfs check --repair :(

NVMe SSD + compression - benchmarking

Hey, all I'm following up on the queries I had last week since I have installed the NVMe SSD into the PCI-e adapter. I'm having difficulty knowing whether or not I'm doing these benchmarks correctly. As a first test, I put together a 4.7GB .tar containing mostly duplicated copies of the

Re: Inconsistent behavior of fsync in btrfs

On 27 Apr 2018, at 10:07, David Sterba wrote: On Thu, Apr 26, 2018 at 07:59:23PM -0500, Jayashree Mohan wrote: Thanks for the response. We are using a tool we developed called CrashMonkey[1] to run crash consistency tests and generate the bug reports above. We'd be happy to guide you through

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On Fri, Apr 27, 2018 at 12:02:13PM -0400, Jeff Mahoney wrote: > >> +static void queue_rescan_worker(struct btrfs_fs_info *fs_info) > >> +{ > > > > And this had to be moved upwards as there was earlier use of > > btrfs_queue_work that matched following the hunk. > > Weird. That must be exactly

Re: [PATCH v2] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

On Fri, Apr 27, 2018 at 12:11:00PM -0400, Jeff Mahoney wrote: > If we fail to allocate memory for a path, don't bother trying to > insert the qgroup status item. We haven't done anything yet and it'll > fail also. Just print an error and be done with it. > > Signed-off-by: Jeff Mahoney

Re: [PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume

On Fri, Apr 27, 2018 at 05:02:45PM +0900, Misono Tomohiro wrote: > Add btrfs test that checks "rmdir" or "rm -r" command can delete a > subvolume like an ordinary drectory. > > This behavior has been restricted long time but becomes allowed by > following patch in the kernel: > btrfs: Allow

Re: [PATCH v2 12/16] btrfs: track running balance in a simpler way

On Fri, Apr 27, 2018 at 10:10:24AM +0800, Anand Jain wrote: > > > On 04/20/2018 12:33 AM, David Sterba wrote: > > Currently fs_info::balance_running is 0 or 1 and does not use the > > semantics of atomics. The pause and cancel check for 0, that can happen > > only after __btrfs_balance exits for

[PATCH v2] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

If we fail to allocate memory for a path, don't bother trying to insert the qgroup status item. We haven't done anything yet and it'll fail also. Just print an error and be done with it. Signed-off-by: Jeff Mahoney --- fs/btrfs/qgroup.c | 9 - 1 file changed, 4

Re: Inconsistent behavior of fsync in btrfs

On Thu, Apr 26, 2018 at 07:59:23PM -0500, Jayashree Mohan wrote: > Thanks for the response. We are using a tool we developed called > CrashMonkey[1] to run crash consistency tests and generate the bug > reports above. We'd be happy to guide you through setting up > CrashMonkey and getting these

Re: [PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

On 4/27/18 11:44 AM, David Sterba wrote: > On Thu, Apr 26, 2018 at 11:39:50PM +0300, Nikolay Borisov wrote: >> On 26.04.2018 22:23, je...@suse.com wrote: >>> From: Jeff Mahoney >>> >>> If we fail to allocate memory for a path, don't bother trying to >>> insert the qgroup status

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On 4/27/18 11:56 AM, David Sterba wrote: > On Thu, Apr 26, 2018 at 03:23:49PM -0400, je...@suse.com wrote: >> From: Jeff Mahoney >> >> Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) >> fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but >> ended

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On 4/27/18 4:48 AM, Filipe Manana wrote: > On Thu, Apr 26, 2018 at 8:23 PM, wrote: >> From: Jeff Mahoney >> >> Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) >> fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but >> ended up

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On Thu, Apr 26, 2018 at 03:23:49PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) > fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but > ended up reintroducing the hang-on-unmount bug that

Re: [PATCH 3/3] btrfs: qgroup, don't try to insert status item after ENOMEM in rescan worker

On Thu, Apr 26, 2018 at 11:39:50PM +0300, Nikolay Borisov wrote: > On 26.04.2018 22:23, je...@suse.com wrote: > > From: Jeff Mahoney > > > > If we fail to allocate memory for a path, don't bother trying to > > insert the qgroup status item. We haven't done anything yet and it'll

Re: Inconsistent behavior of fsync in btrfs

On 26 Apr 2018, at 18:59, Jayashree Mohan wrote: Hi Chris, Thanks for the response. We are using a tool we developed called CrashMonkey[1] to run crash consistency tests and generate the bug reports above. We'd be happy to guide you through setting up CrashMonkey and getting these bugs

Optimal maintenance for RAID5 array

Hi all I am running a RAID5 array built on 5x8TB HD. The filesystem usage is aproximatively 6TB now I rung kernel 4.16.5 and btrfs progs 4.16 (planning to upgrade to 4.16.1) under Ubuntu xenial I am not sure what is the best/safest way to maintain the array, in particular which is the best

[PATCH v2] btrfs: Unexport and rename btrfs_invalidate_inodes

This function is no longer used outside of inode.c so just make it static. At the same time give a more becoming name, since it's not really invalidating the inodes but just calling d_prune_alias. Last, but not least - move the function above the sole caller to avoid introducing

Re: [PATCH v2] Btrfs: set keep_lock when necessary in btrfs_defrag_leaves

On Fri, Apr 27, 2018 at 11:23:23AM +0800, Liu Bo wrote: > >> --- > >> v2: update commit log with more details. > >> > >> fs/btrfs/tree-defrag.c | 3 +-- > >> 1 file changed, 1 insertion(+), 2 deletions(-) > >> > >> diff --git a/fs/btrfs/tree-defrag.c b/fs/btrfs/tree-defrag.c > >> index

Re: [PATCH] btrfs: Take trans lock before access running trans in check_delayed_ref

On 27.04.2018 12:58, ethanwu wrote: > In preivous patch: > Btrfs: kill trans in run_delalloc_nocow and btrfs_cross_ref_exist > We avoid starting btrfs transaction and get this information from > fs_info->running_transaction directly. > > When accessing running_transaction in check_delayed_ref,

[PATCH] btrfs: Take trans lock before access running trans in check_delayed_ref

In preivous patch: Btrfs: kill trans in run_delalloc_nocow and btrfs_cross_ref_exist We avoid starting btrfs transaction and get this information from fs_info->running_transaction directly. When accessing running_transaction in check_delayed_ref, there's a chance that current transaction will be

[PATCH 2/5] btrfs: Split btrfs_del_delalloc_inode into 2 functions

This is in preparation of fixing delalloc inodes leakage on transaction abort. Also export the new function. Signed-off-by: Nikolay Borisov --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/inode.c | 15 --- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git

[PATCH 3/5] btrfs: Add assert in __btrfs_del_delalloc_inode

The invariant is that when nr_delalloc_inodes is 0 then the root mustn't have any inodes on its delalloc inodes list. Signed-off-by: Nikolay Borisov --- fs/btrfs/inode.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index

[PATCH 0/5] Fix delalloc inodes leaking on btrfs unmount

After investigating crashes on generic/176 it turned that the culprit in fact is the random failure induced by generic/019. As it happens, if on unmount the filesystem is in BTRFS_FS_STATE_ERROR then btrfs_error_commit_super is called. This unveiled 2 bugs: 1. btrfs_destroy_delalloc_inodes's

[PATCH 1/5] btrfs: Unexport btrfs_alloc_delalloc_work

It's used only in inode.c so makes no sense to have it exported. Also move the definition of btrfs_delalloc_work to inode.c since it's used only this file. Signed-off-by: Nikolay Borisov --- fs/btrfs/ctree.h | 9 - fs/btrfs/inode.c | 8 2 files changed, 8

[PATCH 5/5] btrfs: Unexport and rename btrfs_invalidate_inodes

This function is no longer used outside of inode.c so just make it static. At the same time give a more becoming name, since it's not really invalidating the inodes but just calling d_prune_alias. Last, but not least - move the function above the sole caller to avoid introducing

[PATCH 4/5] btrfs: Fix delalloc inodes invalidation during transaction abort

When a transaction is aborted btrfs_cleanup_transaction is called to cleanup all the various in-flight bits and pieces which migth be active. One of those is delalloc inodes - inodes which have dirty pages which haven't been persisted yet. Currently the process of freeing such delalloc inodes in

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On Thu, Apr 26, 2018 at 8:23 PM, wrote: > From: Jeff Mahoney > > Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) > fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but > ended up reintroducing the hang-on-unmount bug that the

Re: [PATCH 1/3] btrfs: qgroups, fix rescan worker running races

On 26.04.2018 22:23, je...@suse.com wrote: > From: Jeff Mahoney > > Commit d2c609b834d6 (Btrfs: fix qgroup rescan worker initialization) > fixed the issue with BTRFS_IOC_QUOTA_RESCAN_WAIT being racy, but > ended up reintroducing the hang-on-unmount bug that the commit it >

Re: [PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume

On Fri, Apr 27, 2018 at 05:02:45PM +0900, Misono Tomohiro wrote: > Add btrfs test that checks "rmdir" or "rm -r" command can delete a > subvolume like an ordinary drectory. > > This behavior has been restricted long time but becomes allowed by > following patch in the kernel: > btrfs: Allow

Re: [PATCH] btrfs: modify path->reada in btrfs_shrink_device to READA_BACK

On 27.04.2018 11:22, Gu Jinxiang wrote: > In btrfs_shrink_device, before btrfs_search_slot, path->reada is > set to READA_FORWARD. But I think READA_BACK is correct. > Since, > 1.key.offset is set to (u64)-1 > 2.After btrfs_search_slot, btrfs_previous_item is called. > So, for readahead

[PATCH] btrfs: modify path->reada in btrfs_shrink_device to READA_BACK

In btrfs_shrink_device, before btrfs_search_slot, path->reada is set to READA_FORWARD. But I think READA_BACK is correct. Since, 1.key.offset is set to (u64)-1 2.After btrfs_search_slot, btrfs_previous_item is called. So, for readahead previous items, READA_BACK is correct one. Signed-off-by:

[PATCH v2] btrfs: Add test that checks rmdir(2) can delete a subvolume

Add btrfs test that checks "rmdir" or "rm -r" command can delete a subvolume like an ordinary drectory. This behavior has been restricted long time but becomes allowed by following patch in the kernel: btrfs: Allow rmdir(2) to delete an empty subvolume Signed-off-by: Tomohiro Misono

Re: [PATCH] btrfs: Add test that checks rmdir(2) can delete a subvolume

On 2018/04/27 13:40, Eryu Guan wrote: > On Thu, Apr 19, 2018 at 04:23:35PM +0900, Misono Tomohiro wrote: >> Add btrfs test that checks "rmdir" or "rm -r" command can delete a >> subvolume like an ordinary drectory. >> >> This behavior has been restricted long time but becomes allowed by >>

[PATCH] btrfs: balance dirty metadata pages in btrfs_finish_ordered_io

We should balance dirty metadata pages at the end of btrfs_finish_ordered_io, since a small, unmergeable random write can potentially produce dirty metadata which is multiple times larger than the data itself. For example, a small, unmergeable 4KiB write may produce: 16KiB dirty leaf (and

Re: [PATCH 2/2] btrfs: add balance args info during start and resume

On 26.04.2018 11:01, Anand Jain wrote: > Balance args info is an important information to be reviewed on the > system under audit. So this patch adds that. Would have been good to add example output to the change log. > > Signed-off-by: Anand Jain > --- >