Re: kernel btrfs file system wedged -- is it toast?

Regarding CPU vs disk hanging ... fifteen minutes ago, after doing a "mount -oclear_cache", then a "mount -ospace_cache", to clear some space cache errors reported by btrfs check, I changed a shell's working directory into the top directory of the freshly remounted btrfs file system, and then

Re: kernel btrfs file system wedged -- is it toast?

That shell is still hung in the kernel, on a "disk sleep" fifteen minutes later, and one of my CPU cores is pegged at 100%, and the disk activity light is not coming on at all. .. This is not disk bound activity ... it's kernel code bound. That's insane ... and sad. 'perf top' is my

[PATCH] btrfs: round down size diff when shrinking/growing device

Further testing showed that the fix introduced in 7dfb8be11b5d ("btrfs: Round down values which are written for total_bytes_size") was insufficient and it could still lead to discrepancies between the total_bytes in the super block and the device total bytes. So this patch also ensures that the

Re: [PATCH] btrfs: test if receive with qgroups corrupts metadata

On Thu, Jul 20, 2017 at 04:03:33PM -0700, Justin Maggard wrote: > This test case does some concurrent send/receives with qgroups enabled. > Currently (4.13-rc1) this usually results in btrfs check errors, and > often also results in a WARN_ON in record_root_in_trans(). > > Bisecting points to

Re: [PATCH RESEND] Btrfs: fix early ENOSPC due to delalloc

On Thu, Jul 20, 2017 at 03:10:35PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > If a lot of metadata is reserved for outstanding delayed allocations, we > rely on shrink_delalloc() to reclaim metadata space in order to fulfill > reservation tickets. However,

Re: [PATCH 2/3] btrfs: fix readdir deadlock with pagefault

On Fri, Jul 21, 2017 at 01:29:08PM -0400, jo...@toxicpanda.com wrote: > From: Josef Bacik > > Readdir does dir_emit while under the btree lock. dir_emit can trigger > the page fault which means we can deadlock. Fix this by allocating a > buffer on opening a directory and copying

Re: [PATCH 2/4] btrfs: separate defrag and property compression

On Thu, Jul 20, 2017 at 06:49:19PM +0800, Anand Jain wrote: > > > On 07/18/2017 02:46 AM, David Sterba wrote: > > Add new value for compression to distinguish between defrag and > > property. Previously, a single variable was used and this caused clashes > > when the per-file 'compression' was

Re: [PATCH] btrfs: make use of inode_need_compress()

On Thu, Jul 20, 2017 at 10:22:13AM +0800, Anand Jain wrote: > >> @@ -1189,11 +1189,10 @@ static int cow_file_range_async(struct inode > >> *inode, struct page *locked_page, > >>async_cow->locked_page = locked_page; > >>async_cow->start = start; > >> > >> - if

Re: [PATCH v3] Btrfs: add skeleton code for compression heuristic

On Fri, 21 Jul 2017 13:00:56 +0800 Anand Jain wrote: > > > On 07/18/2017 02:30 AM, David Sterba wrote: > > So it basically looks good, I could not resist and rewrote the changelog > > and comments. There's one code fix: > > > > On Mon, Jul 17, 2017 at 04:52:58PM +0300,

Re: kernel btrfs file system wedged -- is it toast?

My btrfs file system, after doing a "mount -oclear_cache", followed by a "mount -ospace_cache", eleven hours ago now, is still hung. David Goodwin suggested: >> 'perf top' is my first thought it might at least highlight the area >> gobbling up cpu time. Thanks for suggesting that. It has

[PATCH 1/3] btrfs: don't allow trans ioctl on a directory

From: Josef Bacik We need to use file->private_data for readdir on directories, so just don't allow user space transactions on directories. Signed-off-by: Josef Bacik --- fs/btrfs/ioctl.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/ioctl.c

[PATCH 2/3] btrfs: fix readdir deadlock with pagefault

From: Josef Bacik Readdir does dir_emit while under the btree lock. dir_emit can trigger the page fault which means we can deadlock. Fix this by allocating a buffer on opening a directory and copying the readdir into this buffer and doing dir_emit from outside of the tree lock.

[PATCH 3/3] btrfs: increase ctx->pos for delayed dir index

From: Josef Bacik Our dir_context->pos is supposed to hold the next position we're supposed to look. If we successfully insert a delayed dir index we could end up with a duplicate entry because we don't increase ctx->pos after doing the dir_emit. Signed-off-by: Josef Bacik

Re: [PATCH v3 0/4] Add xxhash and zstd modules

On 2017-07-21 07:16, Austin S. Hemmelgarn wrote: On 2017-07-20 17:27, Nick Terrell wrote: Well this is embarrassing, forgot to type anything before hitting send... Hi all, This patch set adds xxhash, zstd compression, and zstd decompression modules. It also adds zstd support to BtrFS and

[PATCH] Btrfs: Do not use data_alloc_cluster in ssd mode

In the first year of btrfs development, around early 2008, btrfs gained a mount option which enables specific functionality for filesystems on solid state devices. The first occurance of this functionality is in commit e18e4809, labeled "Add mount -o ssd, which includes optimizations for seek

Re: [PATCH] btrfs-progs: Use '-t btrfs' mount option in tests.

On Thu, Jul 20, 2017 at 05:42:46PM +0200, David Sterba wrote: > On Thu, Jul 20, 2017 at 05:40:00PM +0800, Qu Wenruo wrote: > > > > > > On 2017年07月13日 05:52, David Sterba wrote: > > > On Wed, Jul 12, 2017 at 01:05:22PM -0700, Adam Buchbinder wrote: > > >> Without it, mount (at least from

Re: kernel btrfs file system wedged -- is it toast?

On 21/07/17 07:06, Paul Jackson wrote: > What in god green's earth can kernel file system code be > doing that takes fifteen minutes (so far, in this case) or > fifty minutes (in the case I first reported on this thread? I find that just doing a balance on a disk with lots of snapshots can cause

Re: [PATCH RESEND] Btrfs: fix early ENOSPC due to delalloc

On Thu, Jul 20, 2017 at 03:10:35PM -0700, Omar Sandoval wrote: > If a lot of metadata is reserved for outstanding delayed allocations, we > rely on shrink_delalloc() to reclaim metadata space in order to fulfill > reservation tickets. However, shrink_delalloc() has a shortcut where if > it

Re: [PATCH v3] Btrfs: add skeleton code for compression heuristic

On Fri, Jul 21, 2017 at 11:37:49PM +0500, Roman Mamedov wrote: > On Fri, 21 Jul 2017 13:00:56 +0800 > Anand Jain wrote: > > On 07/18/2017 02:30 AM, David Sterba wrote: > > > This must stay 'return 1', if force-compress is on, so the change is > > > reverted. > > > >

Re: kernel btrfs file system wedged -- is it toast?

Hey, On 07/21/2017 08:53 PM, Paul Jackson wrote: > > My btrfs file system, after doing a "mount -oclear_cache", followed > by a "mount -ospace_cache", eleven hours ago now, is still hung. > > David Goodwin suggested: >>> 'perf top' is my first thought it might at least highlight the area

Re: btrfs device ready purpose

On Mon, Jul 17, 2017 at 9:54 AM, David Sterba wrote: > On Fri, Jul 07, 2017 at 11:40:04AM -0600, Chris Murphy wrote: >> On Fri, Jul 7, 2017 at 10:59 AM, Andrei Borzenkov >> wrote: >> > 07.07.2017 19:42, Chris Murphy пишет: >> >> I'm digging through piles of

Re: [PATCH] Btrfs: Do not use data_alloc_cluster in ssd mode

On Fri, Jul 21, 2017 at 01:47:11PM +0200, Hans van Kranenburg wrote: > The changes in here do the following: > > 1. Throw out the current ssd_spread behaviour. > 2. Move the current ssd behaviour to the ssd_spread option. > 3. Make ssd mode data allocation identical to tetris mode, like nossd. >

Re: kernel btrfs file system wedged -- is it toast?

Hans wrote: >> Be aware that the attitude displayed in your ramblings is not helping >> the situation. Yes - this is true. I agree that my last post was burning some bridges. You were quite helpful in your earlier reply - thank-you. I stand by my protest. -- Paul Jackson

Re: [PATCH v3] Btrfs: add skeleton code for compression heuristic

On 07/22/2017 05:00 AM, Adam Borowski wrote: On Fri, Jul 21, 2017 at 11:37:49PM +0500, Roman Mamedov wrote: On Fri, 21 Jul 2017 13:00:56 +0800 Anand Jain wrote: On 07/18/2017 02:30 AM, David Sterba wrote: This must stay 'return 1', if force-compress is on, so the

Re: kernel btrfs file system wedged -- is it toast?

> [ ... ] announce loudly and clearly to any potential users, in > multiple places (perhaps a key announcement in a few places > and links to that announcement from many places, https://btrfs.wiki.kernel.org/index.php/Gotchas#Having_many_subvolumes_can_be_very_slow > ... DO expect to first have

Re: [PATCH] Btrfs: Do not use data_alloc_cluster in ssd mode

On 07/21/2017 05:50 PM, Austin S. Hemmelgarn wrote: > On 2017-07-21 07:47, Hans van Kranenburg wrote: >> [...] >> >> Signed-off-by: Hans van Kranenburg > Behaves as advertised, and I'm not seeing any issues in my testing (and > I can confirm from experience that

python-btrfs v8

Last week, I tagged v8 of the python btrfs library: https://github.com/knorrie/python-btrfs Here's the summary of changes: python-btrfs v8, Jul 18 2017 * Easier handling of multiple appended data structures of the dir item and inode ref families easier. (not backwards compatible!) *

Re: kernel btrfs file system wedged -- is it toast?

On Fri, Jul 21, 2017 at 12:53 PM, Paul Jackson wrote: > > My btrfs file system, after doing a "mount -oclear_cache", followed > by a "mount -ospace_cache", eleven hours ago now, is still hung. > > David Goodwin suggested: >>> 'perf top' is my first thought it might at least

Re: btrfs device ready purpose

21.07.2017 17:36, Chris Murphy пишет: >> >> The command is just a simple wrapper around the DEVICES_READY ioctl, but >> now that systemd has it's own wrapper tool, there are probably no users >> of that subcommand in 'btrfs' tool itself. We can enhance the >> documentation to state the expected

Re: [PATCH v2] btrfs: Remove unused parameters from volume.c functions

On Wed, Jul 19, 2017 at 10:48:42AM +0300, Nikolay Borisov wrote: > This also adjusts the respective callers in other files. Those were found with > -Wunused-parameter. > > btrfs_full_stripe_len's mapping_tree - introduced by > 53b381b3abeb ("Btrfs: RAID5 and RAID6") but it was never really used

Re: [PATCH] Btrfs: Do not use data_alloc_cluster in ssd mode

On 2017-07-21 07:47, Hans van Kranenburg wrote: In the first year of btrfs development, around early 2008, btrfs gained a mount option which enables specific functionality for filesystems on solid state devices. The first occurance of this functionality is in commit e18e4809, labeled "Add

Re: [PATCH v3 0/4] Add xxhash and zstd modules

On 2017-07-20 17:27, Nick Terrell wrote: Hi all, This patch set adds xxhash, zstd compression, and zstd decompression modules. It also adds zstd support to BtrFS and SquashFS. Each patch has relevant summaries, benchmarks, and tests. Best, Nick Terrell For patches 2-3, I've compile tested

Re: [PATCH v2] btrfs: Remove unused variables

On Wed, Jul 19, 2017 at 10:47:57AM +0300, Nikolay Borisov wrote: > clear_super - usage was removed in commit > cea67ab92d3d ("btrfs: clean the old superblocks before freeing the device") > but > that change forgot to remove the actual variable. > > max_key - commit 6174d3cb43aa ("Btrfs: remove

Re: [PATCH v3 0/4] Add xxhash and zstd modules

On 2017-07-20 17:27, Nick Terrell wrote: Hi all, This patch set adds xxhash, zstd compression, and zstd decompression modules. It also adds zstd support to BtrFS and SquashFS. Each patch has relevant summaries, benchmarks, and tests. Best, Nick Terrell Changelog: v1 -> v2: - Make pointer in

Re: [PATCH] Btrfs: Do not use data_alloc_cluster in ssd mode

On 07/21/2017 04:49 PM, Adam Borowski wrote: > On Fri, Jul 21, 2017 at 01:47:11PM +0200, Hans van Kranenburg wrote: >> The changes in here do the following: >> >> 1. Throw out the current ssd_spread behaviour. >> 2. Move the current ssd behaviour to the ssd_spread option. >> 3. Make ssd mode data

Re: [PATCH v2] btrfs: Remove never reached error handling code in __add_reloc_root

On Thu, Jul 13, 2017 at 02:11:07PM +0300, Nikolay Borisov wrote: > One of the error handling paths in __add_reloc_root contains btrfs_panic() > followed by some other code. As the name implies what it does is print > some error message and call BUG, naturally what follow afterwards is not >

Re: [PATCH] btrfs: Remove unused variables from btrfs_async_reclaim_metadata_space

On Fri, Jul 14, 2017 at 11:46:24AM +0300, Nikolay Borisov wrote: > The ret from flush_space is ignored so no point in assigning it to a variable. It would be good to document why it's safe to ignore return value of flush_spaces along with removing it. -- To unsubscribe from this list: send the