Re: System completely unresponsive after `btrfs balance start -dconvert=raid0 /` and `btrfs fi show /`

Carmine Paolino posted on Tue, 13 Oct 2015 23:21:49 +0200 as excerpted: > I have an home server with 3 hard drives that I added to the same btrfs > filesystem. Several hours ago I run `btrfs balance start -dconvert=raid0 > /` and as soon as I run `btrfs fi show /` I lost my ssh connection to >

Re: State of Dedup / Defrag

On Tue, Oct 13, 2015 at 02:59:59PM -0400, Rich Freeman wrote: > What is the current state of Dedup and Defrag in btrfs? I seem to > recall there having been problems a few months ago and I've stopped > using it, but I haven't seen much news since. It has been 1 day since a kernel bug leading to

[PATCH v2] btrfs: compress: put variables defined per compress type in struct to make cache friendly

Below variables are defined per compress type. - struct list_head comp_idle_workspace[BTRFS_COMPRESS_TYPES] - spinlock_t comp_workspace_lock[BTRFS_COMPRESS_TYPES] - int comp_num_workspace[BTRFS_COMPRESS_TYPES] - atomic_t comp_alloc_workspace[BTRFS_COMPRESS_TYPES] - wait_queue_head_t

Re: [PATCH] btrfs: fix use after free iterating extrefs

On Tue, Oct 13, 2015 at 12:17:55PM -0700, Mark Fasheh wrote: > On Tue, Oct 13, 2015 at 02:06:48PM -0400, Chris Mason wrote: > > The code for btrfs inode-resolve has never worked properly for > > files with enough hard links to trigger extrefs. It was trying to > > get the leaf out of a path after

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Mon, Oct 12, 2015 at 11:36:31PM -0400, Trond Myklebust wrote: > On Mon, Oct 12, 2015 at 7:17 PM, Darrick J. Wong > wrote: > > On Sun, Oct 11, 2015 at 07:22:03AM -0700, Christoph Hellwig wrote: > >> On Wed, Sep 30, 2015 at 01:26:52PM -0400, Anna Schumaker wrote: > >> >

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Mon, Oct 12, 2015 at 04:17:49PM -0700, Darrick J. Wong wrote: > Hm. Peng's patches only generalize the CLONE and CLONE_RANGE ioctls from > btrfs, however they don't port over the (vastly different) EXTENT_SAME ioctl. > > What does everyone think about generalizing EXTENT_SAME? The interface

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On Mon, Oct 12, 2015 at 11:36:31PM -0400, Trond Myklebust wrote: > How is this supposed to be implemented on something like NFS without > protocol changes? Explicit dedup has no chance of working over NFS or other network protocols without protocol changes. -- To unsubscribe from this list: send

[PATCH 1/2] btrfs: extend balance filter limit to take minimum and maximum

The 'limit' filter is underdesigned, it should have been a range for [min,max], with some relaxed semantics when one of the bounds is missing. Besides that, using a full u64 for a single value is a waste of bytes. Let's fix both by extending the use of the u64 bytes for the [min,max] range. This

[PATCH 0/2] Balance filters: stripes, enhanced limit

Update to balance filters, intended fro 4.4: * new 'stripes=' - process only stripes that cross given number of devices, specified by a range * updated 'limit=' - previously a single number was accepted, it's a range so now we can specify a minimum number of chunks to process There will be

[PATCH 2/2] btrfs: add balance filter for stripes

From: Gabríel Arthúr Pétursson Balance block groups which have the given number of stripes, defined by a range min..max. This is useful to selectively rebalance only chunks that do not span enough devices, applies to RAID0/10/5/6. Signed-off-by: Gabríel Arthúr Pétursson

Re: [PATCH v5 9/9] btrfs: btrfs_copy_file_range() only supports reflinks

On Mon, Oct 12, 2015 at 04:41:06PM -0700, Darrick J. Wong wrote: > One of the patches in last week's XFS reflink patchbomb adds FALLOC_FL_UNSHARE > flag; at the moment it _only_ forces copy-on-write of shared blocks, and it > leaves holes alone. Yes, I've seen the implementation. > Obviously we

[PATCH] btrfs-progs: mkfs: Enable -d dup for single device

Current code don't support dup profile in single device, except it is in mixed mode, because following reason: 1: In some ssd with deduplication function, it have no effect. 2: For a physical device, it the entire disk broken, -d dup can not help. 3: Half performance comparing with single

behavior of BTRFS in relation to inodes when moving/copying files between filesystems

Hi! With BTRFS to XFS/Ext4 the inode number of the target file stays the same in with both cp and mv case (/mnt/zeit is a freshly created XFS in this example): merkaba:~> ls -li foo /mnt/zeit/moo 6609270 foo 99 /mnt/zeit/moo merkaba:~> cp foo /mnt/zeit/moo merkaba:~> ls -li foo

RE: [PATCH] btrfs-progs: mkfs: Enable -d dup for single device

Hi, David Sterba > -Original Message- > From: David Sterba [mailto:dste...@suse.cz] > Sent: Tuesday, October 13, 2015 8:36 PM > To: Zhao Lei > Cc: dste...@suse.cz; linux-btrfs@vger.kernel.org; c...@fb.com > Subject: Re: [PATCH] btrfs-progs: mkfs: Enable -d dup for

Re: [PATCH] btrfs-progs: mkfs: Enable -d dup for single device

On Tue, Oct 13, 2015 at 07:40:30PM +0800, Zhao Lei wrote: > > What I remember from the comment is that "it's slightly offset that would > > lead > > to corruption". > > Before this patch, I had done git blame to search why the condition was added, > and hadn't found the exact reason. found it:

Re: [PATCH RESEND 0/3] btrfs-progs: Introduce device delete by devid

On Sat, Oct 10, 2015 at 10:30:55PM +0800, Anand Jain wrote: > This is the btrfs-progs part of the kernel patch >Btrfs: Introduce device delete by devid Thanks, now in next/delete-by-id-v3, I made some changes so please have a look. Notably, I've dropped the BTRFS_VOL_ARG_V2_FLAGS mask, this

Re: [PATCH] btrfs-progs: mkfs: Enable -d dup for single device

On Tue, Oct 13, 2015 at 06:29:41PM +0800, Zhao Lei wrote: > Current code don't support dup profile in single device, except it > is in mixed mode, because following reason: > 1: In some ssd with deduplication function, it have no effect. > 2: For a physical device, it the entire disk broken, -d

RE: [PATCH] btrfs-progs: mkfs: Enable -d dup for single device

Hi, David Sterba Thanks for review. > -Original Message- > From: David Sterba [mailto:dste...@suse.cz] > Sent: Tuesday, October 13, 2015 7:29 PM > To: Zhao Lei > Cc: linux-btrfs@vger.kernel.org; c...@fb.com > Subject: Re: [PATCH] btrfs-progs: mkfs: Enable -d dup

[PATCH 0/3] btrfs-progs: mkfs: Fix different mixed type by argument sequence

Given a 200G vdd1 and 1G vdd2: In current code: mkfs.btrfs -f /dev/vdd1 /dev/vdd2 and mkfs.btrfs -f /dev/vdd2 /dev/vdd1 will create different "mixed" type. See [PATCH 2/3] for detail. This patchset also include some small fixs. Zhao Lei (3): btrfs-progs: mkfs: Remove saved_optind in

[PATCH 1/3] btrfs-progs: mkfs: Remove saved_optind in mkfs.btrfs

No need to use complex logic for iter devs in mkfs.c, as backup optind, increase/decrease optind and reset dev_cnt. A simple for() loop is enough for above request. Signed-off-by: Zhao Lei --- mkfs.c | 25 +++-- 1 file changed, 7 insertions(+), 18

[PATCH 2/3] btrfs-progs: mkfs: Fix different mixed type by argument sequence

Given a 200G vdd1 and 1G vdd2: In current code: # mkfs.btrfs -f /dev/vdd1 /dev/vdd2 SMALL VOLUME: forcing mixed metadata/data groups btrfs-progs v4.1.2 See http://btrfs.wiki.kernel.org for more information. Label: (null) UUID:

[PATCH 3/3] btrfs-progs: mkfs: Fix inaccurate mixed information

In current code, with a "BIG VOLUME" /dev/vdd2: # ./mkfs.btrfs -f -M /dev/vdd2 SMALL VOLUME: forcing mixed metadata/data groups ... This patch changed above output to: Using mixed metadata/data groups And the "SMALL VOLUME" output only when we exactly using SMALL VOLUME. Signed-off-by: Zhao

[PATCH] Btrfs: fix file corruption and data loss after cloning inline extents

From: Filipe Manana Currently the clone ioctl allows to clone an inline extent from one file to another that already has other (non-inlined) extents. This is a problem because btrfs is not designed to deal with files having inline and regular extents, if a file has an inline

[PATCH 1/2] fstests: btrfs test for cloning of inline extents

From: Filipe Manana Test several cases of cloning inline extents that used to lead to file corruption or data loss. These file corruption and data loss cases are fixed by the linux kernel patch titled: "Btrfs: fix file corruption and data loss after cloning inline extents"

[PATCH 2/2] fstests: update btrfs/035 to check for data loss

From: Filipe Manana The test currently verifies that cloning one file with an inline extent with a size of 10 bytes into a file with an inline extent that has a size of 20 bytes succeeds. But this results in data loss, because the btrfs clone operation drops the 20 bytes

Kernel error in extent-tree, forced readonly

One of my backup disks hit a btrfs bug yesterday, leaving me with a forced readonly filesystem (see kernel trace below). This error is reproducible, and happens on first access after mounting. This disk receives snapshots (incrementally "ssh btrfs send -p | btrfs receive") from several hosts on a

[PATCH 3/7] btrfs-progs: add helpers for parsing 32bit ranges

Signed-off-by: David Sterba --- cmds-balance.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/cmds-balance.c b/cmds-balance.c index 62bee3cc78b6..72714b23b45c 100644 --- a/cmds-balance.c +++ b/cmds-balance.c @@ -159,6 +159,37 @@ static int

[PATCH 5/7] btrfs-progs: extend balance args to take min/max limit filter

Add the overlapping limit and [limit_min, limit_max] members to the balance args. The min/max values are interpreted iff the corresponding flag BTRFS_BALANCE_ARGS_LIMITS is set. Note that the values are only 32bit, but this should be enough for the foreseeable future. Signed-off-by: David Sterba

[PATCH 4/7] btrfs-progs: add helpers to print ranges

Signed-off-by: David Sterba --- cmds-balance.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/cmds-balance.c b/cmds-balance.c index 72714b23b45c..dba6613b1540 100644 --- a/cmds-balance.c +++ b/cmds-balance.c @@ -190,6 +190,25 @@ static int

[PATCH 7/7] btrfs-progs: balance: add stripes filter

From: Gabríel Arthúr Pétursson Add new balance filter 'stripes=' to process only chunks that are spread accross given number of chunks. The range must be specified with both values, but they can be the same to denote exact number of stripes. Signed-off-by: Gabríel Arthúr

[PATCH 6/7] btrfs-progs: balance: enhance the limit fiter with range

We can do more with the balance filer. Enhance it so we can specify also the minimum number of block groups to process. The 'limit' filter now accepts a range (a..b, can be partial) and needs kernel support. The 'limit=value' filter is equivalent to 'limit=..value' but works on older kernels as

[PATCH 2/7] btrfs-progs: extend parse_range API to accept a relaxed range

In some cases we want to accept a range of type [a..a]. Add a new function to do the 'a < b' check for the caller and use it. Signed-off-by: David Sterba --- cmds-balance.c | 30 +- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git

[PATCH 1/7] btrfs-progs: cleanup and comment parse_range

Simplify a check and unindent some code. Signed-off-by: David Sterba --- cmds-balance.c | 63 ++ 1 file changed, 37 insertions(+), 26 deletions(-) diff --git a/cmds-balance.c b/cmds-balance.c index

[PATCH 0/7] Btrfs-progs, balance filters: stripes, limits

Here's the userspace part that enables the use of stripes and enhanced limit filters. David Sterba (6): btrfs-progs: cleanup and comment parse_range btrfs-progs: extend parse_range API to accept a relaxed range btrfs-progs: add helpers for parsing 32bit ranges btrfs-progs: add helpers to

[PATCH] Btrfs: fix double range unlock of hole region when reading page

From: Filipe Manana If when reading a page we find a hole and our caller had already locked the range (bio flags has the bit EXTENT_BIO_PARENT_LOCKED set), we end up unlocking the hole's range and then later our caller unlocks it again, which might have already been locked by

Re: [PATCH 00/11] btrfs-progs: Use btrfs_open_dir to avoid show error of ioctl or tree search

On Mon, Oct 12, 2015 at 09:22:53PM +0800, Zhao Lei wrote: > Use btrfs_open_dir() instead of open_file_or_dir(), to show error before > real action(in ioctl or tree search), to make the error message exact > and unified. > > Zhao Lei (11): > btrfs-progs: subvolume: use btrfs_open_dir for btrfs

[PATCH] btrfs: compress: put variables defined per compress type in struct to make cache friendly

Below variables are defined per compress type. - struct list_head comp_idle_workspace[BTRFS_COMPRESS_TYPES] - spinlock_t comp_workspace_lock[BTRFS_COMPRESS_TYPES] - int comp_num_workspace[BTRFS_COMPRESS_TYPES] - atomic_t comp_alloc_workspace[BTRFS_COMPRESS_TYPES] - wait_queue_head_t

Re: BTRFS with 8TB SMR drives

On Mon, Oct 12, 2015 at 06:25:52PM +0200, Henk Slager wrote: > and looking at this spec: > http://www.seagate.com/files/www-content/product-content/hdd-fam/seagate-archive-hdd/en-us/docs/archive-hdd-dS1834-3-1411us.pdf > > it seems that it is a drive-managed SMR disk. I am not sure why David >

Re: [PATCH] btrfs: compress: put variables defined per compress type in struct to make cache friendly

On Wed, Oct 14, 2015 at 01:13:26AM +0900, Byongho Lee wrote: > Below variables are defined per compress type. > - struct list_head comp_idle_workspace[BTRFS_COMPRESS_TYPES] > - spinlock_t comp_workspace_lock[BTRFS_COMPRESS_TYPES] > - int comp_num_workspace[BTRFS_COMPRESS_TYPES] > - atomic_t

Re: [PATCH] btrfs: fix resending received snapshot with parent

On Tue, Oct 13, 2015 at 1:31 AM, Ed Tomlinson wrote: > On Friday, October 9, 2015 4:24:10 PM EDT, Filipe Manana wrote: >> >> On Wed, Sep 30, 2015 at 8:23 PM, Robin Ruede wrote: >>> >>> This fixes a regression introduced by 37b8d27d between v4.1 and v4.2. >>> >>>

Re: [PATCH] btrfs: fix use after free iterating extrefs

On Tue, Oct 13, 2015 at 02:06:48PM -0400, Chris Mason wrote: > The code for btrfs inode-resolve has never worked properly for > files with enough hard links to trigger extrefs. It was trying to > get the leaf out of a path after freeing the path: > > btrfs_release_path(path); > leaf

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 10/09/2015 07:15 AM, Pádraig Brady wrote: > On 08/10/15 02:40, Neil Brown wrote: >> Anna Schumaker writes: >> >>> @@ -1338,34 +1362,26 @@ ssize_t vfs_copy_file_range(struct file *file_in, >>> loff_t pos_in, >>> struct file *file_out, loff_t

Re: [PATCH v5 8/9] vfs: Add vfs_copy_file_range() support for pagecache copies

On 10/07/2015 09:40 PM, Neil Brown wrote: > Anna Schumaker writes: > >> @@ -1338,34 +1362,26 @@ ssize_t vfs_copy_file_range(struct file *file_in, >> loff_t pos_in, >> struct file *file_out, loff_t pos_out, >> size_t

Re: [PATCH 0/3] btrfs-progs: mkfs: Fix different mixed type by argument sequence

On Tue, Oct 13, 2015 at 08:52:16PM +0800, Zhao Lei wrote: > Given a 200G vdd1 and 1G vdd2: > > In current code: > mkfs.btrfs -f /dev/vdd1 /dev/vdd2 > and > mkfs.btrfs -f /dev/vdd2 /dev/vdd1 > will create different "mixed" type. I think combining large and small devices was not intended use

[PATCH] btrfs: fix use after free iterating extrefs

The code for btrfs inode-resolve has never worked properly for files with enough hard links to trigger extrefs. It was trying to get the leaf out of a path after freeing the path: btrfs_release_path(path); leaf = path->nodes[0]; item_size = btrfs_item_size_nr(leaf, slot);

Re: [PATCH] btrfs: fix use after free iterating extrefs

On Tue, Oct 13, 2015 at 7:06 PM, Chris Mason wrote: > The code for btrfs inode-resolve has never worked properly for > files with enough hard links to trigger extrefs. It was trying to > get the leaf out of a path after freeing the path: > > btrfs_release_path(path); >

State of Dedup / Defrag

What is the current state of Dedup and Defrag in btrfs? I seem to recall there having been problems a few months ago and I've stopped using it, but I haven't seen much news since. I'm interested both in the 3.18 and subsequent kernel series. -- Rich -- To unsubscribe from this list: send the

System completely unresponsive after `btrfs balance start -dconvert=raid0 /` and `btrfs fi show /`

Hi all, I have an home server with 3 hard drives that I added to the same btrfs filesystem. Several hours ago I run `btrfs balance start -dconvert=raid0 /` and as soon as I run `btrfs fi show /` I lost my ssh connection to the machine. The machine is still on, but it doesn’t even respond to

Can't mount btrfs: corrupt leaf, slot offset bad

I rebooted my server last night and discovered that my btrfs filesystem (3 disk raid1) would not mount anymore. After doing some research and getting nowhere I went to IRC and user darkling asked me a few questions and asked for output of btrfs-debug-tree and ultimately sent me here saying I

Re: [PATCH] btrfs: compress: put variables defined per compress type in struct to make cache friendly

David Sterba writes: > >> +static struct { >> +struct list_head idle_workspace; >> +spinlock_t workspace_lock; >> +int num_workspace; >> +atomic_t alloc_workspace; >> +wait_queue_head_t workspace_wait; >> +} comp[BTRFS_COMPRESS_TYPES]; > > The name became too generic, please