Re: [PATCH] fstests: btrfs/150 regression test for reading compressed data

On Wed, Sep 20, 2017 at 05:52:43PM -0600, Liu Bo wrote: > We had a bug in btrfs compression code which could end up with a > kernel panic. > > This is adding a regression test for the bug and I've also sent a > kernel patch to fix the bug. > > The patch is "Btrfs: fix kernel oops while reading

[PATCH v2] btrfs: LOGICAL_INO enhancements (this time based on 4.14-rc1)

The previous patch series was based on v4.12.14, and this introductory text was missing. This patch series fixes some weaknesses in the btrfs LOGICAL_INO ioctl. Background: Suppose we have a file with one extent: root@tester:~# zcat /usr/share/doc/cpio/changelog.gz > /test/a

[PATCH 3/3] btrfs: increase output size for LOGICAL_INO_V2 ioctl

Build-server workloads have hundreds of references per file after dedup. Multiply by a few snapshots and we quickly exhaust the limit of 2730 references per extent that can fit into a 64K buffer. Raise the limit to 16M to be consistent with other btrfs ioctls (e.g. TREE_SEARCH_V2,

[PATCH 2/3] btrfs: add a flags argument to LOGICAL_INO and call it LOGICAL_INO_V2

Now that check_extent_in_eb()'s extent offset filter can be turned off, we need a way to do it from userspace. Add a 'flags' field to the btrfs_logical_ino_args structure to disable extent offset filtering, taking the place of one of the reserved[] fields. Previous versions of LOGICAL_INO

[PATCH 1/3] btrfs: add a flag to iterate_inodes_from_logical to find all extent refs for uncompressed extents

The LOGICAL_INO ioctl provides a backward mapping from extent bytenr and offset (encoded as a single logical address) to a list of extent refs. LOGICAL_INO complements TREE_SEARCH, which provides the forward mapping (extent ref -> extent bytenr and offset, or logical address). These are useful

Re: btrfs-progs: suggestion of removing --commit-after option of subvol delete

On 2017年09月21日 10:49, Misono, Tomohiro wrote: On 2017/09/20 23:21, Qu Wenruo wrote: On 2017年09月20日 22:03, David Sterba wrote: On Wed, Sep 20, 2017 at 08:22:54AM +0800, Qu Wenruo wrote: The costly part will be tracking the filesystems of subvolumes. We must do it for each subvolume and

Re: btrfs-progs: suggestion of removing --commit-after option of subvol delete

On 2017/09/20 23:21, Qu Wenruo wrote: > > > On 2017年09月20日 22:03, David Sterba wrote: >> On Wed, Sep 20, 2017 at 08:22:54AM +0800, Qu Wenruo wrote: >>> The costly part will be tracking the filesystems of subvolumes. >>> We must do it for each subvolume and batch them to address the final >>>

[PATCH] fstests: btrfs/150 regression test for reading compressed data

We had a bug in btrfs compression code which could end up with a kernel panic. This is adding a regression test for the bug and I've also sent a kernel patch to fix the bug. The patch is "Btrfs: fix kernel oops while reading compressed data". Signed-off-by: Liu Bo ---

[PATCH 2/2] Btrfs: skip checksum when reading compressed data if some IO have failed

Currently even if the underlying disk reports failure on IO, compressed read endio still gets to verify checksum and reports it as a checksum error. In fact, if some IO have failed during reading a compressed data extent , there's no way the checksum could match, therefore, we can skip that in

[PATCH 1/2] Btrfs: fix kernel oops while reading compressed data

The kernel oops happens at kernel BUG at fs/btrfs/extent_io.c:2104! ... RIP: clean_io_failure+0x263/0x2a0 [btrfs] It's showing that read-repair code is using an improper mirror index. This is due to the fact that compression read's endio hasn't recorded the failed mirror index in %cb->orig_bio.

[PATCH 1/3] btrfs: add a flag to iterate_inodes_from_logical to find all extent refs for uncompressed extents

The LOGICAL_INO ioctl provides a backward mapping from extent bytenr and offset (encoded as a single logical address) to a list of extent refs. LOGICAL_INO complements TREE_SEARCH, which provides the forward mapping (extent ref -> extent bytenr and offset, or logical address). These are useful

[PATCH 2/3] btrfs: add a flags argument to LOGICAL_INO and call it LOGICAL_INO_V2

Now that check_extent_in_eb()'s extent offset filter can be turned off, we need a way to do it from userspace. Add a 'flags' field to the btrfs_logical_ino_args structure to disable extent offset filtering, taking the place of one of the reserved[] fields. Previous versions of LOGICAL_INO

[PATCH 3/3] btrfs: increase output size for LOGICAL_INO_V2 ioctl

Build-server workloads have hundreds of references per file after dedup. Multiply by a few snapshots and we quickly exhaust the limit of 2730 references per extent that can fit into a 64K buffer. Raise the limit to 16M to be consistent with other btrfs ioctls (e.g. TREE_SEARCH_V2,

Re: ERROR: parent determination failed (btrfs send-receive)

On Tue, Sep 19, 2017 at 3:37 PM, Andrei Borzenkov wrote: > 18.09.2017 09:10, Dave пишет: >> I use snap-sync to create and send snapshots. >> >> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external >> drive >> https://github.com/wesbarnett/snap-sync >> >

Re: RFC: Btrfs does not store checksum type? Only size? O_o

On 2017年09月21日 04:31, Timofey Titovets wrote: I was reading the kernel code and docs and didn't find anything about some info like "cheksum type" as btrfs have for compression code. (I only found some silent "hack" in btrfs-progs, that print "checksum type", but get "checksum size") Checksum

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 2017年09月21日 02:10, Rich Rauenzahn wrote: On 9/20/2017 9:58 AM, Rich Rauenzahn wrote: What's the most direct way to do that?  (Was about to risk breaking the mirror and repartitioning!  I'd rather not!) Hmm -- maybe this worked: $ sudo btrfs filesystem resize -1m /.MEDIA/ Resize

Re: Does btrfs use crc32 for error correction?

2017-09-19 19:51 GMT+03:00 Eric Sandeen : > On 9/19/17 10:35 AM, Timofey Titovets wrote: >> Stupid question: >> Does btrfs use crc32 for error correction? >> If no, why? >> >> (AFAIK if using CRC that possible to fix 1 bit flip) >> >> P.S. I try check that (i create image,

Re: SSD caching an existing btrfs raid1

Am Wed, 20 Sep 2017 17:51:15 +0200 schrieb Psalle : > On 19/09/17 17:47, Austin S. Hemmelgarn wrote: > (...) > > > > A better option if you can afford to remove a single device from > > that array temporarily is to use bcache.  Bcache has one specific > > advantage in this

Re: RFC: Btrfs does not store checksum type? Only size? O_o

On 09/20/2017 10:31 PM, Timofey Titovets wrote: > I was reading the kernel code and docs and didn't find anything about > some info like "cheksum type" as btrfs have for compression code. > (I only found some silent "hack" in btrfs-progs, that print "checksum > type", but get "checksum size") > >

RFC: Btrfs does not store checksum type? Only size? O_o

I was reading the kernel code and docs and didn't find anything about some info like "cheksum type" as btrfs have for compression code. (I only found some silent "hack" in btrfs-progs, that print "checksum type", but get "checksum size") If i understood all correctly btrfs only store checksum

Re: kernel BUG at fs/btrfs/extent_io.c:1989

On Wed, Sep 20, 2017 at 02:53:57PM +0200, David Sterba wrote: > On Tue, Sep 19, 2017 at 10:12:39AM -0600, Liu Bo wrote: > > On Tue, Sep 19, 2017 at 05:07:25PM +0200, David Sterba wrote: > > > On Tue, Sep 19, 2017 at 11:32:46AM +, Paul Jones wrote: > > > > > This 'mirror 0' looks fishy, (as

Re: difference between -c and -p for send-receive?

20.09.2017 22:05, Antoine Belvire пишет: > Hello, > >> All snapshots listed in -c options and snapshot that we want to >> transfer must have the same parent uuid, unless -p is explicitly >> provided. > > It's rather the same mount point than the same parent uuid, like cp > --reflink, isn't it?

Re: difference between -c and -p for send-receive?

Hello, > All snapshots listed in -c options and snapshot that we want to > transfer must have the same parent uuid, unless -p is explicitly > provided. It's rather the same mount point than the same parent uuid, like cp --reflink, isn't it? ~# btrfs subvolume create /test2/ Create subvolume

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 9/20/2017 9:58 AM, Rich Rauenzahn wrote: What's the most direct way to do that?  (Was about to risk breaking the mirror and repartitioning!  I'd rather not!) Hmm -- maybe this worked: $ sudo btrfs filesystem resize -1m /.MEDIA/ Resize '/.MEDIA/' of '-1m' No, doesn't seem to have worked. 

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 9/19/2017 10:39 PM, Qu Wenruo wrote: In v4.13 kernel, the newly added devices are in fact rounded down. But existing device doesn't get the round down. So it's recommended to resize (shrink) your fs for very small size to fix it if you don't want to wait for the kernel fix. What's

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

For the warning -- is there anyway to add in the filesystem/disk causing the issue?  I didn't see any identifier in the message that told me which it was. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More

Re: SSD caching an existing btrfs raid1

On 19/09/17 17:47, Austin S. Hemmelgarn wrote: (...) A better option if you can afford to remove a single device from that array temporarily is to use bcache.  Bcache has one specific advantage in this case, multiple backend devices can share the same cache device. This means you don't

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 2017年09月20日 22:14, nborisov wrote: On 2017-09-20 08:23, Qu Wenruo wrote: On 2017年09月20日 14:11, nborisov wrote: On 2017-09-20 07:39, Qu Wenruo wrote: On 2017年09月20日 13:10, Qu Wenruo wrote: On 2017年09月20日 12:59, Qu Wenruo wrote: On 2017年09月20日 12:49, Rich Rauenzahn wrote: On

Re: btrfs-progs: suggestion of removing --commit-after option of subvol delete

On 2017年09月20日 22:03, David Sterba wrote: On Wed, Sep 20, 2017 at 08:22:54AM +0800, Qu Wenruo wrote: The costly part will be tracking the filesystems of subvolumes. We must do it for each subvolume and batch them to address the final transaction commit for each fs. I didn't see any easy

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 2017-09-20 08:23, Qu Wenruo wrote: On 2017年09月20日 14:11, nborisov wrote: On 2017-09-20 07:39, Qu Wenruo wrote: On 2017年09月20日 13:10, Qu Wenruo wrote: On 2017年09月20日 12:59, Qu Wenruo wrote: On 2017年09月20日 12:49, Rich Rauenzahn wrote: On 9/19/2017 5:31 PM, Qu Wenruo wrote: On

Re: btrfs-progs: suggestion of removing --commit-after option of subvol delete

On Wed, Sep 20, 2017 at 08:22:54AM +0800, Qu Wenruo wrote: > The costly part will be tracking the filesystems of subvolumes. > We must do it for each subvolume and batch them to address the final > transaction commit for each fs. > > I didn't see any easy ioctl to get the UUID from fd, meaning

Re: kernel BUG at fs/btrfs/extent_io.c:1989

On Tue, Sep 19, 2017 at 10:12:39AM -0600, Liu Bo wrote: > On Tue, Sep 19, 2017 at 05:07:25PM +0200, David Sterba wrote: > > On Tue, Sep 19, 2017 at 11:32:46AM +, Paul Jones wrote: > > > > This 'mirror 0' looks fishy, (as mirror comes from > > > > btrfs_io_bio->mirror_num, > > > > which should

Re: [PATCH v2] fs:btrfs: return -ENOMEM on allocation failure.

On 09/20/2017 02:17 PM, Allen Pais wrote: v2: return -ENOMEM from btrfsic_dev_state_alloc() too. Signed-off-by: Allen Pais Reviewed-by: Anand Jain --- fs/btrfs/check-integrity.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

Re: defragmenting best practice?

On 2017-09-20 02:38, Dave wrote: On Thu 2017-08-31 (09:05), Ulli Horlacher wrote: When I do a btrfs filesystem defragment -r /directory does it defragment really all files in this directory tree, even if it contains subvolumes? The man page does not mention subvolumes on this topic. No answer

Re: using fio to test btrfs compression

On Wed, Sep 20, 2017 at 4:00 PM, Timofey Titovets wrote: > 2017-09-20 13:13 GMT+03:00 shally verma : >> On Wed, Sep 20, 2017 at 3:06 PM, Timofey Titovets >> wrote: >>> 2017-09-20 11:59 GMT+03:00 shally verma

Re: using fio to test btrfs compression

2017-09-20 13:13 GMT+03:00 shally verma : > On Wed, Sep 20, 2017 at 3:06 PM, Timofey Titovets > wrote: >> 2017-09-20 11:59 GMT+03:00 shally verma : >>> On Wed, Sep 20, 2017 at 2:25 PM, Timofey Titovets

Re: using fio to test btrfs compression

On Wed, Sep 20, 2017 at 3:06 PM, Timofey Titovets wrote: > 2017-09-20 11:59 GMT+03:00 shally verma : >> On Wed, Sep 20, 2017 at 2:25 PM, Timofey Titovets >> wrote: >>> 2017-09-20 11:44 GMT+03:00 shally verma

Re: using fio to test btrfs compression

2017-09-20 11:59 GMT+03:00 shally verma : > On Wed, Sep 20, 2017 at 2:25 PM, Timofey Titovets > wrote: >> 2017-09-20 11:44 GMT+03:00 shally verma : >>> One more catch... I am initiating fio from non-btrfs filesystem

RFC: crc32c vs xxhash32/64 (ARM)

Hi, About 3 years ago Liu Bo has try add xxhash support to btrfs[1]. For now (Kernel 4.14) kernel have a native library for xxhash. Main reason to not add that support is what: - CRC32C - has hardware support and amazing fast - CRC32C - Can fix 1 bit error (Yes that possible, but that not

Re: using fio to test btrfs compression

On Wed, Sep 20, 2017 at 2:25 PM, Timofey Titovets wrote: > 2017-09-20 11:44 GMT+03:00 shally verma : >> One more catch... I am initiating fio from non-btrfs filesystem i.e. >> pwd is ext4 based fs where as mount point is btrfs. >> Could that make

Re: using fio to test btrfs compression

2017-09-20 11:44 GMT+03:00 shally verma : > One more catch... I am initiating fio from non-btrfs filesystem i.e. > pwd is ext4 based fs where as mount point is btrfs. > Could that make difference? > >> Thanks >> Shally Only matter are where you store test file =) If

Re: using fio to test btrfs compression

On Wed, Sep 20, 2017 at 2:06 PM, shally verma wrote: > On Mon, Sep 18, 2017 at 7:11 PM, Timofey Titovets > wrote: >> 2017-09-18 16:28 GMT+03:00 shally verma : >>> On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets

Re: using fio to test btrfs compression

On Mon, Sep 18, 2017 at 7:11 PM, Timofey Titovets wrote: > 2017-09-18 16:28 GMT+03:00 shally verma : >> On Mon, Sep 18, 2017 at 1:56 PM, Timofey Titovets >> wrote: >>> 2017-09-18 10:36 GMT+03:00 shally verma

Re: defragmenting best practice?

I've had a very similar issue with the performance of my laptop dropping to very low levels, eventually solved by uninstalling Snapper, deleting snapshots, and then defragmenting the drive. This seems to be a common concern, I also had it happen on my desktop. Dmitry --- Thank you, Dmitry

Re: defragmenting best practice?

>On Thu 2017-08-31 (09:05), Ulli Horlacher wrote: >> When I do a >> btrfs filesystem defragment -r /directory >> does it defragment really all files in this directory tree, even if it >> contains subvolumes? >> The man page does not mention subvolumes on this topic. > >No answer so far :-( > >But

[PATCH v2] btrfs: Make check for unaligned device size more gentle

Commit 7dfb8be11b5d ("btrfs: Round down values which are written for total_bytes_size") is fixing the unaligned device size caused by adding/shrinking device. It added a new WARN_ON() when device size is unaligned. This is fine for new device added to btrfs using v4.13 kernel, but not existing

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 2017年09月20日 14:11, nborisov wrote: On 2017-09-20 07:39, Qu Wenruo wrote: On 2017年09月20日 13:10, Qu Wenruo wrote: On 2017年09月20日 12:59, Qu Wenruo wrote: On 2017年09月20日 12:49, Rich Rauenzahn wrote: On 9/19/2017 5:31 PM, Qu Wenruo wrote: On 2017年09月19日 23:56, Rich Rauenzahn wrote: [ 

[PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output

Commit 7dfb8be11b5d ("btrfs: Round down values which are written for total_bytes_size") is fixing the unaligned device size caused by adding/shrinking device. It added a new WARN_ON() when device size is unaligned. This is fine for new device added to btrfs using v4.13 kernel, but not existing

[PATCH v2] fs:btrfs: return -ENOMEM on allocation failure.

v2: return -ENOMEM from btrfsic_dev_state_alloc() too. Signed-off-by: Allen Pais --- fs/btrfs/check-integrity.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index 7d5a9b5..9db1e76 100644 ---

Re: Problems with debian install

On 09/20/2017 02:39 AM, Qu Wenruo wrote: On 2017年09月20日 05:35, Pierre Couderc wrote: I am trying to install stretch on a computer with 2 btrfs disks and EFI. Is there a howto to do that ? Did someone success ? I get problems as soon as the partionning in debian installer. What partitions

Re: WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

On 2017-09-20 07:39, Qu Wenruo wrote: On 2017年09月20日 13:10, Qu Wenruo wrote: On 2017年09月20日 12:59, Qu Wenruo wrote: On 2017年09月20日 12:49, Rich Rauenzahn wrote: On 9/19/2017 5:31 PM, Qu Wenruo wrote: On 2017年09月19日 23:56, Rich Rauenzahn wrote: [    4.747377] WARNING: CPU: 3 PID: 439 at