Re: mount gets stuck - BUG: soft lockup

2017-06-07 Thread Duncan
Thomas Mischke posted on Wed, 07 Jun 2017 09:44:41 + as excerpted: > i tried to convert a JBOD BTRFS consisting of 5 disks (6TB each) to > raid10 (converting from an earlier configuration). > All disk were backed by bcache. > > Because a rebalance takes very long I had to pause the balance fo

Re: [PATCH] btrfs: test Btrfs delalloc accounting overflow

2017-06-07 Thread Eryu Guan
On Wed, Jun 07, 2017 at 10:34:13AM -0700, Omar Sandoval wrote: > On Wed, Jun 07, 2017 at 05:36:45PM +0800, Eryu Guan wrote: > > On Tue, Jun 06, 2017 at 11:57:10PM -0700, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > This is a regression test for "[PATCH] Btrfs: fix delalloc accountin

Re: [PATCH 7/7] Btrfs: end_workqueue_bio use switch case instead of else if

2017-06-07 Thread kbuild test robot
Hi Timofey, [auto build test WARNING on next-20170605] [also build test WARNING on v4.12-rc4] [cannot apply to btrfs/next v4.9-rc8 v4.9-rc7 v4.9-rc6] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commi

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Adam Borowski
On Wed, Jun 07, 2017 at 10:20:53PM +0200, Goffredo Baroncelli wrote: > On 2017-06-07 20:04, Adam Borowski wrote: > > On Wed, Jun 07, 2017 at 07:01:21PM +0200, Goffredo Baroncelli wrote: > >> It might be useful to have a command to handle these situations: read all > >> the good data, read even the

Re: mount gets stuck - BUG: soft lockup

2017-06-07 Thread Liu Bo
On Wed, Jun 07, 2017 at 09:44:41AM +, Thomas Mischke wrote: > Hello, > > i tried to convert a JBOD BTRFS consisting of 5 disks (6TB each) to raid10 > (converting from an earlier configuration). > All disk were backed by bcache. > > Because a rebalance takes very long I had to pause the balan

Re: [PATCH 7/7] Btrfs: warn if total_bytes_pinned is non-zero on unmount

2017-06-07 Thread Liu Bo
On Tue, Jun 06, 2017 at 04:45:32PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Catch any future/remaining leaks or underflows of total_bytes_pinned. > This might be a little bit late, what about checking it after btrfs_finish_extetn_commit()? -liubo > Signed-off-by: Omar Sandoval >

Re: [PATCH 4/7] Btrfs: always account pinned bytes when dropping a tree block ref

2017-06-07 Thread Liu Bo
On Tue, Jun 06, 2017 at 04:45:29PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Currently, we only increment total_bytes_pinned in > btrfs_free_tree_block() when dropping the last reference on the block. > However, when the delayed ref is run later, we will decrement > total_bytes_pinned

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Goffredo Baroncelli
On 2017-06-07 20:04, Adam Borowski wrote: > On Wed, Jun 07, 2017 at 07:01:21PM +0200, Goffredo Baroncelli wrote: >> On 2017-06-07 17:58, Chris Murphy wrote: >>> 3. My take on this would have been to use btrfs restore and go after >>> the file path if I absolutely needed a copy of this file (no back

Re: [PATCH 6/7] Btrfs: rework delayed ref total_bytes_pinned accounting

2017-06-07 Thread Liu Bo
On Tue, Jun 06, 2017 at 04:45:31PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > The total_bytes_pinned counter is completely broken when accounting > delayed refs: > > - If two drops for the same extent are merged, we will decrement > total_bytes_pinned twice but only increment it onc

Re: [PATCH 5/7] Btrfs: return old and new total ref mods when adding delayed refs

2017-06-07 Thread Liu Bo
On Tue, Jun 06, 2017 at 04:45:30PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > We need this to decide when to account pinned bytes. > > Signed-off-by: Omar Sandoval > --- > fs/btrfs/delayed-ref.c | 29 > fs/btrfs/delayed-ref.h | 6 -- > fs/btrfs/exte

[PATCH v2] btrfs: Convert btrfs_fs_info's free_chunk_space to an atomic64_t

2017-06-07 Thread Sargun Dhillon
This patch is a small performance optimization to get rid of a spin lock, where instead an atomic64_t can be used. Signed-off-by: Sargun Dhillon Reviewed-by: Omar Sandoval --- fs/btrfs/ctree.h | 4 ++-- fs/btrfs/disk-io.c | 3 +-- fs/btrfs/extent-tree.c | 4 +--- fs/btrfs/volumes.c

Re: [PATCH] btrfs: Convert btrfs_fs_info's free_chunk_space to an atomic64_t

2017-06-07 Thread Sargun Dhillon
On Wed, Jun 7, 2017 at 11:23 AM, Omar Sandoval wrote: > On Wed, Jun 07, 2017 at 06:09:33PM +, Sargun Dhillon wrote: >> This patch is a small performance optimization to get rid of a spin >> lock, where instead an atomic64_t can be used. >> >> Signed-off-by: Sargun Dhillon > [snip] >> diff --g

Re: [PATCH] btrfs: Convert btrfs_fs_info's free_chunk_space to an atomic64_t

2017-06-07 Thread Omar Sandoval
On Wed, Jun 07, 2017 at 06:09:33PM +, Sargun Dhillon wrote: > This patch is a small performance optimization to get rid of a spin > lock, where instead an atomic64_t can be used. > > Signed-off-by: Sargun Dhillon [snip] > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index 017b67d..0

Re: [PATCH 3/8] btrfs: btrfs_bio_clone never fails, skip error handling

2017-06-07 Thread Omar Sandoval
On Fri, Jun 02, 2017 at 06:58:36PM +0200, David Sterba wrote: > Update direct callers of btrfs_bio_clone that do error handling, that we > can now remove. > > Signed-off-by: David Sterba > --- > fs/btrfs/inode.c | 4 > fs/btrfs/volumes.c | 1 - > 2 files changed, 5 deletions(-) > > >

[PATCH] btrfs: Convert btrfs_fs_info's free_chunk_space to an atomic64_t

2017-06-07 Thread Sargun Dhillon
This patch is a small performance optimization to get rid of a spin lock, where instead an atomic64_t can be used. Signed-off-by: Sargun Dhillon --- fs/btrfs/ctree.h | 4 ++-- fs/btrfs/disk-io.c | 3 +-- fs/btrfs/extent-tree.c | 4 +--- fs/btrfs/volumes.c | 26 +++---

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Adam Borowski
On Wed, Jun 07, 2017 at 07:01:21PM +0200, Goffredo Baroncelli wrote: > On 2017-06-07 17:58, Chris Murphy wrote: > > 3. My take on this would have been to use btrfs restore and go after > > the file path if I absolutely needed a copy of this file (no backup), > > and then copied that back to the fil

Re: [PATCH 0/7] Btrfs: fix total_bytes_pinned counter

2017-06-07 Thread Omar Sandoval
On Wed, Jun 07, 2017 at 05:48:05PM +0200, Holger Hoffstätte wrote: > On 06/07/17 01:45, Omar Sandoval wrote: > > From: Omar Sandoval > > > > This series fixes several problems with the total_bytes_pinned counter. > > Patches 1 and 2 are cleanups. Patches 3 and 4 are straightforward fixes. > > Pat

Re: [PATCH] btrfs: test Btrfs delalloc accounting overflow

2017-06-07 Thread Omar Sandoval
On Wed, Jun 07, 2017 at 05:36:45PM +0800, Eryu Guan wrote: > On Tue, Jun 06, 2017 at 11:57:10PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > This is a regression test for "[PATCH] Btrfs: fix delalloc accounting > > leak caused by u32 overflow". It creates a bunch of delalloc extent

Re: [PATCH 1/3] btrfs-progs: Generic functions to retrieve chunks and their bg info

2017-06-07 Thread Goffredo Baroncelli
Hi, any news about these commands ? BR G.Baroncelli On 2016-10-28 17:44, David Sterba wrote: > On Mon, Oct 17, 2016 at 05:35:13PM -0700, Divya Indi wrote: >> An efficient alternative to retrieving block groups: >> get_chunks(): Walk the chunk tree to retrieve the chunks. >> get_bg_info(): For ea

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Goffredo Baroncelli
On 2017-06-07 17:58, Chris Murphy wrote: > I'm confused about a few items in this thread: > > 1. On normal read, if there's a csum mismatch, there should be a path > to file and EIO, because Btrfs won't propagate bad data up to user > space. I'd expect if there's metadata corruption, there'd be no

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Chris Murphy
I'm confused about a few items in this thread: 1. On normal read, if there's a csum mismatch, there should be a path to file and EIO, because Btrfs won't propagate bad data up to user space. I'd expect if there's metadata corruption, there'd be no path to file, and no EIO. The original email does

Re: [PATCH 0/7] Btrfs: fix total_bytes_pinned counter

2017-06-07 Thread Holger Hoffstätte
On 06/07/17 01:45, Omar Sandoval wrote: > From: Omar Sandoval > > This series fixes several problems with the total_bytes_pinned counter. > Patches 1 and 2 are cleanups. Patches 3 and 4 are straightforward fixes. > Patch 5 is prep for patch 6. Patch 6 is the most complicated fix. > Patches 5 and

[PATCH] fstests: btrfs, test incremental send after renaming and linking file

2017-06-07 Thread fdmanana
From: Filipe Manana Test that an incremental send operation works correctly when an inode A is renamed, a new hard link added to it and some other inode B is renamed to the old name of inode A. The btrfs bug is fixed by the following patch for the linux kernel: "Btrfs: send, fix invalid path

[PATCH] Btrfs: send, fix invalid path after renaming and linking file

2017-06-07 Thread fdmanana
From: Filipe Manana Currently an incremental snapshot can generate link operations which contain an invalid target path. Such case happens when in the send snapshot a file was renamed, a new hard link added for it and some other inode (with a lower number) got renamed to the former name of that f

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Stefan G. Weichinger
Am 2017-06-07 um 15:52 schrieb Adam Borowski: > On Wed, Jun 07, 2017 at 06:48:48PM +0500, Roman Mamedov wrote: >> On Wed, 7 Jun 2017 15:09:02 +0200 >> Adam Borowski wrote: Yes, because btrfs will return -EIO So try dd_rescue >>> >>> Or even plain dd conv=noerror. Both will do a faithful

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Adam Borowski
On Wed, Jun 07, 2017 at 06:48:48PM +0500, Roman Mamedov wrote: > On Wed, 7 Jun 2017 15:09:02 +0200 > Adam Borowski wrote: > > > Yes, because btrfs will return -EIO > > > So try dd_rescue > > > > Or even plain dd conv=noerror. Both will do a faithful analogue of a > > physical disk with a silent

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Roman Mamedov
On Wed, 7 Jun 2017 15:09:02 +0200 Adam Borowski wrote: > On Wed, Jun 07, 2017 at 01:10:26PM +0300, Timofey Titovets wrote: > > 2017-06-07 13:05 GMT+03:00 Stefan G. Weichinger : > > > Am 2017-06-07 um 11:37 schrieb Timofey Titovets: > > > > > >> btrfs scrub start /mnt_path do this trick > > >> > >

mount gets stuck - BUG: soft lockup

2017-06-07 Thread Thomas Mischke
Hello, i tried to convert a JBOD BTRFS consisting of 5 disks (6TB each) to raid10 (converting from an earlier configuration). All disk were backed by bcache. Because a rebalance takes very long I had to pause the balance for a required reboot. After the reboot, the mount is getting stuck and t

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Adam Borowski
On Wed, Jun 07, 2017 at 01:10:26PM +0300, Timofey Titovets wrote: > 2017-06-07 13:05 GMT+03:00 Stefan G. Weichinger : > > Am 2017-06-07 um 11:37 schrieb Timofey Titovets: > > > >> btrfs scrub start /mnt_path do this trick > >> > >> After, you can find info with paths in dmesg > > > > thank you, I t

Re: [PATCH 6/7] Btrfs: __tree_mod_log_insert decrease max compare count

2017-06-07 Thread Timofey Titovets
2017-06-07 12:45 GMT+03:00 Filipe Manana : > On Wed, Jun 7, 2017 at 1:58 AM, Timofey Titovets wrote: >> In worst case code do 4 comparison, >> just add some new checks to switch check branch faster >> now in worst case code do 3 comparison > > Some comments below. > >> >> Signed-off-by: Timofey Ti

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Timofey Titovets
2017-06-07 13:05 GMT+03:00 Stefan G. Weichinger : > Am 2017-06-07 um 11:37 schrieb Timofey Titovets: > >> btrfs scrub start /mnt_path do this trick >> >> After, you can find info with paths in dmesg > > thank you, I think I have the file, it's a qemu-img-file. > I try cp-ing it to another fs first,

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Stefan G. Weichinger
Am 2017-06-07 um 11:37 schrieb Timofey Titovets: > btrfs scrub start /mnt_path do this trick > > After, you can find info with paths in dmesg thank you, I think I have the file, it's a qemu-img-file. I try cp-ing it to another fs first, but assume this will fail, right? -- To unsubscribe from

Re: [PATCH 6/7] Btrfs: __tree_mod_log_insert decrease max compare count

2017-06-07 Thread Filipe Manana
On Wed, Jun 7, 2017 at 1:58 AM, Timofey Titovets wrote: > In worst case code do 4 comparison, > just add some new checks to switch check branch faster > now in worst case code do 3 comparison Some comments below. > > Signed-off-by: Timofey Titovets > --- > fs/btrfs/ctree.c | 20 +++

Re: getting rid of "csum failed" on a hw raid

2017-06-07 Thread Timofey Titovets
2017-06-07 11:48 GMT+03:00 Stefan G. Weichinger : > > greets > > I have a customer server that was set up with a hardware raid and on top > of that /dev/sda exists a btrfs filesystem. > > Yes, I know, that is stupid (can't change that easily). > > In dmesg I get flooding: > > [2329426.792480] BTRFS

Re: [PATCH] btrfs: test Btrfs delalloc accounting overflow

2017-06-07 Thread Eryu Guan
On Tue, Jun 06, 2017 at 11:57:10PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > This is a regression test for "[PATCH] Btrfs: fix delalloc accounting > leak caused by u32 overflow". It creates a bunch of delalloc extents and > merges them together to make sure the accounting is done righ

Re: [PATCH 7/8] btrfs: opencode trivial compressed_bio_alloc, simplify error handling

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: compressed_bio_alloc is now a trivial wrapper around btrfs_bio_alloc, no point keeping it. The error handling can be simplified, as we know btrfs_bio_alloc will never fail. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubsc

Re: [PATCH 8/8] btrfs: pass bytes to btrfs_bio_alloc

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: Most callers of btrfs_bio_alloc convert from bytes to sectors. Hide that in the helper and simplify the logic in the callsers. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to

Re: [PATCH 5/8] btrfs: sink gfp parameter to btrfs_bio_clone

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: All callers pass GFP_NOFS. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/8] btrfs: btrfs_io_bio_alloc never fails, skip error handling

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: Update direct callers of btrfs_io_bio_alloc that do error handling, that we can now remove. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More maj

Re: [PATCH 3/8] btrfs: btrfs_bio_clone never fails, skip error handling

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: Update direct callers of btrfs_bio_clone that do error handling, that we can now remove. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordo

Re: [PATCH 1/8] btrfs: bioset allocations will never fail, adapt our helpers

2017-06-07 Thread Anand Jain
Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/8] btrfs: btrfs_bio_alloc never fails, skip error handling

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: Update direct callers of btrfs_bio_alloc that do error handling, that we can now remove. Reviewed-by: Anand Jain -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordo

getting rid of "csum failed" on a hw raid

2017-06-07 Thread Stefan G. Weichinger
greets I have a customer server that was set up with a hardware raid and on top of that /dev/sda exists a btrfs filesystem. Yes, I know, that is stupid (can't change that easily). In dmesg I get flooding: [2329426.792480] BTRFS warning (device sda): csum failed ino 7454384 off 81708052480 csum

Re: [PATCH 6/8] btrfs: remove redundant parameters from btrfs_bio_alloc

2017-06-07 Thread Anand Jain
On 06/03/17 00:58, David Sterba wrote: All callers pass gfp_flags=GFP_NOFS and nr_vecs=BIO_MAX_PAGES. The line (in the other thread) mentioning the reason to remove __GFP_HIGH can go into the commit log here. - bio = btrfs_bio_alloc(bdev, sector, BIO_MAX_PAGES, -