Re: Spurious "ghost" "parent transid verify failed" messages on 5.0.21 - with call traces

2019-07-02 Thread Qu Wenruo
On 2019/7/3 下午12:32, Zygo Blaxell wrote: > On Mon, Jul 01, 2019 at 01:56:08PM +0800, Qu Wenruo wrote: >> >> >> On 2019/7/1 上午11:39, Zygo Blaxell wrote: >>> On Wed, Apr 03, 2019 at 10:47:16AM -0400, Zygo Blaxell wrote: On Tue, Mar 12, 2019 at 12:00:25AM -0400, Zygo Blaxell wrote: > On 4.1

Re: What are the maintenance recommendation ?

2019-07-02 Thread Zygo Blaxell
On Sat, Jun 29, 2019 at 08:50:03PM +0200, Pierre Couderc wrote: > 1- Is there a summary of btrfs recommendations for maintenance ? > > I have read somewhere that  a monthly  btrfs scrub is recommended. 1. Scrub detects and (when using the DUP or RAID1/10/5/6 profiles) corrects errors introduced

Re: Spurious "ghost" "parent transid verify failed" messages on 5.0.21 - with call traces

2019-07-02 Thread Zygo Blaxell
On Mon, Jul 01, 2019 at 01:56:08PM +0800, Qu Wenruo wrote: > > > On 2019/7/1 上午11:39, Zygo Blaxell wrote: > > On Wed, Apr 03, 2019 at 10:47:16AM -0400, Zygo Blaxell wrote: > >> On Tue, Mar 12, 2019 at 12:00:25AM -0400, Zygo Blaxell wrote: > >>> On 4.14.x and 4.20.14 kernels (probably all the ones

Re: [PATCH][next] btrfs: fix memory leak of path on error return path

2019-07-02 Thread Anand Jain
On 2/7/19 10:10 PM, Colin King wrote: From: Colin Ian King Currently if the allocation of roots or tmp_ulist fails the error handling does not free up the allocation of path causing a memory leak. Fix this by freeing path with a call to btrfs_free_path before taking the error return path. Addr

[PATCH] btrfs: Fix build error while LIBCRC32C is module

2019-07-02 Thread YueHaibing
If CONFIG_BTRFS_FS is y and CONFIG_LIBCRC32C is m, building fails: fs/btrfs/super.o: In function `btrfs_mount_root': super.c:(.text+0xb7f9): undefined reference to `crc32c_impl' fs/btrfs/super.o: In function `init_btrfs_fs': super.c:(.init.text+0x3465): undefined reference to `crc32c_impl' fs/btrf

[PATCH] Btrfs: remove unnecessary condition in btrfs_clone() to avoid too much nesting

2019-07-02 Thread fdmanana
From: Filipe Manana The bulk of the work done when cloning extents, at ioctl.c:btrfs_clone(), is done inside an if statement that checks if the found key has the type BTRFS_EXTENT_DATA_KEY. That if statement is redundant however, because btrfs_search_slot() always leaves us in a leaf slot that po

[PATCH -next] btrfs: remove set but not used variable 'offset'

2019-07-02 Thread YueHaibing
Fixes gcc '-Wunused-but-set-variable' warning: fs/btrfs/volumes.c: In function __btrfs_map_block: fs/btrfs/volumes.c:6023:6: warning: variable offset set but not used [-Wunused-but-set-variable] It is not used any more since commit 343abd1c0ca9 ("btrfs: Use btrfs_get_io_geometry appropriately")

[PATCH][next] btrfs: fix memory leak of path on error return path

2019-07-02 Thread Colin King
From: Colin Ian King Currently if the allocation of roots or tmp_ulist fails the error handling does not free up the allocation of path causing a memory leak. Fix this by freeing path with a call to btrfs_free_path before taking the error return path. Addresses-Coverity: ("Resource leak") Fixes:

Re: btrfs vs write caching firmware bugs (was: Re: BTRFS recovery not possible)

2019-07-02 Thread Andrea Gelmini
On Mon, Jun 24, 2019 at 11:31:35AM -0600, Chris Murphy wrote: > Right. The questions I have: should Btrfs (or any file system) be able > to detect such devices and still protect the data? i.e. for the file I have more than 600 industrial machine all around the world. After a few fs corruption (ext

Re: [PATCH v2 4/4] vfs: don't allow most setxattr to immutable files

2019-07-02 Thread Amir Goldstein
On Mon, Jul 1, 2019 at 7:31 PM Darrick J. Wong wrote: > > From: Darrick J. Wong > > The chattr manpage has this to say about immutable files: > > "A file with the 'i' attribute cannot be modified: it cannot be deleted > or renamed, no link can be created to this file, most of the file's > metadat

[PATCH v2 14/14] btrfs-progs: image: Fix error output to show correct return value

2019-07-02 Thread WenRuo Qu
We can easily get confusing error message like: ERROR: restore failed: Success This is caused by wrong "%m" usage, as we normally use ret to indicate error, without populating errno. This patch will fix it by output the return value directly as normally we have extra error message to show more

[PATCH v2 01/14] btrfs-progs: image: Use SZ_* to replace intermediate size

2019-07-02 Thread WenRuo Qu
Signed-off-by: Qu Wenruo --- image/metadump.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/image/metadump.h b/image/metadump.h index 8ace60f503d0..f85c9bcfb813 100644 --- a/image/metadump.h +++ b/image/metadump.h @@ -23,8 +23,8 @@ #include "ctree.h" #define HEADER_M

[PATCH v2 13/14] btrfs-progs: image: Output error message for chunk tree build error

2019-07-02 Thread WenRuo Qu
Signed-off-by: Qu Wenruo --- image/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/image/main.c b/image/main.c index 2256138df079..0be3b45569ed 100644 --- a/image/main.c +++ b/image/main.c @@ -2811,8 +2811,10 @@ static int restore_metadump(const char *input, FILE *o

[PATCH v2 00/14] btrfs-progs: image: Enhance and bug fixes

2019-07-02 Thread WenRuo Qu
This patchset is based on v5.1.1 tag. With this update, the patchset has the following features: - various small fixes and enhancements for btrfs-image * Fix an indent misalign * Fix an access-beyond-boundary bug * Fix a confusing error message due to unpopulated errno * Output error messa

[PATCH v2 12/14] btrfs-progs: image: Reduce memory usage for chunk tree search

2019-07-02 Thread WenRuo Qu
Just like original restore_worker, search_for_chunk_blocks() will also use a lot of memory for restoring large uncompressed extent or compressed extent with data dump. Reduce the memory usage by: - Use fixed buffer size for uncompressed extent Now we will use fixed 512K as buffer size for readin

[PATCH v2 08/14] btrfs-progs: image: Introduce helper to determine if a tree block is in the range of system chunks

2019-07-02 Thread WenRuo Qu
Introduce a new helper function, is_in_sys_chunks(), to determine if an item is in the range of system chunks. Since btrfs-image will merge adjacent same type extents into one item, this function is designed to return true for any bytes in system chunk range. Signed-off-by: Qu Wenruo --- image/

[PATCH v2 05/14] btrfs-progs: image: Introduce framework for more dump versions

2019-07-02 Thread WenRuo Qu
The original dump format only contains a @magic member to verify the format, this means if we want to introduce new on-disk format or change certain size limit, we can only introduce new magic as kind of version. This patch will introduce the framework to allow multiple magic to co-exist for furth

[PATCH v2 11/14] btrfs-progs: image: Don't waste memory when we're just extracting super block

2019-07-02 Thread WenRuo Qu
There is no need to allocate 2 * max_pending_size (which can be 256M) if we're just extracting super block. We only need to prepare BTRFS_SUPER_INFO_SIZE as buffer size. Signed-off-by: Qu Wenruo --- image/main.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/image/

[PATCH v2 07/14] btrfs-progs: image: Allow restore to record system chunk ranges for later usage

2019-07-02 Thread WenRuo Qu
Currently we are doing a pretty slow search for system chunks before restoring real data. The current behavior is to search all clusters for chunk tree root first, then search all clusters again and again for every chunk tree block. This causes recursive calls and pretty slow start up, the only go

[PATCH v2 10/14] btrfs-progs: image: Reduce memory requirement for decompression

2019-07-02 Thread WenRuo Qu
With recent change to enlarge max_pending_size to 256M for data dump, the decompress code requires quite a lot of memory space. (256M * 4). The main reason behind it is, we're using wrapped uncompress() function call, which needs the buffer to be large enough to contain the decompressed data. Thi

[PATCH v2 09/14] btrfs-progs: image: Rework how we search chunk tree blocks

2019-07-02 Thread WenRuo Qu
Before this patch, we were using a very inefficient way to search chunks: We iterate through all clusters to find the chunk root tree block first, then re-iterate all clusters again to find every child tree blocks. Every time we need to iterate all clusters just to find a chunk tree block. This i

[PATCH v2 06/14] btrfs-progs: image: Introduce -d option to dump data

2019-07-02 Thread WenRuo Qu
This new data dump feature will dump the whole image, not long the existing tree blocks but also all its data extents(*). This feature will rely on the new dump format (_DUmP_v1), as it needs extra large extent size limit, and older btrfs-image dump can't handle such large item/cluster size. Sinc

[PATCH v2 04/14] btrfs-progs: image: Verify the superblock before restore

2019-07-02 Thread WenRuo Qu
This patch will export disk-io.c::check_super() as btrfs_check_super() and use it in btrfs-image for extra verification. Signed-off-by: Qu Wenruo --- disk-io.c| 6 +++--- disk-io.h| 1 + image/main.c | 5 + 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/disk-io.c b/di

[PATCH v2 02/14] btrfs-progs: image: Fix an indent misalign

2019-07-02 Thread WenRuo Qu
Signed-off-by: Qu Wenruo --- image/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/image/main.c b/image/main.c index 86845dadc958..9a07d9455e4f 100644 --- a/image/main.c +++ b/image/main.c @@ -2645,7 +2645,7 @@ int main(int argc, char *argv[]) cr

[PATCH v2 03/14] btrfs-progs: image: Fix an access-beyond-boundary bug when there are 32 online CPUs

2019-07-02 Thread WenRuo Qu
[BUG] When there are over 32 (in my example, 35) online CPUs, btrfs-image -c9 will just hang. [CAUSE] Btrfs-image has a hard coded limit (32) on how many threads we can use. For the "-t" option we do the up limit check. But when we don't specify "-t" option and speicified "-c" option, then btrfs-

Re: [PATCH v7 RESEND Rebased] btrfs-progs: dump-tree: add noscan option

2019-07-02 Thread Anand Jain
Ping? On 26/6/19 4:30 PM, Anand Jain wrote: From: Anand Jain The cli 'btrfs inspect dump-tree ' will scan for the partner devices if any by default. So as of now you can not inspect each mirrored device independently. This patch adds noscan option, which when used won't scan the system for

Re: [PATCH 0/3 RESEND Rebased] readmirror feature

2019-07-02 Thread Anand Jain
Ping? On 26/6/19 4:33 PM, Anand Jain wrote: These patches are tested to be working fine. Function call chain __btrfs_map_block()->find_live_mirror() uses thread pid to determine the %mirror_num when the mirror_num=0. This patch introduces a framework so that we can add policies to determine