Re: [dm-devel] [PATCH 08/15] dm mpath: merge do_end_io_bio into multipath_end_io_bio

On Mon, May 22, 2017 at 08:51:20PM +0200, Martin Wilck wrote: > >   > > - if (!error) > > - return 0; /* I/O complete */ > > + BUG_ON(!mpio); > > You dereferenced mpio already above. Indeed. I removed the BUG_ON for the next version. -- To unsubscribe from this list: send

Re: [PATCH 02/15] scsi/osd: don't save block errors into req_results

On Wed, May 24, 2017 at 04:04:40PM +, Bart Van Assche wrote: > Are you sure that that code is not necessary? From osd_initiator.c: > > static void _put_request(struct request *rq) > { > /* >* If osd_finalize_request() was called but the request was not >* executed

[PATCH v2 1/5] btrfs-progs: check: inode nbytes fix in lowmem

After checking one inode item, we should get the actual nbytes of the inode item. Introduce function 'repair_inode_nbytes_lowmem' to set nbytes in struct btrfs_inode_item to the actual nbytes. After call of the function, the wrong nbytes should have been corrected. Signed-off-by: Su Yue

[PATCH v2 0/5] btrfs-progs: check: simple errors repair in lowmem

The series include following contents: 1) Repair wrong nbytes of file inode item. After traversal of extents in one file, we should get the actual nbytes of the file. If nbytes in the file inode differs from the actual value, set the value to actual one. The wrong nbytes of file

[PATCH v2 4/5] btrfs-progs: fsck-check: Allow fsck check test to repair in lowmem mode for certain test cases

From: Qu Wenruo Since lowmem mode can repair certain corruptions (mostly in fs tree), insert a beacon into each fsck test cases to allow some of them be tested for lowmem mode. With this patch, fsck option override will check the beacon file ".lowmem_repairbale" in the

[PATCH v2 3/5] btrfs-progs: check: enable lowmem repair

Enable btrfsck option '--repair' with option '--mode=lowmem'. Now lowmem mode only repairs wrong nbytes, dir isize. Signed-off-by: Su Yue --- cmds-check.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index

[PATCH v2 2/5] btrfs-progs: check: dir isize fix in lowmem

After traversal of whole directory, we should get the actual isize. Introduce function 'repair_dir_isize_lowmem' to set isize in the directory inode item to actual size. After call of the function, the wrong dir isize should have been corrected. Signed-off-by: Su Yue

[PATCH v2 5/5] btrfs-progs: fsck-check: test cases for nbytes and dir isize

Create test case '026-wrong-dir-inode-isize'. Create becon files '.lowmem_repairable' under tests/fsck-test/016 and 026. Now 'make test-fsck' will test lowmem repairable test cases if TEST_ENABLE_OVERRIDE=true. Signed-off-by: Su Yue ---

Re: [PATCH] Btrfs: skip commit transaction if we don't have enough pinned bytes

On Thu, May 25, 2017 at 06:50:48PM +0200, David Sterba wrote: > On Tue, May 23, 2017 at 12:06:40PM +0300, Nikolay Borisov wrote: > > > > > > On 19.05.2017 20:39, Liu Bo wrote: > > > We commit transaction in order to reclaim space from pinned bytes because > > > it could process delayed refs, and

[PATCH 6/6] Btrfs: add sanity check of extent item in scrub

Currently scrub only verify checksum of both metadata and data and couldn't detect an invalid extent_item. This adds sanity check for extent item, now it can check if extent_inline_ref_type is valid. Signed-off-by: Liu Bo --- fs/btrfs/scrub.c | 43

[PATCH 2/6] Btrfs: convert to use btrfs_get_extent_inline_ref_type

Since we have a helper which can do sanity check, this converts all btrfs_extent_inline_ref_type to it. Signed-off-by: Liu Bo --- fs/btrfs/backref.c | 9 +++-- fs/btrfs/extent-tree.c | 33 ++--- fs/btrfs/relocation.c | 15

[PATCH 1/6] Btrfs: add a helper to retrive extent inline ref type

An invalid value of extent inline ref type may be read from a malicious image which may force btrfs to crash. This adds a helper which does sanity check for the ref type, so we can know if it's sane, return type if so, otherwise return an error. Signed-off-by: Liu Bo ---

[PATCH 5/6] Btrfs: remove BUG() in add_data_reference

Now that we have a helper to report invalid value of extent inline ref type, we need to quit gracefully instead of throwing out a kernel panic. Signed-off-by: Liu Bo --- fs/btrfs/relocation.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git

[PATCH 4/6] Btrfs: remove BUG() in print_extent_item

btrfs_print_leaf() is used in btrfs_get_extent_inline_ref_type, so here we really want to print the invalid value of ref type instead of causing a kernel panic. Signed-off-by: Liu Bo --- fs/btrfs/print-tree.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-)

[PATCH 3/6] Btrfs: remove BUG() in btrfs_extent_inline_ref_size

Now that btrfs_get_extent_inline_ref_type() can report if type is a valid one and all callers can gracefully deal with that, we don't need to crash here. Signed-off-by: Liu Bo --- fs/btrfs/ctree.h | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/ctree.h

[PATCH 0/6] add sanity check for extent inline ref type

An invalid extent inline ref type could be read from a btrfs image and it ends up with a panic[1], this set is to deal with the insane value gracefully in patch 1-2 and clean up BUG() in the code in patch 3-5. Patch 6 adds scrub support to detect the corruption, so users can be noticed when they

[PATCH] Btrfs: let btrfs_print_leaf print more about block group

This adds chunk_objectid and flags, with flags we can recognize whether the block group is about data or metadata. Signed-off-by: Liu Bo --- fs/btrfs/print-tree.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/print-tree.c

Re: [PATCH] btrfs: Use kvzalloc instead of kzalloc/vmalloc in alloc_bitmap

On Thu, May 25, 2017 at 12:18:02PM -0700, Vinnie Magro wrote: > Replace alloc_bitmap with call to kvzalloc. kvzalloc preserves the > same fallback heuristic. Looks good, thanks. Reviewed-by: Omar Sandoval > Signed-off-by: Vinnie Magro > --- >

[PATCH] btrfs: Use kvzalloc instead of kzalloc/vmalloc in alloc_bitmap

Replace alloc_bitmap with call to kvzalloc. kvzalloc preserves the same fallback heuristic. Signed-off-by: Vinnie Magro --- fs/btrfs/free-space-tree.c | 23 ++- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/fs/btrfs/free-space-tree.c

[PATCH v4 2/2] Btrfs: compression must free at least one sector size

Btrfs already skip store of data where compression didn't free at least one byte. Let's make logic better and make check that compression free at least one sector size because in another case it useless to store this data compressed Signed-off-by: Timofey Titovets ---

[PATCH v4 1/2] Btrfs: lzo.c pr_debug() deflate->lzo

Fix copy paste typo in debug message for lzo.c, lzo is not deflate Signed-off-by: Timofey Titovets --- fs/btrfs/lzo.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index f48c8c14..bd0b0938 100644 --- a/fs/btrfs/lzo.c

[PATCH v4 0/2] Btrfs: compression fixes

First patch: Fix copy paste typo in debug message for lzo.c, lzo is not deflate Second patch: Force btrfs to not store data as compressed, if compression will not free at least one sector size, because it's useless in term of saving storage space and reading data from disk, as a result

Re: [PATCH v3 2/2] Btrfs: compression must free at least PAGE_SIZE

2017-05-25 15:51 GMT+03:00 Chandan Rajendra : ... > Apologies for the delayed response. > > I am not really sure if compression code must save atleast one sectorsize > worth of space. But if other developers agree to it, then the above > 'if' condition can be replaced

Re: [PATCH] Btrfs: clear EXTENT_DEFRAG bits in finish_ordered_io

On Fri, May 19, 2017 at 01:01:42PM -0700, Liu Bo wrote: > On Fri, May 19, 2017 at 09:06:42PM +0200, David Sterba wrote: > > On Tue, May 09, 2017 at 05:02:15PM -0600, Liu Bo wrote: > > > Before this, we use 'filled' mode here, ie. if all range has been filled > > > with EXTENT_DEFRAG bits, get to

Re: [PATCH] Btrfs: skip commit transaction if we don't have enough pinned bytes

On Tue, May 23, 2017 at 12:06:40PM +0300, Nikolay Borisov wrote: > > > On 19.05.2017 20:39, Liu Bo wrote: > > We commit transaction in order to reclaim space from pinned bytes because > > it could process delayed refs, and in may_commit_transaction(), we check > > first if pinned bytes are

Re: [PATCH] Btrfs: work around maybe-uninitialized warning

On Fri, May 19, 2017 at 09:20:53PM +0200, Arnd Bergmann wrote: > On Fri, May 19, 2017 at 8:10 PM, Liu Bo wrote: > > On Thu, May 18, 2017 at 03:33:29PM +0200, Arnd Bergmann wrote: > >> A rewrite of btrfs_submit_direct_hook appears to have introduced a warning: > >> > >>

Re: [PATCH v3 2/2] Btrfs: compression must free at least PAGE_SIZE

On Sunday, May 21, 2017 12:10:39 AM IST Timofey Titovets wrote: > Btrfs already skip store of data where compression didn't free at least one > byte. > So make logic better and make check that compression free at least one > PAGE_SIZE, > because in another case it useless to store this data

[PATCH] btrfs: btrfs_wait_tree_block_writeback can be void return

Nothing checks its return value. Signed-off-by: Jeff Layton Reviewed-by: Jan Kara Reviewed-by: Liu Bo --- fs/btrfs/disk-io.c | 6 +++--- fs/btrfs/disk-io.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git

Re: [PATCH 06/10] fs: Introduce IOMAP_NOWAIT

On Wed 24-05-17 11:41:46, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > IOCB_NOWAIT translates to IOMAP_NOWAIT for iomaps. > This is used by XFS in the XFS patch. > > Signed-off-by: Goldwyn Rodrigues > Reviewed-by: Christoph Hellwig

Re: [PATCH 05/10] fs: return if direct write will trigger writeback

On Wed 24-05-17 11:41:45, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Find out if the write will trigger a wait due to writeback. If yes, > return -EAGAIN. > > Return -EINVAL for buffered AIO: there are multiple causes of > delay such as page locks, dirty

Re: [PATCH 04/10] fs: Introduce RWF_NOWAIT

On Wed 24-05-17 11:41:44, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > RWF_NOWAIT informs kernel to bail out if an AIO request will block > for reasons such as file allocations, or a writeback triggered, > or would block while allocating requests while performing >

Re: [PATCH 03/10] fs: Use RWF_* flags for AIO operations

On Wed 24-05-17 11:41:43, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > aio_rw_flags is introduced in struct iocb (using aio_reserved1) which will > carry the RWF_* flags. We cannot use aio_flags because they are not > checked for validity which may break existing

Re: [PATCH 02/10] fs: Introduce filemap_range_has_page()

On Wed 24-05-17 11:41:42, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > filemap_range_has_page() return true if the file's mapping has > a page within the range mentioned. This function will be used > to check if a write() call will cause a writeback of previous >

Re: [PATCH 01/10] fs: Separate out kiocb flags setup based on RWF_* flags

On Wed 24-05-17 11:41:41, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Signed-off-by: Goldwyn Rodrigues > Reviewed-by: Christoph Hellwig Looks good. You can add: Reviewed-by: Jan Kara

[PATCH v4 01/20] btrfs-progs: raid56: Introduce raid56 header for later recovery usage

Introduce a new header, kernel-lib/raid56.h, for later raid56 works. It contains 2 functions, from original btrfs-progs code: void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs); int raid5_gen_result(int nr_devs, size_t stripe_len, int dest, void **data); Will be expanded later and

[PATCH v4 13/20] btrfs-progs: scrub: Introduce function to scrub one data stripe

Introduce new function, scrub_one_data_stripe(), to check all data and tree blocks inside the data stripe. This function will not try to recovery any error, but only check if any data/tree blocks has mismatch csum. If data missing csum, which is completely valid for case like nodatasum, it will

[PATCH v4 07/20] btrfs-progs: Allow __btrfs_map_block_v2 to remove unrelated stripes

For READ, caller normally hopes to get what they request, other than full stripe map. In this case, we should remove unrelated stripe map, just like the following case: 32K 96K |<-request range->| 0 64k 128K RAID0: |

[PATCH v4 18/20] btrfs-progs: scrub: Introduce a function to scrub one full stripe

Introduce a new function, scrub_one_full_stripe(), to check a full stripe. It handles the full stripe scrub in the following steps: 0) Check if we need to check full stripe If full stripe contains no extent, why waste our CPU and IO? 1) Read out full stripe Then we know how many devices

[PATCH v4 19/20] btrfs-progs: scrub: Introduce function to check a whole block group

Introduce new function, scrub_one_block_group(), to scrub a block group. For Single/DUP/RAID0/RAID1/RAID10, we use old mirror number based map_block, and check extent by extent. For parity based profile (RAID5/6), we use new map_block_v2() and check full stripe by full stripe. Signed-off-by: Qu

[PATCH v4 09/20] btrfs-progs: scrub: Introduce structures to support offline scrub for RAID56

Introuduce new local structures, scrub_full_stripe and scrub_stripe, for incoming offline RAID56 scrub support. For pure stripe/mirror based profiles, like raid0/1/10/dup/single, we will follow the original bytenr and mirror number based iteration, so they don't need any extra structures for

[PATCH v4 10/20] btrfs-progs: scrub: Introduce functions to scrub mirror based tree block

Introduce new functions, check/recover_tree_mirror(), to check and recover mirror-based tree blocks (Single/DUP/RAID0/1/10). check_tree_mirror() can also be used on in-memory tree blocks using @data parameter. This is very handy for RAID5/6 case, either checking the data stripe tree block by

[PATCH v4 11/20] btrfs-progs: scrub: Introduce functions to scrub mirror based data blocks

Introduce new function, check/recover_data_mirror(), to check and recover mirror based data blocks. Unlike tree block, data blocks must be recovered sector by sector, so we introduced corrupted_bitmap for check and recover. Signed-off-by: Qu Wenruo Signed-off-by: Su Yue

[PATCH v4 12/20] btrfs-progs: scrub: Introduce function to scrub one mirror-based extent

Introduce a new function, scrub_one_extent(), as a wrapper to check one mirror-based extent. It will accept a btrfs_path parameter @path, which must points to a META/EXTENT_ITEM. And @start, @len, which must be a subset of META/EXTENT_ITEM. Signed-off-by: Qu Wenruo ---

[PATCH v4 00/20] Btrfs-progs offline scrub

For any one who wants to try it, it can be get from my repo: https://github.com/adam900710/btrfs-progs/tree/offline_scrub Several reports on kernel scrub screwing up good data stripes are in ML for sometime. And since kernel scrub won't account P/Q corruption, it makes us quite hard to detect

[PATCH v4 17/20] btrfs-progs: scrub: Introduce helper to write a full stripe

Introduce a internal helper, write_full_stripe() to calculate P/Q and write the whole full stripe. This is useful to recover RAID56 stripes. Signed-off-by: Qu Wenruo --- scrub.c | 44 1 file changed, 44 insertions(+) diff

[PATCH v4 20/20] btrfs-progs: scrub: Introduce offline scrub function

Now, btrfs-progs has a kernel scrub equivalent. A new option, --offline is added to "btrfs scrub start". If --offline is given, btrfs scrub will just act like kernel scrub, to check every copy of extent and do a report on corrupted data and if it's recoverable. The advantage compare to kernel

[PATCH v4 05/20] btrfs-progs: Introduce wrapper to recover raid56 data

Introduce a wrapper to recover raid56 data. The logical is the same with kernel one, but with different interfaces, since kernel ones cares the performance while in btrfs we don't care that much. And the interface is more caller friendly inside btrfs-progs. Signed-off-by: Qu Wenruo

[PATCH v4 16/20] btrfs-progs: scrub: Introduce function to recover data parity

Introduce function, recover_from_parities(), to recover data stripes. It just wraps raid56_recov() with extra check functions to scrub_full_stripe structure. Signed-off-by: Qu Wenruo --- scrub.c | 51 +++ 1 file changed,

[PATCH v4 15/20] btrfs-progs: extent-tree: Introduce function to check if there is any extent in given range.

Introduce a new function, btrfs_check_extent_exists(), to check if there is any extent in the range specified by user. The parameter can be a large range, and if any extent exists in the range, it will return >0 (in fact it will return 1). Or return 0 if no extent is found. Signed-off-by: Qu

[PATCH v4 14/20] btrfs-progs: scrub: Introduce function to verify parities

Introduce new function, verify_parities(), to check if parities matches for full stripe which all data stripes matches with their csum. Caller should fill the scrub_full_stripe structure properly before calling this function. Signed-off-by: Qu Wenruo --- scrub.c | 69

[PATCH v4 04/20] btrfs-progs: raid56: Allow raid6 to recover data and p

Copied from kernel lib/raid6/recov.c. Minor modifications includes: - Rename from raid6_datap_recov_intx() to raid5_recov_datap() - Rename parameter from faila to dest1 Signed-off-by: Qu Wenruo --- kernel-lib/raid56.c | 41 +

[PATCH v4 02/20] btrfs-progs: raid56: Introduce tables for RAID6 recovery

Use kernel RAID6 galois tables for later RAID6 recovery. Galois tables file, kernel-lib/tables.c is generated by user space program, mktable. Galois field tables declaration, in kernel-lib/raid56.h, is completely copied from kernel. The mktables.c is copied from kernel with minor header/macro

[PATCH v4 03/20] btrfs-progs: raid56: Allow raid6 to recover 2 data stripes

Copied from kernel lib/raid6/recov.c raid6_2data_recov_intx1() function. With the following modification: - Rename to raid6_recov_data2() for shorter name - s/kfree/free/g modification Signed-off-by: Qu Wenruo --- Makefile| 4 +-- raid56.c =>

[PATCH v4 08/20] btrfs-progs: csum: Introduce function to read out data csums

Introduce a new function: btrfs_read_data_csums(), to read out csums for sectors in range. This is quite useful for read out data csum so we don't need to do it using open code. Signed-off-by: Qu Wenruo Signed-off-by: Su Yue --- Makefile |

[PATCH v4 06/20] btrfs-progs: Introduce new btrfs_map_block function which returns more unified result.

Introduce a new function, __btrfs_map_block_v2(). Unlike old btrfs_map_block(), which needs different parameter to handle different RAID profile, this new function uses unified btrfs_map_block structure to handle all RAID profile in a more meaningful method: Return physical address along with